Background: Hormonal therapy targeting the androgen receptor inhibits prostate cancer (PCa), but the tumor eventually recurs as castration-resistant prostate cancer (CRPC). Objective: To understand the mechanisms by which subclones within early PCa develop into CRPC. Design, setting, and participants: We isolated epithelial cells from fresh human PCa cases, including primary adenocarcinoma, locally recurrent CRPC, and metastatic CRPC, and utilized single-cell RNA sequencing to identify subpopulations destined to become either CRPC-adeno or small cell neuroendocrine carcinoma (SCNC). Outcome measurements and statistical analysis: We revealed dynamic transcriptional reprogramming that promotes disease progression among 23226 epithelial cells using single-cell RNA sequencing, and validated subset-specific progression using immunohistochemistry and large cohorts of publically available genomic data. Results and limitations: We identified a small fraction of highly plastic CRPC-like cells in hormone-naïve early PCa and demonstrated its correlation with biochemical recurrence and distant metastasis, independent of clinical characteristics. We show that progression toward castration resistance was initiated from subtype-specific lineage plasticity and clonal expansion of pre-existing neuroendocrine and CRPC-like cells in early PCa. Conclusions: CRPC-like cells are present early in the development of PCa and are not exclusively the result of acquired evolutionary selection during androgen deprivation therapy. The lethal CRPC and SCNC phenotypes should be targeted earlier in the disease course of patients with PCa. Patient summary: Here, we report the presence of pre-existing castration-resistant prostate cancer (CRPC)-like cells in primary prostate cancer, which represents a novel castration-resistant mechanism different from the adaptation mechanism after androgen deprivation therapy (ADT). Patients whose tumors harbor increased pre-existing neuroendocrine and CRPC-like cells may become rapidly resistant to ADT and may require aggressive early intervention. Keywords: Castration-resistant prostate cancer; Castration-resistant prostate cancer–like cells; Critical transcription regulator; Evolutionary trajectory; Intratumor heterogeneity; Large population validation; Neuroendocrine differentiation; Primary prostate cancer; Single-cell transcriptomes.