Tissue-resident macrophages adapt to local signals within tissues to acquire specific functions. Neoplasia transforms the tissue, raising the question as to how the environmental perturbations contribute to tumor-associated macrophage (TAM) identity and functions. Combining single-cell RNA sequencing (scRNA-seq) with spatial localization of distinct TAM subsets by imaging, we discover that TAM transcriptomic programs follow two main differentiation paths according to their localization in the stroma or in the neoplastic epithelium of the mammary duct. Furthermore, this diversity is exclusively detected in a spontaneous tumor model and tracks the different tissue territories as well as the type of tumor lesion. These TAM subsets harbor distinct capacity to activate CD8+ T cells and phagocyte tumor cells, supporting that specific tumor regions, rather than defined activation states, are the major drivers of TAM plasticity and heterogeneity. The distinctions created here provide a framework to design cancer treatment targeting specific TAM niches. Keywords: CP: Cancer; CP: Immunology; Multiphoton imaging; breast cancer; scRNA-seq; spatial distribution; tumor-associated macrophages.