Developmental osteogenesis, physiological bone remodelling and fracture healing require removal of matrix and cellular debris. Osteoclasts generated by the fusion of circulating monocytes degrade bone, whereas the identity of the cells responsible for cartilage resorption is a long-standing and controversial question. Here we show that matrix degradation and chondrocyte phagocytosis are mediated by fatty acid binding protein 5-expressing cells representing septoclasts, which have a mesenchymal origin and are not derived from haematopoietic cells. The Notch ligand Delta-like 4, provided by endothelial cells, is necessary for septoclast specification and developmental bone growth. Consistent with the termination of growth, septoclasts disappear in adult and ageing bone, but re-emerge in association with growing vessels during fracture healing. We propose that cartilage degradation is mediated by rare, specialized cells distinct from osteoclasts. Our findings have implications for fracture healing, which is frequently impaired in aging humans.