Age-associated B cells (ABC) accumulate with age and in individuals with different immunological disorders, including cancer patients treated with immune checkpoint blockade and those with inborn errors of immunity. Here, we investigate whether ABCs from different conditions are similar and how they impact the longitudinal level of the COVID-19 vaccine response. Single-cell RNA sequencing indicates that ABCs with distinct aetiologies have common transcriptional profiles and can be categorised according to their expression of immune genes, such as the autoimmune regulator (AIRE). Furthermore, higher baseline ABC frequency correlates with decreased levels of antigen-specific memory B cells and reduced neutralising capacity against SARS-CoV-2. ABCs express high levels of the inhibitory FcγRIIB receptor and are distinctive in their ability to bind immune complexes, which could contribute to diminish vaccine responses either directly, or indirectly via enhanced clearance of immune complexed-antigen. Expansion of ABCs may, therefore, serve as a biomarker identifying individuals at risk of suboptimal responses to vaccination.