Despite mRNA vaccination, elderly individuals remain especially vulnerable to severe consequences of SARS-CoV-2 infection. Here, we compare the memory B cell responses in a cohort of elderly and younger individuals who received mRNA booster vaccinations. Plasma neutralizing potency and breadth were similar between the two groups. By contrast, the absolute number of SARS-CoV-2-specific memory B cells was lower in the elderly. Antibody sequencing revealed that the SARS-CoV-2-specific elderly memory compartments were more clonal and less diverse. Notably, memory antibodies from the elderly preferentially targeted the ACE2-binding site on the RBD, while those from younger individuals targeted less accessible but more conserved epitopes. Nevertheless, individual memory antibodies elicited by booster vaccines in the elderly and younger individuals showed similar levels of neutralizing activity and breadth against SARS-CoV-2 variants. Thus, the relatively diminished protective effects of vaccination against serious disease in the elderly are associated with a smaller number of antigen-specific memory B cells that express altered antibody repertoires.