In this study, we compared the humoral immune characteristics of children, elderly individuals, and pregnant women in Guangzhou, China, who had been infected with the SARS-CoV-2 strains BA.5 and XBB against the currently predominant strains EG.5 and JN.1. It was discovered that the neutralizing antibody titers in children, elderly individuals, and pregnant women against strains EG.5 and JN.1 were low in individuals who had been infected with strain BA.5, irrespective of their vaccination status. There was a significant positive correlation between the neutralization titers against JN.1 and EG.5 in both the acute and convalescent phases of BA.5 infection. For XBB-infected patients, the sera in the acute stage exhibited a low neutralizing titer against EG.5 and JN.1, whereas the convalescent sera demonstrated a significantly higher neutralizing titer against the two viruses, particularly in infected individuals who had been vaccinated. For XBB-infected patients, there was a strong positive correlation between the serum neutralizing antibody titers against EG.5 and JN.1 in both the acute and recovery phases. This finding provides crucial information for judging the epidemic trend of COVID-19 and the development of vaccines, especially for developing customized vaccines and immune strategies for different populations.