Promoting mental health is an important component of the United Nations' Sustainable Development Goal 3. A large body of literature has found that neighborhood environment is associated with promoting mental health. However, longitudinal evidence on the neighborhood environment—mental health associations remain scarce. This study aimed to (1) examine the longitudinal associations between neighborhood characteristics and adults' mental health in China; and (2) investigate the heterogeneity by age and urbanicity separately. The data came from the China Family Panel Studies (CFPS) in 2010 and 2014. Multilevel modeling was employed to investigate the longitudinal associations and the respective moderating effects by age and urbanicity. The results illustrated that higher number of recreational & sports facilities, fewer number of kindergarten & primary schools, and higher levels of neighborhood aesthetics were associated with better mental health, especially for older adults and rural residents. Our findings suggest that adults' mental health could be improved by making recreational and sports facilities more accessible and by making neighborhoods more visually appealing. It is critical to balance children's educational resources with adults' mental health. The age and urban-rural differences will inform policymakers to develop interventions for more vulnerable populations like older adults and rural residents.