Can the All of Us Sample be Reweighted to Mirror a Nationally Representative Sample? A Comparison of Mortality Predictors.

Abstract

Background: Participants in the All of Us (AoU) research study differ from the US population in myriad characteristics, limiting the generalizability of findings. A statistical reweighting tool to improve generalizability would enhance the scientific value of the data.

Methods: To account for differences between AoU and the nationally representative 1999-2018 National Health and Nutrition Examination Survey (NHANES), we generated selection weights using four models incorporating sociodemographic, self-reported health, and clinical characteristics. We assessed covariate balance and compared predictors of all-cause mortality in weighted AoU to NHANES using the ratio of hazard ratios (RHRs), where an RHR of one indicates unbiased estimates in (weighted) AoU relative to NHANES.

Results: Weighting improved balance on measured variables between AoU and NHANES. Among the four weighting models, the most complex model, which included sociodemographic, health, and clinical variables and their interactions, achieved HRs in AoU most similar to those in NHANES. For example, the RHR for hypertension for unweighted AoU vs. NHANES (RHR = 1.5; 95% confidence interval [CI] = 1.4, 1.7) was reduced to 1.2 (95% CI = 0.9, 1.5) after applying weights from the clinical-interaction model. Even in this model, 17 of 35 HRs evaluated diverged by >20% (RHR <0.8 or >1.2) between weighted AoU and NHANES.

Conclusions: Predictors of mortality in AoU differ from those in the US population both in their distribution and in their associations with mortality. Reweighting can mitigate selection bias, but no model we tested comprehensively achieved generalizability.