Uncertainty in the estimated effects of statin initiation on risk of dementia: using a multiverse analysis to assess sources of variability

Mixed evidence on how statins affect dementia risk may reflect variability in model specifications. Alternate specifications are rarely systematically compared. Using an emulated trial design framework, we investigated variation in the estimated effect of statin initiation on dementia across alternative (1) eligibility criteria, (2) confounding variable sets, and (3) outcome definitions. Kaiser Permanente Northern California members’ linked electronic health records from 1996 to 2020 were used to identify statin initiation and dementia diagnoses. Statin initiators were matched on age and low-density lipoprotein cholesterol with up to 5 non-initiators. Possible covariates included clinical (n = 1.4 million); socioeconomic and behavioral (n = 265,224); and genetic (n = 69,573) variables. Using Cox proportional-hazards models, we estimated variation across 1.27 million intent-to-treat estimates for statin initiation varying specification of eligibility, outcome definition, and covariates. Estimated hazard ratios (HRs) for statin initiation on dementia across all specifications ranged from 0.93 to 1.47. The variance of estimates due to model specification differences was 7.6 times larger than the average variance of specific estimates due to finite sample size. Three modeling decisions notably attenuated coefficients [ln(HR)]: requiring a run-in period prior to the emulated trial start date (0.034); adjustment for diabetes (0.030) and cardiovascular disease (0.039); and excluding the first year of follow-up (0.041). HRs from models with all three specifications ranged from 0.99 to 1.15. No specification we evaluated consistently generated protective effects. Estimates of the association between statin initiation and dementia leveraging real world data are sensitive to model specification, especially decisions related to clinical covariates and time-at-risk.