Endpoint assessment via routinely collected data generates estimates comparable to randomized controlled trial data: analysis of a cluster-randomized trial on fall injury prevention

Abstract

Background and Objectives

Routinely collected data (RCD) from healthcare claims and encounters are increasingly used for outcomes in randomized trials; however, methods for estimating the validity and relative precision of RCD-derived outcomes compared to those from conventional outcome ascertainment are limited. We developed an approach to measuring validity and relative precision of RCD and quantifying uncertainty.

Methods

We reanalyzed data from the Strategies to Reduce Injuries and Develop Confidence in Elders (STRIDE) cluster-randomized, controlled trial. Eighty-six primary care practices in 10 US healthcare systems were randomized to either a multifactorial intervention delivered by nurse falls care managers, or enhanced usual care, with 5451 persons age ≥ 70 at increased fall injury risk enrolled in the study. We estimated the hazard ratio (HR) and confidence interval (CI) for STRIDE's primary outcome (time to first serious fall injury) using original study data and RCD. The ratio of the RCD HR to original HR (“ratio of HRs”) measured validity. The confidence limit ratio (CLR; upper divided by lower confidence limits of CI) measured precision, with the ratio of the CLR with RCD to the CLR from the original study data (“ratio of CLRs”), measuring relative precision. We estimated uncertainty around the ratio of HRs and ratio of CLRs using bootstrapped 95% CIs and performed sensitivity analyses to assess the effects of adaptations needed to use RCD.

Results

Among the original sample of 5451 study participants, 5036 (92%) were linked to RCD. The intervention to control HR was 0.91 (95% CI: 0.78–1.07) in RCD, compared to 0.92 (95% CI: 0.80–1.06) in the original data. Using all RCD through STRIDE's administrative end date, the ratio of HRs was 1.00 (95% CI: 0.89–1.11) and ratio of CLRs was 1.03 (95% CI: 0.96–1.06). The CI around ratio of HRs was about three-fold wider for RCD than for the original STRIDE data in individuals who linked to RCD. Relative precision of RCD improved with increased length of follow-up.

Conclusion

Relying solely on RCD to ascertain the primary outcome in STRIDE would have resulted in similar point estimates and confidence limits for the treatment effect as in the original data. However, there was meaningful uncertainty around the estimate of validity. Efforts to validate RCD-derived outcomes for use as clinical trial endpoints should include measurement of uncertainty around validity estimates.