Development and evaluation of the electronic frailty index+ (eFI+) tool for older people: prognostic prediction modelling with integrated decision curve and health economic analysis.

Abstract

Background: Our aim was to develop and evaluate the electronic frailty index+, a prognostic tool, including four integrated prognostic-decision models, to stratify older people into subgroups for targeting key interventions.

Methods: Prognostic model development, internal validation and external validation using large data sets and longitudinal cohort study data, with decision curve and health economic analysis.

Population: Patients aged 65+ years.

Key outcomes: The 12-month outcomes for prognostic models: new home care package care home admission emergency department attendance/hospitalisation with fall/fracture all-cause mortality.

Statistical methods: We developed and internally validated models for our key outcomes in one large data set. We used internal-external cross-validation for the home care model and full external validation for the remaining three models in a second large data set. We used CARE75+ to investigate additional predictive value of clinical measures practical for primary care.

Decision curve analysis: We translated the prognostic models into a framework to support clinical decision-making.

Health economic evaluation: We integrated the falls prediction models with effect size estimates from network meta-analysis to examine potential cost savings.

Results: We used data from 660,417 patients in SAIL, 88,947 in Connected Bradford and 252 CARE75+ participants. Model performance was promising in internal-external cross-validation, with average calibration slope 1.00 (95% confidence interval 0.99 to 1.01), average calibration-in-the-large -0.01 (95% confidence interval -0.02 to 0.01), average observed/expected ratio 0.99 (95% confidence interval 0.98 to 1.01) and average C-statistic 0.81 (95% confidence interval 0.81 to 0.81).

Emergency department attendance/hospitalisation with fall/fracture: Model performance was promising on internal and external validation, although with some evidence for overprediction of falls risk, with calibration slope 1.25 (95% confidence interval 1.24 to 1.27), calibration-in-the-large -0.931 (95% confidence interval -0.938 to -0.920), observed/expected ratio 0.43 (95% confidence interval 0.42 to 0.44), C-statistic 0.83 (0.82 to 0.83).

Care home admission: Model performance was promising on internal validation, but it showed some miscalibration on external validation, with calibration slope 0.75 (95% CI 0.74 to 0.76), calibration-in-the-large -1.60 (-1.62 to -1.58) and observed/expected ratio 0.25 (95% CI 0.24 to 0.25), C-statistic of 0.86 (95% CI 0.86 to 0.86).

All-cause mortality: The model showed excellent performance across the full range of predicted risks on external validation, with average calibration slope 1.00 (0.98 to 1.01), average calibration-in-the-large -0.23 (-0.27 to -0.19), average observed/expected ratio 0.77 (0.75 to 0.79) and average C-statistic 0.83 (0.82 to 0.83).

Economic modelling: Modelling indicated that provision of multifactorial assessment and treatment for people with an annual falls risk of ≥ 40% has the largest cost reduction per targeted person (£1025).

Discussion: All four prediction models have promising predictive performance, although some had evidence of overprediction of risk (miscalibration). Decision curve analysis indicates potential clinical utility, and economic modelling provides novel information for policy-makers and commissioners.

Future work: Future research should include model impact studies to evaluate use of the models in routine care.

Limitations: We were unable to complete external validation of the home care prediction model.

Study registration: This study is registered as ClinicalTrials.gov ID NCT04113174.

Funding: This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 127905) and is published in full in Health Technology Assessment; Vol. 30, No. 61. See the NIHR Funding and Awards website for further award information.