Elizabeth J Williamson, John Tazare, Krishnan Bhaskaran, Helen I McDonald, Alex J Walker, Laurie Tomlinson, Kevin Wing, Sebastian Bacon, Chris Bates, Helen J Curtis, Harriet J Forbes, Caroline Minassian, Caroline E Morton, Emily Nightingale, Amir Mehrkar, David Evans, Brian D Nicholson, David A Leon, Peter Inglesby, Brian MacKenna, Nicholas G Davies, Nicholas J DeVito, Henry Drysdale, Jonathan Cockburn, William J Hulme, Jessica Morley, Ian Douglas, Christopher T Rentsch, Rohini Mathur, Angel Wong, Anna Schultze, Richard Croker, John Parry, Frank Hester, Sam Harper, Richard Grieve, David A Harrison, Ewout W Steyerberg, Rosalind M Eggo, Karla Diaz-Ordaz, Ruth Keogh, Stephen J W Evans, Liam Smeeth, Ben Goldacre
{"title":"Comparison of methods for predicting COVID-19-related death in the general population using the OpenSAFELY platform.","authors":"Elizabeth J Williamson, John Tazare, Krishnan Bhaskaran, Helen I McDonald, Alex J Walker, Laurie Tomlinson, Kevin Wing, Sebastian Bacon, Chris Bates, Helen J Curtis, Harriet J Forbes, Caroline Minassian, Caroline E Morton, Emily Nightingale, Amir Mehrkar, David Evans, Brian D Nicholson, David A Leon, Peter Inglesby, Brian MacKenna, Nicholas G Davies, Nicholas J DeVito, Henry Drysdale, Jonathan Cockburn, William J Hulme, Jessica Morley, Ian Douglas, Christopher T Rentsch, Rohini Mathur, Angel Wong, Anna Schultze, Richard Croker, John Parry, Frank Hester, Sam Harper, Richard Grieve, David A Harrison, Ewout W Steyerberg, Rosalind M Eggo, Karla Diaz-Ordaz, Ruth Keogh, Stephen J W Evans, Liam Smeeth, Ben Goldacre","doi":"10.1186/s41512-022-00120-2","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Obtaining accurate estimates of the risk of COVID-19-related death in the general population is challenging in the context of changing levels of circulating infection.</p><p><strong>Methods: </strong>We propose a modelling approach to predict 28-day COVID-19-related death which explicitly accounts for COVID-19 infection prevalence using a series of sub-studies from new landmark times incorporating time-updating proxy measures of COVID-19 infection prevalence. This was compared with an approach ignoring infection prevalence. The target population was adults registered at a general practice in England in March 2020. The outcome was 28-day COVID-19-related death. Predictors included demographic characteristics and comorbidities. Three proxies of local infection prevalence were used: model-based estimates, rate of COVID-19-related attendances in emergency care, and rate of suspected COVID-19 cases in primary care. We used data within the TPP SystmOne electronic health record system linked to Office for National Statistics mortality data, using the OpenSAFELY platform, working on behalf of NHS England. Prediction models were developed in case-cohort samples with a 100-day follow-up. Validation was undertaken in 28-day cohorts from the target population. We considered predictive performance (discrimination and calibration) in geographical and temporal subsets of data not used in developing the risk prediction models. Simple models were contrasted to models including a full range of predictors.</p><p><strong>Results: </strong>Prediction models were developed on 11,972,947 individuals, of whom 7999 experienced COVID-19-related death. All models discriminated well between individuals who did and did not experience the outcome, including simple models adjusting only for basic demographics and number of comorbidities: C-statistics 0.92-0.94. However, absolute risk estimates were substantially miscalibrated when infection prevalence was not explicitly modelled.</p><p><strong>Conclusions: </strong>Our proposed models allow absolute risk estimation in the context of changing infection prevalence but predictive performance is sensitive to the proxy for infection prevalence. Simple models can provide excellent discrimination and may simplify implementation of risk prediction tools.</p>","PeriodicalId":72800,"journal":{"name":"Diagnostic and prognostic research","volume":"6 1","pages":"6"},"PeriodicalIF":0.0000,"publicationDate":"2022-02-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8865947/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Diagnostic and prognostic research","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1186/s41512-022-00120-2","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0
Abstract
Background: Obtaining accurate estimates of the risk of COVID-19-related death in the general population is challenging in the context of changing levels of circulating infection.
Methods: We propose a modelling approach to predict 28-day COVID-19-related death which explicitly accounts for COVID-19 infection prevalence using a series of sub-studies from new landmark times incorporating time-updating proxy measures of COVID-19 infection prevalence. This was compared with an approach ignoring infection prevalence. The target population was adults registered at a general practice in England in March 2020. The outcome was 28-day COVID-19-related death. Predictors included demographic characteristics and comorbidities. Three proxies of local infection prevalence were used: model-based estimates, rate of COVID-19-related attendances in emergency care, and rate of suspected COVID-19 cases in primary care. We used data within the TPP SystmOne electronic health record system linked to Office for National Statistics mortality data, using the OpenSAFELY platform, working on behalf of NHS England. Prediction models were developed in case-cohort samples with a 100-day follow-up. Validation was undertaken in 28-day cohorts from the target population. We considered predictive performance (discrimination and calibration) in geographical and temporal subsets of data not used in developing the risk prediction models. Simple models were contrasted to models including a full range of predictors.
Results: Prediction models were developed on 11,972,947 individuals, of whom 7999 experienced COVID-19-related death. All models discriminated well between individuals who did and did not experience the outcome, including simple models adjusting only for basic demographics and number of comorbidities: C-statistics 0.92-0.94. However, absolute risk estimates were substantially miscalibrated when infection prevalence was not explicitly modelled.
Conclusions: Our proposed models allow absolute risk estimation in the context of changing infection prevalence but predictive performance is sensitive to the proxy for infection prevalence. Simple models can provide excellent discrimination and may simplify implementation of risk prediction tools.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
