Diagnostic and prognostic research最新文献

筛选
英文 中文
A study protocol for a predictive model to assess population‑based risk of adverse pregnancy outcomes: The Adverse Pregnancy Outcomes Population Risk Tool (PregPoRT). 用于评估基于人群的不良妊娠结局风险的预测模型的研究方案:不良妊娠结局人群风险工具(PregPoRT)。
IF 3.9
Diagnostic and prognostic research Pub Date : 2026-03-03 DOI: 10.1186/s41512-026-00220-3
Sabrina Chiodo, Sonia M Grandi, Jessica Gronsbell, Laura C Rosella
{"title":"A study protocol for a predictive model to assess population‑based risk of adverse pregnancy outcomes: The Adverse Pregnancy Outcomes Population Risk Tool (PregPoRT).","authors":"Sabrina Chiodo, Sonia M Grandi, Jessica Gronsbell, Laura C Rosella","doi":"10.1186/s41512-026-00220-3","DOIUrl":"10.1186/s41512-026-00220-3","url":null,"abstract":"<p><strong>Background: </strong>Adverse pregnancy outcomes (APOs), such as gestational diabetes, preeclampsia, and placental abruption, are major contributors to maternal and fetal morbidity and mortality, with implications for individual long-term health and health system performance. Existing prediction models for APOs rely primarily on clinical or biomarker data, with few incorporating social, behavioral, or environmental determinants that are critical for shaping perinatal outcomes. This study describes the development and validation protocol for the Adverse Pregnancy Outcomes Population Risk Tool (PregPoRT), a novel, population-based prediction model designed to estimate APO risk using population-based and routinely collected survey and administrative data in Canada.</p><p><strong>Methods: </strong>PregPoRT will be developed using a retrospective cohort of female-identifying individuals, aged 15-49, who participated in the Canadian Community Health Survey (CCHS) between 2000 and 2017, and had a subsequent delivery hospitalization within two years recorded in the Discharge Abstract Database (DAD). Pre-pregnancy predictors were selected according to a health equity-informed framework by Kramer and colleagues (2019), and include biomedical, behavioral, social, and environmental variables from the CCHS, the Canadian Marginalization Index (CAN-Marg), the Canadian Urban Environmental Health Research Consortium (CANUE), and the Canadian Active Living Environments (Can-ALE) dataset. The primary outcome is a composite measure of APOs (gestational diabetes, preeclampsia, or placental abruption), identified using validated ICD codes. A Weibull accelerated failure time model will be used to estimate the risk of experiencing an APO. Continuous variables will be modeled with restricted cubic splines. Variable selection will be performed using the Least Absolute Shrinkage and Selection Operator (LASSO), and model performance will be assessed via discrimination, calibration, and overall accuracy. Validation strategies include split-sample, bootstrap, and temporal validation using later CCHS cycles. Survey weights will be applied throughout to ensure national representativeness.</p><p><strong>Discussion: </strong>PregPoRT will be the first Canadian prediction model for APOs that leverages nationally representative, linked survey and administrative data and explicitly integrates social, behavioral, and environmental determinants of health, domains that have been largely absent from prior models. By incorporating modifiable and socially patterned risk factors, the tool is designed to support public health planning, resource allocation, and maternal health equity monitoring.</p>","PeriodicalId":72800,"journal":{"name":"Diagnostic and prognostic research","volume":"10 1","pages":""},"PeriodicalIF":3.9,"publicationDate":"2026-03-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12954943/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147345949","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
The continuous net benefit: assessing the clinical utility of prediction models when informing a continuum of decisions. 持续的净收益:评估预测模型的临床效用时,通知一个连续的决定。
IF 3.9
Diagnostic and prognostic research Pub Date : 2026-02-17 DOI: 10.1186/s41512-026-00224-z
Jose Benitez-Aurioles, Laure Wynants, Niels Peek, Patrick Goodley, Philip Crosbie, Matthew Sperrin
{"title":"The continuous net benefit: assessing the clinical utility of prediction models when informing a continuum of decisions.","authors":"Jose Benitez-Aurioles, Laure Wynants, Niels Peek, Patrick Goodley, Philip Crosbie, Matthew Sperrin","doi":"10.1186/s41512-026-00224-z","DOIUrl":"10.1186/s41512-026-00224-z","url":null,"abstract":"<p><p>BACKGROUND: The net benefit and decision curve analysis are increasingly being used to assess the clinical utility of prognostic models. This metric assesses the value added by a model’s predictions when individuals are treated differently according to whether they are over or under a chosen threshold. Although such ‘treat or not’ decisions are common, prognostic models are also often used to tailor and personalise the care of patients, which implicitly involves the consideration of multiple interventions at different risk thresholds. We aim to extend decision curve analysis to estimate the net benefit of a model over multiple thresholds. METHODS: We take a weighted area under a rescaled version of the net benefit curve, deriving the continuous net benefit. In addition to the consideration of a continuum of interventions, we also show how the continuous net benefit can be used to evaluate single treatments in populations with a range of optimal thresholds, due to individual variations in expected treatment benefit or harm, highlighting limitations of current proposed methods that calculate the area under the decision curve. We propose this not as a substitute for decision curves, but as a complementary evaluation metric, in lieu of single-threshold point estimates. RESULTS: We showcase this metric through two examples of model validation in cardiovascular preventive care. The continuous net benefit brought additional insight over point estimates when comparing models over a range of decisions. CONCLUSIONS: The continuous net benefit informs those looking to validate clinical prediction models of their clinical utility, and helps decision makers understand their usefulness, improving their viability towards implementation. </p>","PeriodicalId":72800,"journal":{"name":"Diagnostic and prognostic research","volume":"10 1","pages":"8"},"PeriodicalIF":3.9,"publicationDate":"2026-02-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12911006/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146215067","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
The impact of violation of the proportional hazards assumption on the discrimination of the Cox proportional hazards model. 违反比例风险假设对Cox比例风险模型判别的影响。
IF 2.6
Diagnostic and prognostic research Pub Date : 2026-02-12 DOI: 10.1186/s41512-026-00223-0
Peter C Austin, Daniele Giardiello
{"title":"The impact of violation of the proportional hazards assumption on the discrimination of the Cox proportional hazards model.","authors":"Peter C Austin, Daniele Giardiello","doi":"10.1186/s41512-026-00223-0","DOIUrl":"10.1186/s41512-026-00223-0","url":null,"abstract":"<p><strong>Background: </strong>The Cox proportional hazards regression model is frequently used to estimate an individual's probability of experiencing an outcome within a specified prediction horizon. A key assumption of this model is that of proportional hazards. An important component of validating a prediction model is assessing its discrimination. Discrimination refers to the ability of predicted risk to separate those who do and do not experience the event. The impact of violation of the proportional hazards assumption on the discrimination of risk estimates obtained from a Cox model has not been examined.</p><p><strong>Methods: </strong>We used Monte Carlo simulations to assess the impact of the magnitude of the violation of the proportional hazards assumption on the discrimination of a Cox model as assessed using the time-varying area under the curve and on predictive accuracy as assessed using the time-varying index of predictive accuracy.</p><p><strong>Results: </strong>Compared to settings in which the proportional hazards assumption was satisfied, discrimination and predictive accuracy decreased in settings in which the log-hazard ratio was positively associated with time. Conversely, compared to settings in which the proportional hazards assumption was satisfied, discrimination and predictive accuracy increased in settings in which the log-hazard ratio was negatively associated with time. Compared with the use of a Cox regression model, the use of accelerated failure time parametric survival models, Royston and Parmar's spline-based parametric survival models, and generalized linear models using pseudo-observations did not result in estimates with improved discrimination or predictive accuracy in settings in which the proportional hazards assumption was violated.</p><p><strong>Conclusions: </strong>Violation of the proportional hazards assumption had an effect on the discrimination of predictions obtained using a Cox regression model.</p>","PeriodicalId":72800,"journal":{"name":"Diagnostic and prognostic research","volume":"10 1","pages":"7"},"PeriodicalIF":2.6,"publicationDate":"2026-02-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12895773/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146183189","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Protocol for development of a reporting guideline (TRIPOD-Code) for code repositories associated with diagnostic and prognostic prediction model studies. 为与诊断和预后预测模型研究相关的代码库制定报告指南的协议(TRIPOD-Code)。
IF 3.9
Diagnostic and prognostic research Pub Date : 2026-02-10 DOI: 10.1186/s41512-025-00217-4
Tom Pollard, Thomas Sounack, Catherine A Gao, Leo Anthony Celi, Charlotta Lindvall, Hyeonhoon Lee, Hyung-Chul Lee, Karel G M Moons, Gary S Collins
{"title":"Protocol for development of a reporting guideline (TRIPOD-Code) for code repositories associated with diagnostic and prognostic prediction model studies.","authors":"Tom Pollard, Thomas Sounack, Catherine A Gao, Leo Anthony Celi, Charlotta Lindvall, Hyeonhoon Lee, Hyung-Chul Lee, Karel G M Moons, Gary S Collins","doi":"10.1186/s41512-025-00217-4","DOIUrl":"10.1186/s41512-025-00217-4","url":null,"abstract":"<p><p>INTRODUCTION: The Transparent Reporting of a multivariable prediction model of Individual Prognosis Or Diagnosis (TRIPOD) statement was published to improve the reporting and critical appraisal of prediction model studies for diagnosis and prognosis. This paper describes the processes and methods that will be used to develop an extension to the TRIPOD statement (TRIPOD-Code) for the management of code associated with prediction model studies. TRIPOD-Code focuses specifically on the transparent reporting of analytical code used in prediction model studies, including code for data preprocessing, model development, and model evaluation. METHODS AND ANALYSIS: TRIPOD-Code will be developed following published guidance from the EQUATOR Network and will comprise five stages. Stage 1 will involve a methodological review of how code availability is reported in published prediction model studies. In Stage 2, we will consult a diverse group of key stakeholders using a Delphi process to identify items to be considered for inclusion in TRIPOD-Code. Stage 3 will consist of virtual consensus meetings to consolidate and prioritise the key items. Stage 4 will involve developing the TRIPOD-Code checklist. In the final stage, Stage 5, we will disseminate TRIPOD-Code via journals, conferences, blogs, websites (including TRIPOD and the EQUATOR Network) and social media. TRIPOD-Code will provide researchers working on prediction model studies with a reporting checklist and accompanying guidance to promote code completeness and availability. ETHICS AND DISSEMINATION: This study has been determined to be exempt from ongoing IRB oversight by the Massachusetts Institute of Technology Committee on the Use of Humans as Experimental Subjects (COUHES) under Exempt ID: E-6675. Findings from this study will be disseminated through peer-reviewed publications. </p>","PeriodicalId":72800,"journal":{"name":"Diagnostic and prognostic research","volume":"10 1","pages":"4"},"PeriodicalIF":3.9,"publicationDate":"2026-02-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12888484/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146151381","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A comparison of methodological approaches to developing clinical prediction models for individuals living with multiple long-term conditions: a protocol for a systematic review. 为患有多种长期疾病的个体开发临床预测模型的方法学方法的比较:一项系统回顾的方案。
IF 3.9
Diagnostic and prognostic research Pub Date : 2026-02-06 DOI: 10.1186/s41512-026-00221-2
Lauren A Anderson, Joie Ensor, Clare L Gillies, Selina T Lock, Kamlesh Khunti, Laura J Gray
{"title":"A comparison of methodological approaches to developing clinical prediction models for individuals living with multiple long-term conditions: a protocol for a systematic review.","authors":"Lauren A Anderson, Joie Ensor, Clare L Gillies, Selina T Lock, Kamlesh Khunti, Laura J Gray","doi":"10.1186/s41512-026-00221-2","DOIUrl":"10.1186/s41512-026-00221-2","url":null,"abstract":"<p><p>INTRODUCTION: Multiple long-term conditions (MLTCs) are being made a priority by funding bodies as prevalence rates increase. Improving early detection of individuals at high risk of developing MLTCs may delay or prevent complications and poor health outcomes. Predicting MLTCs remains a challenge, and methods for singular outcomes have been proven to be inappropriate for MLTC research. The aim of this paper is to present the protocol for a systematic review to identify all published models for prediction of MLTCs, and to summarise methods used for model development. METHODS AND ANALYSIS: MEDLINE (Ovid), Embase (Ovid), CINAHL (EBSCOHost) and CENTRAL (Cochrane Library) will be searched from September 2015 to identify relevant clinical prediction models which predict the development of MLTCs. Screening, data extraction and the risk of bias will be undertaken by two reviewers independently. Data extraction will include primary items for methodology and model outcomes and secondary items including study descriptors, population information, measured outcomes, methodology, model performance measures, clinical usefulness measures and risk of bias. A narrative synthesis will be conducted to summarise current methodological practice and to identify areas for improvement to inform future methodological and model development. ETHICS AND DISSEMINATION: Ethical approval is not required for this systematic review as it will use published literature only. The findings of the review will be submitted for publication in a peer reviewed journal. STRENGTHS AND LIMITATIONS OF THIS STUDY: • A comprehensive review of all methodologies used in clinical prediction modelling for MLTCs will be undertaken. • Methodologies will be included regardless of the outcome conditions predicted. • The protocol is reported according to the Preferred Reporting Items for Systematic Reviews Protocol (PRISMA – P). • Screening, data extraction and risk of bias will be conducted independently by two reviewers. • Only English language papers will be included. </p>","PeriodicalId":72800,"journal":{"name":"Diagnostic and prognostic research","volume":"10 1","pages":"6"},"PeriodicalIF":3.9,"publicationDate":"2026-02-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12879393/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146133627","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Machine learning-based COVID-19 prognostic models lag behind in reporting quality: findings from a TRIPOD/TRIPOD + AI systematic review. 基于机器学习的COVID-19预后模型在报告质量方面落后:来自TRIPOD/TRIPOD + AI系统评价的发现。
IF 2.6
Diagnostic and prognostic research Pub Date : 2026-02-03 DOI: 10.1186/s41512-026-00218-x
Ioannis Partheniadis, Persefoni Talimtzi, Adriani Nikolakopoulou, Anna-Bettina Haidich
{"title":"Machine learning-based COVID-19 prognostic models lag behind in reporting quality: findings from a TRIPOD/TRIPOD + AI systematic review.","authors":"Ioannis Partheniadis, Persefoni Talimtzi, Adriani Nikolakopoulou, Anna-Bettina Haidich","doi":"10.1186/s41512-026-00218-x","DOIUrl":"10.1186/s41512-026-00218-x","url":null,"abstract":"<p><strong>Background: </strong>Reporting of COVID-19 prognostic models frequently falls short of established standards. The TRIPOD checklist and its 2024 AI extension (TRIPOD + AI) provide a comprehensive framework for assessing reporting quality. We therefore evaluated and compared reporting completeness in conventional versus machine-learning models.</p><p><strong>Methods: </strong>Studies reporting the development, and internal and external validation of prognostic prediction models for COVID-19 using either conventional or machine learning-based algorithms were included. Literature searches were conducted in MEDLINE, Epistemonikos.org, and Scopus (up to July 31, 2024). Studies using conventional statistical methods were evaluated under TRIPOD, while machine learning-based studies were assessed using TRIPOD + AI. Data extraction followed TRIPOD and TRIPOD + AI checklists, measuring adherence per article and per checklist item. The protocol was prospectively registered at the Open Science Framework ( https://osf.io/kg9yw ).</p><p><strong>Results: </strong>A total of 53 studies describing 71 prognostic models were identified. Overall, adherence to both guidelines was low, with significantly poorer compliance among machine learning-based studies (TRIPOD + AI) compared to conventional model studies (TRIPOD) (28.4% vs. 38.1%, 95% CI of difference: 4.1-15.4). No study fully adhered to abstract reporting requirements, and appropriate titles were included in only a minority of cases (29.0%, 95% CI: 16.1-46.6 for TRIPOD; 13.6%, 95% CI: 4.8-33.3 for TRIPOD + AI). Sample size calculations were not fully reported in any study. Reporting of methods and results sections was poor across both frameworks.</p><p><strong>Conclusion: </strong>Lower adherence among machine learning studies reflects the relatively recent publication of the TRIPOD + AI guidelines (April 2024), which postdate many of the included studies. Both conventional and machine learning-based prediction models showed insufficient reporting, with major gaps in model description and performance reporting. Greater compliance with reporting guidelines is critical to improving the clarity, reproducibility, and clinical value of prediction model research.</p>","PeriodicalId":72800,"journal":{"name":"Diagnostic and prognostic research","volume":"10 1","pages":"3"},"PeriodicalIF":2.6,"publicationDate":"2026-02-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12866346/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146108706","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Performance of the Leicester risk assessment and Leicester practice risk scores for assessing the risk of undiagnosed type 2 diabetes or prediabetes in diverse populations: protocol for a systematic review of published validations and updates. 在不同人群中评估未确诊的2型糖尿病或前驱糖尿病风险的Leicester风险评估和Leicester实践风险评分的表现:已发表验证和更新的系统评价方案。
IF 2.6
Diagnostic and prognostic research Pub Date : 2026-01-15 DOI: 10.1186/s41512-026-00219-w
Louise Haddon, Joie Ensor, Kamlesh Khunti, Laura J Gray
{"title":"Performance of the Leicester risk assessment and Leicester practice risk scores for assessing the risk of undiagnosed type 2 diabetes or prediabetes in diverse populations: protocol for a systematic review of published validations and updates.","authors":"Louise Haddon, Joie Ensor, Kamlesh Khunti, Laura J Gray","doi":"10.1186/s41512-026-00219-w","DOIUrl":"10.1186/s41512-026-00219-w","url":null,"abstract":"<p><strong>Background: </strong>Approximately one million adults in the UK are estimated to have undiagnosed type 2 diabetes mellitus (T2DM), with a further 5.1 million adults with nondiabetic hyperglycaemia (NDH) that does not meet the threshold for a diabetes diagnosis. The Leicester Risk Assessment score (LRA) and Leicester Practice Risk score (LPR) are diagnostic risk prediction models that estimate an individual's risk of undiagnosed T2DM and NDH, developed for use in community and primary care settings respectively. The LRA is also used as a prognostic model; neither model has been updated since development. This study will systematically review all applications of these models as diagnostic and prognostic tools and any published updates to evaluate their performance in different populations. This review has been registered with PROSPERO (CRD420251005841).</p><p><strong>Methods: </strong>We will implement a citation search strategy to search Scopus, Web of Science and Google Scholar, restricted to full text, English language papers. Eligible papers will validate, update or modify either model. Data will be extracted using a form based on the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist; missing information will be sought from authors or estimated from other available information where possible. Meta-analysis of predictive performance measures will be completed if sufficient data exist. Subgroup and sensitivity analyses will be used to explore between-study heterogeneity and risk-of-bias impact.</p><p><strong>Discussion: </strong>This review will identify studies that have implemented, modified or validated the LRA and LPR for the risk of undiagnosed T2DM and NDH in different populations. This will allow summary measures, including level of uncertainty, of model performance to be calculated, making this highly relevant to individuals and stakeholders who recommend and implement these models. Review conclusions will also inform the potential update and recalibration of the models. This will ultimately lead to improved outcomes through earlier diagnosis and management.</p>","PeriodicalId":72800,"journal":{"name":"Diagnostic and prognostic research","volume":"10 1","pages":"2"},"PeriodicalIF":2.6,"publicationDate":"2026-01-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12810001/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145992086","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Prediction models developed using artificial intelligence: similar predictive performances with highly varying predictions for individuals - an illustration in deep vein thrombosis. 使用人工智能开发的预测模型:类似的预测性能,对个体的预测高度不同-深静脉血栓形成的例子。
IF 2.6
Diagnostic and prognostic research Pub Date : 2026-01-08 DOI: 10.1186/s41512-025-00216-5
Maerziya Yusufujiang, Constanza L Andaur Navarro, Johanna Aa Damen, Toshihiko Takada, Geert-Jan Geersing, Lotty Hooft, Ewoud Schuit, Karel Gm Moons, Valentijn Mt de Jong, Maarten van Smeden
{"title":"Prediction models developed using artificial intelligence: similar predictive performances with highly varying predictions for individuals - an illustration in deep vein thrombosis.","authors":"Maerziya Yusufujiang, Constanza L Andaur Navarro, Johanna Aa Damen, Toshihiko Takada, Geert-Jan Geersing, Lotty Hooft, Ewoud Schuit, Karel Gm Moons, Valentijn Mt de Jong, Maarten van Smeden","doi":"10.1186/s41512-025-00216-5","DOIUrl":"10.1186/s41512-025-00216-5","url":null,"abstract":"&lt;p&gt;&lt;strong&gt;Objectives: &lt;/strong&gt;The rise in popularity and off-the-shelf availability of machine learning (ML) and AI-based methodology to develop new prediction models provides developers with ample choices to compare and select the best performing model out of many possible models. Many studies have shown that such comparisons on any particular dataset, the difference in performance between models developed using different techniques (e.g. logistic regression, vs. random forest or neural networks) can often be small, especially when looking at crude performance measures such as the area under the ROC curve. This may lead to the conclusion that such models are essentially exchangeable, and model selection is arbitrary. However, as we will illustrate using a dataset on deep venous thrombosis, prediction models with similar discriminative performance may nonetheless generate different outcome probability estimates for individual patients and potentially lead to meaningfully different decision making.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Methods: &lt;/strong&gt;We developed diagnostic prediction models to predict the presence of deep venous thrombosis (DVT) in a large dataset of patients with leg symptoms suspected of having DVT, using five modelling techniques: unpenalized logistic regression (ULR), ridge logistic regression (RLR), random forests (RF), support vector machine (SVM) and neural network (NN). Age, sex, d-dimer, history of DVT, diagnosis alternative to DVT, and having cancer were used as a fixed set of predictors. Model performance was evaluated in terms of discrimination, calibration, and stability of individual risk prediction for a set of patients across the models.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Results: &lt;/strong&gt;Of the 6,087 suspected patients, 1,146 (19%) were diagnosed with DVT based on leg ultrasound (reference test). Three prediction models (ULR, RLR, NN) had similar discrimination with AUCs point estimates of 0.84. However, the 6087 individuals' estimated probabilities of DVT varied substantially across the five different modelling techniques, highlighting differences in prediction stability. Notably, the RF model tended to overestimate individual risks, while the SVM model tended to underestimate them compared to the other models. While the estimated probabilities were more similar for ULR, RLR and NN, classification measures (sensitivity, specificity, positive and negative predictive value) did differ because of differences in estimated probabilities of individuals near the risk threshold, illustrating that differences, even when relatively small, could potentially lead to different clinical decisions.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Conclusions: &lt;/strong&gt;Prediction models developed with different modeling techniques yielded very different individuals' outcome probabilities, even though the models had similar discriminative performance in this low-dimensional setting. Part of this variation can be explained by differences in calibration but also from modelling choices as estimated risks","PeriodicalId":72800,"journal":{"name":"Diagnostic and prognostic research","volume":"10 1","pages":"1"},"PeriodicalIF":2.6,"publicationDate":"2026-01-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12784591/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145936459","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A decomposition of Fisher's information to inform sample size for developing or updating fair and precise clinical prediction models - part 2: time-to-event outcomes. 分解Fisher的信息,为开发或更新公平和精确的临床预测模型提供样本量-第2部分:时间到事件结果。
IF 2.6
Diagnostic and prognostic research Pub Date : 2025-12-16 DOI: 10.1186/s41512-025-00204-9
Richard D Riley, Gary S Collins, Lucinda Archer, Rebecca Whittle, Amardeep Legha, Laura Kirton, Paula Dhiman, Mohsen Sadatsafavi, Nicola J Adderley, Joseph Alderman, Glen P Martin, Joie Ensor
{"title":"A decomposition of Fisher's information to inform sample size for developing or updating fair and precise clinical prediction models - part 2: time-to-event outcomes.","authors":"Richard D Riley, Gary S Collins, Lucinda Archer, Rebecca Whittle, Amardeep Legha, Laura Kirton, Paula Dhiman, Mohsen Sadatsafavi, Nicola J Adderley, Joseph Alderman, Glen P Martin, Joie Ensor","doi":"10.1186/s41512-025-00204-9","DOIUrl":"10.1186/s41512-025-00204-9","url":null,"abstract":"<p><strong>Background: </strong>When developing a clinical prediction model using time-to-event data (i.e. with censoring and different lengths of follow-up), previous research focuses on the sample size needed to minimise overfitting and precisely estimating the overall risk. However, instability of individual-level risk estimates may still be large.</p><p><strong>Methods: </strong>We propose using a decomposition of Fisher's information matrix to help examine and calculate the sample size required for developing a model that aims for precise and fair risk estimates. We propose a six-step process which can be used either before data collection or when an existing dataset is available. Steps 1 to 5 require researchers to specify the overall risk in the target population at a key time-point of interest: an assumed pragmatic 'core model' in the form of an exponential regression model, the (anticipated) joint distribution of core predictors included in that model and the distribution of censoring times. The 'core model' can be specified directly or based on a specified C-index and relative effects of (standardised) predictors. The joint distribution of predictors may be available directly in an existing dataset, in a pilot study or in a synthetic dataset provided by other researchers.</p><p><strong>Results: </strong>We derive closed-form solutions that decompose the variance of an individual's estimated event rate into Fisher's unit information matrix, predictor values and total sample size; this allows researchers to calculate and examine uncertainty distributions around individual risk estimates and misclassification probabilities for specified sample sizes. We provide an illustrative example in breast cancer and emphasise the importance of clinical context, including any risk thresholds for decision-making, and examine fairness concerns for pre- and postmenopausal women. Lastly, in two empirical evaluations, we provide reassurance that uncertainty interval widths based on our exponential approach are close to using more flexible parametric models.</p><p><strong>Conclusions: </strong>Our approach allows users to identify the (target) sample size required to develop a prediction model for time-to-event outcomes, via the pmstabilityss module. It aims to facilitate models with improved trust, reliability and fairness in individual-level predictions.</p>","PeriodicalId":72800,"journal":{"name":"Diagnostic and prognostic research","volume":"9 1","pages":"33"},"PeriodicalIF":2.6,"publicationDate":"2025-12-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12709744/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145770082","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Risk of bias in machine learning and statistical models to predict height or weight: a systematic review in fetal and paediatric medicine. 预测身高或体重的机器学习和统计模型的偏倚风险:胎儿和儿科医学的系统综述。
IF 2.6
Diagnostic and prognostic research Pub Date : 2025-12-15 DOI: 10.1186/s41512-025-00215-6
Neil R Lawrence, Irina Bacila, Joseph Tonge, Anthea Tucker, Jeremy Dawson, Zi-Qiang Lang, Nils P Krone, Paula Dhiman, Gary S Collins
{"title":"Risk of bias in machine learning and statistical models to predict height or weight: a systematic review in fetal and paediatric medicine.","authors":"Neil R Lawrence, Irina Bacila, Joseph Tonge, Anthea Tucker, Jeremy Dawson, Zi-Qiang Lang, Nils P Krone, Paula Dhiman, Gary S Collins","doi":"10.1186/s41512-025-00215-6","DOIUrl":"10.1186/s41512-025-00215-6","url":null,"abstract":"<p><strong>Background: </strong>Prediction of suboptimal growth allows early intervention that can improve outcomes for developing fetus' as well as infants and children. We investigate the risk of bias in statistical or machine learning models to predict the height or weight of a fetus, infant or child under 20 years of age to inform the current standard of research and provide insight into why equations developed over 30 years ago are still recommended for use by national professional bodies.</p><p><strong>Methods: </strong>We systematically searched MEDLINE and EMBASE for peer reviewed original research studies published in 2022. We included studies if they developed or validated a multivariable model to predict height or weight of an individual using two or more variables, excluding studies assessing imaging or using genetics or metabolomics information. Risk of bias was assessed for all prediction models and analyses using the Prediction model Risk Of Bias ASsessment Tool (PROBAST).</p><p><strong>Results: </strong>Sixty-four studies were included, in which we assessed the development of 180 models and validation of 61 models. Sample size was only considered in 10% of developed models and 13% of validated models. Despite height and weight being continuous variables, 77% of models developed predicted a dichotomised outcome variable.</p><p><strong>Registration: </strong>The review was registered on PROSPERO (ID: CRD42023421146), the International prospective register of systematic reviews on 26/4/2023.</p>","PeriodicalId":72800,"journal":{"name":"Diagnostic and prognostic research","volume":"9 1","pages":"32"},"PeriodicalIF":2.6,"publicationDate":"2025-12-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12703889/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145758352","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信
小红书