Comparison of Classically and Machine Learning Generated Survival Prediction Models for Patients With Spinal Metastasis - A meta-Analysis of Two Recently Developed Algorithms.
Hung-Kuan Yen, Wei-Hsin Lin, Olivier Quinten Groot, Chih-Wei Chen, Jiun-Jen Yang, Michiel Erik Reinier Bongers, Aditya Karhade, Akash Shah, Tse-Chuan Yang, Bas Jj Bindels, Shih-Hsiang Dai, Jorrit-Jan Verlaan, Joseph Schwab, Shu-Hua Yang, Francis J Hornicek, Ming-Hsiao Hu
{"title":"Comparison of Classically and Machine Learning Generated Survival Prediction Models for Patients With Spinal Metastasis - A meta-Analysis of Two Recently Developed Algorithms.","authors":"Hung-Kuan Yen, Wei-Hsin Lin, Olivier Quinten Groot, Chih-Wei Chen, Jiun-Jen Yang, Michiel Erik Reinier Bongers, Aditya Karhade, Akash Shah, Tse-Chuan Yang, Bas Jj Bindels, Shih-Hsiang Dai, Jorrit-Jan Verlaan, Joseph Schwab, Shu-Hua Yang, Francis J Hornicek, Ming-Hsiao Hu","doi":"10.1177/21925682231162817","DOIUrl":null,"url":null,"abstract":"<p><strong>Study design: </strong>A systemic review and a meta-analysis. We also provided a retrospective cohort for validation in this study.</p><p><strong>Objective: </strong>(1) Using a meta-analysis to determine the pooled discriminatory ability of The Skeletal Oncology Research Group (SORG) classical algorithm (CA) and machine learning algorithms (MLA); and (2) test the hypothesis that SORG-CA has less variability in performance than SORG-MLA in non-American validation cohorts as SORG-CA does not incorporates regional-specific variables such as body mass index as input.</p><p><strong>Methods: </strong>After data extraction from the included studies, logit-transformation was applied for extracted AUCs for further analysis. The discriminatory abilities of both algorithms were directly compared by their logit (AUC)s. Further subgroup analysis by region (America vs non-America) was also conducted by comparing the corresponding logit (AUC).</p><p><strong>Results: </strong>The pooled logit (AUC)s of 90-day SORG-CA was .82 (95% confidence interval [CI], .53-.11), 1-year SORG-CA was 1.11 (95% CI, .74-1.48), 90-day SORG-MLA was 1.36 (95% CI, 1.09-1.63), and 1-year SORG-MLA was 1.57 (95% CI, 1.17-1.98). All the algorithms performed better in United States than in Taiwan (<i>P</i> < .001). The performance of SORG-CA was less influenced by a non-American cohort than SORG-MLA.</p><p><strong>Conclusion: </strong>These observations might highlight the importance of incorporating region-specific variables into existing models to make them generalizable to racially or geographically distinct regions.</p>","PeriodicalId":12680,"journal":{"name":"Global Spine Journal","volume":null,"pages":null},"PeriodicalIF":2.6000,"publicationDate":"2024-07-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Global Spine Journal","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1177/21925682231162817","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"CLINICAL NEUROLOGY","Score":null,"Total":0}
引用次数: 0
Abstract
Study design: A systemic review and a meta-analysis. We also provided a retrospective cohort for validation in this study.
Objective: (1) Using a meta-analysis to determine the pooled discriminatory ability of The Skeletal Oncology Research Group (SORG) classical algorithm (CA) and machine learning algorithms (MLA); and (2) test the hypothesis that SORG-CA has less variability in performance than SORG-MLA in non-American validation cohorts as SORG-CA does not incorporates regional-specific variables such as body mass index as input.
Methods: After data extraction from the included studies, logit-transformation was applied for extracted AUCs for further analysis. The discriminatory abilities of both algorithms were directly compared by their logit (AUC)s. Further subgroup analysis by region (America vs non-America) was also conducted by comparing the corresponding logit (AUC).
Results: The pooled logit (AUC)s of 90-day SORG-CA was .82 (95% confidence interval [CI], .53-.11), 1-year SORG-CA was 1.11 (95% CI, .74-1.48), 90-day SORG-MLA was 1.36 (95% CI, 1.09-1.63), and 1-year SORG-MLA was 1.57 (95% CI, 1.17-1.98). All the algorithms performed better in United States than in Taiwan (P < .001). The performance of SORG-CA was less influenced by a non-American cohort than SORG-MLA.
Conclusion: These observations might highlight the importance of incorporating region-specific variables into existing models to make them generalizable to racially or geographically distinct regions.
期刊介绍:
Global Spine Journal (GSJ) is the official scientific publication of AOSpine. A peer-reviewed, open access journal, devoted to the study and treatment of spinal disorders, including diagnosis, operative and non-operative treatment options, surgical techniques, and emerging research and clinical developments.GSJ is indexed in PubMedCentral, SCOPUS, and Emerging Sources Citation Index (ESCI).