Masoumeh Vali, Hossein Motahari Nezhad, Levente Kovacs, Amir H Gandomi
{"title":"Machine learning algorithms for predicting PTSD: a systematic review and meta-analysis.","authors":"Masoumeh Vali, Hossein Motahari Nezhad, Levente Kovacs, Amir H Gandomi","doi":"10.1186/s12911-024-02754-2","DOIUrl":null,"url":null,"abstract":"<p><p>This study aimed to compare and evaluate the prediction accuracy and risk of bias (ROB) of post-traumatic stress disorder (PTSD) predictive models. We conducted a systematic review and random-effect meta-analysis summarizing predictive model development and validation studies using machine learning in diverse samples to predict PTSD. Model performances were pooled using the area under the curve (AUC) with a 95% confidence interval (CI). Heterogeneity in each meta-analysis was measured using I<sup>2</sup>. The risk of bias in each study was appraised using the PROBAST tool. 48% of the 23 included studies had a high ROB, and the remaining had unclear. Tree-based models were the primarily used algorithms and showed promising results in predicting PTSD outcomes for various groups, as indicated by their pooled AUCs: military incidents (0.745), sexual or physical trauma (0.861), natural disasters (0.771), medical trauma (0.808), firefighters (0.96), and alcohol-related stress (0.935). However, the applicability of these findings is limited due to several factors, such as significant variability among the studies, high and unclear risks of bias, and a shortage of models that maintain accuracy when tested in new settings. Researchers should follow the reporting standards for AI/ML and adhere to the PROBAST guidelines. It is also essential to conduct external validations of these models to ensure they are practical and relevant in real-world settings.</p>","PeriodicalId":9340,"journal":{"name":"BMC Medical Informatics and Decision Making","volume":"25 1","pages":"34"},"PeriodicalIF":3.3000,"publicationDate":"2025-01-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11752770/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"BMC Medical Informatics and Decision Making","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1186/s12911-024-02754-2","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"MEDICAL INFORMATICS","Score":null,"Total":0}
引用次数: 0
Abstract
This study aimed to compare and evaluate the prediction accuracy and risk of bias (ROB) of post-traumatic stress disorder (PTSD) predictive models. We conducted a systematic review and random-effect meta-analysis summarizing predictive model development and validation studies using machine learning in diverse samples to predict PTSD. Model performances were pooled using the area under the curve (AUC) with a 95% confidence interval (CI). Heterogeneity in each meta-analysis was measured using I2. The risk of bias in each study was appraised using the PROBAST tool. 48% of the 23 included studies had a high ROB, and the remaining had unclear. Tree-based models were the primarily used algorithms and showed promising results in predicting PTSD outcomes for various groups, as indicated by their pooled AUCs: military incidents (0.745), sexual or physical trauma (0.861), natural disasters (0.771), medical trauma (0.808), firefighters (0.96), and alcohol-related stress (0.935). However, the applicability of these findings is limited due to several factors, such as significant variability among the studies, high and unclear risks of bias, and a shortage of models that maintain accuracy when tested in new settings. Researchers should follow the reporting standards for AI/ML and adhere to the PROBAST guidelines. It is also essential to conduct external validations of these models to ensure they are practical and relevant in real-world settings.
本研究旨在比较和评价创伤后应激障碍(PTSD)预测模型的预测准确性和偏倚风险(risk of bias, ROB)。我们进行了系统回顾和随机效应荟萃分析,总结了在不同样本中使用机器学习预测创伤后应激障碍的预测模型开发和验证研究。采用曲线下面积(AUC)对模型性能进行汇总,置信区间为95%。每个meta分析的异质性采用I2测量。使用PROBAST工具评估每项研究的偏倚风险。纳入的23项研究中,有48%的研究罗伯值较高,其余研究罗伯值不明确。基于树的模型是主要使用的算法,在预测不同群体的创伤后应激障碍结果方面显示出有希望的结果,如其汇总的auc所示:军事事件(0.745)、性或身体创伤(0.861)、自然灾害(0.771)、医疗创伤(0.808)、消防员(0.96)和酒精相关压力(0.935)。然而,由于几个因素,这些发现的适用性受到限制,例如研究之间的显著变异性,高且不明确的偏倚风险,以及缺乏在新环境中测试时保持准确性的模型。研究人员应遵循人工智能/机器学习的报告标准,并遵守PROBAST指南。还必须对这些模型进行外部验证,以确保它们在实际环境中是实用的和相关的。
期刊介绍:
BMC Medical Informatics and Decision Making is an open access journal publishing original peer-reviewed research articles in relation to the design, development, implementation, use, and evaluation of health information technologies and decision-making for human health.