PLOS digital healthPub Date : 2025-05-29eCollection Date: 2025-05-01DOI: 10.1371/journal.pdig.0000832
Gerko Schaap, Benjamin Butt, Christina Bode
{"title":"Suitability of just-in-time adaptive intervention in post-COVID-19-related symptoms: A systematic scoping review.","authors":"Gerko Schaap, Benjamin Butt, Christina Bode","doi":"10.1371/journal.pdig.0000832","DOIUrl":"10.1371/journal.pdig.0000832","url":null,"abstract":"<p><p>Patients with post-COVID-19-related symptoms require active and timely support in self-management. Just-in-time adaptive interventions (JITAI) seem promising in meeting these needs, as they aim to provide tailored interventions based on patient-centred measures. This systematic scoping review explores the suitability and examines key components of a potential JITAI in post-COVID-19 syndrome. Databases (PsycINFO, PubMed, and Scopus) were searched using terms related to post-COVID-19-related symptom clusters (fatigue and pain; respiratory problems; cognitive dysfunction; psychological problems) and to JITAI. Studies were summarised to identify potential components (interventions options, tailoring variables and decision rules), feasibility and effectiveness, and potential barriers. Out of the 341 screened records, 11 papers were included (five single-armed pilot or feasibility studies, three two-armed randomised controlled trial studies, and three observational studies). Two articles addressed fatigue or pain-related complaints, and nine addressed psychological problems. No articles about JITAI for respiratory problems or cognitive dysfunction clusters were found. Most interventions provided monitoring, education or reinforcement support, using mostly ecological momentary assessments or smartphone-based sensing. JITAIs were found to be acceptable and feasible, and seemingly effective, although evidence is limited. Given these findings, a JITAI for post-COVID-19 syndrome is promising, but needs to fit the complex, multifaceted nature of its symptoms. Future studies should assess the feasibility of machine learning to accurately predict when to execute timely interventions.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"4 5","pages":"e0000832"},"PeriodicalIF":0.0,"publicationDate":"2025-05-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12121805/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144183168","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
PLOS digital healthPub Date : 2025-05-29eCollection Date: 2025-05-01DOI: 10.1371/journal.pdig.0000807
Alexandra Tsalidis, Lakshmi Bharadwaj, Francis X Shen
{"title":"Standardization and accuracy of race and ethnicity data: Equity implications for medical AI.","authors":"Alexandra Tsalidis, Lakshmi Bharadwaj, Francis X Shen","doi":"10.1371/journal.pdig.0000807","DOIUrl":"10.1371/journal.pdig.0000807","url":null,"abstract":"<p><p>The rapid integration of artificial intelligence (AI) into healthcare has raised many concerns about race bias in AI models. Yet, overlooked in this dialogue is the lack of quality control for the accuracy of patient race and ethnicity (r/e) data in electronic health records (EHR). This article critically examines the factors driving inaccurate and unrepresentative r/e datasets. These include conceptual uncertainties about how to categorize races and ethnicity, shortcomings in data collection practices, EHR standards, and the misclassification of patients' race or ethnicity. To address these challenges, we propose a two-pronged action plan. First, we present a set of best practices for healthcare systems and medical AI researchers to improve r/e data accuracy. Second, we call for developers of medical AI models to transparently warrant the quality of their r/e data. Given the ethical and scientific imperatives of ensuring high-quality r/e data in AI-driven healthcare, we argue that these steps should be taken immediately.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"4 5","pages":"e0000807"},"PeriodicalIF":0.0,"publicationDate":"2025-05-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12121746/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144183369","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
PLOS digital healthPub Date : 2025-05-29eCollection Date: 2025-05-01DOI: 10.1371/journal.pdig.0000875
Lauren Schumacher, Rik Crutzen, Kayleigh Kwah, Katherine Brown, Julia V Bailey, Stephen Bremner, Louise J Jackson, Katie Newby
{"title":"Planning for successful participant recruitment and retention in trials of behavioural interventions: Feasibility randomised controlled trial of the Wrapped intervention.","authors":"Lauren Schumacher, Rik Crutzen, Kayleigh Kwah, Katherine Brown, Julia V Bailey, Stephen Bremner, Louise J Jackson, Katie Newby","doi":"10.1371/journal.pdig.0000875","DOIUrl":"10.1371/journal.pdig.0000875","url":null,"abstract":"<p><p>Randomised controlled trials (RCTs) must have sufficient power if planned analyses are to be performed and strong conclusions drawn. A prerequisite of this is successful participant recruitment and retention. Designing a comprehensive plan for participant recruitment and retention prior to trial commencement is recommended, but evidence concerning successful strategies, and how to go about developing a comprehensive plan, is lacking. This paper reports on the application of a six-stage process to develop a recruitment and retention strategy for a future RCT. Stage 1) Rapid evidence review: strategies used in previous trials were identified through database searching. This informed Stage 2) PPI workshop: workshops with public and patient involvement (PPI) group were used to select a sub-set of these strategies based on their potential to be successful and acceptable with the target audience. Stage 3) Focus groups with the target audience: the sub-set was refined through feedback from 15 young people (data subjected to content analysis). Strategies the PPI and focus groups mutually agreed upon proceeded directly to Stage 5; those without consensus proceeded to Stage 4. Stage 4) PPI workshop: PPI members voted on the remaining strategies; those without consensus were discarded. Stage 5) Observation of strategies during feasibility RCT (fRCT): the retained set of strategies were observed in practice in a fRCT in which recruitment and retention data and qualitative feedback from participants was collected. Stage 6) PPI workshop: the fRCT findings were reviewed and strategies for use in the future RCT were finalised. The finalised strategy included set of adverts; schedule of financial incentives; instructions to send survey invite by email, one prompt by SMS prior to data collection, and up to three SMS reminders; procedure to keep participants engaged (e.g., newsletters, personalisation of communications); and procedure if participants fail to complete a research activity (follow-up email/phone call).</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"4 5","pages":"e0000875"},"PeriodicalIF":0.0,"publicationDate":"2025-05-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12121807/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144183526","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"AI-driven healthcare: Fairness in AI healthcare: A survey.","authors":"Sribala Vidyadhari Chinta, Zichong Wang, Avash Palikhe, Xingyu Zhang, Ayesha Kashif, Monique Antoinette Smith, Jun Liu, Wenbin Zhang","doi":"10.1371/journal.pdig.0000864","DOIUrl":"10.1371/journal.pdig.0000864","url":null,"abstract":"<p><p>Artificial intelligence (AI) is rapidly advancing in healthcare, enhancing the efficiency and effectiveness of services across various specialties, including cardiology, ophthalmology, dermatology, emergency medicine, etc. AI applications have significantly improved diagnostic accuracy, treatment personalization, and patient outcome predictions by leveraging technologies such as machine learning, neural networks, and natural language processing. However, these advancements also introduce substantial ethical and fairness challenges, particularly related to biases in data and algorithms. These biases can lead to disparities in healthcare delivery, affecting diagnostic accuracy and treatment outcomes across different demographic groups. This review paper examines the integration of AI in healthcare, highlighting critical challenges related to bias and exploring strategies for mitigation. We emphasize the necessity of diverse datasets, fairness-aware algorithms, and regulatory frameworks to ensure equitable healthcare delivery. The paper concludes with recommendations for future research, advocating for interdisciplinary approaches, transparency in AI decision-making, and the development of innovative and inclusive AI applications.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"4 5","pages":"e0000864"},"PeriodicalIF":0.0,"publicationDate":"2025-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12091740/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144112934","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
PLOS digital healthPub Date : 2025-05-20eCollection Date: 2025-05-01DOI: 10.1371/journal.pdig.0000835
Tianhao Zhu, Kexin Xu, Wonchan Son, Kristofer Linton-Reid, Marc Boubnovski-Martell, Matt Grech-Sollars, Antoine D Lain, Joram M Posma
{"title":"Designing a computer-assisted diagnosis system for cardiomegaly detection and radiology report generation.","authors":"Tianhao Zhu, Kexin Xu, Wonchan Son, Kristofer Linton-Reid, Marc Boubnovski-Martell, Matt Grech-Sollars, Antoine D Lain, Joram M Posma","doi":"10.1371/journal.pdig.0000835","DOIUrl":"10.1371/journal.pdig.0000835","url":null,"abstract":"<p><p>Chest X-ray (CXR) is a diagnostic tool for cardiothoracic assessment. They make up 50% of all diagnostic imaging tests. With hundreds of images examined every day, radiologists can suffer from fatigue. This fatigue may reduce diagnostic accuracy and slow down report generation. We describe a prototype computer-assisted diagnosis (CAD) pipeline employing computer vision (CV) and Natural Language Processing (NLP). It was trained and evaluated on the publicly available MIMIC-CXR dataset. We perform image quality assessment, view labelling, and segmentation-based cardiomegaly severity classification. We use the output of the severity classification for large language model-based report generation. Four board-certified radiologists assessed the output accuracy of our CAD pipeline. Across the dataset composed of 377,100 CXR images and 227,827 free-text radiology reports, our system identified 0.18% of cases with mixed-sex mentions, 0.02% of poor quality images (F1 = 0.81), and 0.28% of wrongly labelled views (accuracy 99.4%). We assigned views for 4.18% of images which have unlabelled views. Our binary cardiomegaly classification model has 95.2% accuracy. The inter-radiologist agreement on evaluating the generated report's semantics and correctness for radiologist-MIMIC is 0.62 (strict agreement) and 0.85 (relaxed agreement) similar to the radiologist-CAD agreement of 0.55 (strict) and 0.93 (relaxed). Our work found and corrected several incorrect or missing metadata annotations for the MIMIC-CXR dataset. The performance of our CAD system suggests performance on par with human radiologists. Future improvements revolve around improved text generation and the development of CV tools for other diseases.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"4 5","pages":"e0000835"},"PeriodicalIF":0.0,"publicationDate":"2025-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12091825/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144112940","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
PLOS digital healthPub Date : 2025-05-16eCollection Date: 2025-05-01DOI: 10.1371/journal.pdig.0000851
Pietro Arina, Davide Ferrari, Maciej R Kaczorek, Nicholas Tetlow, Amy Dewar, Robert Stephens, Daniel Martin, Ramani Moonesinghe, Mervyn Singer, John Whittle, Evangelos B Mazomenos
{"title":"Assessing perioperative risks in a mixed elderly surgical population using machine learning: A multi-objective symbolic regression approach to cardiorespiratory fitness derived from cardiopulmonary exercise testing.","authors":"Pietro Arina, Davide Ferrari, Maciej R Kaczorek, Nicholas Tetlow, Amy Dewar, Robert Stephens, Daniel Martin, Ramani Moonesinghe, Mervyn Singer, John Whittle, Evangelos B Mazomenos","doi":"10.1371/journal.pdig.0000851","DOIUrl":"10.1371/journal.pdig.0000851","url":null,"abstract":"<p><p>Accurate preoperative risk assessment is of great value to both patients and clinical teams. Several risk scores have been developed but are often not calibrated to the local institution, limited in terms of data input into the underlying models, and/or lack individual precision. Machine Learning (ML) models have the potential to address limitations in existing scoring systems. A database of 1190 elderly patients who underwent major elective surgery was analyzed retrospectively. Preoperative cardiorespiratory fitness data from cardiopulmonary exercise testing (CPET), demographic and clinical data were extracted and integrated into advanced machine learning (ML) algorithms. Multi-Objective-Symbolic-Regression (MOSR), a novel algorithm utilizing Genetic Programming to generate mathematical formulae for learning tasks, was employed to predict patient morbidity at Postoperative Day 3, as defined by the PostOperative Morbidity Survey (POMS). Shapley-Additive-exPlanations (SHAP) was subsequently used to analyze feature contributions. Model performance was benchmarked against existing risk prediction scores, namely the Portsmouth-Physiological-and-Operative-Severity-Score-for-the-Enumeration-of-Mortality-and-Morbidity (PPOSSUM) and the Duke-Activity-Status-Index, as well as linear regression using CPET features. A model was also developed for the same task using data directly extracted from the CPET time-series. The incorporation of cardiorespiratory fitness data enhanced the performance of all models for predicting postoperative morbidity by 20% compared to sole reliance on clinical data. Cardiorespiratory fitness features demonstrated greater importance than clinical features in the SHAP analysis. Models utilizing data taken directly from the CPET time-series demonstrated a 12% improvement over the cardiorespiratory fitness models. MOSR model surpassed all other models in every experiment, demonstrating excellent robustness and generalization capabilities. Integrating cardiorespiratory fitness data with ML models enables improved preoperative prediction of postoperative morbidity in elective surgical patients. The MOSR model stands out for its capacity to pinpoint essential features and build models that are both simple and accurate, showing excellent generalizability.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"4 5","pages":"e0000851"},"PeriodicalIF":0.0,"publicationDate":"2025-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12084048/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144082571","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A non-specialist worker delivered digital assessment of cognitive development (DEEP) in young children: A longitudinal validation study in rural India.","authors":"Supriya Bhavnani, Alok Ranjan, Debarati Mukherjee, Gauri Divan, Amit Prakash, Astha Yadav, Chaman Lal, Diksha Gajria, Hiba Irfan, Kamal Kant Sharma, Smita Todkar, Vikram Patel, Gareth McCray","doi":"10.1371/journal.pdig.0000824","DOIUrl":"10.1371/journal.pdig.0000824","url":null,"abstract":"<p><p>Cognitive development in early childhood is critical for life-long well-being. Existing cognitive development surveillance tools require lengthy parental interviews and observations of children. Developmental Assessment on an E-Platform (DEEP) is a digital tool designed to address this gap by providing a gamified, direct assessment of cognition in young children which can be delivered by front-line providers in community settings. This longitudinal study recruited children from the SPRING trial in rural Haryana, India. DEEP was administered at 39 (SD 1; N = 1359), 60 (SD 5; N = 1234) and 95 (SD 4; N = 600) months and scores were derived using item response theory. Criterion validity was examined by correlating DEEP-score with age, Bayley's Scales of Infant Development (BSID-III) cognitive domain score at age 3 and Raven's Coloured Progressive Matrices (CPM) at age 8; predictive validity was examined by correlating DEEP-scores at preschool-age with academic performance at age 8 and convergent validity through correlations with height-for-age z-scores (HAZ), socioeconomic status (SES) and early life adversities. DEEP-score correlated strongly with age (r = 0.83, 95% CI 0.82 0.84) and moderately with BSID-III (r = 0.50, 0.39 - 0.60) and CPM (r = 0.37; 0.30 - 0.44). DEEP-score at preschool-age predicted academic outcomes at school-age (0.32; 0.25 - 0.41) and correlated positively with HAZ and SES and negatively with early life adversities. DEEP provides a valid, scalable method for cognitive assessment. It's integration into developmental surveillance programs could aid in monitoring and early detection of cognitive delays, enabling timely interventions.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"4 5","pages":"e0000824"},"PeriodicalIF":0.0,"publicationDate":"2025-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12084064/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144082563","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
PLOS digital healthPub Date : 2025-05-15eCollection Date: 2025-05-01DOI: 10.1371/journal.pdig.0000853
Indu Ilanchezian, Valentyn Boreiko, Laura Kühlewein, Ziwei Huang, Murat Seçkin Ayhan, Matthias Hein, Lisa Koch, Philipp Berens
{"title":"Development and validation of an AI algorithm to generate realistic and meaningful counterfactuals for retinal imaging based on diffusion models.","authors":"Indu Ilanchezian, Valentyn Boreiko, Laura Kühlewein, Ziwei Huang, Murat Seçkin Ayhan, Matthias Hein, Lisa Koch, Philipp Berens","doi":"10.1371/journal.pdig.0000853","DOIUrl":"https://doi.org/10.1371/journal.pdig.0000853","url":null,"abstract":"<p><p>Counterfactual reasoning is often used by humans in clinical settings. For imaging based specialties such as ophthalmology, it would be beneficial to have an AI model that can create counterfactual images, illustrating answers to questions like \"If the subject had had diabetic retinopathy, how would the fundus image have looked?\". Such an AI model could aid in training of clinicians or in patient education through visuals that answer counterfactual queries. We used large-scale retinal image datasets containing color fundus photography (CFP) and optical coherence tomography (OCT) images to train ordinary and adversarially robust classifiers that classify healthy and disease categories. In addition, we trained an unconditional diffusion model to generate diverse retinal images including ones with disease lesions. During sampling, we then combined the diffusion model with classifier guidance to achieve realistic and meaningful counterfactual images maintaining the subject's retinal image structure. We found that our method generated counterfactuals by introducing or removing the necessary disease-related features. We conducted an expert study to validate that generated counterfactuals are realistic and clinically meaningful. Generated color fundus images were indistinguishable from real images and were shown to contain clinically meaningful lesions. Generated OCT images appeared realistic, but could be identified by experts with higher than chance probability. This shows that combining diffusion models with classifier guidance can achieve realistic and meaningful counterfactuals even for high-resolution medical images such as CFP images. Such images could be used for patient education or training of medical professionals.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"4 5","pages":"e0000853"},"PeriodicalIF":0.0,"publicationDate":"2025-05-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12080772/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144082574","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
PLOS digital healthPub Date : 2025-05-15eCollection Date: 2025-05-01DOI: 10.1371/journal.pdig.0000647
Mohammad Bakhtiari
{"title":"Utilizing process mining in quality management: A case study in radiation oncology.","authors":"Mohammad Bakhtiari","doi":"10.1371/journal.pdig.0000647","DOIUrl":"https://doi.org/10.1371/journal.pdig.0000647","url":null,"abstract":"<p><p>Radiation oncology is known for its complexity, inherent risks, and sheer volume of data. Adopting a process-oriented management approach and systemic thinking is essential for ensuring safety, efficiency, and the highest quality of care. Process mining offers a data-centric method for analyzing and improving clinical workflows to ensure optimal patient outcomes. This study utilizes process mining techniques along with a quality management system to analyze event logs obtained from an electronic medical record system. Conformance checking and process improvement methodologies were utilized to detect inefficiencies and bottlenecks. Examining the treatment planning process through process mining revealed two principal bottlenecks-OAR contouring and physics chart checks. This led to specific interventions that markedly decreased the time to complete treatment planning processes. Additionally, applying organizational mining methods provided valuable information on how resources are utilized and how teams collaborate within the organization. Process mining is a useful tool for improving efficiency, quality, and decision-making in radiation oncology. By transitioning from traditional management to a data-driven leadership approach, radiation oncology departments can optimize workflows, enhance patient care, and adapt to the evolving demands of modern healthcare.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"4 5","pages":"e0000647"},"PeriodicalIF":0.0,"publicationDate":"2025-05-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12080779/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144082542","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
PLOS digital healthPub Date : 2025-05-14eCollection Date: 2025-05-01DOI: 10.1371/journal.pdig.0000755
Shreya Chappidi, Mason J Belue, Stephanie A Harmon, Sarisha Jagasia, Ying Zhuge, Erdal Tasci, Baris Turkbey, Jatinder Singh, Kevin Camphausen, Andra V Krauze
{"title":"From manual clinical criteria to machine learning algorithms: Comparing outcome endpoints derived from diverse electronic health record data modalities.","authors":"Shreya Chappidi, Mason J Belue, Stephanie A Harmon, Sarisha Jagasia, Ying Zhuge, Erdal Tasci, Baris Turkbey, Jatinder Singh, Kevin Camphausen, Andra V Krauze","doi":"10.1371/journal.pdig.0000755","DOIUrl":"https://doi.org/10.1371/journal.pdig.0000755","url":null,"abstract":"<p><strong>Background: </strong>Progression free survival (PFS) is a critical clinical outcome endpoint during cancer management and treatment evaluation. Yet, PFS is often missing from publicly available datasets due to the current subjective, expert, and time-intensive nature of generating PFS metrics. Given emerging research in multi-modal machine learning (ML), we explored the benefits and challenges associated with mining different electronic health record (EHR) data modalities and automating extraction of PFS metrics via ML algorithms.</p><p><strong>Methods: </strong>We analyzed EHR data from 92 pathology-proven GBM patients, obtaining 233 corticosteroid prescriptions, 2080 radiology reports, and 743 brain MRI scans. Three methods were developed to derive clinical PFS: 1) frequency analysis of corticosteroid prescriptions, 2) natural language processing (NLP) of reports, and 3) computer vision (CV) volumetric analysis of imaging. Outputs from these methods were compared to manually annotated clinical guideline PFS metrics.</p><p><strong>Results: </strong>Employing data-driven methods, standalone progression rates were 63% (prescription), 78% (NLP), and 54% (CV), compared to the 99% progression rate from manually applied clinical guidelines using integrated data sources. The prescription method identified progression an average of 5.2 months later than the clinical standard, while the CV and NLP algorithms identified progression earlier by 2.6 and 6.9 months, respectively. While lesion growth is a clinical guideline progression indicator, only half of patients exhibited increasing contrast-enhancing tumor volumes during scan-based CV analysis.</p><p><strong>Conclusion: </strong>Our results indicate that data-driven algorithms can extract tumor progression outcomes from existing EHR data. However, ML methods are subject to varying availability bias, supporting contextual information, and pre-processing resource burdens that influence the extracted PFS endpoint distributions. Our scan-based CV results also suggest that the automation of clinical criteria may not align with human intuition. Our findings indicate a need for improved data source integration, validation, and revisiting of clinical criteria in parallel to multi-modal ML algorithm development.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"4 5","pages":"e0000755"},"PeriodicalIF":0.0,"publicationDate":"2025-05-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12077705/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144082577","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}