JAMIA OpenPub Date : 2024-12-11eCollection Date: 2024-12-01DOI: 10.1093/jamiaopen/ooae142
Liu Yang, Mudan Ren, Shuifa Sun, Ji Lu, Yirong Wu
{"title":"Investigation on the preferences for data quality assessment indicators of electronic health records: user-oriented perspective.","authors":"Liu Yang, Mudan Ren, Shuifa Sun, Ji Lu, Yirong Wu","doi":"10.1093/jamiaopen/ooae142","DOIUrl":"10.1093/jamiaopen/ooae142","url":null,"abstract":"<p><strong>Objectives: </strong>This study aims to investigate whether different types of electronic health record (EHR) users have distinct preferences for data quality assessment indicators (DQAI) and explore how these preferences can guide the enhancement of EHR systems and the optimization of related policies.</p><p><strong>Materials and methods: </strong>High-frequency indicators were identified by a systematic literature review to construct a DQAI system, which was assessed by a user-oriented investigation involving doctors, nurses, hospital supervisors, and clinical researchers. The entropy weight method and fuzzy comprehensive evaluation model were employed for the system comprehensive evaluation. Exploratory factor analysis was used to construct dimensions, and visualization analysis was utilized to explore preferences at both the indicator and dimension levels.</p><p><strong>Results: </strong>Sixteen indicators were identified to construct the DQAI system and grouped into 2 dimensions: structural and relational. The DQAI system achieved a comprehensive evaluation score of 90.445, corresponding to a \"very important\" membership level (62.5%). Doctors and nurses exhibited a higher score mean (4.43-4.66 out of 5) than supervisors (3.73-4.55 out of 5). Researchers emphasized credibility, with a score mean of 4.79 out of 5.</p><p><strong>Discussion: </strong>The findings reveal that different types of EHR users exhibit distinct preferences for the DQAI at both indicator and dimension levels. Doctors and nurses thought that all indicators were important, clinical researchers emphasized credibility, and supervisors focused mainly on accuracy. Indicators in the relational dimension were generally more valued than structural ones. Doctors and nurses prioritized indicators of relational dimension, while researchers and supervisors leaned towards indicators of structural dimension. These insights suggest that tailored approaches in EHR system development and policy-making could enhance EHR data quality.</p><p><strong>Conclusion: </strong>This study underscores the importance of user-centered approaches in optimizing EHR systems, highlighting diverse user preferences at both indicator and dimension levels.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"7 4","pages":"ooae142"},"PeriodicalIF":2.5,"publicationDate":"2024-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11633948/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142814452","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2024-12-10eCollection Date: 2024-12-01DOI: 10.1093/jamiaopen/ooae130
Maryam Zolnoori, Sasha Vergez, Zidu Xu, Elyas Esmaeili, Ali Zolnour, Krystal Anne Briggs, Jihye Kim Scroggins, Seyed Farid Hosseini Ebrahimabad, James M Noble, Maxim Topaz, Suzanne Bakken, Kathryn H Bowles, Ian Spens, Nicole Onorato, Sridevi Sridharan, Margaret V McDonald
{"title":"Decoding disparities: evaluating automatic speech recognition system performance in transcribing Black and White patient verbal communication with nurses in home healthcare.","authors":"Maryam Zolnoori, Sasha Vergez, Zidu Xu, Elyas Esmaeili, Ali Zolnour, Krystal Anne Briggs, Jihye Kim Scroggins, Seyed Farid Hosseini Ebrahimabad, James M Noble, Maxim Topaz, Suzanne Bakken, Kathryn H Bowles, Ian Spens, Nicole Onorato, Sridevi Sridharan, Margaret V McDonald","doi":"10.1093/jamiaopen/ooae130","DOIUrl":"10.1093/jamiaopen/ooae130","url":null,"abstract":"<p><strong>Objectives: </strong>As artificial intelligence evolves, integrating speech processing into home healthcare (HHC) workflows is increasingly feasible. Audio-recorded communications enhance risk identification models, with automatic speech recognition (ASR) systems as a key component. This study evaluates the transcription accuracy and equity of 4 ASR systems-Amazon Web Services (AWS) General, AWS Medical, Whisper, and Wave2Vec-in transcribing patient-nurse communication in US HHC, focusing on their ability in accurate transcription of speech from Black and White English-speaking patients.</p><p><strong>Materials and methods: </strong>We analyzed audio recordings of patient-nurse encounters from 35 patients (16 Black and 19 White) in a New York City-based HHC service. Overall, 860 utterances were available for study, including 475 drawn from Black patients and 385 from White patients. Automatic speech recognition performance was measured using word error rate (WER), benchmarked against a manual gold standard. Disparities were assessed by comparing ASR performance across racial groups using the linguistic inquiry and word count (LIWC) tool, focusing on 10 linguistic dimensions, as well as specific speech elements including repetition, filler words, and proper nouns (medical and nonmedical terms).</p><p><strong>Results: </strong>The average age of participants was 67.8 years (SD = 14.4). Communication lasted an average of 15 minutes (range: 11-21 minutes) with a median of 1186 words per patient. Of 860 total utterances, 475 were from Black patients and 385 from White patients. Amazon Web Services General had the highest accuracy, with a median WER of 39%. However, all systems showed reduced accuracy for Black patients, with significant discrepancies in LIWC dimensions such as \"Affect,\" \"Social,\" and \"Drives.\" Amazon Web Services Medical performed best for medical terms, though all systems have difficulties with filler words, repetition, and nonmedical terms, with AWS General showing the lowest error rates at 65%, 64%, and 53%, respectively.</p><p><strong>Discussion: </strong>While AWS systems demonstrated superior accuracy, significant disparities by race highlight the need for more diverse training datasets and improved dialect sensitivity. Addressing these disparities is critical for ensuring equitable ASR performance in HHC settings and enhancing risk prediction models through audio-recorded communication.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"7 4","pages":"ooae130"},"PeriodicalIF":2.5,"publicationDate":"2024-12-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11631515/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142808080","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2024-12-04eCollection Date: 2024-12-01DOI: 10.1093/jamiaopen/ooae125
Katharine Bradley, James McCormack, Megan Addis, Leah K Hamilton, Gwen T Lapham, Daniel Jonas, Dawn Bishop, Darla Parsons, Cheryl Budimir, Victoria Sanchez, Jennifer Bannon, Gabriela Villalobos, Alex H Krist, Theresa Walunas, Anya Day
{"title":"Do electronic health records used by primary care practices support recommended alcohol-related care?","authors":"Katharine Bradley, James McCormack, Megan Addis, Leah K Hamilton, Gwen T Lapham, Daniel Jonas, Dawn Bishop, Darla Parsons, Cheryl Budimir, Victoria Sanchez, Jennifer Bannon, Gabriela Villalobos, Alex H Krist, Theresa Walunas, Anya Day","doi":"10.1093/jamiaopen/ooae125","DOIUrl":"10.1093/jamiaopen/ooae125","url":null,"abstract":"<p><strong>Objective: </strong>The quality of alcohol-related prevention and treatment in US primary care is poor. The purpose of this study was to describe the extent to which Electronic Health Records (EHRs) used by 167 primary care practices across 7 states currently include the necessary prompts, clinical support, and performance reporting essential for improving alcohol-related prevention and treatment in primary care.</p><p><strong>Materials and methods: </strong>Experts from five regional quality improvement programs identified basic EHR features needed to support evidence-based alcohol-related prevention (ie, screening and brief intervention) and treatment of alcohol use disorders (AUD). Data were collected regarding whether EHRs included these features.</p><p><strong>Results: </strong>EHRs from 21 vendors were used by the primary care practices. For prevention, 62% of the 167 practices' EHRs included a validated screening questionnaire, 46% automatically scored the screening instrument, 62% could report the percent screened, and 37% could report the percent screening positive. Only 7% could report the percent offered brief intervention. For alcohol treatment, 49% of practices could report the percent diagnosed with AUD, 58% and 91% allowed documentation of referral and treatment with AUD medication, respectively. Only 3% could report the percent of patients diagnosed with AUD who received treatment.</p><p><strong>Discussion: </strong>Most EHRs observed across 167 primary care practices across 7 US states lacked basic functionality necessary to support evidence-based alcohol-related prevention and AUD treatment. Only 3% and 7% of EHRs, respectively, included the ability to report widely recommended quality measures needed to improve the quality of recommended alcohol-related prevention and treatment in primary care.</p><p><strong>Conclusion: </strong>Improving EHR functionality is likely necessary before alcohol-related primary care can be improved.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"7 4","pages":"ooae125"},"PeriodicalIF":2.5,"publicationDate":"2024-12-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11630038/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142808085","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2024-11-29eCollection Date: 2024-12-01DOI: 10.1093/jamiaopen/ooae132
Bartha Alexandra Nantongo, Josephine Nabukenya, Peter Nabende, John Kamulegeya
{"title":"A retrospective cohort study on predicting infants at a risk of defaulting routine immunization in Uganda using machine learning models.","authors":"Bartha Alexandra Nantongo, Josephine Nabukenya, Peter Nabende, John Kamulegeya","doi":"10.1093/jamiaopen/ooae132","DOIUrl":"10.1093/jamiaopen/ooae132","url":null,"abstract":"<p><strong>Objectives: </strong>Using machine learning models to predict infants at risk of defaulting routine immunization (RI) and identify significant features for Uganda.</p><p><strong>Materials and methods: </strong>Principal component analysis reduced dimensionality. Datasets were balanced using synthetic minority over-sampling technique. k-Nearest Neighbors, Decision Trees, Random Forests (RFs), Support Vector Machine (SVM), Naïve-Bayes, Logistic Regression (LR), XGBoost, Adoptive-Boosting, and Gradient-Boosting were used on Uganda's 2016 Demographic and health survey data with social-economic and demographic factors as predictors. Experiments with and without K-fold cross-validation were performed. Models were evaluated for accuracy, recall, precision, and area under a curve (AUC).</p><p><strong>Results and discussion: </strong>Experimental results revealed that the rate of defaulting increases as an infant's age increases at 5.3% Bacille Calmette-Guérin (BCG), 7.3% pentavalentI, 22.9% pentavalentIII, and 22.1% for measles. Significant predictors for BCG were immunization card, polio0, cluster altitude. Reception of pneumococcal1, BCG, and district for pentavalentI; polio3, pentavalentII for pentavalentIII; polio active and pentavalentIII for measles.RF had the best performance at predicting vaccine defaulting with 96%, 95%, 94%, 84% accuracy for BCG, PentavalentI, pentavalentIII, measles, respectively. Similarly, RF had the same precision, recall, AUC at 1.0. However, XGBoost, SVM, LR displayed the worst discriminatory power among infants who received the vaccine from defaulters with AUC ≤0.57.</p><p><strong>Conclusion: </strong>Immunization card, preceding vaccines reception, and district were the most influential predictors. RF was the best classifier among the 9 models to predict defaulting RI. The study recommends regular outreaches, daily vaccination, provision of immunization cards, and accessible water sources to reduce defaulting.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"7 4","pages":"ooae132"},"PeriodicalIF":2.5,"publicationDate":"2024-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11645499/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142829960","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2024-11-27eCollection Date: 2024-12-01DOI: 10.1093/jamiaopen/ooae122
Els Roorda, Marc Bruijnzeels, Jeroen Struijs, Marco Spruit
{"title":"Business intelligence systems for population health management: a scoping review.","authors":"Els Roorda, Marc Bruijnzeels, Jeroen Struijs, Marco Spruit","doi":"10.1093/jamiaopen/ooae122","DOIUrl":"10.1093/jamiaopen/ooae122","url":null,"abstract":"<p><strong>Objective: </strong>Population health management (PHM) is a promising data-driven approach to address the challenges faced by health care systems worldwide. Although Business Intelligence (BI) systems are known to be relevant for a data-driven approach, the usage for PHM is limited in its elaboration. To explore available scientific publications, a systematic review guided by PRISMA was conducted of mature BI initiatives to investigate their decision contexts and BI capabilities.</p><p><strong>Materials and methods: </strong>PubMed, Embase, and Web of Science were searched for articles published from January 2012 through November 2023. Articles were included if they described a (potential) BI system for PHM goals. Additional relevant publications were identified through snowballing. Technological Readiness Levels were evaluated to select mature initiatives from the 29 initiatives found. From the 11 most mature systems the decision context (eg, patient identification, risk stratification) and BI capabilities (eg, data warehouse, linked biobank) were extracted.</p><p><strong>Results: </strong>The initiatives found are highly fragmented in decision context and BI capabilities. Varied terminology is used and much information is missing. Impact on population's health is currently limited for most initiatives. Care Link, CommunityRx, and Gesundes Kinzigtal currently stand out in aligning BI capabilities with their decision contexts.</p><p><strong>Discussion and conclusion: </strong>PHM is a data-driven approach that requires a coherent data strategy and understanding of decision contexts and user needs. Effective BI capabilities depend on this understanding. Designing public-private partnerships to protect intellectual property while enabling rapid knowledge development is crucial. Development of a framework is proposed for systematic knowledge building.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"7 4","pages":"ooae122"},"PeriodicalIF":2.5,"publicationDate":"2024-11-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11602128/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142740813","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2024-11-18eCollection Date: 2024-12-01DOI: 10.1093/jamiaopen/ooae134
Ibrahim George-Sankoh, Laura E MacMullen, Asif T Chinwalla, Deanne Taylor, Rebecca D Ganetzky, Katelynn Stanley, Elizabeth M McCormick, Zarazuela Zolkipli-Cunningham, Marni J Falk
{"title":"MMFP-Tableau: enabling precision mitochondrial medicine through integration, visualization, and analytics of clinical and research health system electronic data.","authors":"Ibrahim George-Sankoh, Laura E MacMullen, Asif T Chinwalla, Deanne Taylor, Rebecca D Ganetzky, Katelynn Stanley, Elizabeth M McCormick, Zarazuela Zolkipli-Cunningham, Marni J Falk","doi":"10.1093/jamiaopen/ooae134","DOIUrl":"10.1093/jamiaopen/ooae134","url":null,"abstract":"<p><strong>Objective: </strong>To describe a novel data integration workflow developed to automate clinical and research electronic health system data integration and harmonization from siloed sources for centralized access, visualization and analysis by clinicians and researchers in an end user-friendly customized analytic platform.</p><p><strong>Materials and methods: </strong>A centralized, semi-automated framework provides data provenance and user access to integrated data sources. Data models are implemented leveraging a centralized server (Alteryx) for high-level analytics including scheduling, integration, and modeling. Data are then sent to a secure Tableau instance for end-user visualization and interaction, with minimal software development required.</p><p><strong>Results: </strong>MMFP-Tableau, named for its origin in the Mitochondrial Medicine Frontier Program (MMFP) at the Children's Hospital of Philadelphia (CHOP), has advanced efforts to realize precision medicine by facilitating expert clinician and researcher end-user direct access to integrated, highly robust health system datasets. This scalable data solution enables translational researchers to link subject and cohort clinical and research parameters with research samples; enhances external biopharma collaborations for clinical trial design, subject recruitment, and data tracking; accelerates retrospective clinical cohort data analysis; and improves complex data visualization for clinicians and researchers.</p><p><strong>Discussion: </strong>MMFP-Tableau promotes complex data integration, visibility, and advanced analytic capabilities to facilitate seamless multidisciplinary research, benefitting clinical care and research in rare disease patients and cohorts.</p><p><strong>Conclusion: </strong>The MMFP-Tableau data platform integrates multiple data sources across various siloed data platforms to transform complex data into readily accessible datasets. This approach represents a generalizable workflow concept readily adaptable to implement across diverse fields of medicine.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"7 4","pages":"ooae134"},"PeriodicalIF":2.5,"publicationDate":"2024-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11570990/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142669281","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2024-11-15eCollection Date: 2024-12-01DOI: 10.1093/jamiaopen/ooae108
Benjamin X Collins, Jean-Christophe Bélisle-Pipon, Barbara J Evans, Kadija Ferryman, Xiaoqian Jiang, Camille Nebeker, Laurie Novak, Kirk Roberts, Martin Were, Zhijun Yin, Vardit Ravitsky, Joseph Coco, Rachele Hendricks-Sturrup, Ishan Williams, Ellen W Clayton, Bradley A Malin
{"title":"Addressing ethical issues in healthcare artificial intelligence using a lifecycle-informed process.","authors":"Benjamin X Collins, Jean-Christophe Bélisle-Pipon, Barbara J Evans, Kadija Ferryman, Xiaoqian Jiang, Camille Nebeker, Laurie Novak, Kirk Roberts, Martin Were, Zhijun Yin, Vardit Ravitsky, Joseph Coco, Rachele Hendricks-Sturrup, Ishan Williams, Ellen W Clayton, Bradley A Malin","doi":"10.1093/jamiaopen/ooae108","DOIUrl":"10.1093/jamiaopen/ooae108","url":null,"abstract":"<p><strong>Objectives: </strong>Artificial intelligence (AI) proceeds through an iterative and evaluative process of development, use, and refinement which may be characterized as a lifecycle. Within this context, stakeholders can vary in their interests and perceptions of the ethical issues associated with this rapidly evolving technology in ways that can fail to identify and avert adverse outcomes. Identifying issues throughout the AI lifecycle in a systematic manner can facilitate better-informed ethical deliberation.</p><p><strong>Materials and methods: </strong>We analyzed existing lifecycles from within the current literature for ethical issues of AI in healthcare to identify themes, which we relied upon to create a lifecycle that consolidates these themes into a more comprehensive lifecycle. We then considered the potential benefits and harms of AI through this lifecycle to identify ethical questions that can arise at each step and to identify where conflicts and errors could arise in ethical analysis. We illustrated the approach in 3 case studies that highlight how different ethical dilemmas arise at different points in the lifecycle.</p><p><strong>Results discussion and conclusion: </strong>Through case studies, we show how a systematic lifecycle-informed approach to the ethical analysis of AI enables mapping of the effects of AI onto different steps to guide deliberations on benefits and harms. The lifecycle-informed approach has broad applicability to different stakeholders and can facilitate communication on ethical issues for patients, healthcare professionals, research participants, and other stakeholders.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"7 4","pages":"ooae108"},"PeriodicalIF":2.5,"publicationDate":"2024-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11565898/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142648917","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2024-10-16eCollection Date: 2024-12-01DOI: 10.1093/jamiaopen/ooae092
Min-Jeoung Kang, Sarah C Rossetti, Graham Lowenthal, Christopher Knaplund, Li Zhou, Kumiko O Schnock, Kenrick D Cato, Patricia C Dykes
{"title":"Designing and testing clinical simulations of an early warning system for implementation in acute care settings.","authors":"Min-Jeoung Kang, Sarah C Rossetti, Graham Lowenthal, Christopher Knaplund, Li Zhou, Kumiko O Schnock, Kenrick D Cato, Patricia C Dykes","doi":"10.1093/jamiaopen/ooae092","DOIUrl":"10.1093/jamiaopen/ooae092","url":null,"abstract":"<p><strong>Objectives: </strong>Conducting simulation testing with end-users is essential for facilitating successful implementation of new health information technologies. This study designed a standardized simulation testing process with a system prototype prior to implementation to help study teams identify the system's interpretability and feasibility from the end-user perspective and to effectively integrate new innovations into real-world clinical settings and workflows.</p><p><strong>Materials and methods: </strong>A clinical simulation model was developed to test a new Clinical Decision Support (CDS) system outside of the clinical environment while maintaining high fidelity. A web-based CDS prototype, the \"CONCERN Smart Application,\" which leverages clinical data to measure and express a patient's risk of deterioration on a 3-level scale (\"low,\" \"moderate,\" or \"high\"), and audiovisual-integrated materials, were used to lead simulation sessions.</p><p><strong>Results: </strong>A total of 6 simulation sessions with 17 nurses were held to investigate how nurses interact with the CONCERN Smart application and how it influences their critical thinking, and clinical responses. Four themes were extracted from the simulation debriefing sessions and used to inform implementation strategies. The strategies include how the CDS should be improved for practical real-world use.</p><p><strong>Discussion and conclusions: </strong>Standardized simulation testing procedures identified and informed the necessary CDS improvements, the enhancements needed for real-world use, and the training requirements to effectively prepare end-users for system go-live.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"7 4","pages":"ooae092"},"PeriodicalIF":2.5,"publicationDate":"2024-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11483109/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142476671","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2024-10-09eCollection Date: 2024-12-01DOI: 10.1093/jamiaopen/ooae111
Will Ke Wang, Hayoung Jeong, Leeor Hershkovich, Peter Cho, Karnika Singh, Lauren Lederer, Ali R Roghanizad, Md Mobashir Hasan Shandhi, Warren Kibbe, Jessilyn Dunn
{"title":"Tree-based classification model for Long-COVID infection prediction with age stratification using data from the National COVID Cohort Collaborative.","authors":"Will Ke Wang, Hayoung Jeong, Leeor Hershkovich, Peter Cho, Karnika Singh, Lauren Lederer, Ali R Roghanizad, Md Mobashir Hasan Shandhi, Warren Kibbe, Jessilyn Dunn","doi":"10.1093/jamiaopen/ooae111","DOIUrl":"10.1093/jamiaopen/ooae111","url":null,"abstract":"<p><strong>Objectives: </strong>We propose and validate a domain knowledge-driven classification model for diagnosing post-acute sequelae of SARS-CoV-2 infection (PASC), also known as Long COVID, using Electronic Health Records (EHRs) data.</p><p><strong>Materials and methods: </strong>We developed a robust model that incorporates features strongly indicative of PASC or associated with the severity of COVID-19 symptoms as identified in our literature review. The XGBoost tree-based architecture was chosen for its ability to handle class-imbalanced data and its potential for high interpretability. Using the training data provided by the Long COVID Computation Challenge (L3C), which was a sample of the National COVID Cohort Collaborative (N3C), our models were fine-tuned and calibrated to optimize Area Under the Receiver Operating characteristic curve (AUROC) and the F1 score, following best practices for the class-imbalanced N3C data.</p><p><strong>Results: </strong>Our age-stratified classification model demonstrated strong performance with an average 5-fold cross-validated AUROC of 0.844 and F1 score of 0.539 across the young adult, mid-aged, and older-aged populations in the training data. In an independent testing dataset, which was made available after the challenge was over, we achieved an overall AUROC score of 0.814 and F1 score of 0.545.</p><p><strong>Discussion: </strong>The results demonstrated the utility of knowledge-driven feature engineering in a sparse EHR data and demographic stratification in model development to diagnose a complex and heterogeneously presenting condition like PASC. The model's architecture, mirroring natural clinician decision-making processes, contributed to its robustness and interpretability, which are crucial for clinical translatability. Further, the model's generalizability was evaluated over a new cross-sectional data as provided in the later stages of the L3C challenge.</p><p><strong>Conclusion: </strong>The study proposed and validated the effectiveness of age-stratified, tree-based classification models to diagnose PASC. Our approach highlights the potential of machine learning in addressing the diagnostic challenges posed by the heterogeneity of Long-COVID symptoms.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"7 4","pages":"ooae111"},"PeriodicalIF":2.5,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11547948/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142629987","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2024-09-27eCollection Date: 2024-10-01DOI: 10.1093/jamiaopen/ooae099
Antonio Parraga-Leo, Tomiko T Oskotsky, Boris Oskotsky, Camilla Wibrand, Alennie Roldan, Alice S Tang, Connie W Y Ha, Ronald J Wong, Samuel S Minot, Gaia Andreoletti, Idit Kosti, Kevin R Theis, Sherrianne Ng, Yun S Lee, Patricia Diaz-Gimeno, Phillip R Bennett, David A MacIntyre, Susan V Lynch, Roberto Romero, Adi L Tarca, David K Stevenson, Nima Aghaeepour, Jonathan L Golob, Marina Sirota
{"title":"VMAP: Vaginal Microbiome Atlas during Pregnancy.","authors":"Antonio Parraga-Leo, Tomiko T Oskotsky, Boris Oskotsky, Camilla Wibrand, Alennie Roldan, Alice S Tang, Connie W Y Ha, Ronald J Wong, Samuel S Minot, Gaia Andreoletti, Idit Kosti, Kevin R Theis, Sherrianne Ng, Yun S Lee, Patricia Diaz-Gimeno, Phillip R Bennett, David A MacIntyre, Susan V Lynch, Roberto Romero, Adi L Tarca, David K Stevenson, Nima Aghaeepour, Jonathan L Golob, Marina Sirota","doi":"10.1093/jamiaopen/ooae099","DOIUrl":"10.1093/jamiaopen/ooae099","url":null,"abstract":"<p><strong>Objectives: </strong>To enable interactive visualization of the vaginal microbiome across the pregnancy and facilitate discovery of novel insights and generation of new hypotheses.</p><p><strong>Material and methods: </strong>Vaginal Microbiome Atlas during Pregnancy (VMAP) was created with R shiny to generate visualizations of structured vaginal microbiome data from multiple studies.</p><p><strong>Results: </strong>VMAP (http://vmapapp.org) visualizes 3880 vaginal microbiome samples of 1402 pregnant individuals from 11 studies, aggregated via open-source tool MaLiAmPi. Visualized features include diversity measures, VALENCIA community state types, and composition (phylotypes, taxonomy) that can be filtered by various categories.</p><p><strong>Discussion: </strong>This work represents one of the largest and most geographically diverse aggregations of the vaginal microbiome in pregnancy to date and serves as a user-friendly resource to further analyze vaginal microbiome data and better understand pregnancies and associated outcomes.</p><p><strong>Conclusion: </strong>VMAP can be obtained from https://github.com/msirota/vmap.git and is currently deployed as an online app for non-R users.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"7 3","pages":"ooae099"},"PeriodicalIF":2.5,"publicationDate":"2024-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11430916/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142355733","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}