JMIR Medical Informatics最新文献

{"title":"Clinical Performance and Communication Skills of ChatGPT Versus Physicians in Emergency Medicine: Simulated Patient Study.","authors":"ChulHyoung Park, Min Ho An, Gyubeom Hwang, Rae Woong Park, Juho An","doi":"10.2196/68409","DOIUrl":"https://doi.org/10.2196/68409","url":null,"abstract":"<p><strong>Background: </strong>Emergency medicine can benefit from artificial intelligence (AI) due to its unique challenges, such as high patient volume and the need for urgent interventions. However, it remains difficult to assess the applicability of AI systems to real-world emergency medicine practice, which requires not only medical knowledge but also adaptable problem-solving and effective communication skills.</p><p><strong>Objective: </strong>We aimed to evaluate ChatGPT's (OpenAI) performance in comparison to human doctors in simulated emergency medicine settings, using the framework of clinical performance examination and written examinations.</p><p><strong>Methods: </strong>In total, 12 human doctors were recruited to represent the medical professionals. Both ChatGPT and the human doctors were instructed to manage each case like real clinical settings with 12 simulated patients. After the clinical performance examination sessions, the conversation records were evaluated by an emergency medicine professor on history taking, clinical accuracy, and empathy on a 5-point Likert scale. Simulated patients completed a 5-point scale survey including overall comprehensibility, credibility, and concern reduction for each case. In addition, they evaluated whether the doctor they interacted with was similar to a human doctor. An additional evaluation was performed using vignette-based written examinations to assess diagnosis, investigation, and treatment planning. The mean scores from ChatGPT were then compared with those of the human doctors.</p><p><strong>Results: </strong>ChatGPT scored significantly higher than the physicians in both history-taking (mean score 3.91, SD 0.67 vs mean score 2.67, SD 0.78, P<.001) and empathy (mean score 4.50, SD 0.67 vs mean score 1.75, SD 0.62, P<.001). However, there was no significant difference in clinical accuracy. In the survey conducted with simulated patients, ChatGPT scored higher for concern reduction (mean score 4.33, SD 0.78 vs mean score 3.58, SD 0.90, P=.04). For comprehensibility and credibility, ChatGPT showed better performance, but the difference was not significant. In the similarity assessment score, no significant difference was observed (mean score 3.50, SD 1.78 vs mean score 3.25, SD 1.86, P=.71).</p><p><strong>Conclusions: </strong>ChatGPT's performance highlights its potential as a valuable adjunct in emergency medicine, demonstrating comparable proficiency in knowledge application, efficiency, and empathetic patient interaction. These results suggest that a collaborative health care model, integrating AI with human expertise, could enhance patient care and outcomes.</p>","PeriodicalId":56334,"journal":{"name":"JMIR Medical Informatics","volume":"13 ","pages":"e68409"},"PeriodicalIF":3.1,"publicationDate":"2025-07-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144661131","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Detection and Analysis of Circadian Biomarkers for Metabolic Syndrome Using Wearable Data: Cross-Sectional Study.","authors":"Jeong-Kyun Kim, Sujeong Mun, Siwoo Lee","doi":"10.2196/69328","DOIUrl":"https://doi.org/10.2196/69328","url":null,"abstract":"<p><strong>Background: </strong>Wearable devices are increasingly used for monitoring health and detecting digital biomarkers related to chronic diseases such as metabolic syndrome (MetS). Although circadian rhythm disturbances are known to contribute to MetS, few studies have explored wearable-derived circadian biomarkers for MetS identification.</p><p><strong>Objective: </strong>This study aimed to detect and analyze sleep and circadian rhythm biomarkers associated with MetS using step count and heart rate data from wearable devices and to identify the key biomarkers using explainable artificial intelligence (XAI).</p><p><strong>Methods: </strong>Data were analyzed from 272 participants in the Korean Medicine Daejeon Citizen Cohort, collected between 2020 and 2023, including 88 participants with MetS and 184 without any MetS diagnostic criteria. Participants wore Fitbit Versa or Inspire 2 devices for at least 5 weekdays, providing minute-level heart rate, step count, and sleep data. A total of 26 indicators were derived, including sleep markers (midsleep time and total sleep time) and circadian rhythm markers (midline estimating statistic of rhythm, amplitude, interdaily stability, and relative amplitude). In addition, a novel circadian rhythm marker, continuous wavelet circadian rhythm energy (CCE), was proposed using continuous wavelet transform of heart rate signals. Statistical tests (t test and the Wilcoxon rank sum test) and machine learning models-Shapley Additive Explanations, explainable boosting machine, and tabular neural network-were applied to evaluate marker significance and importance.</p><p><strong>Results: </strong>Circadian rhythm markers, especially heart rate-based indicators, showed stronger associations with MetS than sleep markers. The newly proposed CCE demonstrated the highest importance for MetS identification across all XAI models, with significantly lower values observed in the MetS group (P<.001). Other heart rate-based markers, including relative amplitude and low activity period, were also identified as important contributors. Although sleep markers did not reach statistical significance, some were recognized as secondary predictors in XAI-based analyses. The CCE marker maintained a high predictive value even when adjusting for age, sex, and BMI.</p><p><strong>Conclusions: </strong>This study identified CCE and relative amplitude of heart rate as key circadian rhythm biomarkers for MetS monitoring, demonstrating their high importance across multiple XAI models. In contrast, traditional sleep markers showed limited significance, suggesting that circadian rhythm analysis may offer additional insights into MetS beyond sleep-related indicators. These findings highlight the potential of wearable-based circadian biomarkers for improving MetS assessment and management.</p>","PeriodicalId":56334,"journal":{"name":"JMIR Medical Informatics","volume":"13 ","pages":"e69328"},"PeriodicalIF":3.1,"publicationDate":"2025-07-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144651296","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Machine Learning Approach to Differentiate Cold and Hot Syndrome in Viral Pneumonia Integrating Traditional Chinese Medicine and Modern Medicine: Machine Learning Model Development and Validation.","authors":"Xiaojie Jin, Yanru Wang, Jiarui Wang, Qian Gao, Yuhan Huang, Lingyu Shao, Jiali Zhao, Jintian Li, Ling Li, Zhiming Zhang, Shuyan Li, Yongqi Liu","doi":"10.2196/64725","DOIUrl":"https://doi.org/10.2196/64725","url":null,"abstract":"<p><strong>Background: </strong>Syndrome differentiation in traditional Chinese medicine (TCM) is an ancient principle that guides disease diagnosis and treatment. Among these, the cold and hot syndromes play a crucial role in identifying the nature of the disease and guiding the treatment of viral pneumonia. However, differentiating between cold and hot syndromes is often considered esoteric. Machine learning offers a promising avenue for clinicians to identify these syndromes more accurately, thereby supporting more informed clinical decision-making in the treatment.</p><p><strong>Objective: </strong>This study aims to construct a diagnostic model for differentiating cold and hot syndromes in viral pneumonia by integrating TCM and modern medical features using machine learning methods.</p><p><strong>Methods: </strong>The application of 8 machine learning algorithms (gradient boosting machine [GBM], logistic regression, random forest, extreme gradient boosting [XGB], light gradient boosting machine [LGB], ridge regression, least absolute shrinkage and selection operator, and support vector machine) generated and validated (both internally and externally) a model for differentiating cold and hot syndromes in viral pneumonia, based on clinical data from 1484 patient samples collected at 2 medical centers between 2021 and 2022.</p><p><strong>Results: </strong>The GBM model, which combines TCM and modern medicine features, outperformed models using only TCM features or only modern medicine features in distinguishing cold and hot syndromes in patients with viral pneumonia. The optimal discrimination model comprised 13 optimal features (temperature, red cell distribution width-SD, creatinine, total bilirubin, globulin, C-reactive protein, unconjugated bilirubin, white blood cell, neutrophil percentage, aspartate transaminase/alanine transaminase, total cholesterol, thrombocytocrit, and age) and the GBM algorithm, achieving an area under the curve (AUC) of 0.7788. Under internal and external testing, the AUCs were 0.7645 and 0.8428, respectively. Moreover, significant differences were observed between the cold and hot syndrome groups in temperature (P=.02), red cell distribution width-SD (P<.001), neutrophil percentage (P=.01), total cholesterol (P=.003), thrombocytocrit (P<.001), and age (P<.001).</p><p><strong>Conclusions: </strong>This pioneering study integrates the theory of TCM cold and hot syndromes with modern laboratory-based tests through machine learning. The developed model offers a novel approach for differentiating cold and hot syndromes in viral pneumonia, enabling practitioners to identify the syndrome quickly and efficiently, thereby supporting more informed clinical decision-making. Additionally, this research provides new insights into the modernization and scientific interpretation of TCM syndrome differentiation.</p>","PeriodicalId":56334,"journal":{"name":"JMIR Medical Informatics","volume":"13 ","pages":"e64725"},"PeriodicalIF":3.1,"publicationDate":"2025-07-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144651295","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Implementing Electronic Health Records in Philippine Primary Care Settings: Mixed-Methods Pilot Study.","authors":"Anton Elepaño, Carol Stephanie Tan-Lim, Mark Anthony Javelosa, Regine Ynez De Mesa, Mia Rey, Josephine Sanchez, Leonila Dans, Antonio Miguel Dans","doi":"10.2196/63036","DOIUrl":"https://doi.org/10.2196/63036","url":null,"abstract":"<p><strong>Background: </strong>Between 2020 and 2022, the Philippine Primary Care Studies program, a government-funded initiative supporting universal health care implementation, piloted two electronic health records (EHR) systems across urban, rural, and remote primary care sites.</p><p><strong>Objective: </strong>The study aimed to evaluate the implementation of two EHR systems in diverse primary care settings in the Philippines over a three-year period.</p><p><strong>Methods: </strong>This implementation study used an explanatory mixed methods design. Two EHR systems were deployed: an Open Medical Records System (OpenMRS)-based platform in 2016, and a Microsoft-based system in 2021. Both systems integrated clinical documentation, pharmacy, laboratory, and reporting modules. Implementation strategies included training workshops and materials, iterative user feedback loops, and infrastructure cofinancing with local governments. Surveys were administered yearly to all end users. The primary outcome was behavioral intention to use the system. Quantitative data were supplemented by inductive content analysis of qualitative responses to explain observed trends.</p><p><strong>Results: </strong>A total of 351 survey responses were collected from 2020 to 2022. In 2020, the intention to use the OpenMRS-based EHR was high across all sites. By 2022, following the launch of the Microsoft-based EHR, acceptability declined significantly among doctors and administrative staff, particularly at the urban site. In contrast, the remote site which retained the OpenMRS-based system maintained high acceptability levels. Qualitative findings revealed that while the new EHR system provided a more privacy-focused design, users preferred a cross-platform EHR to allow more flexible access to patient data. At the rural site where the EHR was used to facilitate task-shifting among nurses involved in clinical management, users were less impacted by this shift.</p><p><strong>Conclusions: </strong>The disparities in EHR acceptability across urban, rural, and remote sites were influenced by contextual, technical, and demographic factors. The decline in acceptability following the EHR system transition highlights the importance of implementation strategies that reflect the specific needs and capacities of each setting. These findings offer practical insights for adapting EHR systems to diverse primary care contexts.</p>","PeriodicalId":56334,"journal":{"name":"JMIR Medical Informatics","volume":"13 ","pages":"e63036"},"PeriodicalIF":3.1,"publicationDate":"2025-07-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144644255","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Performance of Natural Language Processing versus International Classification of Diseases Codes in Building Registries for Patients With Fall Injury: Retrospective Analysis.","authors":"Atta Taseh, Souri Sasanfar, Michelle Chan, Evan Sirls, Ara Nazarian, Kayhan Batmanghelich, Jonathan F Bean, Soheil Ashkani-Esfahani","doi":"10.2196/66973","DOIUrl":"https://doi.org/10.2196/66973","url":null,"abstract":"<p><strong>Background: </strong>Standardized registries, such as the International Classification of Diseases (ICD) codes, are commonly built using administrative codes assigned to patient encounters. However, patients with fall injury are often coded using subsequent injury codes, such as hip fractures. This necessitates manual screening to ensure the accuracy of data registries.</p><p><strong>Objective: </strong>This study aimed to automate the extraction of fall incidents and mechanisms using natural language processing (NLP) and compare this approach with the ICD method.</p><p><strong>Methods: </strong>Clinical notes for patients with fall-induced hip fractures were retrospectively reviewed by medical experts. Fall incidences were detected, annotated, and classified among patients who had a fall-induced hip fracture (case group). The control group included patients with hip fractures without any evidence of falls. NLP models were developed using the annotated notes of the study groups to fulfill two separate tasks: fall occurrence detection and fall mechanism classification. The performances of the models were compared using accuracy, sensitivity, specificity, positive predictive value, negative predictive value, F1-score, and area under the receiver operating characteristic curve.</p><p><strong>Results: </strong>A total of 1769 clinical notes were included in the final analysis for the fall occurrence task, and 783 clinical notes were analyzed for the fall mechanism classification task. The highest F1-score using NLP for fall occurrence was 0.97 (specificity=0.96; sensitivity=0.97), and for fall mechanism classification was 0.61 (specificity=0.56; sensitivity=0.62). Natural language processing could detect up to 98% of the fall occurrences and 65% of the fall mechanisms accurately, compared to 26% and 12%, respectively, by ICD codes.</p><p><strong>Conclusions: </strong>Our findings showed promising performance with higher accuracy of NLP algorithms compared to the conventional method for detecting fall occurrence and mechanism in developing disease registries using clinical notes. Our approach can be introduced to other registries that are based on large data and are in need of accurate annotation and classification.</p>","PeriodicalId":56334,"journal":{"name":"JMIR Medical Informatics","volume":"13 ","pages":"e66973"},"PeriodicalIF":3.1,"publicationDate":"2025-07-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144638811","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Regulatory Insights From 27 Years of Artificial Intelligence/Machine Learning-Enabled Medical Device Recalls in the United States: Implications for Future Governance.","authors":"Wei-Pin Chen, Wei-Guang Teng, C Benson Kuo, Yu-Jui Yen, Jian-Yu Lian, Matthew Sing, Peng-Ting Chen","doi":"10.2196/67552","DOIUrl":"10.2196/67552","url":null,"abstract":"<p><strong>Background: </strong>Artificial intelligence/machine learning (AI/ML) has revolutionized the health care industry, particularly in the development and use of medical devices. The US Food and Drug Administration (FDA) has authorized over 878 AI/ML-enabled medical devices, reflecting a growing trend in both quantity and application scope. Understanding the distinct challenges these devices present in terms of FDA regulation violations is crucial for effectively avoiding recalls. This is particularly pertinent for proactive measures regarding medical devices.</p><p><strong>Objective: </strong>This study explores the impact of AI/ML on medical device recalls, focusing on the distinct causes associated with AI/ML-enabled devices compared with other device types. Recall information associated with 510(k)-cleared devices was obtained from openFDA. Three recall cohorts were established: \"All 510(k) devices recall,\" \"software-related devices recall,\" and \"AI/ML devices recall.\"</p><p><strong>Methods: </strong>Recall information for 510(k)-cleared devices was obtained from openFDA. AI/ML-enabled medical devices were identified based on FDA listings. Three cohorts were established: \"All 510(k) devices recall,\" \"software-related devices recall,\" and \"AI/ML devices recall.\" Root cause analysis was conducted for each recall event.</p><p><strong>Results: </strong>The results indicate that while the top 5 recall root causes are relatively similar across the 3 control groups, the proportions vary, with AI/ML devices showing a higher impact for 87% of all recalls. Design and development-related factors play a significant role in recalls of AI/ML devices with root causes related to device design and software design accounting for 50% of recalls, emphasizing the importance of thorough planning, user feedback incorporation, and validation during the development process to reduce the probability of recalls. In addition, changes in software, including design changes and control changes, also contribute substantially to recalls in AI/ML devices.</p><p><strong>Conclusions: </strong>In conclusion, this study provides valuable insights into the unique challenges and considerations associated with AI/ML-enabled medical device recalls, offering guidance for manufacturers to enhance verification plans and mitigate risks in this rapidly evolving technological landscape.</p>","PeriodicalId":56334,"journal":{"name":"JMIR Medical Informatics","volume":"13 ","pages":"e67552"},"PeriodicalIF":3.1,"publicationDate":"2025-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12274014/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144612354","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Combining Machine Learning With Real-World Data to Identify Gaps in Clinical Practice Guidelines: Feasibility Study Using the Prospective German Stroke Registry and the National Acute Ischemic Stroke Guidelines.","authors":"Sandrine Müller, Susanne Diekmann, Markus Wenzel, Horst Karl Hahn, Johannes Tuennerhoff, Ulrike Ernemann, Florian Hennersdorf, Max Westphal, Sven Poli","doi":"10.2196/69282","DOIUrl":"10.2196/69282","url":null,"abstract":"&lt;p&gt;&lt;strong&gt;Background: &lt;/strong&gt;Clinical practice guidelines (CPGs) serve as essential tools for guiding clinicians in providing appropriate patient care. However, clinical practice does not always reflect CPGs. This is particularly critical in acute diseases requiring immediate treatment, such as acute ischemic stroke, one of the leading causes of morbidity and mortality worldwide. Adherence to CPGs improves patient outcomes, yet guidelines may not address all patient scenarios, resulting in variability in treatment decisions. Identifying such gaps would augment CPGs but is challenging when using traditional methods.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Objective: &lt;/strong&gt;This study aims to leverage real-world data coupled with machine learning (ML) techniques to systematically identify and quantify gaps in German thrombolysis-in-stroke guidelines.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Methods: &lt;/strong&gt;We analyzed observational data from the German Stroke Registry - Endovascular Treatment (GSR-ET), a prospective national registry involving 18,069 patients from 25 stroke centers in whom endovascular treatment of a large vessel occlusion was attempted between 2015 and 2023. Key variables included demographic, clinical and imaging information, treatment details, and outcomes. A random forest model was used to predict intravenous thrombolysis treatment decisions based on three different sets of features: (1) guideline-recommended features, (2) clinician-selected features, and (3) features as documented in the GSR-ET before thrombolytic treatment. Feature importance scores, permutation importance, and Shapley Additive Explanations values were used, with clinician guidance, to interpret the model and identify key factors associated with guideline deviations and independent clinician judgments.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Results: &lt;/strong&gt;Of all GSR-ET patients, 13,440 (74.4%) were analyzed after excluding those with incomplete or implausible data. The random forest model's performance, measured by area under the receiver operating characteristics curve, was 0.71 (95% CI 0.68-0.73), 0.74 (95% CI 0.73-0.75), and 0.77 (95% CI 0.76-0.78) for the guideline-recommended, clinician-selected, and GSR-ET feature sets, respectively. Across all sets, time from symptom onset to admission was the most important predictor of thrombolysis treatment decisions. Age, which according to the German guidelines is not to be considered for thrombolysis administration, emerged as a significant predictor in the GSR-ET feature set, suggesting a potential gap between guidelines and clinical practice.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Conclusions: &lt;/strong&gt;In our study, we introduce an innovative approach that combines real-world data with ML techniques to identify discrepancies between CPGs and actual clinical decision-making. Using intravenous thrombolysis in large vessel occlusion stroke as a model, our findings suggest that treatment decisions may be influenced by factors not explicitly included in the current German guideline, such as patient age and ","PeriodicalId":56334,"journal":{"name":"JMIR Medical Informatics","volume":"13 ","pages":"e69282"},"PeriodicalIF":3.1,"publicationDate":"2025-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12274016/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144627864","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Identifying People Living With or Those at Risk for HIV in a Nationally Sampled Electronic Health Record Repository Called the National Clinical Cohort Collaborative: Computational Phenotyping Study.","authors":"Eric Hurwitz, Cara D Varley, A Jerrod Anzolone, Vithal Madhira, Amy L Olex, Jing Sun, Dimple Vaidya, Nada Fadul, Jessica Y Islam, Lesley E Jackson, Kenneth J Wilkins, Zachary Butzin-Dozier, Dongmei Li, Sandra E Safo, Julie A McMurry, Pooja Maheria, Tommy Williams, Shukri A Hassan, Melissa A Haendel, Rena C Patel","doi":"10.2196/68143","DOIUrl":"https://doi.org/10.2196/68143","url":null,"abstract":"<p><strong>Background: </strong>Electronic health records (EHRs) provide valuable insights to address clinical and epidemiological research concerning HIV, including the disproportionate impact of the COVID-19 pandemic on people living with HIV. To identify this population, most studies using EHR or claims databases start with diagnostic codes, which can result in misclassification without further refinement using drug or laboratory data. Furthermore, given that antiretrovirals now have indications for both HIV and COVID-19 (ie, ritonavir in nirmatrelvir/ritonavir), new phenotyping methods are needed to better capture people living with HIV. Therefore, we created a generalizable and innovative method to robustly identify people living with HIV, preexposure prophylaxis (PrEP) users, postexposure prophylaxis (PEP) users, and people not living with HIV using granular clinical data after the emergence of COVID-19.</p><p><strong>Objective: </strong>The primary aim of this study was to use computational phenotyping in EHR data to identify people living with HIV (cohort 1), PrEP users (cohort 2), PEP users (cohort 3), or \"none of the above\" (people not living with HIV; cohort 4) and describe COVID-19-related characteristics among these cohorts.</p><p><strong>Methods: </strong>We used diagnostic and laboratory measurements and drug concepts in the National Clinical Cohort Collaborative to create a computational phenotype for the 4 cohorts with confidence levels. For robustness, we conducted a randomly sampled, blinded clinician annotation to assess precision. We calculated the distribution of demographics, comorbidities, and COVID-19 variables among the 4 cohorts.</p><p><strong>Results: </strong>We identified 132,664 people living with HIV with a high level of confidence, 36,088 PrEP users, 4120 PEP users, and 20,639,675 people not living with HIV. Most people living with HIV were identified by a combination of medical conditions, laboratory measurements, and drug exposures (74,809/132,664, 56.4%), followed by laboratory measurements and drug exposures (15,241/132,664, 11.5%) and then by medical conditions and drug exposures (14,595/132,664, 11%). A higher proportion of people living with HIV experienced COVID-19-related hospitalization (4650,132,664, 3.5%) or mortality (828/132,664, 0.6%) and all-cause mortality (2083/132,664, 1.6%) compared to other cohorts.</p><p><strong>Conclusions: </strong>Using an extensive phenotyping algorithm leveraging granular data in an EHR repository, we have identified people living with HIV, people not living with HIV, PrEP users, and PEP users. Our findings offer transferable lessons to optimize future EHR phenotyping for these cohorts.</p>","PeriodicalId":56334,"journal":{"name":"JMIR Medical Informatics","volume":"13 ","pages":"e68143"},"PeriodicalIF":3.1,"publicationDate":"2025-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144612353","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Predicting Drug-Side Effect Relationships From Parametric Knowledge Embedded in Biomedical BERT Models: Methodological Study With a Natural Language Processing Approach.","authors":"Woohyuk Jeon, Minjae Park, Doyeon An, Wonshik Nam, Ju-Young Shin, Seunghee Lee, Suehyun Lee","doi":"10.2196/67513","DOIUrl":"https://doi.org/10.2196/67513","url":null,"abstract":"&lt;p&gt;&lt;strong&gt;Background: &lt;/strong&gt;Adverse drug reactions (ADRs) pose serious risks to patient health, and effectively predicting and managing them is an important public health challenge. Given the complexity and specificity of biomedical text data, the traditional context-independent word embedding model, Word2Vec, has limitations in fully reflecting the domain specificity of such data. Although Bidirectional Encoder Representations from Transformers (BERT)-based models pretrained on biomedical corpora have demonstrated high performance in ADR-related studies, research using these models to predict previously unknown drug-side effect relationships remains insufficient.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Objective: &lt;/strong&gt;This study proposes a method for predicting drug-side effect relationships by leveraging the parametric knowledge embedded in biomedical BERT models. Through this approach, we predict promising candidates for potential drug-side effect relationships with unknown causal mechanisms by leveraging parametric knowledge from biomedical BERT models and embedding vector similarities of known relationships.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Methods: &lt;/strong&gt;We used 158,096 pairs of drug-side effect relationships from the side effect resource (SIDER) database to generate an adjacency matrix and calculate the cosine similarity between word embedding vectors of drugs and side effects. Relation scores were calculated for 8,235,435 drug-side effect pairs using this similarity. To evaluate the prediction accuracy of drug-side effect relationships, the area under the curve (AUC) value was measured using the calculated relation score and 158,096 known drug-side effect relationships from SIDER.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Results: &lt;/strong&gt;The clagator/biobert_v1.1 model achieved an AUC of 0.915 at an optimal threshold of 0.289, outperforming the existing Word2Vec model with an AUC of 0.848. The BERT-based models pretrained on the biomedical corpus outperformed the vanilla BERT model with an AUC of 0.857. External validation with the FDA (Food and Drug Administration) Adverse Event Reporting System data, using Fisher exact test based on 8,235,435 predicted drug-side effect pairs and 901,361 known relationships, confirmed high statistical significance (P&lt;.001) with an odds ratio of 4.822. In addition, a literature review of predicted drug-side effect relationships not confirmed in the SIDER database revealed that these relationships have been reported in recent studies published after 2016.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Conclusions: &lt;/strong&gt;This study introduces a method for extracting drug-side effect relationships embedded in parameters of language models pretrained on biomedical corpora and using this information to predict previously unknown drug-side effect relationships. We found that BERT-based models pretrained with biomedical corpora consider contextual information and achieve better performance in drug-side effect relationship prediction. External validation using the FDA Adverse Event Reporting System da","PeriodicalId":56334,"journal":{"name":"JMIR Medical Informatics","volume":"13 ","pages":"e67513"},"PeriodicalIF":3.1,"publicationDate":"2025-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144610430","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Leveraging AI to Drive Timely Improvements in Patient Experience Feedback: Algorithm Validation.","authors":"Mustafa Khanbhai, Catalina Carenzo, Sarindi Aryasinghe, David Manton, Erik Mayer","doi":"10.2196/60900","DOIUrl":"10.2196/60900","url":null,"abstract":"&lt;p&gt;&lt;strong&gt;Background: &lt;/strong&gt;Understanding and improving patient care is pivotal for health care providers. With increasing volumes of the Friends and Family Test (FFT) data in England, manual analysis of this patient feedback poses challenges for many health care organizations. This underscores the importance of automated text analysis, particularly in predicting sentiments and themes in real time.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Objective: &lt;/strong&gt;Leveraging machine learning and natural language processing, this study explores the utility of a supervised algorithm to systematically test and refine the algorithm's cross-contextual performance in diverse health care settings, addressing variations in population characteristics, geographical locations, and care settings, ultimately driving improvements based on patient feedback.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Methods: &lt;/strong&gt;The text analytics algorithm initially developed in a large acute trust in London was further tested in 9 health care organizations with diverse care settings across England. These trusts varied in technical capacity and resource, population demographics, and FFT free text datasets. Testing and validation of the algorithm were performed, including manual coding of a subset of retrospective comments. Technical infrastructure, including coding environments and packages for algorithm testing and deployment, was optimized. The algorithm was iteratively trained using bag of words from anonymized data, tailored to accommodate contextual variations, and tested for change in algorithm performance while simultaneously rectifying issues identified.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Results: &lt;/strong&gt;The algorithm demonstrated satisfactory overall accuracy (&gt;75%) in predicting themes and sentiments embedded within free-text responses across a variety of care settings and population demographics. While the algorithm yielded strong and reusable models in relatively stable environments, such as adult inpatient care settings, the initial accuracy was notably lower in organizations providing services such as pediatrics and mental health. However, the accuracy of our algorithm significantly improved when individual trust coding templates were applied. Thematic saturation was reached after the fifth organization was recruited, and no further coding was required for the last 4 organizations. Subsequently, a framework and pipeline for deployment of the algorithm were developed to provide a standardized approach for implementation and analysis of FFT free text, ensuring ease of use.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Conclusions: &lt;/strong&gt;This study represents a significant step forward in leveraging free-text FFT data for valuable insights in diverse health care settings through the testing and development of a robust supervised learning text analytics algorithm. The disparity in some care settings was anticipated, given that the lexicon and phraseology used was inherently different from those prevalent in adult inpatient care (where the algorithm was developed). H","PeriodicalId":56334,"journal":{"name":"JMIR Medical Informatics","volume":"13 ","pages":"e60900"},"PeriodicalIF":3.1,"publicationDate":"2025-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12270031/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144610382","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}