Journal of the American Medical Informatics Association最新文献

{"title":"Evaluating gradient-based explanation methods for neural network ECG analysis using heatmaps.","authors":"Andrea Marheim Storås, Steffen Mæland, Jonas L Isaksen, Steven Alexander Hicks, Vajira Thambawita, Claus Graff, Hugo Lewi Hammer, Pål Halvorsen, Michael Alexander Riegler, Jørgen K Kanters","doi":"10.1093/jamia/ocae280","DOIUrl":"10.1093/jamia/ocae280","url":null,"abstract":"<p><strong>Objective: </strong>Evaluate popular explanation methods using heatmap visualizations to explain the predictions of deep neural networks for electrocardiogram (ECG) analysis and provide recommendations for selection of explanations methods.</p><p><strong>Materials and methods: </strong>A residual deep neural network was trained on ECGs to predict intervals and amplitudes. Nine commonly used explanation methods (Saliency, Deconvolution, Guided backpropagation, Gradient SHAP, SmoothGrad, Input × gradient, DeepLIFT, Integrated gradients, GradCAM) were qualitatively evaluated by medical experts and objectively evaluated using a perturbation-based method.</p><p><strong>Results: </strong>No single explanation method consistently outperformed the other methods, but some methods were clearly inferior. We found considerable disagreement between the human expert evaluation and the objective evaluation by perturbation.</p><p><strong>Discussion: </strong>The best explanation method depended on the ECG measure. To ensure that future explanations of deep neural networks for medical data analyses are useful to medical experts, data scientists developing new explanation methods should collaborate tightly with domain experts. Because there is no explanation method that performs best in all use cases, several methods should be applied.</p><p><strong>Conclusion: </strong>Several explanation methods should be used to determine the most suitable approach.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"79-88"},"PeriodicalIF":4.7,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11648713/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142591550","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A machine learning framework to adjust for learning effects in medical device safety evaluation.","authors":"Jejo D Koola, Karthik Ramesh, Jialin Mao, Minyoung Ahn, Sharon E Davis, Usha Govindarajulu, Amy M Perkins, Dax Westerman, Henry Ssemaganda, Theodore Speroff, Lucila Ohno-Machado, Craig R Ramsay, Art Sedrakyan, Frederic S Resnic, Michael E Matheny","doi":"10.1093/jamia/ocae273","DOIUrl":"10.1093/jamia/ocae273","url":null,"abstract":"<p><strong>Objectives: </strong>Traditional methods for medical device post-market surveillance often fail to accurately account for operator learning effects, leading to biased assessments of device safety. These methods struggle with non-linearity, complex learning curves, and time-varying covariates, such as physician experience. To address these limitations, we sought to develop a machine learning (ML) framework to detect and adjust for operator learning effects.</p><p><strong>Materials and methods: </strong>A gradient-boosted decision tree ML method was used to analyze synthetic datasets that replicate the complexity of clinical scenarios involving high-risk medical devices. We designed this process to detect learning effects using a risk-adjusted cumulative sum method, quantify the excess adverse event rate attributable to operator inexperience, and adjust for these alongside patient factors in evaluating device safety signals. To maintain integrity, we employed blinding between data generation and analysis teams. Synthetic data used underlying distributions and patient feature correlations based on clinical data from the Department of Veterans Affairs between 2005 and 2012. We generated 2494 synthetic datasets with widely varying characteristics including number of patient features, operators and institutions, and the operator learning form. Each dataset contained a hypothetical study device, Device B, and a reference device, Device A. We evaluated accuracy in identifying learning effects and identifying and estimating the strength of the device safety signal. Our approach also evaluated different clinically relevant thresholds for safety signal detection.</p><p><strong>Results: </strong>Our framework accurately identified the presence or absence of learning effects in 93.6% of datasets and correctly determined device safety signals in 93.4% of cases. The estimated device odds ratios' 95% confidence intervals were accurately aligned with the specified ratios in 94.7% of datasets. In contrast, a comparative model excluding operator learning effects significantly underperformed in detecting device signals and in accuracy. Notably, our framework achieved 100% specificity for clinically relevant safety signal thresholds, although sensitivity varied with the threshold applied.</p><p><strong>Discussion: </strong>A machine learning framework, tailored for the complexities of post-market device evaluation, may provide superior performance compared to standard parametric techniques when operator learning is present.</p><p><strong>Conclusion: </strong>Demonstrating the capacity of ML to overcome complex evaluative challenges, our framework addresses the limitations of traditional statistical methods in current post-market surveillance processes. By offering a reliable means to detect and adjust for learning effects, it may significantly improve medical device safety evaluation.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"206-217"},"PeriodicalIF":4.7,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11648715/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142548633","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Research for all: building a diverse researcher community for the All of Us Research Program.","authors":"Rubin Baskir, Minnkyong Lee, Sydney J McMaster, Jessica Lee, Faith Blackburne-Proctor, Romuladus Azuine, Nakia Mack, Sheri D Schully, Martin Mendoza, Janeth Sanchez, Yong Crosby, Erica Zumba, Michael Hahn, Naomi Aspaas, Ahmed Elmi, Shanté Alerté, Elizabeth Stewart, Danielle Wilfong, Meag Doherty, Margaret M Farrell, Grace B Hébert, Sula Hood, Cheryl M Thomas, Debra D Murray, Brendan Lee, Louisa A Stark, Megan A Lewis, Jen D Uhrig, Laura R Bartlett, Edgar Gil Rico, Adolph Falcón, Elizabeth Cohn, Mitchell R Lunn, Juno Obedin-Maliver, Linda Cottler, Milton Eder, Fornessa T Randal, Jason Karnes, KiTani Lemieux, Nelson Lemieux, Nelson Lemieux, Lilanta Bradley, Ronnie Tepp, Meredith Wilson, Monica Rodriguez, Chris Lunt, Karriem Watson","doi":"10.1093/jamia/ocae270","DOIUrl":"10.1093/jamia/ocae270","url":null,"abstract":"<p><strong>Objectives: </strong>The NIH All of Us Research Program (All of Us) is engaging a diverse community of more than 10 000 registered researchers using a robust engagement ecosystem model. We describe strategies used to build an ecosystem that attracts and supports a diverse and inclusive researcher community to use the All of Us dataset and provide metrics on All of Us researcher usage growth.</p><p><strong>Materials and methods: </strong>Researcher audiences and diversity categories were defined to guide a strategy. A researcher engagement strategy was codeveloped with program partners to support a researcher engagement ecosystem. An adapted ecological model guided the ecosystem to address multiple levels of influence to support All of Us data use. Statistics from the All of Us Researcher Workbench demographic survey describe trends in researchers' and institutional use of the Workbench and publication numbers.</p><p><strong>Results: </strong>From 2022 to 2024, some 13 partner organizations and their subawardees conducted outreach, built capacity, or supported researchers and institutions in using the data. Trends indicate that Workbench registrations and use have increased over time, including among researchers underrepresented in the biomedical workforce. Data Use and Registration Agreements from minority-serving institutions also increased.</p><p><strong>Discussion: </strong>All of Us built a diverse, inclusive, and growing research community via intentional engagement with researchers and via partnerships to address systemic data access issues. Future programs will provide additional support to researchers and institutions to ameliorate All of Us data use challenges.</p><p><strong>Conclusion: </strong>The approach described helps address structural inequities in the biomedical research field to advance health equity.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"38-50"},"PeriodicalIF":4.7,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11648709/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142631475","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Correction to: Measuring interpersonal firearm violence: natural language processing methods to address limitations in criminal charge data.","authors":"","doi":"10.1093/jamia/ocae268","DOIUrl":"10.1093/jamia/ocae268","url":null,"abstract":"","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"264"},"PeriodicalIF":4.7,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11648706/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142479223","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Trending in the right direction: critical access hospitals increased adoption of advanced electronic health record functions from 2018 to 2023.","authors":"Nate C Apathy, A Jay Holmgren, Paige Nong, Julia Adler-Milstein, Jordan Everson","doi":"10.1093/jamia/ocae267","DOIUrl":"10.1093/jamia/ocae267","url":null,"abstract":"<p><strong>Objectives: </strong>We analyzed trends in adoption of advanced patient engagement and clinical data analytics functionalities among critical access hospitals (CAHs) and non-CAHs to assess how historical gaps have changed.</p><p><strong>Materials and methods: </strong>We used 2014, 2018, and 2023 data from the American Hospital Association Annual Survey IT Supplement to measure differences in adoption rates (ie, the \"adoption gap\") of patient engagement and clinical data analytics functionalities across CAHs and non-CAHs. We measured changes over time in CAH and non-CAH adoption of 6 \"core\" clinical data analytics functionalities, 5 \"core\" patient engagement functionalities, 5 new patient engagement functionalities, and 3 bulk data export use cases. We constructed 2 composite measures for core functionalities and analyzed adoption for other functionalities individually.</p><p><strong>Results: </strong>Core functionality adoption increased from 21% of CAHs in 2014 to 56% in 2023 for clinical data analytics and 18% to 49% for patient engagement. The CAH adoption gap in both domains narrowed from 2018 to 2023 (both P < .01). More than 90% of all hospitals had adopted viewing and downloading electronic data and clinical notes by 2023. The largest CAH adoption gaps in 2023 were for Fast Healthcare Interoperability Resources (FHIR) bulk export use cases (eg, analytics and reporting: 63% of CAHs, 81% of non-CAHs, P < .001).</p><p><strong>Discussion: </strong>Adoption of advanced electronic health record functionalities has increased for CAHs and non-CAHs, and some adoption gaps have been closed since 2018. However, CAHs may continue to struggle with clinical data analytics and FHIR-based functionalities.</p><p><strong>Conclusion: </strong>Some crucial patient engagement functionalities have reached near-universal adoption; however, policymakers should consider programs to support CAHs in closing remaining adoption gaps.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"71-78"},"PeriodicalIF":4.7,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11648721/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142591482","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Correction to: Are medical history data fit for risk stratification of patients with chest pain in emergency care? Comparing data collected from patients using computerized history taking with data documented by physicians in the electronic health record in the CLEOS-CPDS prospective cohort study.","authors":"","doi":"10.1093/jamia/ocae252","DOIUrl":"10.1093/jamia/ocae252","url":null,"abstract":"","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"261-263"},"PeriodicalIF":4.7,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11648703/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142331358","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Comparative analysis of personal protective equipment nonadherence detection: computer vision versus human observers.","authors":"Mary S Kim, Beomseok Park, Genevieve J Sippel, Aaron H Mun, Wanzhao Yang, Kathleen H McCarthy, Emely Fernandez, Marius George Linguraru, Aleksandra Sarcevic, Ivan Marsic, Randall S Burd","doi":"10.1093/jamia/ocae262","DOIUrl":"10.1093/jamia/ocae262","url":null,"abstract":"<p><strong>Objectives: </strong>Human monitoring of personal protective equipment (PPE) adherence among healthcare providers has several limitations, including the need for additional personnel during staff shortages and decreased vigilance during prolonged tasks. To address these challenges, we developed an automated computer vision system for monitoring PPE adherence in healthcare settings. We assessed the system performance against human observers detecting nonadherence in a video surveillance experiment.</p><p><strong>Materials and methods: </strong>The automated system was trained to detect 15 classes of eyewear, masks, gloves, and gowns using an object detector and tracker. To assess how the system performs compared to human observers in detecting nonadherence, we designed a video surveillance experiment under 2 conditions: variations in video durations (20, 40, and 60 seconds) and the number of individuals in the videos (3 versus 6). Twelve nurses participated as human observers. Performance was assessed based on the number of detections of nonadherence.</p><p><strong>Results: </strong>Human observers detected fewer instances of nonadherence than the system (parameter estimate -0.3, 95% CI -0.4 to -0.2, P < .001). Human observers detected more nonadherence during longer video durations (parameter estimate 0.7, 95% CI 0.4-1.0, P < .001). The system achieved a sensitivity of 0.86, specificity of 1, and Matthew's correlation coefficient of 0.82 for detecting PPE nonadherence.</p><p><strong>Discussion: </strong>An automated system simultaneously tracks multiple objects and individuals. The system performance is also independent of observation duration, an improvement over human monitoring.</p><p><strong>Conclusion: </strong>The automated system presents a potential solution for scalable monitoring of hospital-wide infection control practices and improving PPE usage in healthcare settings.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"163-171"},"PeriodicalIF":4.7,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11648733/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142479222","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The journey to building a diverse, equitable, and inclusive American Medical Informatics Association.","authors":"Tiffani J Bright, Oliver J Bear Don't Walk Iv, Carl Erwin Johnson, Carolyn Petersen, Patricia C Dykes, Krista G Martin, Kevin B Johnson, Lois Walters-Threat, Catherine K Craven, Robert J Lucero, Gretchen P Jackson, Rubina F Rizvi","doi":"10.1093/jamia/ocae258","DOIUrl":"10.1093/jamia/ocae258","url":null,"abstract":"<p><strong>Objective: </strong>The American Medical Informatics Association (AMIA) Task Force on Diversity, Equity, and Inclusion (DEI) was established to address systemic racism and health disparities in biomedical and health informatics, aligning with AMIA's mission to transform healthcare. AMIA's DEI initiatives were spurred by member voices responding to police brutality and COVID-19's impact on Black/African American communities.</p><p><strong>Materials and methods: </strong>The Task Force, consisting of 20 members across 3 groups aligned with AMIA's 2020-2025 Strategic Plan, met biweekly to develop DEI recommendations with the help of 16 additional volunteers. These recommendations were reviewed, prioritized, and presented to the AMIA Board of Directors for approval.</p><p><strong>Results: </strong>In 9 months, the Task Force (1) created a logic model to support workforce diversity and raise AMIA's DEI awareness, (2) conducted an environmental scan of other associations' DEI activities, (3) developed a DEI framework for AMIA meetings, (4) gathered member feedback, (5) cultivated DEI educational resources, (6) created a Board nominations and diversity session, (7) reviewed the Board's Strategic Planning for DEI alignment, (8) led a program to increase diversity at the 2020 AMIA Virtual Annual Symposium, and (9) standardized socially-assigned race and ethnicity data collection.</p><p><strong>Discussion: </strong>The Task Force proposed actionable recommendations that focused on AMIA's role in addressing systemic racism and health equity, helping the organization understand its member diversity.</p><p><strong>Conclusion: </strong>This work supported marginalized groups, broadened the research agenda, and positioned AMIA as a DEI leader while reinforcing the need for ongoing transformation within informatics.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"3-8"},"PeriodicalIF":4.7,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11648708/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142479236","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The role of routine and structured social needs data collection in improving care in US hospitals.","authors":"Chelsea Richwine, Vaishali Patel, Jordan Everson, Bradley Iott","doi":"10.1093/jamia/ocae279","DOIUrl":"10.1093/jamia/ocae279","url":null,"abstract":"<p><strong>Objectives: </strong>To understand how health-related social needs (HRSN) data are collected at US hospitals and implications for use.</p><p><strong>Materials and methods: </strong>Using 2023 nationally representative survey data on US hospitals (N = 2775), we described hospitals' routine and structured collection and use of HRSN data and examined the relationship between methods of data collection and specific uses. Multivariate logistic regression was used to identify characteristics associated with data collection and use and understand how methods of data collection relate to use.</p><p><strong>Results: </strong>In 2023, 88% of hospitals collected HRSN data (64% routinely, 72% structured). While hospitals commonly used data for internal purposes (eg, discharge planning, 79%), those that collected data routinely and in a structured format (58%) used data for purposes involving coordination or exchange with other organizations (eg, making referrals, 74%) at higher rates than hospitals that collected data but not routinely or in a non-structured format (eg, 93% vs 67% for referrals, P< .05). In multivariate regression, routine and structured data collection was positively associated with all uses of data examined. Hospital location, ownership, system-affiliation, value-based care participation, and critical access designation were associated with HRSN data collection, but only system-affiliation was consistently (positively) associated with use.</p><p><strong>Discussion: </strong>While most hospitals screen for social needs, fewer collect data routinely and in a structured format that would facilitate downstream use. Routine and structured data collection was associated with greater use, particularly for secondary purposes.</p><p><strong>Conclusion: </strong>Routine and structured screening may result in more actionable data that facilitates use for various purposes that support patient care and improve community and population health, indicating the importance of continuing efforts to increase routine screening and standardize HRSN data collection.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"28-37"},"PeriodicalIF":4.7,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11648711/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142591563","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Correction to: Artificial intelligence for optimizing recruitment and retention in clinical trials: a scoping review.","authors":"","doi":"10.1093/jamia/ocae283","DOIUrl":"10.1093/jamia/ocae283","url":null,"abstract":"","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"260"},"PeriodicalIF":4.7,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11648702/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142583537","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}