JAMIA OpenPub Date : 2023-09-22eCollection Date: 2023-10-01DOI: 10.1093/jamiaopen/ooad082
Fagen Xie, Susan Wang, Lori Viveros, Allegra Rich, Huong Q Nguyen, Ariadna Padilla, Lindsey Lyons, Claudia L Nau
{"title":"Using natural language processing to identify the status of homelessness and housing instability among serious illness patients from clinical notes in an integrated healthcare system.","authors":"Fagen Xie, Susan Wang, Lori Viveros, Allegra Rich, Huong Q Nguyen, Ariadna Padilla, Lindsey Lyons, Claudia L Nau","doi":"10.1093/jamiaopen/ooad082","DOIUrl":"https://doi.org/10.1093/jamiaopen/ooad082","url":null,"abstract":"<p><strong>Background: </strong>Efficiently identifying the social risks of patients with serious illnesses (SIs) is the critical first step in providing patient-centered and value-driven care for this medically vulnerable population.</p><p><strong>Objective: </strong>To apply and further hone an existing natural language process (NLP) algorithm that identifies patients who are homeless/at risk of homeless to a SI population.</p><p><strong>Methods: </strong>Patients diagnosed with SI between 2019 and 2020 were identified using an adapted list of diagnosis codes from the Center for Advance Palliative Care from the Kaiser Permanente Southern California electronic health record. Clinical notes associated with medical encounters within 6 months before and after the diagnosis date were processed by a previously developed NLP algorithm to identify patients who were homeless/at risk of homelessness. To improve the generalizability to the SI population, the algorithm was refined by multiple iterations of chart review and adjudication. The updated algorithm was then applied to the SI population.</p><p><strong>Results: </strong>Among 206 993 patients with a SI diagnosis, 1737 (0.84%) were identified as homeless/at risk of homelessness. These patients were more likely to be male (51.1%), age among 45-64 years (44.7%), and have one or more emergency visit (65.8%) within a year of their diagnosis date. Validation of the updated algorithm yielded a sensitivity of 100.0% and a positive predictive value of 93.8%.</p><p><strong>Conclusions: </strong>The improved NLP algorithm effectively identified patients with SI who were homeless/at risk of homelessness and can be used to target interventions for this vulnerable group.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 3","pages":"ooad082"},"PeriodicalIF":2.1,"publicationDate":"2023-09-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10517738/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41151029","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2023-09-19eCollection Date: 2023-10-01DOI: 10.1093/jamiaopen/ooad083
Kristen Petros De Guex, Tabor E Flickinger, Lisa Mayevsky, Hannah Zaveri, Michael Goncalves, Helen Reed, Lazaro Pesina, Rebecca Dillingham
{"title":"Optimizing usability of a mobile health intervention for Spanish-speaking Latinx people with HIV through user-centered design: a post-implementation study.","authors":"Kristen Petros De Guex, Tabor E Flickinger, Lisa Mayevsky, Hannah Zaveri, Michael Goncalves, Helen Reed, Lazaro Pesina, Rebecca Dillingham","doi":"10.1093/jamiaopen/ooad083","DOIUrl":"10.1093/jamiaopen/ooad083","url":null,"abstract":"<p><strong>Objective: </strong>Latinx people comprise 30% of all new human immunodeficiency virus (HIV) infections in the United States and face many challenges to accessing and engaging with HIV care. To bridge these gaps in care, a Spanish-language mobile health (mHealth) intervention known as ConexionesPositivas (CP) was adapted from an established English-language platform called PositiveLinks (PL) to help improve engagement in care and reduce viral nonsuppression among its users. We aimed to determine how CP can address the challenges that Latinx people with HIV (PWH) in the United States face.</p><p><strong>Materials and methods: </strong>We conducted a post-implementation study of the CP mHealth platform, guided by principles of user-centered design. We enrolled 20 Spanish-speaking CP users in the study, who completed the previously validated System Usability Scale (SUS) and semistructured interviews. Interviews were transcribed and translated for analysis. We performed thematic coding of interview transcripts in Dedoose.</p><p><strong>Results: </strong>The SUS composite score was 75, which is within the range of good usability. Four categories of themes were identified in the interviews: client context, strengths of CP, barriers to use and dislikes, and suggestions to improve CP. Positive impacts included encouraging self-monitoring of medication adherence, mood and stress, connection to professional care, and development of a support system for PWH.</p><p><strong>Discussion: </strong>While CP is an effective and easy-to-use application, participants expressed a desire for improved personalization and interactivity, which will guide further iteration.</p><p><strong>Conclusion: </strong>This study highlights the importance of tailoring mHealth interventions to improve equity of access, especially for populations with limited English proficiency.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 3","pages":"ooad083"},"PeriodicalIF":2.1,"publicationDate":"2023-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10508965/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41172420","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2023-09-14eCollection Date: 2023-10-01DOI: 10.1093/jamiaopen/ooad080
Jeana M Holt, AkkeNeel Talsma, Teresa S Johnson, Timothy Ehlinger
{"title":"Artificial neural network approaches to identify maternal and infant risk and asset factors using Peridata.Net: a WI-MIOS study.","authors":"Jeana M Holt, AkkeNeel Talsma, Teresa S Johnson, Timothy Ehlinger","doi":"10.1093/jamiaopen/ooad080","DOIUrl":"10.1093/jamiaopen/ooad080","url":null,"abstract":"<p><strong>Objective: </strong>To analyze PeriData.Net, a clinical registry with linked maternal-infant hospital data of Milwaukee County residents, to demonstrate a predictive analytic approach to perinatal infant risk assessment.</p><p><strong>Materials and methods: </strong>Using unsupervised learning, we identified infant birth clusters with similar multivariate health indicator patterns, measured using perinatal variables from 2008 to 2019 from <i>n</i> = 43 969 clinical registry records in Milwaukee County, WI, followed by supervised learning risk-propagation modeling to identify key maternal factors. To understand the relationship between socioeconomic status (SES) and birth outcome cluster assignment, we recoded zip codes in Peridata.Net according to SES level.</p><p><strong>Results: </strong>Three self-organizing map clusters describe infant birth outcome patterns that are similar in the multivariate space. Birth outcome clusters showed higher hazard birth outcome patterns in cluster 3 than clusters 1 and 2. Cluster 3 was associated with lower Apgar scores at 1 and 5 min after birth, shorter infant length, and premature birth. Prediction profiles of birth clusters indicate the most sensitivity to pregnancy weight loss and prenatal visits. Majority of infants assigned to cluster 3 were in the 2 lowest SES levels.</p><p><strong>Discussion: </strong>Using an extensive perinatal clinical registry, we found that the strongest predictive performance, when considering cluster membership using supervised learning, was achieved by incorporating social and behavioral risk factors. There were inequalities in infant birth outcomes based on SES.</p><p><strong>Conclusion: </strong>Identifying infant risk hazard profiles can contribute to knowledge discovery and guide future research directions. Additionally, presenting the results to community members can build consensus for community-identified health and risk indicator prioritization for intervention development.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 3","pages":"ooad080"},"PeriodicalIF":2.1,"publicationDate":"2023-09-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/97/3c/ooad080.PMC10500218.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10286996","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2023-09-11DOI: 10.1093/jamiaopen/ooad081
Ralph Ward, Jihad S Obeid, Lindsey Jennings, Elizabeth Szwast, William Garrett Hayes, Royal Pipaliya, Cameron Bailey, Skylar Faul, Brianna Polyak, George Hamilton Baker, Jenna L McCauley, Leslie A Lenert
{"title":"Enhanced phenotypes for identifying opioid overdose in emergency department visit electronic health record data","authors":"Ralph Ward, Jihad S Obeid, Lindsey Jennings, Elizabeth Szwast, William Garrett Hayes, Royal Pipaliya, Cameron Bailey, Skylar Faul, Brianna Polyak, George Hamilton Baker, Jenna L McCauley, Leslie A Lenert","doi":"10.1093/jamiaopen/ooad081","DOIUrl":"https://doi.org/10.1093/jamiaopen/ooad081","url":null,"abstract":"Abstract Background Accurate identification of opioid overdose (OOD) cases in electronic healthcare record (EHR) data is an important element in surveillance, empirical research, and clinical intervention. We sought to improve existing OOD electronic phenotypes by incorporating new data types beyond diagnostic codes and by applying several statistical and machine learning methods. Materials and Methods We developed an EHR dataset of emergency department visits involving OOD cases or patients considered at risk for an OOD and ascertained true OOD status through manual chart reviews. We developed and validated prediction models using Random Forest, Extreme Gradient Boost, and Elastic Net models that incorporated 717 features involving primary and second diagnoses, chief complaints, medications prescribed, vital signs, laboratory results, and procedural codes. We also developed models limited to single data types. Results A total of 1718 records involving 1485 patients were manually reviewed; 541 (36.4%) patients had one or more OOD. Prediction performance was similar for all models; sensitivity varied from 94% to 97%; and area under the receiver operating characteristic curve (AUC) was 98% for all methods. The primary diagnosis and chief complaint were the most important contributors to AUC performance; primary diagnoses and medication class contributed most to sensitivity; chief complaint, primary diagnosis, and vital signs were most important for specificity. Models limited to decision support data types available in real time demonstrated robust prediction performance. Conclusions Substantial prediction performance improvements were demonstrated for identifying OODs in EHR data. Our e-phenotypes could be applied in surveillance, retrospective empirical applications, or clinical decision support systems.","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136070908","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2023-09-08eCollection Date: 2023-10-01DOI: 10.1093/jamiaopen/ooad073
Ashley C Griffin, Saif Khairat, Stacy C Bailey, Arlene E Chung
{"title":"A chatbot for hypertension self-management support: user-centered design, development, and usability testing.","authors":"Ashley C Griffin, Saif Khairat, Stacy C Bailey, Arlene E Chung","doi":"10.1093/jamiaopen/ooad073","DOIUrl":"10.1093/jamiaopen/ooad073","url":null,"abstract":"<p><strong>Objectives: </strong>Health-related chatbots have demonstrated early promise for improving self-management behaviors but have seldomly been utilized for hypertension. This research focused on the design, development, and usability evaluation of a chatbot for hypertension self-management, called \"Medicagent.\"</p><p><strong>Materials and methods: </strong>A user-centered design process was used to iteratively design and develop a text-based chatbot using Google Cloud's Dialogflow natural language understanding platform. Then, usability testing sessions were conducted among patients with hypertension. Each session was comprised of: (1) background questionnaires, (2) 10 representative tasks within Medicagent, (3) System Usability Scale (SUS) questionnaire, and (4) a brief semi-structured interview. Sessions were video and audio recorded using Zoom. Qualitative and quantitative analyses were used to assess effectiveness, efficiency, and satisfaction of the chatbot.</p><p><strong>Results: </strong>Participants (<i>n</i> = 10) completed nearly all tasks (98%, 98/100) and spent an average of 18 min (SD = 10 min) interacting with Medicagent. Only 11 (8.6%) utterances were not successfully mapped to an intent. Medicagent achieved a mean SUS score of 78.8/100, which demonstrated acceptable usability. Several participants had difficulties navigating the conversational interface without menu and back buttons, felt additional information would be useful for redirection when utterances were not recognized, and desired a health professional persona within the chatbot.</p><p><strong>Discussion: </strong>The text-based chatbot was viewed favorably for assisting with blood pressure and medication-related tasks and had good usability.</p><p><strong>Conclusion: </strong>Flexibility of interaction styles, handling unrecognized utterances gracefully, and having a credible persona were highlighted as design components that may further enrich the user experience of chatbots for hypertension self-management.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 3","pages":"ooad073"},"PeriodicalIF":2.1,"publicationDate":"2023-09-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10491950/pdf/ooad073.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10570803","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2023-08-31eCollection Date: 2023-10-01DOI: 10.1093/jamiaopen/ooad077
Kelly T Gleason, Danielle S Powell, Aleksandra Wec, Xingyuan Zou, Mary Jo Gamper, Danielle Peereboom, Jennifer L Wolff
{"title":"Patient portal interventions: a scoping review of functionality, automation used, and therapeutic elements of patient portal interventions.","authors":"Kelly T Gleason, Danielle S Powell, Aleksandra Wec, Xingyuan Zou, Mary Jo Gamper, Danielle Peereboom, Jennifer L Wolff","doi":"10.1093/jamiaopen/ooad077","DOIUrl":"10.1093/jamiaopen/ooad077","url":null,"abstract":"<p><strong>Objectives: </strong>We sought to understand the objectives, targeted populations, therapeutic elements, and delivery characteristics of patient portal interventions.</p><p><strong>Materials and methods: </strong>Following Arksey and O-Malley's methodological framework, we conducted a scoping review of manuscripts published through June 2022 by hand and systematically searching PubMed, PSYCHInfo, Embase, and Web of Science. The search yielded 5403 manuscripts; 248 were selected for full-text review; 81 met the eligibility criteria for examining outcomes of a patient portal intervention.</p><p><strong>Results: </strong>The 81 articles described: trials involving comparison groups (<i>n</i> = 37; 45.7%), quality improvement initiatives (<i>n</i> = 15; 18.5%), pilot studies (<i>n</i> = 7; 8.6%), and single-arm studies (<i>n</i> = 22; 27.2%). Studies were conducted in primary care (<i>n</i> = 33, 40.7%), specialty outpatient (<i>n</i> = 24, 29.6%), or inpatient settings (<i>n</i> = 4, 4.9%)-or they were deployed system wide (<i>n</i> = 9, 11.1%). Interventions targeted specific health conditions (<i>n</i> = 35, 43.2%), promoted preventive services (<i>n</i> = 19, 23.5%), or addressed communication (<i>n</i> = 19, 23.4%); few specifically sought to improve the patient experience (<i>n</i> = 3, 3.7%). About half of the studies (<i>n</i> = 40, 49.4%) relied on human involvement, and about half involved personalized (vs exclusively standardized) elements (<i>n</i> = 42, 51.8%). Interventions commonly collected patient-reported information (<i>n</i> = 36, 44.4%), provided education (<i>n</i> = 35, 43.2%), or deployed preventive service reminders (<i>n</i> = 14, 17.3%).</p><p><strong>Discussion: </strong>This scoping review finds that most patient portal interventions have delivered education or facilitated collection of patient-reported information. Few interventions have involved pragmatic designs or been deployed system wide.</p><p><strong>Conclusion: </strong>The patient portal is an important tool in real-world efforts to more effectively support patients, but interventions to date rely largely on evidence from consented participants rather than pragmatically implemented systems-level initiatives.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 3","pages":"ooad077"},"PeriodicalIF":2.5,"publicationDate":"2023-08-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/a9/a9/ooad077.PMC10469545.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10208248","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2023-08-29eCollection Date: 2023-10-01DOI: 10.1093/jamiaopen/ooad078
Emily L Graul, Philip W Stone, Georgie M Massen, Sara Hatam, Alexander Adamson, Spiros Denaxas, Nicholas S Peters, Jennifer K Quint
{"title":"Determining prescriptions in electronic healthcare record data: methods for development of standardized, reproducible drug codelists.","authors":"Emily L Graul, Philip W Stone, Georgie M Massen, Sara Hatam, Alexander Adamson, Spiros Denaxas, Nicholas S Peters, Jennifer K Quint","doi":"10.1093/jamiaopen/ooad078","DOIUrl":"10.1093/jamiaopen/ooad078","url":null,"abstract":"<p><strong>Objective: </strong>To develop a standardizable, reproducible method for creating drug codelists that incorporates clinical expertise and is adaptable to other studies and databases.</p><p><strong>Materials and methods: </strong>We developed methods to generate drug codelists and tested this using the Clinical Practice Research Datalink (CPRD) Aurum database, accounting for missing data in the database. We generated codelists for: (1) cardiovascular disease and (2) inhaled Chronic Obstructive Pulmonary Disease (COPD) therapies, applying them to a sample cohort of 335 931 COPD patients. We compared searching all drug dictionary variables (A) against searching only (B) chemical or (C) ontological variables.</p><p><strong>Results: </strong>In Search A, we identified 165 150 patients prescribed cardiovascular drugs (49.2% of cohort), and 317 963 prescribed COPD inhalers (94.7% of cohort). Evaluating output per search strategy, Search C missed numerous prescriptions, including vasodilator anti-hypertensives (A and B:19 696 prescriptions; C:1145) and SAMA inhalers (A and B:35 310; C:564).</p><p><strong>Discussion: </strong>We recommend the full search (A) for comprehensiveness. There are special considerations when generating adaptable and generalizable drug codelists, including fluctuating status, cohort-specific drug indications, underlying hierarchical ontology, and statistical analyses.</p><p><strong>Conclusions: </strong>Methods must have end-to-end clinical input, and be standardizable, reproducible, and understandable to all researchers across data contexts.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 3","pages":"ooad078"},"PeriodicalIF":2.1,"publicationDate":"2023-08-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/d2/17/ooad078.PMC10463548.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10127493","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2023-08-18eCollection Date: 2023-10-01DOI: 10.1093/jamiaopen/ooad065
Seuli Bose-Brill, Rachel D'Amico, Adam Bartley, Robert Ashmead, Paola Flores-Beamon, Shadia Jallaq, Kevin Li, Shengyi Mao, Shannon Gillespie, Naleef Fareed, Kartik K Venkatesh, Norah L Crossnohere, Jody Davis, Alicia C Bunger, Allison Lorenz
{"title":"Establishing a clinical informatics umbilical cord: lessons learned in launching infrastructure to support dyadic mother/infant primary care.","authors":"Seuli Bose-Brill, Rachel D'Amico, Adam Bartley, Robert Ashmead, Paola Flores-Beamon, Shadia Jallaq, Kevin Li, Shengyi Mao, Shannon Gillespie, Naleef Fareed, Kartik K Venkatesh, Norah L Crossnohere, Jody Davis, Alicia C Bunger, Allison Lorenz","doi":"10.1093/jamiaopen/ooad065","DOIUrl":"10.1093/jamiaopen/ooad065","url":null,"abstract":"<p><p>The Multimodal Maternal Infant Perinatal Outpatient Delivery System (MOMI PODS) was developed to facilitate the pregnancy to postpartum primary care transition, particularly for individuals at risk for severe maternal morbidity, via a unique multidisciplinary model of mother/infant dyadic primary care. Specialized clinical informatics platforms are critical to ensuring the feasibility and scalability of MOMI PODS and a smooth perinatal transition into longitudinal postpartum primary care. In this manuscript, we describe the MOMI PODS transition and management clinical informatics platforms developed to facilitate MOMI PODS referrals, scheduling, evidence-based multidisciplinary care, and program evaluation. We discuss opportunities and lessons learned associated with our applied methods, as advances in clinical informatics have considerable potential to enhance the quality and evaluation of innovative maternal health programs like MOMI PODS.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 3","pages":"ooad065"},"PeriodicalIF":2.1,"publicationDate":"2023-08-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10438959/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10051098","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2023-08-17eCollection Date: 2023-10-01DOI: 10.1093/jamiaopen/ooad069
Timothy Lee, Paul J Lukac, Sitaram Vangala, Kamran Kowsari, Vu Vu, Spencer Fogelman, Michael A Pfeffer, Douglas S Bell
{"title":"Evaluating the predictive ability of natural language processing in identifying tertiary/quaternary cases in prioritization workflows for interhospital transfer.","authors":"Timothy Lee, Paul J Lukac, Sitaram Vangala, Kamran Kowsari, Vu Vu, Spencer Fogelman, Michael A Pfeffer, Douglas S Bell","doi":"10.1093/jamiaopen/ooad069","DOIUrl":"10.1093/jamiaopen/ooad069","url":null,"abstract":"<p><strong>Objectives: </strong>Tertiary and quaternary (TQ) care refers to complex cases requiring highly specialized health services. Our study aimed to compare the ability of a natural language processing (NLP) model to an existing human workflow in predictively identifying TQ cases for transfer requests to an academic health center.</p><p><strong>Materials and methods: </strong>Data on interhospital transfers were queried from the electronic health record for the 6-month period from July 1, 2020 to December 31, 2020. The NLP model was allowed to generate predictions on the same cases as the human predictive workflow during the study period. These predictions were then retrospectively compared to the true TQ outcomes.</p><p><strong>Results: </strong>There were 1895 transfer cases labeled by both the human predictive workflow and the NLP model, all of which had retrospective confirmation of the true TQ label. The NLP model receiver operating characteristic curve had an area under the curve of 0.91. Using a model probability threshold of ≥0.3 to be considered TQ positive, accuracy was 81.5% for the NLP model versus 80.3% for the human predictions (<i>P </i>=<i> </i>.198) while sensitivity was 83.6% versus 67.7% (<i>P</i><.001).</p><p><strong>Discussion: </strong>The NLP model was as accurate as the human workflow but significantly more sensitive. This translated to 15.9% more TQ cases identified by the NLP model.</p><p><strong>Conclusion: </strong>Integrating an NLP model into existing workflows as automated decision support could translate to more TQ cases identified at the onset of the transfer process.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 3","pages":"ooad069"},"PeriodicalIF":2.1,"publicationDate":"2023-08-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/fd/7d/ooad069.PMC10435371.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10049396","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2023-08-16eCollection Date: 2023-10-01DOI: 10.1093/jamiaopen/ooad067
Sara E Jones, Katie R Bradwell, Lauren E Chan, Julie A McMurry, Courtney Olson-Chen, Jessica Tarleton, Kenneth J Wilkins, Victoria Ly, Saad Ljazouli, Qiuyuan Qin, Emily Groene Faherty, Yan Kwan Lau, Catherine Xie, Yu-Han Kao, Michael N Liebman, Federico Mariona, Anup P Challa, Li Li, Sarah J Ratcliffe, Melissa A Haendel, Rena C Patel, Elaine L Hill
{"title":"Who is pregnant? Defining real-world data-based pregnancy episodes in the National COVID Cohort Collaborative (N3C).","authors":"Sara E Jones, Katie R Bradwell, Lauren E Chan, Julie A McMurry, Courtney Olson-Chen, Jessica Tarleton, Kenneth J Wilkins, Victoria Ly, Saad Ljazouli, Qiuyuan Qin, Emily Groene Faherty, Yan Kwan Lau, Catherine Xie, Yu-Han Kao, Michael N Liebman, Federico Mariona, Anup P Challa, Li Li, Sarah J Ratcliffe, Melissa A Haendel, Rena C Patel, Elaine L Hill","doi":"10.1093/jamiaopen/ooad067","DOIUrl":"10.1093/jamiaopen/ooad067","url":null,"abstract":"<p><strong>Objectives: </strong>To define pregnancy episodes and estimate gestational age within electronic health record (EHR) data from the National COVID Cohort Collaborative (N3C).</p><p><strong>Materials and methods: </strong>We developed a comprehensive approach, named Hierarchy and rule-based pregnancy episode Inference integrated with Pregnancy Progression Signatures (HIPPS), and applied it to EHR data in the N3C (January 1, 2018-April 7, 2022). HIPPS combines: (1) an extension of a previously published pregnancy episode algorithm, (2) a novel algorithm to detect gestational age-specific signatures of a progressing pregnancy for further episode support, and (3) pregnancy start date inference. Clinicians performed validation of HIPPS on a subset of episodes. We then generated pregnancy cohorts based on gestational age precision and pregnancy outcomes for assessment of accuracy and comparison of COVID-19 and other characteristics.</p><p><strong>Results: </strong>We identified 628 165 pregnant persons with 816 471 pregnancy episodes, of which 52.3% were live births, 24.4% were other outcomes (stillbirth, ectopic pregnancy, abortions), and 23.3% had unknown outcomes. Clinician validation agreed 98.8% with HIPPS-identified episodes. We were able to estimate start dates within 1 week of precision for 475 433 (58.2%) episodes. 62 540 (7.7%) episodes had incident COVID-19 during pregnancy.</p><p><strong>Discussion: </strong>HIPPS provides measures of support for pregnancy-related variables such as gestational age and pregnancy outcomes based on N3C data. Gestational age precision allows researchers to find time to events with reasonable confidence.</p><p><strong>Conclusion: </strong>We have developed a novel and robust approach for inferring pregnancy episodes and gestational age that addresses data inconsistency and missingness in EHR data.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 3","pages":"ooad067"},"PeriodicalIF":2.5,"publicationDate":"2023-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10432357/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10200000","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}