JAMIA OpenPub Date : 2023-08-17eCollection Date: 2023-10-01DOI: 10.1093/jamiaopen/ooad069
Timothy Lee, Paul J Lukac, Sitaram Vangala, Kamran Kowsari, Vu Vu, Spencer Fogelman, Michael A Pfeffer, Douglas S Bell
{"title":"Evaluating the predictive ability of natural language processing in identifying tertiary/quaternary cases in prioritization workflows for interhospital transfer.","authors":"Timothy Lee, Paul J Lukac, Sitaram Vangala, Kamran Kowsari, Vu Vu, Spencer Fogelman, Michael A Pfeffer, Douglas S Bell","doi":"10.1093/jamiaopen/ooad069","DOIUrl":"10.1093/jamiaopen/ooad069","url":null,"abstract":"<p><strong>Objectives: </strong>Tertiary and quaternary (TQ) care refers to complex cases requiring highly specialized health services. Our study aimed to compare the ability of a natural language processing (NLP) model to an existing human workflow in predictively identifying TQ cases for transfer requests to an academic health center.</p><p><strong>Materials and methods: </strong>Data on interhospital transfers were queried from the electronic health record for the 6-month period from July 1, 2020 to December 31, 2020. The NLP model was allowed to generate predictions on the same cases as the human predictive workflow during the study period. These predictions were then retrospectively compared to the true TQ outcomes.</p><p><strong>Results: </strong>There were 1895 transfer cases labeled by both the human predictive workflow and the NLP model, all of which had retrospective confirmation of the true TQ label. The NLP model receiver operating characteristic curve had an area under the curve of 0.91. Using a model probability threshold of ≥0.3 to be considered TQ positive, accuracy was 81.5% for the NLP model versus 80.3% for the human predictions (<i>P </i>=<i> </i>.198) while sensitivity was 83.6% versus 67.7% (<i>P</i><.001).</p><p><strong>Discussion: </strong>The NLP model was as accurate as the human workflow but significantly more sensitive. This translated to 15.9% more TQ cases identified by the NLP model.</p><p><strong>Conclusion: </strong>Integrating an NLP model into existing workflows as automated decision support could translate to more TQ cases identified at the onset of the transfer process.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 3","pages":"ooad069"},"PeriodicalIF":2.1,"publicationDate":"2023-08-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/fd/7d/ooad069.PMC10435371.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10049396","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2023-08-16eCollection Date: 2023-10-01DOI: 10.1093/jamiaopen/ooad067
Sara E Jones, Katie R Bradwell, Lauren E Chan, Julie A McMurry, Courtney Olson-Chen, Jessica Tarleton, Kenneth J Wilkins, Victoria Ly, Saad Ljazouli, Qiuyuan Qin, Emily Groene Faherty, Yan Kwan Lau, Catherine Xie, Yu-Han Kao, Michael N Liebman, Federico Mariona, Anup P Challa, Li Li, Sarah J Ratcliffe, Melissa A Haendel, Rena C Patel, Elaine L Hill
{"title":"Who is pregnant? Defining real-world data-based pregnancy episodes in the National COVID Cohort Collaborative (N3C).","authors":"Sara E Jones, Katie R Bradwell, Lauren E Chan, Julie A McMurry, Courtney Olson-Chen, Jessica Tarleton, Kenneth J Wilkins, Victoria Ly, Saad Ljazouli, Qiuyuan Qin, Emily Groene Faherty, Yan Kwan Lau, Catherine Xie, Yu-Han Kao, Michael N Liebman, Federico Mariona, Anup P Challa, Li Li, Sarah J Ratcliffe, Melissa A Haendel, Rena C Patel, Elaine L Hill","doi":"10.1093/jamiaopen/ooad067","DOIUrl":"10.1093/jamiaopen/ooad067","url":null,"abstract":"<p><strong>Objectives: </strong>To define pregnancy episodes and estimate gestational age within electronic health record (EHR) data from the National COVID Cohort Collaborative (N3C).</p><p><strong>Materials and methods: </strong>We developed a comprehensive approach, named Hierarchy and rule-based pregnancy episode Inference integrated with Pregnancy Progression Signatures (HIPPS), and applied it to EHR data in the N3C (January 1, 2018-April 7, 2022). HIPPS combines: (1) an extension of a previously published pregnancy episode algorithm, (2) a novel algorithm to detect gestational age-specific signatures of a progressing pregnancy for further episode support, and (3) pregnancy start date inference. Clinicians performed validation of HIPPS on a subset of episodes. We then generated pregnancy cohorts based on gestational age precision and pregnancy outcomes for assessment of accuracy and comparison of COVID-19 and other characteristics.</p><p><strong>Results: </strong>We identified 628 165 pregnant persons with 816 471 pregnancy episodes, of which 52.3% were live births, 24.4% were other outcomes (stillbirth, ectopic pregnancy, abortions), and 23.3% had unknown outcomes. Clinician validation agreed 98.8% with HIPPS-identified episodes. We were able to estimate start dates within 1 week of precision for 475 433 (58.2%) episodes. 62 540 (7.7%) episodes had incident COVID-19 during pregnancy.</p><p><strong>Discussion: </strong>HIPPS provides measures of support for pregnancy-related variables such as gestational age and pregnancy outcomes based on N3C data. Gestational age precision allows researchers to find time to events with reasonable confidence.</p><p><strong>Conclusion: </strong>We have developed a novel and robust approach for inferring pregnancy episodes and gestational age that addresses data inconsistency and missingness in EHR data.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 3","pages":"ooad067"},"PeriodicalIF":2.5,"publicationDate":"2023-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10432357/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10200000","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2023-08-16eCollection Date: 2023-10-01DOI: 10.1093/jamiaopen/ooad070
Kevin Xie, Samuel W Terman, Ryan S Gallagher, Chloe E Hill, Kathryn A Davis, Brian Litt, Dan Roth, Colin A Ellis
{"title":"Generalization of finetuned transformer language models to new clinical contexts.","authors":"Kevin Xie, Samuel W Terman, Ryan S Gallagher, Chloe E Hill, Kathryn A Davis, Brian Litt, Dan Roth, Colin A Ellis","doi":"10.1093/jamiaopen/ooad070","DOIUrl":"10.1093/jamiaopen/ooad070","url":null,"abstract":"<p><strong>Objective: </strong>We have previously developed a natural language processing pipeline using clinical notes written by epilepsy specialists to extract seizure freedom, seizure frequency text, and date of last seizure text for patients with epilepsy. It is important to understand how our methods generalize to new care contexts.</p><p><strong>Materials and methods: </strong>We evaluated our pipeline on unseen notes from nonepilepsy-specialist neurologists and non-neurologists without any additional algorithm training. We tested the pipeline out-of-institution using epilepsy specialist notes from an outside medical center with only minor preprocessing adaptations. We examined reasons for discrepancies in performance in new contexts by measuring physical and semantic similarities between documents.</p><p><strong>Results: </strong>Our ability to classify patient seizure freedom decreased by at least 0.12 agreement when moving from epilepsy specialists to nonspecialists or other institutions. On notes from our institution, textual overlap between the extracted outcomes and the gold standard annotations attained from manual chart review decreased by at least 0.11 F<sub>1</sub> when an answer existed but did not change when no answer existed; here our models generalized on notes from the outside institution, losing at most 0.02 agreement. We analyzed textual differences and found that syntactic and semantic differences in both clinically relevant sentences and surrounding contexts significantly influenced model performance.</p><p><strong>Discussion and conclusion: </strong>Model generalization performance decreased on notes from nonspecialists; out-of-institution generalization on epilepsy specialist notes required small changes to preprocessing but was especially good for seizure frequency text and date of last seizure text, opening opportunities for multicenter collaborations using these outcomes.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 3","pages":"ooad070"},"PeriodicalIF":2.5,"publicationDate":"2023-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10432353/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10049398","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2023-08-09eCollection Date: 2023-10-01DOI: 10.1093/jamiaopen/ooad063
Olena Mazurenko, Emma McCord, Cara McDonnell, Nate C Apathy, Lindsey Sanner, Meredith C B Adams, Burke W Mamlin, Joshua R Vest, Robert W Hurley, Christopher A Harle
{"title":"Examining primary care provider experiences with using a clinical decision support tool for pain management.","authors":"Olena Mazurenko, Emma McCord, Cara McDonnell, Nate C Apathy, Lindsey Sanner, Meredith C B Adams, Burke W Mamlin, Joshua R Vest, Robert W Hurley, Christopher A Harle","doi":"10.1093/jamiaopen/ooad063","DOIUrl":"10.1093/jamiaopen/ooad063","url":null,"abstract":"<p><strong>Objective: </strong>To evaluate primary care provider (PCP) experiences using a clinical decision support (CDS) tool over 16 months following a user-centered design process and implementation.</p><p><strong>Materials and methods: </strong>We conducted a qualitative evaluation of the Chronic Pain OneSheet (OneSheet), a chronic pain CDS tool. OneSheet provides pain- and opioid-related risks, benefits, and treatment information for patients with chronic pain to PCPs. Using the 5 Rights of CDS framework, we conducted and analyzed semi-structured interviews with 19 PCPs across 2 academic health systems.</p><p><strong>Results: </strong>PCPs stated that OneSheet mostly contained the right information required to treat patients with chronic pain and was correctly located in the electronic health record. PCPs used OneSheet for distinct subgroups of patients with chronic pain, including patients prescribed opioids, with poorly controlled pain, or new to a provider or clinic. PCPs reported variable workflow integration and selective use of certain OneSheet features driven by their preferences and patient population. PCPs recommended broadening OneSheet access to clinical staff and patients for data entry to address clinician time constraints.</p><p><strong>Discussion: </strong>Differences in patient subpopulations and workflow preferences had an outsized effect on CDS tool use even when the CDS contained the right information identified in a user-centered design process.</p><p><strong>Conclusions: </strong>To increase adoption and use, CDS design and implementation processes may benefit from increased tailoring that accommodates variation and dynamics among patients, visits, and providers.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 3","pages":"ooad063"},"PeriodicalIF":2.1,"publicationDate":"2023-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/fe/32/ooad063.PMC10412405.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9998689","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2023-07-28eCollection Date: 2023-10-01DOI: 10.1093/jamiaopen/ooad064
{"title":"Correction to: Predicting risk of metastases and recurrence in soft-tissue sarcomas via Radiomics and Formal Methods.","authors":"","doi":"10.1093/jamiaopen/ooad064","DOIUrl":"10.1093/jamiaopen/ooad064","url":null,"abstract":"<p><p>[This corrects the article DOI: 10.1093/jamiaopen/ooad025.].</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 3","pages":"ooad064"},"PeriodicalIF":2.1,"publicationDate":"2023-07-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10382253/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10274180","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2023-07-19eCollection Date: 2023-10-01DOI: 10.1093/jamiaopen/ooad044
Oliver Aalami, Mike Hittle, Vishnu Ravi, Ashley Griffin, Paul Schmiedmayer, Varun Shenoy, Santiago Gutierrez, Ross Venook
{"title":"CardinalKit: open-source standards-based, interoperable mobile development platform to help translate the promise of digital health.","authors":"Oliver Aalami, Mike Hittle, Vishnu Ravi, Ashley Griffin, Paul Schmiedmayer, Varun Shenoy, Santiago Gutierrez, Ross Venook","doi":"10.1093/jamiaopen/ooad044","DOIUrl":"10.1093/jamiaopen/ooad044","url":null,"abstract":"<p><p>Smartphone devices capable of monitoring users' health, physiology, activity, and environment revolutionize care delivery, medical research, and remote patient monitoring. Such devices, laden with clinical-grade sensors and cloud connectivity, allow clinicians, researchers, and patients to monitor health longitudinally, passively, and persistently, shifting the paradigm of care and research from low-resolution, intermittent, and discrete to one of persistent, continuous, and high resolution. The collection, transmission, and storage of sensitive health data using mobile devices presents unique challenges that serve as significant barriers to entry for care providers and researchers alike. Compliance with standards like HIPAA and GDPR requires unique skills and practices. These requirements make off-the-shelf technologies insufficient for use in the digital health space. As a result, budget, timeline, talent, and resource constraints are the largest barriers to new digital technologies. The CardinalKit platform is an open-source project addressing these challenges by focusing on reducing these barriers and accelerating the innovation, adoption, and use of digital health technologies. CardinalKit provides a mobile template application and web dashboard to enable an interoperable foundation for developing digital health applications. We demonstrate the applicability of CardinalKit to a wide variety of digital health applications across 18 innovative digital health prototypes.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 3","pages":"ooad044"},"PeriodicalIF":2.1,"publicationDate":"2023-07-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10356573/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9919009","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2023-07-11eCollection Date: 2023-10-01DOI: 10.1093/jamiaopen/ooad050
Marika Dy, Kristan Olazo, Courtney R Lyles, Sarah Lisker, Jessica Weinberg, Christine Lee, Michelle E Tarver, Anindita Saha, Kimberly Kontson, Richardae Araojo, Ellenor Brown, Urmimala Sarkar
{"title":"Usability and acceptability of virtual reality for chronic pain management among diverse patients in a safety-net setting: a qualitative analysis.","authors":"Marika Dy, Kristan Olazo, Courtney R Lyles, Sarah Lisker, Jessica Weinberg, Christine Lee, Michelle E Tarver, Anindita Saha, Kimberly Kontson, Richardae Araojo, Ellenor Brown, Urmimala Sarkar","doi":"10.1093/jamiaopen/ooad050","DOIUrl":"10.1093/jamiaopen/ooad050","url":null,"abstract":"<p><strong>Objective: </strong>The aim of this study was to understand the usability and acceptability of virtual reality (VR) among a racially and ethnically diverse group of patients who experience chronic pain.</p><p><strong>Materials and methods: </strong>Using the Technology Acceptance Model theory, we conducted semistructured interviews and direct observation of VR use with English-speaking patients who experience chronic pain treated in a public healthcare system (<i>n</i> = 15), using a commercially available VR technology platform. Interviews included questions about current pain management strategies, technology use, experiences and opinions with VR, and motivators for future use.</p><p><strong>Results: </strong>Before the study, none of the 15 participants had heard about or used VR for pain management. Common motivators for VR use included a previous history of substance use and having exhausted many other options to manage their pain and curiosity. Most participants had a positive experience with VR and 47% found that the VR modules distracted them from their pain. When attempting the navigation-based usability tasks, most participants (73%-92%) were able to complete them independently.</p><p><strong>Discussion: </strong>VR is a usable tool for diverse patients with chronic pain. Our findings suggest that the usability of VR is not a barrier and perhaps a focus on improving the <i>accessibility</i> of VR in safety-net settings is needed to reduce disparities in health technology use.</p><p><strong>Conclusions: </strong>The usability and acceptability of VR are rarely studied in diverse patient populations. We found that participants had a positive experience using VR, showed interest in future use, and would recommend VR to family and friends.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 3","pages":"ooad050"},"PeriodicalIF":2.5,"publicationDate":"2023-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10336187/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9820491","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2023-07-08eCollection Date: 2023-10-01DOI: 10.1093/jamiaopen/ooad048
Meghan Reading Turchioe, Sarah Harkins, Pooja Desai, Shiveen Kumar, Jessica Kim, Alison Hermann, Rochelle Joly, Yiye Zhang, Jyotishman Pathak, Natalie C Benda
{"title":"Women's perspectives on the use of artificial intelligence (AI)-based technologies in mental healthcare.","authors":"Meghan Reading Turchioe, Sarah Harkins, Pooja Desai, Shiveen Kumar, Jessica Kim, Alison Hermann, Rochelle Joly, Yiye Zhang, Jyotishman Pathak, Natalie C Benda","doi":"10.1093/jamiaopen/ooad048","DOIUrl":"10.1093/jamiaopen/ooad048","url":null,"abstract":"<p><p>This study aimed to evaluate women's attitudes towards artificial intelligence (AI)-based technologies used in mental health care. We conducted a cross-sectional, online survey of U.S. adults reporting female sex at birth focused on bioethical considerations for AI-based technologies in mental healthcare, stratifying by previous pregnancy. Survey respondents (<i>n</i> = 258) were open to AI-based technologies in mental healthcare but concerned about medical harm and inappropriate data sharing. They held clinicians, developers, healthcare systems, and the government responsible for harm. Most reported it was \"very important\" for them to understand AI output. More previously pregnant respondents reported being told AI played a small role in mental healthcare was \"very important\" versus those not previously pregnant (<i>P</i> = .03). We conclude that protections against harm, transparency around data use, preservation of the patient-clinician relationship, and patient comprehension of AI predictions may facilitate trust in AI-based technologies for mental healthcare among women.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 3","pages":"ooad048"},"PeriodicalIF":2.5,"publicationDate":"2023-07-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10329494/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9865239","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2023-07-05eCollection Date: 2023-10-01DOI: 10.1093/jamiaopen/ooad047
Lijing Wang, Amy R Zipursky, Alon Geva, Andrew J McMurry, Kenneth D Mandl, Timothy A Miller
{"title":"A computable case definition for patients with SARS-CoV2 testing that occurred outside the hospital.","authors":"Lijing Wang, Amy R Zipursky, Alon Geva, Andrew J McMurry, Kenneth D Mandl, Timothy A Miller","doi":"10.1093/jamiaopen/ooad047","DOIUrl":"10.1093/jamiaopen/ooad047","url":null,"abstract":"<p><strong>Objective: </strong>To identify a cohort of COVID-19 cases, including when evidence of virus positivity was only mentioned in the clinical text, not in structured laboratory data in the electronic health record (EHR).</p><p><strong>Materials and methods: </strong>Statistical classifiers were trained on feature representations derived from unstructured text in patient EHRs. We used a proxy dataset of patients <i>with</i> COVID-19 polymerase chain reaction (PCR) tests for training. We selected a model based on performance on our proxy dataset and applied it to instances without COVID-19 PCR tests. A physician reviewed a sample of these instances to validate the classifier.</p><p><strong>Results: </strong>On the test split of the proxy dataset, our best classifier obtained 0.56 F1, 0.6 precision, and 0.52 recall scores for SARS-CoV2 positive cases. In an expert validation, the classifier correctly identified 97.6% (81/84) as COVID-19 positive and 97.8% (91/93) as not SARS-CoV2 positive. The classifier labeled an additional 960 cases as not having SARS-CoV2 lab tests in hospital, and only 177 of those cases had the ICD-10 code for COVID-19.</p><p><strong>Discussion: </strong>Proxy dataset performance may be worse because these instances sometimes include discussion of pending lab tests. The most predictive features are meaningful and interpretable. The type of external test that was performed is rarely mentioned.</p><p><strong>Conclusion: </strong>COVID-19 cases that had testing done outside of the hospital can be reliably detected from the text in EHRs. Training on a proxy dataset was a suitable method for developing a highly performant classifier without labor-intensive labeling efforts.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 3","pages":"ooad047"},"PeriodicalIF":2.1,"publicationDate":"2023-07-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10322650/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9832600","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
JAMIA OpenPub Date : 2023-07-01DOI: 10.1093/jamiaopen/ooad023
Jiancheng Ye, Jiarui Hai, Zidan Wang, Chumei Wei, Jiacheng Song
{"title":"Leveraging natural language processing and geospatial time series model to analyze COVID-19 vaccination sentiment dynamics on Tweets.","authors":"Jiancheng Ye, Jiarui Hai, Zidan Wang, Chumei Wei, Jiacheng Song","doi":"10.1093/jamiaopen/ooad023","DOIUrl":"https://doi.org/10.1093/jamiaopen/ooad023","url":null,"abstract":"<p><strong>Objective: </strong>To develop and apply a natural language processing (NLP)-based approach to analyze public sentiments on social media and their geographic pattern in the United States toward coronavirus disease 2019 (COVID-19) vaccination. We also aim to provide insights to facilitate the understanding of the public attitudes and concerns regarding COVID-19 vaccination.</p><p><strong>Methods: </strong>We collected Tweet posts by the residents in the United States after the dissemination of the COVID-19 vaccine. We performed sentiment analysis based on the Bidirectional Encoder Representations from Transformers (BERT) and qualitative content analysis. Time series models were leveraged to describe sentiment trends. Key topics were analyzed longitudinally and geospatially.</p><p><strong>Results: </strong>A total of 3 198 686 Tweets related to COVID-19 vaccination were extracted from January 2021 to February 2022. 2 358 783 Tweets were identified to contain clear opinions, among which 824 755 (35.0%) expressed negative opinions towards vaccination while 1 534 028 (65.0%) demonstrated positive opinions. The accuracy of the BERT model was 79.67%. The key hashtag-based topics include Pfizer, breaking, wearamask, and smartnews. The sentiment towards vaccination across the states showed manifest variability. Key barriers to vaccination include mistrust, hesitancy, safety concern, misinformation, and inequity.</p><p><strong>Conclusion: </strong>We found that opinions toward the COVID-19 vaccination varied across different places and over time. This study demonstrates the potential of an analytical pipeline, which integrates NLP-enabled modeling, time series, and geospatial analyses of social media data. Such analyses could enable real-time assessment, at scale, of public confidence and trust in COVID-19 vaccination, help address the concerns of vaccine skeptics, and provide support for developing tailored policies and communication strategies to maximize uptake.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 2","pages":"ooad023"},"PeriodicalIF":2.1,"publicationDate":"2023-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10097455/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9316856","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}