PLOS digital health最新文献

{"title":"Use of a continuous single lead electrocardiogram analytic to predict patient deterioration requiring rapid response team activation.","authors":"Sooin Lee, Bryce Benson, Ashwin Belle, Richard P Medlin, David Jerkins, Foster Goss, Ashish K Khanna, Michael A DeVita, Kevin R Ward","doi":"10.1371/journal.pdig.0000465","DOIUrl":"https://doi.org/10.1371/journal.pdig.0000465","url":null,"abstract":"<p><p>Identifying the onset of patient deterioration is challenging despite the potential to respond to patients earlier with better vital sign monitoring and rapid response team (RRT) activation. In this study an ECG based software as a medical device, the Analytic for Hemodynamic Instability Predictive Index (AHI-PI), was compared to the vital signs of heart rate, blood pressure, and respiratory rate, evaluating how early it indicated risk before an RRT activation. A higher proportion of the events had risk indication by AHI-PI (92.71%) than by vital signs (41.67%). AHI-PI indicated risk early, with an average of over a day before RRT events. In events whose risks were indicated by both AHI-PI and vital signs, AHI-PI demonstrated earlier recognition of deterioration compared to vital signs. A case-control study showed that situations requiring RRTs were more likely to have AHI-PI risk indication than those that did not. The study derived several insights in support of AHI-PI's efficacy as a clinical decision support system. The findings demonstrated AHI-PI's potential to serve as a reliable predictor of future RRT events. It could potentially help clinicians recognize early clinical deterioration and respond to those unnoticed by vital signs, thereby helping clinicians improve clinical outcomes.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"3 10","pages":"e0000465"},"PeriodicalIF":0.0,"publicationDate":"2024-10-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11500862/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142514299","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Conceptualizing bias in EHR data: A case study in performance disparities by demographic subgroups for a pediatric obesity incidence classifier.","authors":"Elizabeth A Campbell, Saurav Bose, Aaron J Masino","doi":"10.1371/journal.pdig.0000642","DOIUrl":"https://doi.org/10.1371/journal.pdig.0000642","url":null,"abstract":"<p><p>Electronic Health Records (EHRs) are increasingly used to develop machine learning models in predictive medicine. There has been limited research on utilizing machine learning methods to predict childhood obesity and related disparities in classifier performance among vulnerable patient subpopulations. In this work, classification models are developed to recognize pediatric obesity using temporal condition patterns obtained from patient EHR data in a U.S. study population. We trained four machine learning algorithms (Logistic Regression, Random Forest, Gradient Boosted Trees, and Neural Networks) to classify cases and controls as obesity positive or negative, and optimized hyperparameter settings through a bootstrapping methodology. To assess the classifiers for bias, we studied model performance by population subgroups then used permutation analysis to identify the most predictive features for each model and the demographic characteristics of patients with these features. Mean AUC-ROC values were consistent across classifiers, ranging from 0.72-0.80. Some evidence of bias was identified, although this was through the models performing better for minority subgroups (African Americans and patients enrolled in Medicaid). Permutation analysis revealed that patients from vulnerable population subgroups were over-represented among patients with the most predictive diagnostic patterns. We hypothesize that our models performed better on under-represented groups because the features more strongly associated with obesity were more commonly observed among minority patients. These findings highlight the complex ways that bias may arise in machine learning models and can be incorporated into future research to develop a thorough analytical approach to identify and mitigate bias that may arise from features and within EHR datasets when developing more equitable models.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"3 10","pages":"e0000642"},"PeriodicalIF":0.0,"publicationDate":"2024-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11498669/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142514295","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Impact of observability period on the classification of COPD diagnosis timing among Medicare beneficiaries with lung cancer.","authors":"Eman Metwally, Sarah E Soppe, Jennifer L Lund, Sharon Peacock Hinton, Caroline A Thompson","doi":"10.1371/journal.pdig.0000633","DOIUrl":"https://doi.org/10.1371/journal.pdig.0000633","url":null,"abstract":"<p><strong>Background: </strong>Investigators often use claims data to estimate the diagnosis timing of chronic conditions. However, misclassification of chronic conditions is common due to variability in healthcare utilization and in claims history across patients.</p><p><strong>Objective: </strong>We aimed to quantify the effect of various Medicare fee-for-service continuous enrollment period and lookback period (LBP) on misclassification of COPD and sample size.</p><p><strong>Methods: </strong>A stepwise tutorial to classify COPD, based on its diagnosis timing relative to lung cancer diagnosis using the Surveillance Epidemiology and End Results cancer registry linked to Medicare insurance claims. We used 3 approaches varying the LBP and required continuous enrollment (i.e., observability) period between 1 to 5 years. Patients with lung cancer were classified based on their COPD related healthcare utilization into 3 groups: pre-existing COPD (diagnosis at least 3 months before lung cancer diagnosis), concurrent COPD (diagnosis during the -/+ 3months of lung cancer diagnosis), and non-COPD. Among those with 5 years of continuous enrollment, we estimated the sensitivity of the LBP to ascertain COPD diagnosis as the number of patients with pre-existing COPD using a shorter LBP divided by the number of patients with pre-existing COPD using a longer LBP.</p><p><strong>Results: </strong>Extending the LBP from 1 to 5 years increased prevalence of pre-existing COPD from ~ 36% to 51%, decreased both concurrent COPD from ~ 34% to 23% and non-COPD from ~ 29% to 25%. There was minimal effect of extending the required continuous enrollment period beyond one year across various LBPs. In those with 5 years of continuous enrollment, sensitivity of COPD classification (95% CI) increased with longer LBP from 70.1% (69.7% to 70.4%) for one-year LBP to 100% for 5-years LBP.</p><p><strong>Conclusion: </strong>The length of optimum LBP and continuous enrollment period depends on the context of the research question and the data generating mechanisms. Among Medicare beneficiaries, the best approach to identify diagnosis timing of COPD relative to lung cancer diagnosis is to use all available LBP with at least one year of required continuous enrollment.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"3 10","pages":"e0000633"},"PeriodicalIF":0.0,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11495636/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142514297","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Learning and diSentangling patient static information from time-series Electronic hEalth Records (STEER).","authors":"Wei Liao, Joel Voldman","doi":"10.1371/journal.pdig.0000640","DOIUrl":"10.1371/journal.pdig.0000640","url":null,"abstract":"<p><p>Recent work in machine learning for healthcare has raised concerns about patient privacy and algorithmic fairness. Previous work has shown that self-reported race can be predicted from medical data that does not explicitly contain racial information. However, the extent of data identification is unknown, and we lack ways to develop models whose outcomes are minimally affected by such information. Here we systematically investigated the ability of time-series electronic health record data to predict patient static information. We found that not only the raw time-series data, but also learned representations from machine learning models, can be trained to predict a variety of static information with area under the receiver operating characteristic curve as high as 0.851 for biological sex, 0.869 for binarized age and 0.810 for self-reported race. Such high predictive performance can be extended to various comorbidity factors and exists even when the model was trained for different tasks, using different cohorts, using different model architectures and databases. Given the privacy and fairness concerns these findings pose, we develop a variational autoencoder-based approach that learns a structured latent space to disentangle patient-sensitive attributes from time-series data. Our work thoroughly investigates the ability of machine learning models to encode patient static information from time-series electronic health records and introduces a general approach to protect patient-sensitive information for downstream tasks.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"3 10","pages":"e0000640"},"PeriodicalIF":0.0,"publicationDate":"2024-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11493250/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142482584","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Derivation and validation of an algorithm to predict transitions from community to residential long-term care among persons with dementia-A retrospective cohort study.","authors":"Wenshan Li, Luke Turcotte, Amy T Hsu, Robert Talarico, Danial Qureshi, Colleen Webber, Steven Hawken, Peter Tanuseputro, Douglas G Manuel, Greg Huyer","doi":"10.1371/journal.pdig.0000441","DOIUrl":"https://doi.org/10.1371/journal.pdig.0000441","url":null,"abstract":"<p><strong>Objectives: </strong>To develop and validate a model to predict time-to-LTC admissions among individuals with dementia.</p><p><strong>Design: </strong>Population-based retrospective cohort study using health administrative data.</p><p><strong>Setting and participants: </strong>Community-dwelling older adults (65+) in Ontario living with dementia and assessed with the Resident Assessment Instrument for Home Care (RAI-HC) between April 1, 2010 and March 31, 2017.</p><p><strong>Methods: </strong>Individuals in the derivation cohort (n = 95,813; assessed before March 31, 2015) were followed for up to 360 days after the index RAI-HC assessment for admission into LTC. We used a multivariable Fine Gray sub-distribution hazard model to predict the cumulative incidence of LTC entry while accounting for all-cause mortality as a competing risk. The model was validated in 34,038 older adults with dementia with an index RAI-HC assessment between April 1, 2015 and March 31, 2017.</p><p><strong>Results: </strong>Within one year of a RAI-HC assessment, 35,513 (37.1%) individuals in the derivation cohort and 10,735 (31.5%) in the validation cohort entered LTC. Our algorithm was well-calibrated (Emax = 0.119, ICIavg = 0.057) and achieved a c-statistic of 0.707 (95% confidence interval: 0.703-0.712) in the validation cohort.</p><p><strong>Conclusions and implications: </strong>We developed an algorithm to predict time to LTC entry among individuals living with dementia. This tool can inform care planning for individuals with dementia and their family caregivers.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"3 10","pages":"e0000441"},"PeriodicalIF":0.0,"publicationDate":"2024-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11488705/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142482582","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Using interpretable machine learning to predict bloodstream infection and antimicrobial resistance in patients admitted to ICU: Early alert predictors based on EHR data to guide antimicrobial stewardship.","authors":"Davide Ferrari, Pietro Arina, Jonathan Edgeworth, Vasa Curcin, Veronica Guidetti, Federica Mandreoli, Yanzhong Wang","doi":"10.1371/journal.pdig.0000641","DOIUrl":"https://doi.org/10.1371/journal.pdig.0000641","url":null,"abstract":"<p><p>Nosocomial infections and Antimicrobial Resistance (AMR) stand as formidable healthcare challenges on a global scale. To address these issues, various infection control protocols and personalized treatment strategies, guided by laboratory tests, aim to detect bloodstream infections (BSI) and assess the potential for AMR. In this study, we introduce a machine learning (ML) approach based on Multi-Objective Symbolic Regression (MOSR), an evolutionary approach to create ML models in the form of readable mathematical equations in a multi-objective way to overcome the limitation of standard single-objective approaches. This method leverages readily available clinical data collected upon admission to intensive care units, with the goal of predicting the presence of BSI and AMR. We further assess its performance by comparing it to established ML algorithms using both naturally imbalanced real-world data and data that has been balanced through oversampling techniques. Our findings reveal that traditional ML models exhibit subpar performance across all training scenarios. In contrast, MOSR, specifically configured to minimize false negatives by optimizing also for the F1-Score, outperforms other ML algorithms and consistently delivers reliable results, irrespective of the training set balance with F1-Score.22 and.28 higher than any other alternative. This research signifies a promising path forward in enhancing Antimicrobial Stewardship (AMS) strategies. Notably, the MOSR approach can be readily implemented on a large scale, offering a new ML tool to find solutions to these critical healthcare issues affected by limited data availability.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"3 10","pages":"e0000641"},"PeriodicalIF":0.0,"publicationDate":"2024-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11482717/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142482599","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Exploring the relationship between telehealth utilization and treatment burden among patients with chronic conditions: A cross-sectional study in Ontario, Canada.","authors":"Farah Tahsin, Carolyn Steele Gray, Jay Shaw, Aviv Shachak","doi":"10.1371/journal.pdig.0000610","DOIUrl":"https://doi.org/10.1371/journal.pdig.0000610","url":null,"abstract":"<p><p>One in five Canadians lives with one or more chronic conditions. Patients with chronic conditions often experience a high treatment burden because of the work associated with managing care. Telehealth is considered a useful solution to reduce the treatment burden among patients with chronic conditions. However, telehealth can also increase the treatment burden by offloading responsibilities on patients. This cross-sectional study conducted in Ontario, Canada examines the association between telehealth utilization and treatment burden among patients with chronic conditions. This study aimed to explore whether and to what extent, telehealth use is associated with treatment burden among patients with chronic conditions. The secondary objective was to explore which sociodemographic variables are associated with patients' treatment burden. An online survey was administered to community-dwelling patients with one or more chronic conditions. The Treatment Burden Questionnaire (TBQ-15) was used to measure the patient's level of treatment burden, and a modified telehealth usage scale was developed and used to measure the frequency of telehealth use. Data was analyzed using descriptive statistics, correlations, analyses of variance, and hierarchical linear regression analysis. A total of 75 patients completed the survey. The participants' mean age was 64 (SD = 18.93) and 79% were female. The average reported treatment burden was 72.15 out of 150 (a higher score indicating a higher level of burden). When adjusted for demographic variables, a higher frequency of telehealth use was associated with experiencing a higher treatment burden, but the association was not statistically significant. Additionally, when adjusted for demographic variables, younger age, and the presence of an unpaid caregiver were positively related to a high treatment burden score. This finding demonstrates that some patient populations are more at risk of experiencing high treatment burden in the context of telehealth use; and hence, may require extra support to utilize telehealth technologies. The study highlights the need for further research to explore how to minimize the treatment burden among individuals with higher healthcare needs.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"3 10","pages":"e0000610"},"PeriodicalIF":0.0,"publicationDate":"2024-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11478863/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142482583","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Towards women's digital health equity: A qualitative inquiry into attitude and adoption of reproductive mHealth services in Bangladesh.","authors":"M Jonayed, Maruf Hasan Rumi","doi":"10.1371/journal.pdig.0000637","DOIUrl":"https://doi.org/10.1371/journal.pdig.0000637","url":null,"abstract":"&lt;p&gt;&lt;p&gt;Health equity in Bangladesh faces a large chasm over the economic conditions, socio-cultural factors and geographic location despite the push for digitalization of the health sector. While some research has been conducted assessing the viability of digital health solutions in Bangladesh, gender dynamics of digital healthcare have been absent. This study dived into healthcare equity for women with a focus on reproductive health services delivered through mobile devices. This paper reported the findings of a qualitative study employing in-depth interviews conducted among 26 women about their behavioral intention to use mHealth services for reproductive health and the underlying factors influencing this intention with the help of the Integrative Model of Planned Behavior (IMPB). A snowball sampling technique were used to interview those university educated women, aged 21-31, based on their familiarity and exposure of mHealth services from seven universities in Bangladesh. The findings suggested that users of mHealth services find it more convenient and secure compared to visiting healthcare facilities, especially for trivial issues and inquiries regarding their reproductive health. Although promoting such services is lagging behind traditional healthcare, the attitude toward reproductive health services in Bangladesh is generally favorable resulting increasing adoption and use. Because such information-related mobile services (apps, websites, and social media) served as a first base of knowledge on reproductive health among many young girls and women in Bangladesh, who are generally shy to share or talk about their menstruation or personal health problems with family members, peers, or even health professionals due to socio-cultural factors and stigmatization. Conversely, urban centric services, availability of experts, quality management, security of privacy, authenticity of the information, digital divide, lack of campaign initiatives, lack of equipment and technology, lack of sex education, and outdated apps and websites were identified as obstacles that constrain the widespread use of reproductive mHealth services in Bangladesh. This study also concluded that promotion will be crucial in reforming conservative norms, taboos, and misconceptions about women's health and recommended such endeavors to be initiated by the policy makers as there is a substantive need for a specific policy regulating emerging digital health market in Bangladesh. Notwithstanding, women-only sample, low sample size, narrow focus on mHealth users and absence of perspectives from healthcare providers were among shortcomings of this study which could be addressed in future research. Further quantitative explorations are must to determine the usage patterns of reproductive mHealth services and their effectiveness that would identify implementation challenges in terms of customization and personalization in reproductive healthcare in a developing country like Bangladesh","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"3 10","pages":"e0000637"},"PeriodicalIF":0.0,"publicationDate":"2024-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11478865/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142482585","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Unsupervised clustering of longitudinal clinical measurements in electronic health records.","authors":"Arshiya Mariam, Hamed Javidi, Emily C Zabor, Ran Zhao, Tomas Radivoyevitch, Daniel M Rotroff","doi":"10.1371/journal.pdig.0000628","DOIUrl":"https://doi.org/10.1371/journal.pdig.0000628","url":null,"abstract":"<p><p>Longitudinal electronic health records (EHR) can be utilized to identify patterns of disease development and progression in real-world settings. Unsupervised temporal matching algorithms are being repurposed to EHR from signal processing- and protein-sequence alignment tasks where they have shown immense promise for gaining insight into disease. The robustness of these algorithms for classifying EHR clinical data remains to be determined. Timeseries compiled from clinical measurements, such as blood pressure, have far more irregularity in sampling and missingness than the data for which these algorithms were developed, necessitating a systematic evaluation of these methods. We applied 30 state-of-the-art unsupervised machine learning algorithms to 6,912 systematically generated simulated clinical datasets across five parameters. These algorithms included eight temporal matching algorithms with fourteen partitional and eight fuzzy clustering methods. Nemenyi tests were used to determine differences in accuracy using the Adjusted Rand Index (ARI). Dynamic time warping and its lower-bound variants had the highest accuracies across all cohorts (median ARI>0.70). All 30 methods were better at discriminating classes with differences in magnitude compared to differences in trajectory shapes. Missingness impacted accuracies only when classes were different by trajectory shape. The method with the highest ARI was then used to cluster a large pediatric metabolic syndrome (MetS) cohort (N = 43,426). We identified three unique childhood BMI patterns with high average cluster consensus (>70%). The algorithm identified a cluster with consistently high BMI which had the greatest risk of MetS, consistent with prior literature (OR = 4.87, 95% CI: 3.93-6.12). While these algorithms have been shown to have similar accuracies for regular timeseries, their accuracies in clinical applications vary substantially in discriminating differences in shape and especially with moderate to high missingness (>10%). This systematic assessment also shows that the most robust algorithms tested here can derive meaningful insights from longitudinal clinical data.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"3 10","pages":"e0000628"},"PeriodicalIF":0.0,"publicationDate":"2024-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11478862/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142482586","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Use of large language models as a scalable approach to understanding public health discourse.","authors":"Laura Espinosa, Marcel Salathé","doi":"10.1371/journal.pdig.0000631","DOIUrl":"https://doi.org/10.1371/journal.pdig.0000631","url":null,"abstract":"<p><p>Online public health discourse is becoming more and more important in shaping public health dynamics. Large Language Models (LLMs) offer a scalable solution for analysing the vast amounts of unstructured text found on online platforms. Here, we explore the effectiveness of Large Language Models (LLMs), including GPT models and open-source alternatives, for extracting public stances towards vaccination from social media posts. Using an expert-annotated dataset of social media posts related to vaccination, we applied various LLMs and a rule-based sentiment analysis tool to classify the stance towards vaccination. We assessed the accuracy of these methods through comparisons with expert annotations and annotations obtained through crowdsourcing. Our results demonstrate that few-shot prompting of best-in-class LLMs are the best performing methods, and that all alternatives have significant risks of substantial misclassification. The study highlights the potential of LLMs as a scalable tool for public health professionals to quickly gauge public opinion on health policies and interventions, offering an efficient alternative to traditional data analysis methods. With the continuous advancement in LLM development, the integration of these models into public health surveillance systems could substantially improve our ability to monitor and respond to changing public health attitudes.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"3 10","pages":"e0000631"},"PeriodicalIF":0.0,"publicationDate":"2024-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11472907/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142482587","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}