BMJ Health & Care Informatics最新文献

筛选
英文 中文
Evaluating the implementation of a digital coordination centre in an Australian hospital setting: a mixed method study protocol.
IF 4.1
BMJ Health & Care Informatics Pub Date : 2025-02-06 DOI: 10.1136/bmjhci-2024-101300
Sam Freeman, Colin Malone, Wynona Black, Daniel Capurro, Wendy W Chapman, Timothy N Fazio, Jana Gazarek, Meredith J Layton, Kayley Lyons, Laura Pumo, Samantha Plumb, Brad Astbury
{"title":"Evaluating the implementation of a digital coordination centre in an Australian hospital setting: a mixed method study protocol.","authors":"Sam Freeman, Colin Malone, Wynona Black, Daniel Capurro, Wendy W Chapman, Timothy N Fazio, Jana Gazarek, Meredith J Layton, Kayley Lyons, Laura Pumo, Samantha Plumb, Brad Astbury","doi":"10.1136/bmjhci-2024-101300","DOIUrl":"10.1136/bmjhci-2024-101300","url":null,"abstract":"<p><strong>Introduction: </strong>This protocol outlines a mixed methods study evaluating a new Digital Coordination Centre (DCC) at the Royal Melbourne Hospital (RMH), Melbourne, Australia. While coordination centres show potential for impact, evidence on effective implementation in the Australian context remains scarce. This study aims to address this gap.</p><p><strong>Methods and analysis: </strong>The evaluation involves a two-stage approach: a process evaluation to clarify DCC design and identify implementation factors, and an initial outcome evaluation to assess short and medium term outcomes. A developmental approach will support continuous improvement, and implementation science theories applied to unpack change processes. Data sources will include interviews, project documentation and observations, with qualitative and quantitative analyses targeting metrics like emergency department boarding and length of stay.</p><p><strong>Ethics and dissemination: </strong>This study has been approved by the RMH Human Research Ethics Committee (QA2023089). Findings will be shared through peer-reviewed publications and conference presentations.</p>","PeriodicalId":9050,"journal":{"name":"BMJ Health & Care Informatics","volume":"32 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-02-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11804194/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143363483","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Biodesign in the generative AI era: enhancing innovation and equity with NLP and LLM tools.
IF 4.1
BMJ Health & Care Informatics Pub Date : 2025-02-05 DOI: 10.1136/bmjhci-2024-101409
Jowy Tani
{"title":"Biodesign in the generative AI era: enhancing innovation and equity with NLP and LLM tools.","authors":"Jowy Tani","doi":"10.1136/bmjhci-2024-101409","DOIUrl":"10.1136/bmjhci-2024-101409","url":null,"abstract":"","PeriodicalId":9050,"journal":{"name":"BMJ Health & Care Informatics","volume":"32 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-02-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11800217/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143363480","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Evaluation of a pragmatic approach to predicting COVID-19-positive hospital bed occupancy.
IF 4.1
BMJ Health & Care Informatics Pub Date : 2025-02-05 DOI: 10.1136/bmjhci-2024-101055
Derryn Lovett, Thomas Woodcock, Jacques Naude, Julian Redhead, Azeem Majeed, Paul Aylin
{"title":"Evaluation of a pragmatic approach to predicting COVID-19-positive hospital bed occupancy.","authors":"Derryn Lovett, Thomas Woodcock, Jacques Naude, Julian Redhead, Azeem Majeed, Paul Aylin","doi":"10.1136/bmjhci-2024-101055","DOIUrl":"10.1136/bmjhci-2024-101055","url":null,"abstract":"<p><strong>Study objectives: </strong>This study evaluates the feasibility and accuracy of a pragmatic approach to predicting hospital bed occupancy for COVID-19-positive patients, using only simple methods accessible to typical health system teams.</p><p><strong>Methods: </strong>We used an observational forecasting design for the study period 1st June 2021 to -21st January 2022. Evaluation data covered individuals registered with a general practitioner in North West London, through the Whole Systems Integrated Care deidentified dataset. We extracted data on COVID-19-positive tests, vaccination records and admissions to hospitals with confirmed COVID-19 within the study period. We used linear regression models to predict bed occupancy, using lagged, smoothed numbers of COVID-19 cases among unvaccinated individuals in the community as the predictor. We used mean absolute percentage error (MAPE) to assess model accuracy.</p><p><strong>Results: </strong>Model accuracy varied throughout the study period, with a MAPE of 10.8% from 12 July 2021 to 18 October 2021, rising to 20.0% over the subsequent period to 15 December 2021. After that, model accuracy deteriorated considerably, with MAPE 110.4% from December 2021 to 21 January 2022. Model outputs were used by senior healthcare system leaders to aid the planning, organisation and provision of healthcare services to meet demand for hospital beds.</p><p><strong>Conclusions: </strong>The model produced useful predictions of COVID-19-positive bed occupancy prior to the emergence of the Omicron variant, but accuracy deteriorated after this. In practice, the model offers a pragmatic approach to predicting bed occupancy within a pandemic wave. However, this approach requires continual monitoring of errors to ensure that the periods of poor performance are identified quickly.</p>","PeriodicalId":9050,"journal":{"name":"BMJ Health & Care Informatics","volume":"32 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-02-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11800226/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143363487","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Engaging with patients with diabetes: the role of social media in low-income healthcare organisations. 与糖尿病患者互动:社交媒体在低收入医疗机构中的作用。
IF 4.1
BMJ Health & Care Informatics Pub Date : 2025-02-04 DOI: 10.1136/bmjhci-2024-101193
Andrea Cano, Mohy Uddin, Fernanda Caceres, José Rodriguez, Shabbir Syed-Abdul
{"title":"Engaging with patients with diabetes: the role of social media in low-income healthcare organisations.","authors":"Andrea Cano, Mohy Uddin, Fernanda Caceres, José Rodriguez, Shabbir Syed-Abdul","doi":"10.1136/bmjhci-2024-101193","DOIUrl":"10.1136/bmjhci-2024-101193","url":null,"abstract":"<p><strong>Background: </strong>Type 2 diabetes is the fastest-growing global health concern, and its global prevalence is projected to affect 643 million individuals by 2030. Social media platforms, like Facebook, have become crucial channels for healthcare organisations to engage with the public to promote prevention and disease management, especially in low-resource settings like Honduras. This study aims to perform a retrospective analysis of Honduran healthcare organisations' Facebook posts to understand how effectively they engage diabetes-related content with their followers.</p><p><strong>Methods: </strong>The top 10 followed healthcare organisations' Facebook pages were taken as a sample. Data were retrieved from October 2023 to March 2024. Diabetic-related posts were identified using keywords and categorised based on their contents and features.</p><p><strong>Results: </strong>Findings reveal significant disparities in the frequencies of posts and public engagement among different types of organisations. The majority of posts were classified under the miscellaneous category and text+image feature. Recipes and food-related posts were liked and shared the most among the followers.</p><p><strong>Conclusion: </strong>The results of the study found that patients' engagement with diabetes-related content was low in social media. The gap between patients' participation and engagement highlights the need for reassessment and refinement of social media communication strategies for healthcare organisations to empower patients with diabetes through social media and increase public engagement.</p>","PeriodicalId":9050,"journal":{"name":"BMJ Health & Care Informatics","volume":"32 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-02-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11795404/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143188268","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
ConciliaMed: an interactive mobile and web tool to reconcile chronic medications of patients undergoing elective surgery.
IF 4.1
BMJ Health & Care Informatics Pub Date : 2025-01-30 DOI: 10.1136/bmjhci-2024-101256
Pablo Ciudad-Gutiérrez, Paloma Suárez-Casillas, Eva Rocío Alfaro-Lara, Maria Dolores Santos-Rubio, Bernardo Santos-Ramos, Ana Belén Guisado-Gil
{"title":"ConciliaMed: an interactive mobile and web tool to reconcile chronic medications of patients undergoing elective surgery.","authors":"Pablo Ciudad-Gutiérrez, Paloma Suárez-Casillas, Eva Rocío Alfaro-Lara, Maria Dolores Santos-Rubio, Bernardo Santos-Ramos, Ana Belén Guisado-Gil","doi":"10.1136/bmjhci-2024-101256","DOIUrl":"10.1136/bmjhci-2024-101256","url":null,"abstract":"<p><strong>Objective: </strong>The last decade has seen exponential growth in electronic health tools. However, only a limited number of electronic medication reconciliation tools have been developed and implemented in healthcare settings. Here, we present ConciliaMed, a mobile and web-based tool for healthcare professionals to reconcile the chronic medications of patients undergoing elective surgery.</p><p><strong>Methods: </strong>A research team of pharmacists and internists worked together with a technology company to design and develop ConciliaMed. Evidence-based guidelines were collected for inclusion in the tool. A group of experts conducted a simulation with a preliminary version of ConciliaMed to identify bugs and technical improvements and to assess their satisfaction with the application. The final prototype of the tool was disseminated through clinical meetings and the Google Store.</p><p><strong>Results: </strong>Four easy-to-use and interactive modules can be used to reconcile chronic medications through the app, while the web platform is designed for consultation and learning. A higher level of satisfaction with the tool was achieved by the test participants (4.67±0.58). The triggering of dose and duplication alerts for users or the integration of ConciliaMed with electronic prescription systems were some of the more requested adaptations by the test participants.</p><p><strong>Discussion: </strong>The ability to generate an editable reconciliation report or transfer information between users are some of the features of ConciliaMed that encourage its use. The integration of ConciliaMed into the healthcare workflow is expected.</p><p><strong>Conclusion: </strong>The web platform is freely available online (https://conciliamed.chronic-pharma.com), as is the mobile application through the Google Store, making it easily accessible to healthcare professionals.</p>","PeriodicalId":9050,"journal":{"name":"BMJ Health & Care Informatics","volume":"32 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-01-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11784385/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143063661","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Diagnostic prediction model for screening of elevated low-density and non-high-density lipoproteins in young Thai adults between 20 and 40 years of age.
IF 4.1
BMJ Health & Care Informatics Pub Date : 2025-01-30 DOI: 10.1136/bmjhci-2024-101180
Wuttipat Kiratipaisarl, Vithawat Surawattanasakul, Wachiranun Sirikul, Phichayut Phinyo
{"title":"Diagnostic prediction model for screening of elevated low-density and non-high-density lipoproteins in young Thai adults between 20 and 40 years of age.","authors":"Wuttipat Kiratipaisarl, Vithawat Surawattanasakul, Wachiranun Sirikul, Phichayut Phinyo","doi":"10.1136/bmjhci-2024-101180","DOIUrl":"10.1136/bmjhci-2024-101180","url":null,"abstract":"<p><strong>Background: </strong>Low-density lipoprotein cholesterol (LDL-C) and non-high-density lipoprotein cholesterol (non-HDL-C) levels are paramount in atherosclerotic cardiovascular disease risk management. However, 94.4% of Thai young adult are unaware of their condition. A diagnostic prediction model may assist in screening and alleviating underdiagnosis.</p><p><strong>Objectives: </strong>Development and internal validation of diagnostic prediction models on elevated LDL-C (≥160 mg/dL) and non-HDL-C (≥160 mg/dL).</p><p><strong>Methods: </strong>Retrospective, single-centre, tertiary-care hospital annual health examination data from 29 March 2018 to 30 August 2023 was analysed. Two models with 11 predictors from anthropometry and bioimpedance are fitted with multivariable binary logistic regression predicting elevated LDL-C and non-HDL-C. Predictor selection used the backward stepwise elimination. Four performance metrics were quantified: discrimination using area under the receiver-operating characteristic curve (AuROC); calibration by calibration plot; utility by decision curve analysis and instability by performance instability plots. Internal validation was carried out using 500 repetitions of bootstrap-resampling.</p><p><strong>Results: </strong>Dataset included 2222 LDL-C and 5149 non-HDL-C investigations, 303 were classed as elevated LDL-C (13.6%) and 1013 as elevated non-HDL-C cases (19.7%). Two predictors, gender and metabolic age, were identified in the LDL-C model with AuROC 0.639 (95% CI 0.617 to 0.661), poor calibration, and utility in the 7%-25% probability range. Three predictors-gender, diastolic blood pressure and metabolic age-were identified in the non-HDL-C model with AuROC 0.722 (95% CI 0.705 to 0.738), good calibration and utility in 9%-55% probability range.</p><p><strong>Discussion and conclusion: </strong>Overall results demonstrated acceptable discrimination for non-HDL-C model but inadequate performance of LDL-C model for clinical practice. An external validation study should be planned for non-HDL-C model.</p>","PeriodicalId":9050,"journal":{"name":"BMJ Health & Care Informatics","volume":"32 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-01-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11784327/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143063664","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Barcode medication administration system use and safety implications: a data-driven longitudinal study supported by clinical observation. 条形码给药系统的使用和安全性影响:一项由临床观察支持的数据驱动的纵向研究。
IF 4.1
BMJ Health & Care Informatics Pub Date : 2025-01-19 DOI: 10.1136/bmjhci-2024-101214
Rachel Williams, Kumud Kantilal, Kenneth K C Man, Ann Blandford, Yogini Jani
{"title":"Barcode medication administration system use and safety implications: a data-driven longitudinal study supported by clinical observation.","authors":"Rachel Williams, Kumud Kantilal, Kenneth K C Man, Ann Blandford, Yogini Jani","doi":"10.1136/bmjhci-2024-101214","DOIUrl":"10.1136/bmjhci-2024-101214","url":null,"abstract":"<p><strong>Objectives: </strong>Barcode medication administration (BCMA) systems may improve patient safety with successful integration and use. This study aimed to explore the barriers and enablers for the successful use of a BCMA system by examining the patterns of medication and patient scanning over time and potential safety implications.</p><p><strong>Methods: </strong>Retrospective longitudinal study informed by prospective clinical observations using data extracted from five hospital wards over the first 16 months after implementation to determine trends in medication and patient scanning rates, reasons for non-compliance and scanning mismatch alerts. Regression models were applied to explore factors influencing medication scanning rates across wards of different specialties.</p><p><strong>Results: </strong>Electronic data on 613 868 medication administrations showed overall medication scanning rates per ward ranged from 5.6% to 67% and patient scanning rates from 4.6% to 89%. Reported reasons for not scanning medications were 'barcode not readable' and 'unavailability of scanners'. Scanning rates declined over time and the pattern of reason codes for not scanning also changed. Factors associated with higher scanning rates included a locally led quality improvement (QI) initiative, the medication administration time and the medication formulation, for example, tablets and liquids. Overall, 37% of scanning alerts resulted in a change in user action. Staff tried to comply with the BCMA system workflow, but workarounds were observed.</p><p><strong>Discussion: </strong>Compliance with BCMA systems varied across wards and changed over time. QI initiatives hold promise to ensure sustained use of BCMA systems.</p><p><strong>Conclusions: </strong>BCMA systems may help to improve medication safety, but further research is needed to confirm sustained safety benefits.</p>","PeriodicalId":9050,"journal":{"name":"BMJ Health & Care Informatics","volume":"32 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-01-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11784319/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142999863","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Analysing expression of loneliness and insomnia through social intelligence analysis. 通过社会智力分析分析孤独和失眠的表现。
IF 4.1
BMJ Health & Care Informatics Pub Date : 2025-01-19 DOI: 10.1136/bmjhci-2024-101116
Hurmat Ali Shah, Mowafa Househ
{"title":"Analysing expression of loneliness and insomnia through social intelligence analysis.","authors":"Hurmat Ali Shah, Mowafa Househ","doi":"10.1136/bmjhci-2024-101116","DOIUrl":"10.1136/bmjhci-2024-101116","url":null,"abstract":"<p><strong>Background: </strong>Loneliness and insomnia are mutually occurring conditions. This paper investigates whether keywords depicting loneliness and insomnia are expressed together on social media. Understanding loneliness through data fills the gaps or validates the literature on loneliness from sociological and psychological perspectives. Loneliness is associated with various physical and mental health conditions but there are opportunities to understand it from the perspectives and lens of health informatics through social media data. Because loneliness is a subjective phenomenon, therefore, the self-reporting nature of social media data can provide an intimate glimpse into the feelings associated with loneliness.</p><p><strong>Methods: </strong>This study uses sentiment analysis of collected tweets on loneliness and insomnia to filter out the tweets that have negative connotations. Those tweets are then further analysed to find out categories and themes associated with loneliness and insomnia.</p><p><strong>Results: </strong>Through the frequency of word occurrence analysis, it was seen that the association between loneliness and insomnia can be found. The association, in the tweets analysed, is mediated by words denoting depressive symptoms. Moreover, the themes and categories which are associated with the expression of both loneliness and insomnia are those of personal feelings and time.</p>","PeriodicalId":9050,"journal":{"name":"BMJ Health & Care Informatics","volume":"32 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-01-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11751960/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142999862","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
The feasibility and effectiveness of telecare consultations in a nurse-led post-acute stroke clinic. 护士主导的急性脑卒中后门诊远程会诊的可行性和有效性。
IF 4.1
BMJ Health & Care Informatics Pub Date : 2025-01-19 DOI: 10.1136/bmjhci-2024-101082
Arkers Kwan Ching Wong, Robbie Mian Wang, Frances Kam Yuet Wong, Bernard Man Kam Yuen, Ching Sing Fong, Shun Tim Chan, Vivian Wai Yan Kwok
{"title":"The feasibility and effectiveness of telecare consultations in a nurse-led post-acute stroke clinic.","authors":"Arkers Kwan Ching Wong, Robbie Mian Wang, Frances Kam Yuet Wong, Bernard Man Kam Yuen, Ching Sing Fong, Shun Tim Chan, Vivian Wai Yan Kwok","doi":"10.1136/bmjhci-2024-101082","DOIUrl":"10.1136/bmjhci-2024-101082","url":null,"abstract":"<p><strong>Background: </strong>Telecare may provide an alternative to maintaining post-acute stroke care services in making benefit to both the providers and the stroke survivors, although study is needed to investigate its feasibility and effectiveness in integrating this innovative delivery mode into a routine.</p><p><strong>Objectives: </strong>The objectives of this study are to assess the feasibility and effectiveness of telecare consultations in a nurse-led post-acute stroke clinic.</p><p><strong>Methods: </strong>A pre- and post-test one group quasi-experimental design was adopted. Subjects were recruited in the clinic and received three secondary stroke care consultations in 3 months via telecare from stroke nurses. Data were collected at pre- and post-intervention. A Wilcoxon signed-rank test was used to compare the two time-points for differences in effectiveness.</p><p><strong>Results: </strong>Ninety-two stroke survivors participated. The drop-out rate was 27%. The majority perceived the programme as time-friendly and cost-saving and as alleviating their health-related worries. At the 3-month follow-up, notable improvements were observed in the activities of daily living and the strength domain of stroke-specific quality of life.</p><p><strong>Conclusions: </strong>Integrating telecare consultations within nurse-led stroke clinics is a feasible and acceptable strategy for monitoring the health and fostering the self-care abilities of individuals following their discharge from hospital after an acute stroke episode.</p>","PeriodicalId":9050,"journal":{"name":"BMJ Health & Care Informatics","volume":"32 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-01-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11751813/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142999867","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Large language models for data extraction from unstructured and semi-structured electronic health records: a multiple model performance evaluation. 用于从非结构化和半结构化电子健康记录中提取数据的大型语言模型:多模型性能评估。
IF 4.1
BMJ Health & Care Informatics Pub Date : 2025-01-19 DOI: 10.1136/bmjhci-2024-101139
Vasileios Ntinopoulos, Hector Rodriguez Cetina Biefer, Igor Tudorache, Nestoras Papadopoulos, Dragan Odavic, Petar Risteski, Achim Haeussler, Omer Dzemali
{"title":"Large language models for data extraction from unstructured and semi-structured electronic health records: a multiple model performance evaluation.","authors":"Vasileios Ntinopoulos, Hector Rodriguez Cetina Biefer, Igor Tudorache, Nestoras Papadopoulos, Dragan Odavic, Petar Risteski, Achim Haeussler, Omer Dzemali","doi":"10.1136/bmjhci-2024-101139","DOIUrl":"10.1136/bmjhci-2024-101139","url":null,"abstract":"<p><strong>Objectives: </strong>We aimed to evaluate the performance of multiple large language models (LLMs) in data extraction from unstructured and semi-structured electronic health records.</p><p><strong>Methods: </strong>50 synthetic medical notes in English, containing a structured and an unstructured part, were drafted and evaluated by domain experts, and subsequently used for LLM-prompting. 18 LLMs were evaluated against a baseline transformer-based model. Performance assessment comprised four entity extraction and five binary classification tasks with a total of 450 predictions for each LLM. LLM-response consistency assessment was performed over three same-prompt iterations.</p><p><strong>Results: </strong>Claude 3.0 Opus, Claude 3.0 Sonnet, Claude 2.0, GPT 4, Claude 2.1, Gemini Advanced, PaLM 2 chat-bison and Llama 3-70b exhibited an excellent overall accuracy >0.98 (0.995, 0.988, 0.988, 0.988, 0.986, 0.982, 0.982, and 0.982, respectively), significantly higher than the baseline RoBERTa model (0.742). Claude 2.0, Claude 2.1, Claude 3.0 Opus, PaLM 2 chat-bison, GPT 4, Claude 3.0 Sonnet and Llama 3-70b showed a marginally higher and Gemini Advanced a marginally lower multiple-run consistency than the baseline model RoBERTa (Krippendorff's alpha value 1, 0.998, 0.996, 0.996, 0.992, 0.991, 0.989, 0.988, and 0.985, respectively).</p><p><strong>Discussion: </strong>Claude 3.0 Opus, Claude 3.0 Sonnet, Claude 2.0, GPT 4, Claude 2.1, Gemini Advanced, PaLM 2 chat bison and Llama 3-70b performed the best, exhibiting outstanding performance in both entity extraction and binary classification, with highly consistent responses over multiple same-prompt iterations. Their use could leverage data for research and unburden healthcare professionals. Real-data analyses are warranted to confirm their performance in a real-world setting.</p><p><strong>Conclusion: </strong>Claude 3.0 Opus, Claude 3.0 Sonnet, Claude 2.0, GPT 4, Claude 2.1, Gemini Advanced, PaLM 2 chat-bison and Llama 3-70b seem to be able to reliably extract data from unstructured and semi-structured electronic health records. Further analyses using real data are warranted to confirm their performance in a real-world setting.</p>","PeriodicalId":9050,"journal":{"name":"BMJ Health & Care Informatics","volume":"32 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-01-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11751965/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142999865","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信