Exploring Korean Medicine professions' perspectives on the applicability of ChatGPT in facial palsy practice: A web-based survey

IF 1.9 4区医学 Q3 INTEGRATIVE & COMPLEMENTARY MEDICINE

European Journal of Integrative Medicine Pub Date : 2024-12-01 DOI:10.1016/j.eujim.2024.102422

Ji-Su Lee , Sung-A Kim , Taegon Kim , Seunghoon Lee , Tae-Hun Kim , Jung Won Kang

{"title":"Exploring Korean Medicine professions' perspectives on the applicability of ChatGPT in facial palsy practice: A web-based survey","authors":"Ji-Su Lee , Sung-A Kim , Taegon Kim , Seunghoon Lee , Tae-Hun Kim , Jung Won Kang","doi":"10.1016/j.eujim.2024.102422","DOIUrl":null,"url":null,"abstract":"<div><h3>Introduction</h3><div>Since November 2022, when Chat Generative Pre-trained Transformer (ChatGPT) was released, studies have been conducted to explore its potential use in various text-based areas. In the field of Korean Medicine (KM), studies evaluating the feasibility of ChatGPT are mainly focused on the educational domain. This study aimed to 1) identify the current level of ChatGPT-generated responses in answering questions from patients and KM practitioners, specifically exemplified by facial palsy and 2) explore the potential applicability of ChatGPT in KM practice through a perception survey of experts.</div></div><div><h3>Methods</h3><div>This study evaluates the applicability of ChatGPT concerning facial palsy through a survey of board-certified acupuncturists. The survey comprises two parts: assessing response quality for patients and for KM doctors in clinical practice. Ethical approval was obtained, and a 36-item questionnaire was administered online to 33 participants. Responses were analyzed for quality, relevance, and applicability using a combination of Likert scales and a standardized assessment tool.</div></div><div><h3>Results</h3><div>Among 33 eligible participants, 30 board-certified acupuncturists agreed to participate (response rate: 90.9 %). Survey results showed that the applicability of ChatGPT was lower for specialized KM practitioner inquiries than for general patient inquiries. Although the responses were considered useful to the readers (part 1: 96.7 %, part 2: 63.3 %) and understandable (part 1: 66.7 %, part 2: 60 %), the proportion of positive evaluations was relatively low in the domains assessing reliability and sufficiency.</div></div><div><h3>Conclusions</h3><div>Though ChatGPT is generally viewed positively for its potential utility, its direct application in KM clinical practice treating facial palsy at its current level seems challenging. To improve reliability, the information generated by ChatGPT should be critically reviewed by qualified expert medical personnel. To enhance sufficiency, further training of the artificial intelligence model with additional KM information is required.</div></div>","PeriodicalId":11932,"journal":{"name":"European Journal of Integrative Medicine","volume":"72 ","pages":"Article 102422"},"PeriodicalIF":1.9000,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"European Journal of Integrative Medicine","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S187638202400091X","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"INTEGRATIVE & COMPLEMENTARY MEDICINE","Score":null,"Total":0}

引用次数: 0

Abstract

Introduction

Since November 2022, when Chat Generative Pre-trained Transformer (ChatGPT) was released, studies have been conducted to explore its potential use in various text-based areas. In the field of Korean Medicine (KM), studies evaluating the feasibility of ChatGPT are mainly focused on the educational domain. This study aimed to 1) identify the current level of ChatGPT-generated responses in answering questions from patients and KM practitioners, specifically exemplified by facial palsy and 2) explore the potential applicability of ChatGPT in KM practice through a perception survey of experts.

Methods

This study evaluates the applicability of ChatGPT concerning facial palsy through a survey of board-certified acupuncturists. The survey comprises two parts: assessing response quality for patients and for KM doctors in clinical practice. Ethical approval was obtained, and a 36-item questionnaire was administered online to 33 participants. Responses were analyzed for quality, relevance, and applicability using a combination of Likert scales and a standardized assessment tool.

Results

Among 33 eligible participants, 30 board-certified acupuncturists agreed to participate (response rate: 90.9 %). Survey results showed that the applicability of ChatGPT was lower for specialized KM practitioner inquiries than for general patient inquiries. Although the responses were considered useful to the readers (part 1: 96.7 %, part 2: 63.3 %) and understandable (part 1: 66.7 %, part 2: 60 %), the proportion of positive evaluations was relatively low in the domains assessing reliability and sufficiency.

Conclusions

Though ChatGPT is generally viewed positively for its potential utility, its direct application in KM clinical practice treating facial palsy at its current level seems challenging. To improve reliability, the information generated by ChatGPT should be critically reviewed by qualified expert medical personnel. To enhance sufficiency, further training of the artificial intelligence model with additional KM information is required.

Abstract Image

查看原文本刊更多论文

探讨韩国医学专业人士对ChatGPT在面瘫实践中的适用性的看法：一项基于网络的调查

自2022年11月聊天生成预训练转换器（ChatGPT）发布以来，已经进行了研究，以探索其在各种基于文本的领域的潜在用途。在韩医学领域，评估ChatGPT可行性的研究主要集中在教育领域。本研究旨在1)确定ChatGPT在回答患者和KM从业者的问题时产生的反应的当前水平，特别是以面瘫为例；2)通过对专家的感知调查，探索ChatGPT在KM实践中的潜在适用性。方法通过对执业针灸师的调查，评价ChatGPT在面瘫治疗中的适用性。调查包括两个部分：评估患者的反应质量和临床实践中的KM医生。获得了伦理许可，并对33名参与者进行了一份36项的在线问卷调查。使用李克特量表和标准化评估工具的组合来分析回答的质量、相关性和适用性。结果在33名符合条件的参与者中，有30名针灸师同意参与（回复率：90.9%）。调查结果显示，ChatGPT在专业KM从业者查询中的适用性低于一般患者查询。虽然回答被认为对读者有用（第1部分：96.7%，第2部分：63.3%）和可理解（第1部分：66.7%，第2部分：60%），但在评估可靠性和充分性方面，积极评价的比例相对较低。结论虽然ChatGPT的潜在效用被普遍认为是积极的，但其在KM临床实践中治疗面瘫的直接应用在目前的水平似乎具有挑战性。为提高可靠性，ChatGPT生成的信息应由合格的专业医务人员进行严格审查。为了提高充分性，需要使用额外的KM信息进一步训练人工智能模型。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

European Journal of Integrative Medicine INTEGRATIVE & COMPLEMENTARY MEDICINE-

CiteScore

4.70

自引率

4.00%

发文量

102

审稿时长

33 days

期刊介绍： The European Journal of Integrative Medicine (EuJIM) considers manuscripts from a wide range of complementary and integrative health care disciplines, with a particular focus on whole systems approaches, public health, self management and traditional medical systems. The journal strives to connect conventional medicine and evidence based complementary medicine. We encourage submissions reporting research with relevance for integrative clinical practice and interprofessional education. EuJIM aims to be of interest to both conventional and integrative audiences, including healthcare practitioners, researchers, health care organisations, educationalists, and all those who seek objective and critical information on integrative medicine. To achieve this aim EuJIM provides an innovative international and interdisciplinary platform linking researchers and clinicians. The journal focuses primarily on original research articles including systematic reviews, randomized controlled trials, other clinical studies, qualitative, observational and epidemiological studies. In addition we welcome short reviews, opinion articles and contributions relating to health services and policy, health economics and psychology.