ChatGPT 与睡眠障碍专家对常见睡眠问题的回答：专家和普通人的评分。

IF 3.4 2区医学 Q2 CLINICAL NEUROLOGY

Sleep Health Pub Date : 2024-12-01 DOI:10.1016/j.sleh.2024.08.011

Jiyoung Kim MD, PhD , Seo-Young Lee MD, PhD , Jee Hyun Kim MD, PhD , Dong-Hyeon Shin MD , Eun Hye Oh MD, PhD , Jin A Kim BSc , Jae Wook Cho MD, PhD

{"title":"ChatGPT 与睡眠障碍专家对常见睡眠问题的回答：专家和普通人的评分。","authors":"Jiyoung Kim MD, PhD , Seo-Young Lee MD, PhD , Jee Hyun Kim MD, PhD , Dong-Hyeon Shin MD , Eun Hye Oh MD, PhD , Jin A Kim BSc , Jae Wook Cho MD, PhD","doi":"10.1016/j.sleh.2024.08.011","DOIUrl":null,"url":null,"abstract":"<div><h3>Background</h3><div>Many individuals use the Internet, including generative artificial intelligence like ChatGPT, for sleep-related information before consulting medical professionals. This study compared responses from sleep disorder specialists and ChatGPT to common sleep queries, with experts and laypersons evaluating the responses' accuracy and clarity.</div></div><div><h3>Methods</h3><div>We assessed responses from sleep medicine specialists and ChatGPT-4 to 140 sleep-related questions from the Korean Sleep Research Society's website. In a blinded study design, sleep disorder experts and laypersons rated the medical helpfulness, emotional supportiveness, and sentence comprehensibility of the responses on a 1-5 scale.</div></div><div><h3>Results</h3><div>Laypersons rated ChatGPT higher for medical helpfulness (3.79 ± 0.90 vs. 3.44 ± 0.99, <em>p</em> < .001), emotional supportiveness (3.48 ± 0.79 vs. 3.12 ± 0.98, <em>p</em> < .001), and sentence comprehensibility (4.24 ± 0.79 vs. 4.14 ± 0.96, <em>p</em> = .028). Experts also rated ChatGPT higher for emotional supportiveness (3.33 ± 0.62 vs. 3.01 ± 0.67, <em>p</em> < .001) but preferred specialists' responses for sentence comprehensibility (4.15 ± 0.74 vs. 3.94 ± 0.90, <em>p</em> < .001). When it comes to medical helpfulness, the experts rated the specialists' answers slightly higher than the laypersons did (3.70 ± 0.84 vs. 3.63 ± 0.87, <em>p</em> = .109). Experts slightly preferred specialist responses overall (56.0%), while laypersons favored ChatGPT (54.3%; <em>p</em> < .001). ChatGPT's responses were significantly longer (186.76 ± 39.04 vs. 113.16 ± 95.77 words, <em>p</em> < .001).</div></div><div><h3>Discussion</h3><div>Generative artificial intelligence like ChatGPT may help disseminate sleep-related medical information online. Laypersons appear to prefer ChatGPT's detailed, emotionally supportive responses over those from sleep disorder specialists.</div></div>","PeriodicalId":48545,"journal":{"name":"Sleep Health","volume":"10 6","pages":"Pages 665-670"},"PeriodicalIF":3.4000,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"ChatGPT vs. sleep disorder specialist responses to common sleep queries: Ratings by experts and laypeople\",\"authors\":\"Jiyoung Kim MD, PhD , Seo-Young Lee MD, PhD , Jee Hyun Kim MD, PhD , Dong-Hyeon Shin MD , Eun Hye Oh MD, PhD , Jin A Kim BSc , Jae Wook Cho MD, PhD\",\"doi\":\"10.1016/j.sleh.2024.08.011\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><h3>Background</h3><div>Many individuals use the Internet, including generative artificial intelligence like ChatGPT, for sleep-related information before consulting medical professionals. This study compared responses from sleep disorder specialists and ChatGPT to common sleep queries, with experts and laypersons evaluating the responses' accuracy and clarity.</div></div><div><h3>Methods</h3><div>We assessed responses from sleep medicine specialists and ChatGPT-4 to 140 sleep-related questions from the Korean Sleep Research Society's website. In a blinded study design, sleep disorder experts and laypersons rated the medical helpfulness, emotional supportiveness, and sentence comprehensibility of the responses on a 1-5 scale.</div></div><div><h3>Results</h3><div>Laypersons rated ChatGPT higher for medical helpfulness (3.79 ± 0.90 vs. 3.44 ± 0.99, <em>p</em> < .001), emotional supportiveness (3.48 ± 0.79 vs. 3.12 ± 0.98, <em>p</em> < .001), and sentence comprehensibility (4.24 ± 0.79 vs. 4.14 ± 0.96, <em>p</em> = .028). Experts also rated ChatGPT higher for emotional supportiveness (3.33 ± 0.62 vs. 3.01 ± 0.67, <em>p</em> < .001) but preferred specialists' responses for sentence comprehensibility (4.15 ± 0.74 vs. 3.94 ± 0.90, <em>p</em> < .001). When it comes to medical helpfulness, the experts rated the specialists' answers slightly higher than the laypersons did (3.70 ± 0.84 vs. 3.63 ± 0.87, <em>p</em> = .109). Experts slightly preferred specialist responses overall (56.0%), while laypersons favored ChatGPT (54.3%; <em>p</em> < .001). ChatGPT's responses were significantly longer (186.76 ± 39.04 vs. 113.16 ± 95.77 words, <em>p</em> < .001).</div></div><div><h3>Discussion</h3><div>Generative artificial intelligence like ChatGPT may help disseminate sleep-related medical information online. Laypersons appear to prefer ChatGPT's detailed, emotionally supportive responses over those from sleep disorder specialists.</div></div>\",\"PeriodicalId\":48545,\"journal\":{\"name\":\"Sleep Health\",\"volume\":\"10 6\",\"pages\":\"Pages 665-670\"},\"PeriodicalIF\":3.4000,\"publicationDate\":\"2024-12-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Sleep Health\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S2352721824001876\",\"RegionNum\":2,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"CLINICAL NEUROLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Sleep Health","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2352721824001876","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"CLINICAL NEUROLOGY","Score":null,"Total":0}

引用次数: 0

摘要

背景介绍许多人在咨询医疗专家之前都会使用互联网，包括像 ChatGPT 这样的人工智能生成器，来获取与睡眠相关的信息。本研究比较了睡眠障碍专家和 ChatGPT 对常见睡眠问题的回答，由专家和普通人对回答的准确性和清晰度进行评估：我们评估了睡眠医学专家和 ChatGPT-4 对韩国睡眠研究协会网站上 140 个睡眠相关问题的回复。在盲法研究设计中，睡眠障碍专家和非专业人士对回答的医疗帮助性、情感支持性和句子可理解性进行了 1-5 级评分：结果：普通人对 ChatGPT 的医疗帮助性评分更高（3.79 ± 0.90 vs. 3.44 ± 0.99，p 讨论）：像 ChatGPT 这样的生成式人工智能可能有助于在网上传播与睡眠相关的医疗信息。与睡眠障碍专家的回复相比，普通人似乎更喜欢 ChatGPT 详细的、情感支持性的回复。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

ChatGPT vs. sleep disorder specialist responses to common sleep queries: Ratings by experts and laypeople

Background

Many individuals use the Internet, including generative artificial intelligence like ChatGPT, for sleep-related information before consulting medical professionals. This study compared responses from sleep disorder specialists and ChatGPT to common sleep queries, with experts and laypersons evaluating the responses' accuracy and clarity.

Methods

We assessed responses from sleep medicine specialists and ChatGPT-4 to 140 sleep-related questions from the Korean Sleep Research Society's website. In a blinded study design, sleep disorder experts and laypersons rated the medical helpfulness, emotional supportiveness, and sentence comprehensibility of the responses on a 1-5 scale.

Results

Laypersons rated ChatGPT higher for medical helpfulness (3.79 ± 0.90 vs. 3.44 ± 0.99, p < .001), emotional supportiveness (3.48 ± 0.79 vs. 3.12 ± 0.98, p < .001), and sentence comprehensibility (4.24 ± 0.79 vs. 4.14 ± 0.96, p = .028). Experts also rated ChatGPT higher for emotional supportiveness (3.33 ± 0.62 vs. 3.01 ± 0.67, p < .001) but preferred specialists' responses for sentence comprehensibility (4.15 ± 0.74 vs. 3.94 ± 0.90, p < .001). When it comes to medical helpfulness, the experts rated the specialists' answers slightly higher than the laypersons did (3.70 ± 0.84 vs. 3.63 ± 0.87, p = .109). Experts slightly preferred specialist responses overall (56.0%), while laypersons favored ChatGPT (54.3%; p < .001). ChatGPT's responses were significantly longer (186.76 ± 39.04 vs. 113.16 ± 95.77 words, p < .001).

Discussion

Generative artificial intelligence like ChatGPT may help disseminate sleep-related medical information online. Laypersons appear to prefer ChatGPT's detailed, emotionally supportive responses over those from sleep disorder specialists.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Sleep Health CLINICAL NEUROLOGY-

CiteScore

6.30

自引率

9.80%

发文量

114

审稿时长

54 days

期刊介绍： Sleep Health Journal of the National Sleep Foundation is a multidisciplinary journal that explores sleep''s role in population health and elucidates the social science perspective on sleep and health. Aligned with the National Sleep Foundation''s global authoritative, evidence-based voice for sleep health, the journal serves as the foremost publication for manuscripts that advance the sleep health of all members of society.The scope of the journal extends across diverse sleep-related fields, including anthropology, education, health services research, human development, international health, law, mental health, nursing, nutrition, psychology, public health, public policy, fatigue management, transportation, social work, and sociology. The journal welcomes original research articles, review articles, brief reports, special articles, letters to the editor, editorials, and commentaries.