Chasing sleep physicians: ChatGPT-4o on the interpretation of polysomnographic results.

IF 1.9 3区医学 Q2 OTORHINOLARYNGOLOGY

European Archives of Oto-Rhino-Laryngology Pub Date : 2024-10-20 DOI:10.1007/s00405-024-08985-3

Christopher Seifen, Tilman Huppertz, Haralampos Gouveris, Katharina Bahr-Hamm, Johannes Pordzik, Jonas Eckrich, Harry Smith, Tom Kelsey, Andrew Blaikie, Christoph Matthias, Sebastian Kuhn, Christoph Raphael Buhr

{"title":"Chasing sleep physicians: ChatGPT-4o on the interpretation of polysomnographic results.","authors":"Christopher Seifen, Tilman Huppertz, Haralampos Gouveris, Katharina Bahr-Hamm, Johannes Pordzik, Jonas Eckrich, Harry Smith, Tom Kelsey, Andrew Blaikie, Christoph Matthias, Sebastian Kuhn, Christoph Raphael Buhr","doi":"10.1007/s00405-024-08985-3","DOIUrl":null,"url":null,"abstract":"Background: From a healthcare professional's perspective, the use of ChatGPT (Open AI), a large language model (LLM), offers huge potential as a practical and economic digital assistant. However, ChatGPT has not yet been evaluated for the interpretation of polysomnographic results in patients with suspected obstructive sleep apnea (OSA).Aims/objectives: To evaluate the agreement of polysomnographic result interpretation between ChatGPT-4o and a board-certified sleep physician and to shed light into the role of ChatGPT-4o in the field of medical decision-making in sleep medicine.Material and methods: For this proof-of-concept study, 40 comprehensive patient profiles were designed, which represent a broad and typical spectrum of cases, ensuring a balanced distribution of demographics and clinical characteristics. After various prompts were tested, one prompt was used for initial diagnosis of OSA and a further for patients with positive airway pressure (PAP) therapy intolerance. Each polysomnographic result was independently evaluated by ChatGPT-4o and a board-certified sleep physician. Diagnosis and therapy suggestions were analyzed for agreement.Results: ChatGPT-4o and the sleep physician showed 97% (29/30) concordance in the diagnosis of the simple cases. For the same cases the two assessment instances unveiled 100% (30/30) concordance regarding therapy suggestions. For cases with intolerance of treatment with positive airway pressure (PAP) ChatGPT-4o and the sleep physician revealed 70% (7/10) concordance in the diagnosis and 44% (22/50) concordance for therapy suggestions.Conclusion and significance: Precise prompting improves the output of ChatGPT-4o and provides sleep physician-like polysomnographic result interpretation. Although ChatGPT shows some shortcomings in offering treatment advice, our results provide evidence for AI assisted automation and economization of polysomnographic interpretation by LLMs. Further research should explore data protection issues and demonstrate reproducibility with real patient data on a larger scale.","PeriodicalId":11952,"journal":{"name":"European Archives of Oto-Rhino-Laryngology","volume":null,"pages":null},"PeriodicalIF":1.9000,"publicationDate":"2024-10-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"European Archives of Oto-Rhino-Laryngology","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1007/s00405-024-08985-3","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"OTORHINOLARYNGOLOGY","Score":null,"Total":0}

引用次数: 0

Abstract

Background: From a healthcare professional's perspective, the use of ChatGPT (Open AI), a large language model (LLM), offers huge potential as a practical and economic digital assistant. However, ChatGPT has not yet been evaluated for the interpretation of polysomnographic results in patients with suspected obstructive sleep apnea (OSA).

Aims/objectives: To evaluate the agreement of polysomnographic result interpretation between ChatGPT-4o and a board-certified sleep physician and to shed light into the role of ChatGPT-4o in the field of medical decision-making in sleep medicine.

Material and methods: For this proof-of-concept study, 40 comprehensive patient profiles were designed, which represent a broad and typical spectrum of cases, ensuring a balanced distribution of demographics and clinical characteristics. After various prompts were tested, one prompt was used for initial diagnosis of OSA and a further for patients with positive airway pressure (PAP) therapy intolerance. Each polysomnographic result was independently evaluated by ChatGPT-4o and a board-certified sleep physician. Diagnosis and therapy suggestions were analyzed for agreement.

Results: ChatGPT-4o and the sleep physician showed 97% (29/30) concordance in the diagnosis of the simple cases. For the same cases the two assessment instances unveiled 100% (30/30) concordance regarding therapy suggestions. For cases with intolerance of treatment with positive airway pressure (PAP) ChatGPT-4o and the sleep physician revealed 70% (7/10) concordance in the diagnosis and 44% (22/50) concordance for therapy suggestions.

Conclusion and significance: Precise prompting improves the output of ChatGPT-4o and provides sleep physician-like polysomnographic result interpretation. Although ChatGPT shows some shortcomings in offering treatment advice, our results provide evidence for AI assisted automation and economization of polysomnographic interpretation by LLMs. Further research should explore data protection issues and demonstrate reproducibility with real patient data on a larger scale.

查看原文本刊更多论文

追逐睡眠医生：ChatGPT-4o 关于多导睡眠图结果的解释。

背景：从医疗专业人员的角度来看，使用大型语言模型（LLM）ChatGPT（开放式人工智能）作为实用、经济的数字助理具有巨大的潜力。然而，目前尚未对 ChatGPT 如何解释疑似阻塞性睡眠呼吸暂停（OSA）患者的多导睡眠图结果进行评估：评估 ChatGPT-4o 与经委员会认证的睡眠医师对多导睡眠图结果判读的一致性，并揭示 ChatGPT-4o 在睡眠医学医疗决策领域的作用：在这项概念验证研究中，我们设计了 40 份全面的患者档案，这些档案代表了广泛而典型的病例，确保了人口统计学和临床特征的均衡分布。在对各种提示进行测试后，一个提示用于 OSA 的初步诊断，另一个提示用于不耐受正气压疗法的患者。每项多导睡眠图结果都由 ChatGPT-4o 和一位经委员会认证的睡眠医师进行独立评估。对诊断和治疗建议的一致性进行了分析：结果：ChatGPT-4o 和睡眠医师对简单病例的诊断一致率为 97%（29/30）。对于同样的病例，两种评估方法在治疗建议方面的一致性为 100%（30/30）。对于不耐受气道正压（PAP）治疗的病例，ChatGPT-4o 和睡眠医师在诊断上的一致性为 70%（7/10），在治疗建议上的一致性为 44%（22/50）：精确的提示提高了 ChatGPT-4o 的输出效果，并提供了类似于睡眠医师的多导睡眠图结果解释。虽然 ChatGPT 在提供治疗建议方面存在一些不足，但我们的研究结果为人工智能辅助多导睡眠图解释的自动化和经济化提供了证据。进一步的研究应探讨数据保护问题，并用更大规模的真实患者数据证明其可重复性。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

European Archives of Oto-Rhino-Laryngology 医学-耳鼻喉科学

CiteScore

5.30

自引率

7.70%

发文量

537

审稿时长

2-4 weeks

期刊介绍： Official Journal of European Union of Medical Specialists – ORL Section and Board Official Journal of Confederation of European Oto-Rhino-Laryngology Head and Neck Surgery "European Archives of Oto-Rhino-Laryngology" publishes original clinical reports and clinically relevant experimental studies, as well as short communications presenting new results of special interest. With peer review by a respected international editorial board and prompt English-language publication, the journal provides rapid dissemination of information by authors from around the world. This particular feature makes it the journal of choice for readers who want to be informed about the continuing state of the art concerning basic sciences and the diagnosis and management of diseases of the head and neck on an international level. European Archives of Oto-Rhino-Laryngology was founded in 1864 as "Archiv für Ohrenheilkunde" by A. von Tröltsch, A. Politzer and H. Schwartze.