Patient Support in Obstructive Sleep Apnoea by a Large Language Model - ChatGPT 4o on Answering Frequently Asked Questions on First Line Positive Airway Pressure and Second Line Hypoglossal Nerve Stimulation Therapy: A Pilot Study.

IF 3 2区医学 Q2 CLINICAL NEUROLOGY

Nature and Science of Sleep Pub Date : 2024-12-27 eCollection Date: 2024-01-01 DOI:10.2147/NSS.S495654

Johannes Pordzik, Katharina Bahr-Hamm, Tilman Huppertz, Haralampos Gouveris, Christopher Seifen, Andrew Blaikie, Christoph Matthias, Sebastian Kuhn, Jonas Eckrich, Christoph R Buhr

{"title":"Patient Support in Obstructive Sleep Apnoea by a Large Language Model - ChatGPT 4o on Answering Frequently Asked Questions on First Line Positive Airway Pressure and Second Line Hypoglossal Nerve Stimulation Therapy: A Pilot Study.","authors":"Johannes Pordzik, Katharina Bahr-Hamm, Tilman Huppertz, Haralampos Gouveris, Christopher Seifen, Andrew Blaikie, Christoph Matthias, Sebastian Kuhn, Jonas Eckrich, Christoph R Buhr","doi":"10.2147/NSS.S495654","DOIUrl":null,"url":null,"abstract":"Purpose: Obstructive sleep apnoea (OSA) is a common disease that benefits from early treatment and patient support in order to prevent secondary illnesses. This study assesses the capability of the large language model (LLM) ChatGPT-4o to offer patient support regarding first line positive airway pressure (PAP) and second line hypoglossal nerve stimulation (HGNS) therapy.Methods: Seventeen questions, each regarding PAP and HGNS therapy, were posed to ChatGPT-4o. Answers were rated by experienced experts in sleep medicine on a 6-point Likert scale in the categories of medical adequacy, conciseness, coherence, and comprehensibility. Completeness of medical information and potential hazard for patients were rated using a binary system.Results: Overall, ChatGPT-4o achieved reasonably high ratings in all categories. In medical adequacy, it performed significantly better on PAP questions (mean 4.9) compared to those on HGNS (mean 4.6) (p < 0.05). Scores for coherence, comprehensibility and conciseness showed similar results for both HGNS and PAP answers. Raters confirmed completeness of responses in 45 of 51 ratings (88.24%) for PAP answers and 28 of 51 ratings (54.9%) for HGNS answers. Potential hazards for patients were stated in 2 of 52 ratings (0.04%) for PAP answers and none for HGNS answers.Conclusion: ChatGPT-4o has potential as a valuable patient-oriented support tool in sleep medicine therapy that can enhance subsequent face-to-face consultations with a sleep specialist. However, some substantial flaws regarding second line HGNS therapy are most likely due to recent advances in HGNS therapy and the consequent limited information available in LLM training data.","PeriodicalId":18896,"journal":{"name":"Nature and Science of Sleep","volume":"16 ","pages":"2269-2277"},"PeriodicalIF":3.0000,"publicationDate":"2024-12-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11687119/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Nature and Science of Sleep","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.2147/NSS.S495654","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/1/1 0:00:00","PubModel":"eCollection","JCR":"Q2","JCRName":"CLINICAL NEUROLOGY","Score":null,"Total":0}

引用次数: 0

Abstract

Purpose: Obstructive sleep apnoea (OSA) is a common disease that benefits from early treatment and patient support in order to prevent secondary illnesses. This study assesses the capability of the large language model (LLM) ChatGPT-4o to offer patient support regarding first line positive airway pressure (PAP) and second line hypoglossal nerve stimulation (HGNS) therapy.

Methods: Seventeen questions, each regarding PAP and HGNS therapy, were posed to ChatGPT-4o. Answers were rated by experienced experts in sleep medicine on a 6-point Likert scale in the categories of medical adequacy, conciseness, coherence, and comprehensibility. Completeness of medical information and potential hazard for patients were rated using a binary system.

Results: Overall, ChatGPT-4o achieved reasonably high ratings in all categories. In medical adequacy, it performed significantly better on PAP questions (mean 4.9) compared to those on HGNS (mean 4.6) (p < 0.05). Scores for coherence, comprehensibility and conciseness showed similar results for both HGNS and PAP answers. Raters confirmed completeness of responses in 45 of 51 ratings (88.24%) for PAP answers and 28 of 51 ratings (54.9%) for HGNS answers. Potential hazards for patients were stated in 2 of 52 ratings (0.04%) for PAP answers and none for HGNS answers.

Conclusion: ChatGPT-4o has potential as a valuable patient-oriented support tool in sleep medicine therapy that can enhance subsequent face-to-face consultations with a sleep specialist. However, some substantial flaws regarding second line HGNS therapy are most likely due to recent advances in HGNS therapy and the consequent limited information available in LLM training data.

查看原文本刊更多论文

求助全文

约1分钟内获得全文求助全文

来源期刊

Nature and Science of Sleep Neuroscience-Behavioral Neuroscience

CiteScore

5.70

自引率

5.90%

发文量

245

审稿时长

16 weeks

期刊介绍： Nature and Science of Sleep is an international, peer-reviewed, open access journal covering all aspects of sleep science and sleep medicine, including the neurophysiology and functions of sleep, the genetics of sleep, sleep and society, biological rhythms, dreaming, sleep disorders and therapy, and strategies to optimize healthy sleep. Specific topics covered in the journal include: The functions of sleep in humans and other animals Physiological and neurophysiological changes with sleep The genetics of sleep and sleep differences The neurotransmitters, receptors and pathways involved in controlling both sleep and wakefulness Behavioral and pharmacological interventions aimed at improving sleep, and improving wakefulness Sleep changes with development and with age Sleep and reproduction (e.g., changes across the menstrual cycle, with pregnancy and menopause) The science and nature of dreams Sleep disorders Impact of sleep and sleep disorders on health, daytime function and quality of life Sleep problems secondary to clinical disorders Interaction of society with sleep (e.g., consequences of shift work, occupational health, public health) The microbiome and sleep Chronotherapy Impact of circadian rhythms on sleep, physiology, cognition and health Mechanisms controlling circadian rhythms, centrally and peripherally Impact of circadian rhythm disruptions (including night shift work, jet lag and social jet lag) on sleep, physiology, cognition and health Behavioral and pharmacological interventions aimed at reducing adverse effects of circadian-related sleep disruption Assessment of technologies and biomarkers for measuring sleep and/or circadian rhythms Epigenetic markers of sleep or circadian disruption.