Mehmet Fatih Şahin, Anil Keleş, Rıdvan Özcan, Çağrı Doğan, Erdem Can Topkaç, Murat Akgül, Cenk Murat Yazıci
{"title":"Evaluation of information accuracy and clarity: ChatGPT responses to the most frequently asked questions about premature ejaculation.","authors":"Mehmet Fatih Şahin, Anil Keleş, Rıdvan Özcan, Çağrı Doğan, Erdem Can Topkaç, Murat Akgül, Cenk Murat Yazıci","doi":"10.1093/sexmed/qfae036","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Premature ejaculation (PE) is the most prevalent sexual dysfunction in men, and like many diseases and conditions, patients use Internet sources like ChatGPT, which is a popular artificial intelligence-based language model, for queries about this andrological disorder.</p><p><strong>Aim: </strong>The objective of this research was to evaluate the quality, readability, and understanding of texts produced by ChatGPT in response to frequently requested inquiries on PE.</p><p><strong>Methods: </strong>In this study we used Google Trends to identify the most frequently searched phrases related to PE. Subsequently, the discovered keywords were methodically entered into ChatGPT, and the resulting replies were assessed for quality using the Ensuring Quality Information for Patients (EQIP) program. The produced texts were assessed for readability using the Flesch-Kincaid Grade Level (FKGL), Flesch Reading Ease Score (FRES), and DISCERN metrics.</p><p><strong>Outcomes: </strong>This investigation has identified substantial concerns about the quality of texts produced by ChatGPT, highlighting severe problems with reading and understanding.</p><p><strong>Results: </strong>The mean EQIP score for the texts was determined to be 45.93 ± 4.34, while the FRES was 15.8 ± 8.73. Additionally, the FKGL score was computed to be 15.68 ± 1.67 and the DISCERN score was 38.1 ± 3.78. The comparatively low average EQIP and DISCERN scores suggest that improvements are required to increase the quality and dependability of the presented information. In addition, the FKGL scores indicate a significant degree of linguistic intricacy, requiring a level of knowledge comparable to about 14 to 15 years of formal schooling in order to understand. The texts about treatment, which are the most frequently searched items, are more difficult to understand compared to other texts about other categories.</p><p><strong>Clinical implications: </strong>The results of this research suggest that compared to texts on other topics the PE texts produced by ChatGPT exhibit a higher degree of complexity, which exceeds the recommended reading threshold for effective health communication. Currently, ChatGPT is cannot be considered a substitute for comprehensive medical consultations.</p><p><strong>Strengths and limitations: </strong>This study is to our knowledge the first reported research investigating the quality and comprehensibility of information generated by ChatGPT in relation to frequently requested queries about PE. The main limitation is that the investigation included only the first 25 popular keywords in English.</p><p><strong>Conclusion: </strong>ChatGPT is incapable of replacing the need for thorough medical consultations.</p>","PeriodicalId":21782,"journal":{"name":"Sexual Medicine","volume":"12 3","pages":"qfae036"},"PeriodicalIF":2.6000,"publicationDate":"2024-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11144523/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Sexual Medicine","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1093/sexmed/qfae036","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/6/1 0:00:00","PubModel":"eCollection","JCR":"Q1","JCRName":"MEDICINE, GENERAL & INTERNAL","Score":null,"Total":0}
引用次数: 0
Abstract
Background: Premature ejaculation (PE) is the most prevalent sexual dysfunction in men, and like many diseases and conditions, patients use Internet sources like ChatGPT, which is a popular artificial intelligence-based language model, for queries about this andrological disorder.
Aim: The objective of this research was to evaluate the quality, readability, and understanding of texts produced by ChatGPT in response to frequently requested inquiries on PE.
Methods: In this study we used Google Trends to identify the most frequently searched phrases related to PE. Subsequently, the discovered keywords were methodically entered into ChatGPT, and the resulting replies were assessed for quality using the Ensuring Quality Information for Patients (EQIP) program. The produced texts were assessed for readability using the Flesch-Kincaid Grade Level (FKGL), Flesch Reading Ease Score (FRES), and DISCERN metrics.
Outcomes: This investigation has identified substantial concerns about the quality of texts produced by ChatGPT, highlighting severe problems with reading and understanding.
Results: The mean EQIP score for the texts was determined to be 45.93 ± 4.34, while the FRES was 15.8 ± 8.73. Additionally, the FKGL score was computed to be 15.68 ± 1.67 and the DISCERN score was 38.1 ± 3.78. The comparatively low average EQIP and DISCERN scores suggest that improvements are required to increase the quality and dependability of the presented information. In addition, the FKGL scores indicate a significant degree of linguistic intricacy, requiring a level of knowledge comparable to about 14 to 15 years of formal schooling in order to understand. The texts about treatment, which are the most frequently searched items, are more difficult to understand compared to other texts about other categories.
Clinical implications: The results of this research suggest that compared to texts on other topics the PE texts produced by ChatGPT exhibit a higher degree of complexity, which exceeds the recommended reading threshold for effective health communication. Currently, ChatGPT is cannot be considered a substitute for comprehensive medical consultations.
Strengths and limitations: This study is to our knowledge the first reported research investigating the quality and comprehensibility of information generated by ChatGPT in relation to frequently requested queries about PE. The main limitation is that the investigation included only the first 25 popular keywords in English.
Conclusion: ChatGPT is incapable of replacing the need for thorough medical consultations.
期刊介绍：
Sexual Medicine is an official publication of the International Society for Sexual Medicine, and serves the field as the peer-reviewed, open access journal for rapid dissemination of multidisciplinary clinical and basic research in all areas of global sexual medicine, and particularly acts as a venue for topics of regional or sub-specialty interest. The journal is focused on issues in clinical medicine and epidemiology but also publishes basic science papers with particular relevance to specific populations. Sexual Medicine offers clinicians and researchers a rapid route to publication and the opportunity to publish in a broadly distributed and highly visible global forum. The journal publishes high quality articles from all over the world and actively seeks submissions from countries with expanding sexual medicine communities. Sexual Medicine relies on the same expert panel of editors and reviewers as The Journal of Sexual Medicine and Sexual Medicine Reviews.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
