Jonathan M Carnino, Nicholas Y K Chong, Henry Bayly, Lindsay R Salvati, Hardeep S Tiwana, Jessica R Levi
{"title":"AI-generated text in otolaryngology publications: a comparative analysis before and after the release of ChatGPT.","authors":"Jonathan M Carnino, Nicholas Y K Chong, Henry Bayly, Lindsay R Salvati, Hardeep S Tiwana, Jessica R Levi","doi":"10.1007/s00405-024-08834-3","DOIUrl":null,"url":null,"abstract":"<p><strong>Purpose: </strong>This study delves into the broader implications of artificial intelligence (AI) text generation technologies, including large language models (LLMs) and chatbots, on the scientific literature of otolaryngology. By observing trends in AI-generated text within published otolaryngology studies, this investigation aims to contextualize the impact of AI-driven tools that are reshaping scientific writing and communication.</p><p><strong>Methods: </strong>Text from 143 original articles published in JAMA Otolaryngology - Head and Neck Surgery was collected, representing periods before and after ChatGPT's release in November 2022. The text from each article's abstract, introduction, methods, results, and discussion were entered into ZeroGPT.com to estimate the percentage of AI-generated content. Statistical analyses, including T-Tests and Fligner-Killeen's tests, were conducted using R.</p><p><strong>Results: </strong>A significant increase was observed in the mean percentage of AI-generated text post-ChatGPT release, especially in the abstract (from 34.36 to 46.53%, p = 0.004), introduction (from 32.43 to 45.08%, p = 0.010), and discussion sections (from 15.73 to 25.03%, p = 0.015). Publications of authors from non-English speaking countries demonstrated a higher percentage of AI-generated text.</p><p><strong>Conclusion: </strong>This study found that the advent of ChatGPT has significantly impacted writing practices among researchers publishing in JAMA Otolaryngology - Head and Neck Surgery, raising concerns over the accuracy of AI-created content and potential misinformation risks. This manuscript highlights the evolving dynamics between AI technologies, scientific communication, and publication integrity, emphasizing the urgent need for continued research in this dynamic field. The findings also suggest an increasing reliance on AI tools like ChatGPT, raising questions about their broader implications for scientific publishing.</p>","PeriodicalId":1,"journal":{"name":"Accounts of Chemical Research","volume":null,"pages":null},"PeriodicalIF":16.4000,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11513233/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Accounts of Chemical Research","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1007/s00405-024-08834-3","RegionNum":1,"RegionCategory":"化学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/7/17 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"CHEMISTRY, MULTIDISCIPLINARY","Score":null,"Total":0}
引用次数: 0
Abstract
Purpose: This study delves into the broader implications of artificial intelligence (AI) text generation technologies, including large language models (LLMs) and chatbots, on the scientific literature of otolaryngology. By observing trends in AI-generated text within published otolaryngology studies, this investigation aims to contextualize the impact of AI-driven tools that are reshaping scientific writing and communication.
Methods: Text from 143 original articles published in JAMA Otolaryngology - Head and Neck Surgery was collected, representing periods before and after ChatGPT's release in November 2022. The text from each article's abstract, introduction, methods, results, and discussion were entered into ZeroGPT.com to estimate the percentage of AI-generated content. Statistical analyses, including T-Tests and Fligner-Killeen's tests, were conducted using R.
Results: A significant increase was observed in the mean percentage of AI-generated text post-ChatGPT release, especially in the abstract (from 34.36 to 46.53%, p = 0.004), introduction (from 32.43 to 45.08%, p = 0.010), and discussion sections (from 15.73 to 25.03%, p = 0.015). Publications of authors from non-English speaking countries demonstrated a higher percentage of AI-generated text.
Conclusion: This study found that the advent of ChatGPT has significantly impacted writing practices among researchers publishing in JAMA Otolaryngology - Head and Neck Surgery, raising concerns over the accuracy of AI-created content and potential misinformation risks. This manuscript highlights the evolving dynamics between AI technologies, scientific communication, and publication integrity, emphasizing the urgent need for continued research in this dynamic field. The findings also suggest an increasing reliance on AI tools like ChatGPT, raising questions about their broader implications for scientific publishing.
期刊介绍：
Accounts of Chemical Research presents short, concise and critical articles offering easy-to-read overviews of basic research and applications in all areas of chemistry and biochemistry. These short reviews focus on research from the author’s own laboratory and are designed to teach the reader about a research project. In addition, Accounts of Chemical Research publishes commentaries that give an informed opinion on a current research problem. Special Issues online are devoted to a single topic of unusual activity and significance.
Accounts of Chemical Research replaces the traditional article abstract with an article "Conspectus." These entries synopsize the research affording the reader a closer look at the content and significance of an article. Through this provision of a more detailed description of the article contents, the Conspectus enhances the article's discoverability by search engines and the exposure for the research.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
