Text and Audio Simplification: Human vs. ChatGPT.

AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science Pub Date : 2024-05-31 eCollection Date: 2024-01-01

Gondy Leroy, David Kauchak, Philip Harber, Ankit Pal, Akash Shukla

{"title":"Text and Audio Simplification: Human vs. ChatGPT.","authors":"Gondy Leroy, David Kauchak, Philip Harber, Ankit Pal, Akash Shukla","doi":"","DOIUrl":null,"url":null,"abstract":"<p><p>Text and audio simplification to increase information comprehension are important in healthcare. With the introduction of ChatGPT, evaluation of its simplification performance is needed. We provide a systematic comparison of human and ChatGPT simplified texts using fourteen metrics indicative of text difficulty. We briefly introduce our online editor where these simplification tools, including ChatGPT, are available. We scored twelve corpora using our metrics: six text, one audio, and five ChatGPT simplified corpora (using five different prompts). We then compare these corpora with texts simplified and verified in a prior user study. Finally, a medical domain expert evaluated the user study texts and five, new ChatGPT simplified versions. We found that simple corpora show higher similarity with the human simplified texts. ChatGPT simplification moves metrics in the right direction. The medical domain expert's evaluation showed a preference for the ChatGPT style, but the text itself was rated lower for content retention.</p>","PeriodicalId":72181,"journal":{"name":"AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11141852/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science","FirstCategoryId":"1085","ListUrlMain":"","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/1/1 0:00:00","PubModel":"eCollection","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 0

Abstract

Text and audio simplification to increase information comprehension are important in healthcare. With the introduction of ChatGPT, evaluation of its simplification performance is needed. We provide a systematic comparison of human and ChatGPT simplified texts using fourteen metrics indicative of text difficulty. We briefly introduce our online editor where these simplification tools, including ChatGPT, are available. We scored twelve corpora using our metrics: six text, one audio, and five ChatGPT simplified corpora (using five different prompts). We then compare these corpora with texts simplified and verified in a prior user study. Finally, a medical domain expert evaluated the user study texts and five, new ChatGPT simplified versions. We found that simple corpora show higher similarity with the human simplified texts. ChatGPT simplification moves metrics in the right direction. The medical domain expert's evaluation showed a preference for the ChatGPT style, but the text itself was rated lower for content retention.

本刊更多论文

文本和音频简化：人类与 ChatGPT。

简化文本和音频以提高信息理解能力在医疗保健领域非常重要。随着 ChatGPT 的推出，需要对其简化性能进行评估。我们使用 14 个文本难度指标对人类和 ChatGPT 简化文本进行了系统比较。我们简要介绍了我们的在线编辑器，包括 ChatGPT 在内的这些简化工具都可以在这里使用。我们使用我们的指标对 12 个语料库进行了评分：6 个文本、1 个音频和 5 个 ChatGPT 简化语料库（使用 5 种不同的提示）。然后，我们将这些语料与之前用户研究中简化和验证过的文本进行比较。最后，一位医学领域专家对用户研究文本和五个新的 ChatGPT 简化版本进行了评估。我们发现，简单的语料库与人类简化文本的相似度更高。ChatGPT 简化版的度量方向是正确的。医学领域专家的评估结果显示了对 ChatGPT 风格的偏好，但文本本身在内容保留方面的评分较低。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science

自引率

0.00%

发文量

文献相关原料

公司名称	产品信息	采购帮参考价格