Exploring ChatGPT's abilities in medical article writing and peer review.

IF 1.5 4区医学 Q2 MEDICINE, GENERAL & INTERNAL

Croatian Medical Journal Pub Date : 2024-04-30

Gültekin Kadi, Mehmet Ali Aslaner

{"title":"Exploring ChatGPT's abilities in medical article writing and peer review.","authors":"Gültekin Kadi, Mehmet Ali Aslaner","doi":"","DOIUrl":null,"url":null,"abstract":"Aim: To evaluate the quality of ChatGPT-generated case reports and assess the ability of ChatGPT to peer review medical articles.Methods: This study was conducted from February to April 2023. First, ChatGPT 3.0 was used to generate 15 case reports, which were then peer-reviewed by expert human reviewers. Second, ChatGPT 4.0 was employed to peer review 15 published short articles.Results: ChatGPT was capable of generating case reports, but these reports exhibited inaccuracies, particularly when it came to referencing. The case reports received mixed ratings from peer reviewers, with 33.3% of professionals recommending rejection. The reports' overall merit score was 4.9±1.8 out of 10. The review capabilities of ChatGPT were weaker than its text generation abilities. The AI as a peer reviewer did not recognize major inconsistencies in articles that had undergone significant content changes.Conclusion: While ChatGPT demonstrated proficiency in generating case reports, there were limitations in terms of consistency and accuracy, especially in referencing.","PeriodicalId":10796,"journal":{"name":"Croatian Medical Journal","volume":"65 2","pages":"93-100"},"PeriodicalIF":1.5000,"publicationDate":"2024-04-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11074943/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Croatian Medical Journal","FirstCategoryId":"3","ListUrlMain":"","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"MEDICINE, GENERAL & INTERNAL","Score":null,"Total":0}

引用次数: 0

Abstract

Aim: To evaluate the quality of ChatGPT-generated case reports and assess the ability of ChatGPT to peer review medical articles.

Methods: This study was conducted from February to April 2023. First, ChatGPT 3.0 was used to generate 15 case reports, which were then peer-reviewed by expert human reviewers. Second, ChatGPT 4.0 was employed to peer review 15 published short articles.

Results: ChatGPT was capable of generating case reports, but these reports exhibited inaccuracies, particularly when it came to referencing. The case reports received mixed ratings from peer reviewers, with 33.3% of professionals recommending rejection. The reports' overall merit score was 4.9±1.8 out of 10. The review capabilities of ChatGPT were weaker than its text generation abilities. The AI as a peer reviewer did not recognize major inconsistencies in articles that had undergone significant content changes.

Conclusion: While ChatGPT demonstrated proficiency in generating case reports, there were limitations in terms of consistency and accuracy, especially in referencing.

本刊更多论文

探索 ChatGPT 在医学文章写作和同行评审方面的能力。

目的：评估 ChatGPT 生成的病例报告的质量，并评估 ChatGPT 对医学论文进行同行评审的能力：本研究于 2023 年 2 月至 4 月进行。首先，使用 ChatGPT 3.0 生成 15 份病例报告，然后由专家评审员进行同行评审。其次，使用 ChatGPT 4.0 对 15 篇已发表的短文进行同行评审：结果：ChatGPT 能够生成病例报告，但这些报告存在不准确之处，尤其是在引用文献方面。同行评审者对这些病例报告的评价不一，33.3%的专业人士建议拒绝。报告的总体优点得分是 4.9±1.8（满分 10 分）。ChatGPT 的审阅能力弱于其文本生成能力。作为同行评审员的人工智能无法识别内容发生重大变化的文章中的主要不一致之处：虽然 ChatGPT 能熟练生成病例报告，但在一致性和准确性方面存在局限性，尤其是在参考文献方面。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Croatian Medical Journal 医学-医学：内科

CiteScore

3.00

自引率

5.30%

发文量

105

审稿时长

6-12 weeks

期刊介绍： Croatian Medical Journal (CMJ) is an international peer reviewed journal open to scientists from all fields of biomedicine and health related research. Although CMJ welcomes all contributions that increase and expand on medical knowledge, the two areas are of the special interest: topics globally relevant for biomedicine and health and medicine in developing and emerging countries.