人工智能能写科学吗?对人类写作和人工智能生成的科学著作进行比较分析。

IF 3.1 2区 医学 Q2 CLINICAL NEUROLOGY
Karim Rizwan Nathani, Ali-Muhammad Nathani, Maliya Delawan, Aleeza Safdar, Mohamad Bydon
{"title":"人工智能能写科学吗?对人类写作和人工智能生成的科学著作进行比较分析。","authors":"Karim Rizwan Nathani, Ali-Muhammad Nathani, Maliya Delawan, Aleeza Safdar, Mohamad Bydon","doi":"10.3171/2025.4.SPINE25519","DOIUrl":null,"url":null,"abstract":"<p><strong>Objective: </strong>Artificial intelligence (AI) is increasingly capable of academic writing, with large language models such as ChatGPT showing potential to assist or even generate scientific manuscripts. However, concerns remain regarding the quality, reliability, and interpretive capabilities of AI-generated content. The authors' study aimed to compare the quality of a human-written versus an AI-generated scientific manuscript to evaluate the strengths and limitations of AI in the context of academic publishing.</p><p><strong>Methods: </strong>Two manuscripts were developed using identical titles, abstracts, and tables of a simulated analysis: one authored by a physician with multiple publications, and the other generated by ChatGPT-4o. Three independent and blinded reviewers-two human and one AI-assessed each manuscript across five domains: clarity and readability, coherence and flow, technical accuracy, depth, and conciseness and precision. Each category was scored on a 10-point scale, and qualitative feedback was collected to highlight specific strengths and weaknesses. Additionally, all reviewers were asked to deduce authorship of the manuscripts.</p><p><strong>Results: </strong>The AI-generated manuscript scored higher in clarity and readability (mean 9.0 vs 7.2), but lower in technical accuracy (mean 6.3 vs 9.3) and depth (mean 5.5 vs 7.5). However, reviewers noted that the AI version lacked depth, critical analysis, and contextual interpretation. All reviewers accurately identified the authorship of each manuscript and tended to rate the version more favorably when it aligned with their own origin (human or AI); i.e., human reviewers assigned higher scores to the human-written manuscript, while the AI reviewer scored the AI-generated manuscript higher.</p><p><strong>Conclusions: </strong>Although AI models can improve some aspects of scientific writing, particularly clarity and readability, they fall short in critical reasoning and contextual understanding. This reinforces the importance of human authorship and oversight in maintaining the critical analysis and scientific accuracy essential for academic publishing. AI may be used as a complementary tool to support, rather than replace, human-led scientific writing.</p>","PeriodicalId":16562,"journal":{"name":"Journal of neurosurgery. Spine","volume":" ","pages":"1-6"},"PeriodicalIF":3.1000,"publicationDate":"2025-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Can artificial intelligence write science? A comparative analysis of human-written and artificial intelligence-generated scientific writings.\",\"authors\":\"Karim Rizwan Nathani, Ali-Muhammad Nathani, Maliya Delawan, Aleeza Safdar, Mohamad Bydon\",\"doi\":\"10.3171/2025.4.SPINE25519\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Objective: </strong>Artificial intelligence (AI) is increasingly capable of academic writing, with large language models such as ChatGPT showing potential to assist or even generate scientific manuscripts. However, concerns remain regarding the quality, reliability, and interpretive capabilities of AI-generated content. The authors' study aimed to compare the quality of a human-written versus an AI-generated scientific manuscript to evaluate the strengths and limitations of AI in the context of academic publishing.</p><p><strong>Methods: </strong>Two manuscripts were developed using identical titles, abstracts, and tables of a simulated analysis: one authored by a physician with multiple publications, and the other generated by ChatGPT-4o. Three independent and blinded reviewers-two human and one AI-assessed each manuscript across five domains: clarity and readability, coherence and flow, technical accuracy, depth, and conciseness and precision. Each category was scored on a 10-point scale, and qualitative feedback was collected to highlight specific strengths and weaknesses. Additionally, all reviewers were asked to deduce authorship of the manuscripts.</p><p><strong>Results: </strong>The AI-generated manuscript scored higher in clarity and readability (mean 9.0 vs 7.2), but lower in technical accuracy (mean 6.3 vs 9.3) and depth (mean 5.5 vs 7.5). However, reviewers noted that the AI version lacked depth, critical analysis, and contextual interpretation. All reviewers accurately identified the authorship of each manuscript and tended to rate the version more favorably when it aligned with their own origin (human or AI); i.e., human reviewers assigned higher scores to the human-written manuscript, while the AI reviewer scored the AI-generated manuscript higher.</p><p><strong>Conclusions: </strong>Although AI models can improve some aspects of scientific writing, particularly clarity and readability, they fall short in critical reasoning and contextual understanding. This reinforces the importance of human authorship and oversight in maintaining the critical analysis and scientific accuracy essential for academic publishing. AI may be used as a complementary tool to support, rather than replace, human-led scientific writing.</p>\",\"PeriodicalId\":16562,\"journal\":{\"name\":\"Journal of neurosurgery. Spine\",\"volume\":\" \",\"pages\":\"1-6\"},\"PeriodicalIF\":3.1000,\"publicationDate\":\"2025-08-22\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of neurosurgery. Spine\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.3171/2025.4.SPINE25519\",\"RegionNum\":2,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"CLINICAL NEUROLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of neurosurgery. Spine","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.3171/2025.4.SPINE25519","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"CLINICAL NEUROLOGY","Score":null,"Total":0}
引用次数: 0

摘要

人工智能(AI)在学术写作方面的能力越来越强,ChatGPT等大型语言模型显示出协助甚至生成科学稿件的潜力。然而,关于人工智能生成内容的质量、可靠性和解释能力的担忧仍然存在。作者的研究旨在比较人类撰写的科学手稿和人工智能生成的科学手稿的质量,以评估人工智能在学术出版领域的优势和局限性。方法:使用相同的标题、摘要和模拟分析的表格开发了两份手稿:一份由有多篇出版物的医生撰写,另一份由chatgpt - 40生成。三名独立的盲法审稿人——两名人类审稿人和一名人工智能审稿人——从五个方面评估了每份手稿:清晰度和可读性、连贯性和流畅性、技术准确性、深度、简洁性和精确性。每个类别都以10分制进行评分,并收集定性反馈以突出特定的优势和劣势。此外,所有审稿人都被要求推断手稿的作者。结果:人工智能生成的稿件在清晰度和可读性方面得分较高(平均9.0 vs 7.2),但在技术准确性(平均6.3 vs 9.3)和深度(平均5.5 vs 7.5)方面得分较低。然而,评论者指出,人工智能版本缺乏深度、批判性分析和上下文解释。所有审稿人都能准确地识别出每一份手稿的作者,并且当它与他们自己的来源(人类或人工智能)一致时,他们倾向于对该版本进行更有利的评价;也就是说,人类审稿人给人类写的手稿更高的分数,而人工智能审稿人给人工智能生成的手稿更高的分数。结论:尽管人工智能模型可以提高科学写作的某些方面,特别是清晰度和可读性,但它们在批判性推理和上下文理解方面存在不足。这加强了人类作者和监督的重要性,以保持学术出版所必需的批判性分析和科学准确性。人工智能可以作为辅助工具来支持,而不是取代人类主导的科学写作。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Can artificial intelligence write science? A comparative analysis of human-written and artificial intelligence-generated scientific writings.

Objective: Artificial intelligence (AI) is increasingly capable of academic writing, with large language models such as ChatGPT showing potential to assist or even generate scientific manuscripts. However, concerns remain regarding the quality, reliability, and interpretive capabilities of AI-generated content. The authors' study aimed to compare the quality of a human-written versus an AI-generated scientific manuscript to evaluate the strengths and limitations of AI in the context of academic publishing.

Methods: Two manuscripts were developed using identical titles, abstracts, and tables of a simulated analysis: one authored by a physician with multiple publications, and the other generated by ChatGPT-4o. Three independent and blinded reviewers-two human and one AI-assessed each manuscript across five domains: clarity and readability, coherence and flow, technical accuracy, depth, and conciseness and precision. Each category was scored on a 10-point scale, and qualitative feedback was collected to highlight specific strengths and weaknesses. Additionally, all reviewers were asked to deduce authorship of the manuscripts.

Results: The AI-generated manuscript scored higher in clarity and readability (mean 9.0 vs 7.2), but lower in technical accuracy (mean 6.3 vs 9.3) and depth (mean 5.5 vs 7.5). However, reviewers noted that the AI version lacked depth, critical analysis, and contextual interpretation. All reviewers accurately identified the authorship of each manuscript and tended to rate the version more favorably when it aligned with their own origin (human or AI); i.e., human reviewers assigned higher scores to the human-written manuscript, while the AI reviewer scored the AI-generated manuscript higher.

Conclusions: Although AI models can improve some aspects of scientific writing, particularly clarity and readability, they fall short in critical reasoning and contextual understanding. This reinforces the importance of human authorship and oversight in maintaining the critical analysis and scientific accuracy essential for academic publishing. AI may be used as a complementary tool to support, rather than replace, human-led scientific writing.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Journal of neurosurgery. Spine
Journal of neurosurgery. Spine 医学-临床神经学
CiteScore
5.10
自引率
10.70%
发文量
396
审稿时长
6 months
期刊介绍: Primarily publish original works in neurosurgery but also include studies in clinical neurophysiology, organic neurology, ophthalmology, radiology, pathology, and molecular biology.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信