Ryan Y Sanii, Johnny K Kasto, Wade B Wines, Jared M Mahylis, Stephanie J Muh
{"title":"Utility of Artificial Intelligence in Orthopedic Surgery Literature Review: A Comparative Pilot Study.","authors":"Ryan Y Sanii, Johnny K Kasto, Wade B Wines, Jared M Mahylis, Stephanie J Muh","doi":"10.3928/01477447-20231220-02","DOIUrl":null,"url":null,"abstract":"<p><strong>Objective: </strong>Literature reviews are essential to the scientific process and allow clinician researchers to advance general knowledge. The purpose of this study was to evaluate if the artificial intelligence (AI) programs ChatGPT and Perplexity.AI can perform an orthopedic surgery literature review.</p><p><strong>Materials and methods: </strong>Five different search topics of varying specificity within orthopedic surgery were chosen for each search arm to investigate. A consolidated list of unique articles for each search topic was recorded for the experimental AI search arms and compared with the results of the control arm of two independent reviewers. Articles in the experimental arms were examined by the two independent reviewers for relevancy and validity.</p><p><strong>Results: </strong>ChatGPT was able to identify a total of 61 unique articles. Four articles were not relevant to the search topic and 51 articles were deemed to be fraudulent, resulting in 6 valid articles. Perplexity.AI was able to identify a total of 43 unique articles. Nineteen were not relevant to the search topic but all articles were able to be verified, resulting in 24 valid articles. The control arm was able to identify 132 articles. Success rates for ChatGPT and Perplexity. AI were 4.6% (6 of 132) and 18.2% (24 of 132), respectively.</p><p><strong>Conclusion: </strong>The current iteration of ChatGPT cannot perform a reliable literature review, and Perplexity.AI is only able to perform a limited review of the medical literature. Any utilization of these open AI programs should be done with caution and human quality assurance to promote responsible use and avoid the risk of using fabricated search results. [<i>Orthopedics</i>. 2024;47(3):e125-e130.].</p>","PeriodicalId":19631,"journal":{"name":"Orthopedics","volume":null,"pages":null},"PeriodicalIF":1.1000,"publicationDate":"2024-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Orthopedics","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.3928/01477447-20231220-02","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2023/12/28 0:00:00","PubModel":"Epub","JCR":"Q3","JCRName":"ORTHOPEDICS","Score":null,"Total":0}
引用次数: 0
Abstract
Objective: Literature reviews are essential to the scientific process and allow clinician researchers to advance general knowledge. The purpose of this study was to evaluate if the artificial intelligence (AI) programs ChatGPT and Perplexity.AI can perform an orthopedic surgery literature review.
Materials and methods: Five different search topics of varying specificity within orthopedic surgery were chosen for each search arm to investigate. A consolidated list of unique articles for each search topic was recorded for the experimental AI search arms and compared with the results of the control arm of two independent reviewers. Articles in the experimental arms were examined by the two independent reviewers for relevancy and validity.
Results: ChatGPT was able to identify a total of 61 unique articles. Four articles were not relevant to the search topic and 51 articles were deemed to be fraudulent, resulting in 6 valid articles. Perplexity.AI was able to identify a total of 43 unique articles. Nineteen were not relevant to the search topic but all articles were able to be verified, resulting in 24 valid articles. The control arm was able to identify 132 articles. Success rates for ChatGPT and Perplexity. AI were 4.6% (6 of 132) and 18.2% (24 of 132), respectively.
Conclusion: The current iteration of ChatGPT cannot perform a reliable literature review, and Perplexity.AI is only able to perform a limited review of the medical literature. Any utilization of these open AI programs should be done with caution and human quality assurance to promote responsible use and avoid the risk of using fabricated search results. [Orthopedics. 2024;47(3):e125-e130.].
期刊介绍：
For over 40 years, Orthopedics, a bimonthly peer-reviewed journal, has been the preferred choice of orthopedic surgeons for clinically relevant information on all aspects of adult and pediatric orthopedic surgery and treatment. Edited by Robert D''Ambrosia, MD, Chairman of the Department of Orthopedics at the University of Colorado, Denver, and former President of the American Academy of Orthopaedic Surgeons, as well as an Editorial Board of over 100 international orthopedists, Orthopedics is the source to turn to for guidance in your practice.
The journal offers access to current articles, as well as several years of archived content. Highlights also include Blue Ribbon articles published full text in print and online, as well as Tips & Techniques posted with every issue.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
