To what extent does ChatGPT understand genetics?

Abstract

ABSTRACTOpenAI’s ChatGPT, is a conversational chatbot that uses Generative Pre-trained Transformer or GPT language model to mimic human-like responses. Here we evaluated its performance in providing responses to genetics questions across five different tasks including solid genetic basics, identifying inheritance pattern based on described pedigrees, interpreting genetic mutation notations, solving genetic population problems, and taking a medical genetics Ph.D. entrance exam. Our results showed that ChatGPT was able to generate correct answers to approximately 70% of questions (n = 145). Its performance on descriptive and memorisation tasks showed more accuracy compared to analytical and critical thinking ones. Failure to capture human writing nuances in questions, and applying genetic basics to solve problems, alongside providing false information were the most notable drawbacks. However, overall results were promising suggesting that ChatGPT could be a well-prepared assistant for genetic educators and healthcare providers.KEYWORDS: ChatGPTgenerative pre-trained transformergeneticsartificial intelligencelarge language model AcknowledgmentsThis study was supported by the Golestan University of Medical Sciences (Grant Number: 13350).Disclosure statementNo potential conflict of interest was reported by the author(s).Data availability statementNo additional data were created or used in this study beyond what is presented in the manuscript.EthicsThis study was ethically approved by Ethics Committee of Golestan University of Medical Sciences (Ethics Code: IR.GOUMS.REC.1401.522)Supplementary materialSupplemental data for this article can be accessed online at https://doi.org/10.1080/14703297.2023.2258842Additional informationFundingThis work was supported by the Golestan University of Medical Sciences and Health Services [113350].Notes on contributorsTeymoor KhosraviTeymoor Khosravi is a postgraduate student at Golestan university of medical sciences studying human genetics.Zainab M. Al SudaniZainab M. Al Sudani is a medical student at Golestan university of medical sciences.Morteza OladnabiMorteza Oladnabi is an associate professor of medical genetics working at Gorgan congenital malformations research center.