{"title":"人工智能在皮肤科考试中的表现：ChatGPT的考试成功与限制","authors":"Neşe Göçer Gürok, Savaş Öztürk","doi":"10.1111/jocd.70244","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Background</h3>\n \n <p>Artificial intelligence holds significant potential in dermatology.</p>\n </section>\n \n <section>\n \n <h3> Objectives</h3>\n \n <p>This study aimed to explore the potential and limitations of artificial intelligence applications in dermatology education by evaluating ChatGPT's performance on questions from the dermatology residency exam.</p>\n </section>\n \n <section>\n \n <h3> Method</h3>\n \n <p>In this study, the dermatology residency exam results for ChatGPT versions 3.5 and 4.0 were compared with those of resident doctors across various seniority levels. Dermatology resident doctors were categorized into four seniority levels based on their education, and a total of 100 questions—25 multiple-choice questions for each seniority level—were included in the exam. The same questions were also administered to ChatGPT versions 3.5 and 4.0, and the scores were analyzed statistically.</p>\n </section>\n \n <section>\n \n <h3> Results</h3>\n \n <p>ChatGPT 3.5 performed poorly, especially when compared to senior residents. Second (<i>p</i> = 0.038), third (<i>p</i> = 0.041), and fourth-year senior resident physicians (<i>p</i> = 0.020) scored significantly higher than ChatGPT 3.5. ChatGPT 4.0 showed similar performance compared to first- and third-year senior resident physicians, but performed worse in comparison to second (<i>p</i> = 0.037) and fourth-year senior resident physicians (<i>p</i> = 0.029). Both versions scored lower as seniority and exam difficulty increased. ChatGPT 3.5 passed the first and second-year exams but failed the third and fourth-year exams. ChatGPT 4.0 passed the first, second, and third-year exams but failed the fourth-year exam. These findings suggest that ChatGPT was not on par with senior resident physicians, particularly on topics requiring advanced knowledge; however, version 4.0 proved to be more effective than version 3.5.</p>\n </section>\n \n <section>\n \n <h3> Conclusion</h3>\n \n <p>In the future, as ChatGPT's language support and knowledge of medicine improve, it can be used more effectively in educational processes.</p>\n </section>\n </div>","PeriodicalId":15546,"journal":{"name":"Journal of Cosmetic Dermatology","volume":"24 5","pages":""},"PeriodicalIF":2.3000,"publicationDate":"2025-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/jocd.70244","citationCount":"0","resultStr":"{\"title\":\"The Performance of AI in Dermatology Exams: The Exam Success and Limits of ChatGPT\",\"authors\":\"Neşe Göçer Gürok, Savaş Öztürk\",\"doi\":\"10.1111/jocd.70244\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div>\\n \\n \\n <section>\\n \\n <h3> Background</h3>\\n \\n <p>Artificial intelligence holds significant potential in dermatology.</p>\\n </section>\\n \\n <section>\\n \\n <h3> Objectives</h3>\\n \\n <p>This study aimed to explore the potential and limitations of artificial intelligence applications in dermatology education by evaluating ChatGPT's performance on questions from the dermatology residency exam.</p>\\n </section>\\n \\n <section>\\n \\n <h3> Method</h3>\\n \\n <p>In this study, the dermatology residency exam results for ChatGPT versions 3.5 and 4.0 were compared with those of resident doctors across various seniority levels. Dermatology resident doctors were categorized into four seniority levels based on their education, and a total of 100 questions—25 multiple-choice questions for each seniority level—were included in the exam. The same questions were also administered to ChatGPT versions 3.5 and 4.0, and the scores were analyzed statistically.</p>\\n </section>\\n \\n <section>\\n \\n <h3> Results</h3>\\n \\n <p>ChatGPT 3.5 performed poorly, especially when compared to senior residents. Second (<i>p</i> = 0.038), third (<i>p</i> = 0.041), and fourth-year senior resident physicians (<i>p</i> = 0.020) scored significantly higher than ChatGPT 3.5. ChatGPT 4.0 showed similar performance compared to first- and third-year senior resident physicians, but performed worse in comparison to second (<i>p</i> = 0.037) and fourth-year senior resident physicians (<i>p</i> = 0.029). Both versions scored lower as seniority and exam difficulty increased. ChatGPT 3.5 passed the first and second-year exams but failed the third and fourth-year exams. ChatGPT 4.0 passed the first, second, and third-year exams but failed the fourth-year exam. These findings suggest that ChatGPT was not on par with senior resident physicians, particularly on topics requiring advanced knowledge; however, version 4.0 proved to be more effective than version 3.5.</p>\\n </section>\\n \\n <section>\\n \\n <h3> Conclusion</h3>\\n \\n <p>In the future, as ChatGPT's language support and knowledge of medicine improve, it can be used more effectively in educational processes.</p>\\n </section>\\n </div>\",\"PeriodicalId\":15546,\"journal\":{\"name\":\"Journal of Cosmetic Dermatology\",\"volume\":\"24 5\",\"pages\":\"\"},\"PeriodicalIF\":2.3000,\"publicationDate\":\"2025-05-19\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://onlinelibrary.wiley.com/doi/epdf/10.1111/jocd.70244\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Cosmetic Dermatology\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1111/jocd.70244\",\"RegionNum\":4,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"DERMATOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Cosmetic Dermatology","FirstCategoryId":"3","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/jocd.70244","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"DERMATOLOGY","Score":null,"Total":0}
The Performance of AI in Dermatology Exams: The Exam Success and Limits of ChatGPT
Background
Artificial intelligence holds significant potential in dermatology.
Objectives
This study aimed to explore the potential and limitations of artificial intelligence applications in dermatology education by evaluating ChatGPT's performance on questions from the dermatology residency exam.
Method
In this study, the dermatology residency exam results for ChatGPT versions 3.5 and 4.0 were compared with those of resident doctors across various seniority levels. Dermatology resident doctors were categorized into four seniority levels based on their education, and a total of 100 questions—25 multiple-choice questions for each seniority level—were included in the exam. The same questions were also administered to ChatGPT versions 3.5 and 4.0, and the scores were analyzed statistically.
Results
ChatGPT 3.5 performed poorly, especially when compared to senior residents. Second (p = 0.038), third (p = 0.041), and fourth-year senior resident physicians (p = 0.020) scored significantly higher than ChatGPT 3.5. ChatGPT 4.0 showed similar performance compared to first- and third-year senior resident physicians, but performed worse in comparison to second (p = 0.037) and fourth-year senior resident physicians (p = 0.029). Both versions scored lower as seniority and exam difficulty increased. ChatGPT 3.5 passed the first and second-year exams but failed the third and fourth-year exams. ChatGPT 4.0 passed the first, second, and third-year exams but failed the fourth-year exam. These findings suggest that ChatGPT was not on par with senior resident physicians, particularly on topics requiring advanced knowledge; however, version 4.0 proved to be more effective than version 3.5.
Conclusion
In the future, as ChatGPT's language support and knowledge of medicine improve, it can be used more effectively in educational processes.
期刊介绍：
The Journal of Cosmetic Dermatology publishes high quality, peer-reviewed articles on all aspects of cosmetic dermatology with the aim to foster the highest standards of patient care in cosmetic dermatology. Published quarterly, the Journal of Cosmetic Dermatology facilitates continuing professional development and provides a forum for the exchange of scientific research and innovative techniques.
The scope of coverage includes, but will not be limited to: healthy skin; skin maintenance; ageing skin; photodamage and photoprotection; rejuvenation; biochemistry, endocrinology and neuroimmunology of healthy skin; imaging; skin measurement; quality of life; skin types; sensitive skin; rosacea and acne; sebum; sweat; fat; phlebology; hair conservation, restoration and removal; nails and nail surgery; pigment; psychological and medicolegal issues; retinoids; cosmetic chemistry; dermopharmacy; cosmeceuticals; toiletries; striae; cellulite; cosmetic dermatological surgery; blepharoplasty; liposuction; surgical complications; botulinum; fillers, peels and dermabrasion; local and tumescent anaesthesia; electrosurgery; lasers, including laser physics, laser research and safety, vascular lasers, pigment lasers, hair removal lasers, tattoo removal lasers, resurfacing lasers, dermal remodelling lasers and laser complications.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
