Investigating ChatGPT-4’s performance in solving physics problems and its potential implications for education

Abstract

Artificial intelligence (AI) technologies have been consistently influencing the progress of education for an extended period, with its impact becoming more significant especially after the launch of ChatGPT-3.5 at the end of November 2022. In the field of physics education, recent research regarding the performance of ChatGPT-3.5 in solving physics problems discovered that its problem-solving abilities were only at the level of novice students, insufficient to cause outstanding alarm in the field of physics education. However, the release of ChatGPT-4 presented substantial improvements in reasoning and conciseness. How does this translate to performance in solving physics problems, and what kind of impact might it have on education? This study undertakes a comprehensive assessment of ChatGPT-4’s performance in solving physics problems from the perspective of physics conceptual understanding and reasoning, and compares its performance with that of students. It is concluded that ChatGPT-4’s performance in solving physics problems has significantly improved compared to ChatGPT-3.5, and was notably superior to the majority of middle school and high school students. This finding presents both a challenge and an opportunity for physics education and the broader educational field, and triggers immediate considerations for coping with this challenge in future teaching and assessment environments.