Code suggestions and explanations in programming learning: Use of ChatGPT and performance

Abstract

This study investigates the role of generative artificial intelligence (AI) chatbots, particularly ChatGPT, in enhancing programming education for university students, specifically in big data analytics. The research addresses the growing need for innovative educational practices, especially in developed East Asian countries like South Korea, where declining university enrollment presents new challenges. Using a sample size of N = 343 students, this mixed-methods research employed controlled experiments and surveys to compare student performance in programming tasks across three groups: those using ChatGPT, those using Stack Overflow, and a control group without external assistance. Results showed that students using ChatGPT significantly outperformed those relying on Stack Overflow or no assistance, particularly in hands-on coding tasks. This research contributes to the ongoing discourse on AI in education by providing empirical evidence of generative AI's effectiveness in improving learning outcomes and engagement, while also highlighting the challenges associated with integrating AI into educational settings. The findings emphasize the potential of ChatGPT to personalize learning experiences, improve performance, and offer real-time support, underscoring the need for a balanced curriculum design that incorporates AI while maintaining academic integrity and human oversight.