A ChatGPT-based dynamic assessment chatbot

Research on information tools for learning support is abundant, with many studies integrating natural language analysis and chatbots. However, existing research still struggles to provide differentiated instructional guidance tailored to the varying levels of student understanding. Therefore, this study introduces a chatbot named ChatDAC designed as educational material for university programming classes. The system utilizes a GPT-4 model, the same model as ChatGPT, to progressively provide learners with tiered hints based on categorized response types representing different levels of understanding during practice sessions, rather than solely assessing right or wrong answers conventionally. Analysis of the data revealed a significant increase in average post-test scores compared to pre-test scores for all participants, indicating that ChatDAC effectively enhances students’ programming skills. Additionally, a notable positive correlation was found between the proportion of positive responses and post-test scores. Finally, insights from questionnaires and interviews with students about ChatDAC were gathered, along with suggestions for future improvements.