Promoting online evaluation skills through educational chatbots

Online evaluation skills such as assessing the credibility and relevance of Internet sources are crucial for students' self-regulated learning on the Internet, yet many struggle to identify reliable information online. While AI-based chatbots have made progress in teaching various skills, their application in improving online evaluation skills remains underexplored. In this study, we present an educational chatbot designed to train university students to evaluate online information. Participants were assigned to one of three conditions: (1) training with the interactive chatbot, (2) training with a static checklist, or (3) no additional training (i.e., baseline condition). In an ecologically valid test that provided a simulated web environment, participants had to identify the most reliable and relevant websites among several non-target websites to solve given problems. Participants in the chatbot condition outperformed those in the baseline condition on this test, while participants in the checklist condition showed no significant advantage over the baseline condition. These findings suggest the potential of educational chatbots as effective tools for improving critical evaluation skills. The implications of using chatbots for scalable educational interventions are discussed, particularly in light of recent advances such as the integration of large language models into search engines and the potential for hybrid intelligence paradigms that combine human oversight with AI-driven learning tools.