ChatGPT-enhanced mobile instant messaging in online learning: Effects on student outcomes and perceptions

Abstract

Mobile Instant Messaging is prevalent in online learning discussions but has inherent limitations in fostering higher-order thinking skills and managing information overload. This study investigates the pedagogical integration of ChatGPT within Mobile Instant Messaging platforms to enhance online learning, addressing a gap in artificial intelligence-enhanced educational technologies. In a 16-week randomized controlled trial with 63 graduate students enrolled in an Advanced Digital Learning course, this study examined the efficacy of a ChatGPT-enhanced Mobile Instant Messaging (ChatMIM) on student engagement and higher-order thinking skills development. Participants were randomly assigned to a treatment group (n = 33) using ChatMIM or a control group (n = 30) using traditional Mobile Instant Messaging. A mixed-methods research design incorporated pre- and post-intervention assessments using validated instruments for engagement and higher-order thinking skills, systematic content analysis of discussion logs, and semi-structured interviews grounded in the Technology Acceptance Model. Results showed considerable improvements in the experimental group across behavioral, cognitive, and emotional engagement dimensions. Enhancements were also observed in higher-order thinking skills domains, particularly in critical thinking, problem-solving, and creativity. Qualitative findings indicated favorable perceptions of ChatMIM, with participants reporting enhanced learning performance and strong intentions for future use. This study provides empirical evidence supporting the effectiveness of artificial intelligence-enhanced messaging systems in online learning, specifically in fostering student engagement and higher-order cognitive development. The findings advance understanding of artificial intelligence integration in educational technology through psychological theories of cognitive load and feedback.