Evaluating the academic outcome of AI-powered joint support for at-risk students

Abstract

Despite the growing global adoption of AI-powered academic advice, its impact remains underexplored. This study addresses this gap by evaluating the academic performance of at-risk students who receive AI-driven joint support. We proposed a practical framework for assessing academic outcomes by leveraging AI as a decision-making tool for advisors. Our study employs a quasi-experimental difference-in-differences (DID) methodology over three semesters. We compare the academic outcomes of at-risk students in pilot departments who received the intervention (treatment group, N = 516) with all other students in the analytic sample (control group, N = 12,004), supplemented with robustness checks. The results revealed significant positive outcomes associated with AI-powered joint support, including a reduced proportion of at-risk students and higher grade point average (GPA). Specifically, students guided through the collaborative Center for Student Learning and Development (CSLD) in pilot departments achieved an average GPA increase of 0.4 points compared to their peers who were not supported. This approach not only enhances advisors' efficiency but also provides actionable insights into how AI-powered interventions can help at-risk students overcome academic challenges. Implications for advisors and program managers are discussed, emphasizing the potential for scalable data-driven academic support solutions.