Predictive insights into U.S. students’ mathematics performance on PISA 2022 using ensemble tree-based machine learning models

Abstract

Background

In the latest Program for International Student Assessment (PISA) 2022 results, U.S. students earned the lowest math scores in two decades. Educators and stakeholders have endeavored to identify key malleable factors in an attempt to raise scores. Although more researchers are gradually incorporating machine learning (ML) techniques, most still rely on literature reviews by humans to identify important predictors. Here we focus on providing innovative insights into how to use ML models to identify predictors most strongly associated with students’ math performance.

Methods and Results

The dataset comprises 4,552 U.S. students in 154 schools from the PISA 2022. We used three ensemble tree-based ML models (Random Forest, XGBoost, and LightGBM) to select most influential predictors from 143 derived variables of student and school questionnaires. All three models showed high accuracy in predicting students’ math performance, with XGBoost performing best (rMSE = 69.82, training time = 4.14 s) and identifying 10 significant predictors. According to the accumulated local effects (ALEs) plots, three of them have general positive effects, five have roughly negative effects, and two have mixed effects on students’ math performance. When comparing these ML-identified predictors to those identified by literature review, the ML method has significantly improved the accuracy of predictor selection (p-value < 0.05) but offered lower interpretability.

Conclusions

We conclude that ML predictor selection is an effective alternative to LR for obtaining influential factors affecting student learning outcomes. Among the factors identified, math self-efficacy, ESCS, and math anxiety are strongly correlate to students’ math performance. The results provide valuable insights to implement shifts in instructional practices, targeted interventions, curriculum development, and policy decisions, ultimately contributing to enhancing the overall quality of U.S. math education.