Virtual Reality in Education: A Stratified Meta-Analysis of Experimental Rigor and Bias

Abstract

Research on VR-based learning is characterized by substantial heterogeneity, inconsistent findings, and uneven methodological quality. While benefits have been reported, they vary considerably across meta-analyses (MAs), raising concerns about methodological quality and making it difficult to draw firm conclusions for research and practice. To critically assess the evidential strength of this literature, we conducted a MA stratified by methodological quality, covering four decades of research (1993–2022) and integrating 26 first-order MAs, 180 primary studies, 281 comparisons between VR and alternative instructional methods, and a total of 18,792 participants. Four MAs were performed: one including all comparisons, and three restricted to studies grouped by assessed risk of bias. When aggregating all available evidence, the results indicated a statistically significant moderate effect favoring VR (Standardized Mean Difference, SMD = 0.55), but also a modest publication bias. Statistically significant effects were observed mainly in studies classified as having a high risk of bias or some concerns (SMDs = 0.38 and 0.43, respectively). In contrast, the subset of studies rated as low risk of bias (representing the highest methodological quality) showed a nonsignificant effect (SMD = 0.35). Analyses restricted to the most rigorous studies therefore suggest both limited statistical power and inconsistent evidence. All analyses showed substantial heterogeneity, largely driven by between-study variance. To reduce methodological confounds, moderator analyses were restricted to studies of moderate to high methodological quality and focused on four theoretically motivated moderators: type of skill, outcome measure, level of VR immersion, and type of alternative instructional method. None of them significantly explained for the observed heterogeneity. Importantly, analyses based on the best-available evidence did not yield consistent or robust support for a reliable learning advantage of VR. Effects reported in the broader literature appear to be driven largely by studies of lower methodological quality, whereas higher-quality studies provide limited and inconsistent evidence. Overall, these findings indicate substantial uncertainty and suggest that current evidence is insufficient to support firm, general conclusions regarding when, how, and for whom VR enhances learning.