Protocol: Machine learning for selecting moderators in meta-analysis: A systematic review of methods and their applications, and an evaluation using data on tutoring interventions

Abstract

Objectives

This is the protocol for a Campbell systematic review. The objectives are as follows: The first objective is to find and describe machine and statistical learning (ML) methods designed for moderator meta-analysis. The second objective is to find and describe applications of such ML methods in moderator meta-analyses of health, medical, and social science interventions. These two parts of the meta-review will primarily involve a systematic review and will be conducted according to guidelines specified by the Campbell Collaboration (MECCIR guidelines). The outcomes will be a list of ML methods that are designed for moderator meta-analysis (first objective), and a description of how (some of) these methods have been applied in the health, medical, and social sciences (second objective). The third objective is to examine how the ML methods identified in the meta-review can help researchers formulate new hypotheses or select among existing ones, and compare the identified methods to one another and to regular meta-regression methods for moderator analysis. To compare the performance of different moderator meta-analysis methods, we will apply the methods to data on tutoring interventions from two systematic reviews of interventions to improve academic achievement for students with or at risk-of academic difficulties, and to an independent test sample of tutoring studies published after the search period in the two reviews.