Scale Mean and Variance Differences in MTurk and Non-MTurk Samples

Abstract. We meta-analytically examined mean and variance differences between MTurk and non-MTurk samples for a variety of scales used in the organizational sciences. The influence of moderators (i.e., construct domain and valence, sample type, use of qualifications, and data cleaning procedures) was also examined. Across all scales (120 scales, N = 110,090), we found that, overall, MTurk and non-MTurk samples do not have significantly different scale means or variances. Our moderator analyses, however, indicated that MTurk samples may have larger variances than non-MTurk samples under certain conditions. Additionally, our results highlight the importance of data screening and cleaning procedures such that MTurk samples had larger variances than non-MTurk samples when MTurk samples were subjected to more stringent data screening and cleaning procedures.