Self-regulation of peer feedback quality aspects through different dimensions of experience within prior peer feedback assignments

Abstract

Although peer review is a widely-used pedagogical technique, its value depends upon the quality of the reviews that students produce, and much research remains to be done to systematically study the nature, causes, and consequences of variation in peer review quality. We propose a new framework that conceptualizes five larger dimensions of peer review quality and then present a study that investigated three specific peer review quality constructs in a large dataset and further explored how these constructs change through different types of self-regulation peer reviewing experiences. Peer review data across multiple assignments were analyzed from 2,092 undergraduate students enrolled in one of three offerings of a biology course at a large public research university in the United States. Peer review quality was measured in terms of comment amount, comment accuracy, and rating accuracy; the measures of reviewing experience focused upon self-regulated learning factors such as practice, feedback, others’ modeling, and relative performance. Meta-correlation (for testing reliability, separability, and stability) and meta-regression (as a time-series analysis for testing the relationship of changes across assignments in reviewing quality with experiences as reviewer and reviewee) are used to establish the robustness of effects and meaningful variation of effects across course offerings and assignments. Results showed that there were three meaningful review quality constructs (i.e., were measured reliably, separable, and semi-stable over time). Further, all three showed changes in response to previous reviewer and reviewee experiences, but only feedback helpfulness, in particular, showed effects of all four examined types of self-regluation experiences (practice, feedback, others’ modeling, and relative performance). The findings suggest that instructors can improve review quality by providing comment prompt scaffolds that lead to longer comments as well as by matching authors with similarly performing reviewers.