SC-PA: A spot-checking model based on Stackelberg game theory for improving peer assessment

Abstract

During a peer assessment activity in teaching practices, students may not have sufficient motivation to review their peers’ submissions, which makes the peer assessment results questionable. To this end, this paper proposes a novel spot-checking based peer assessment model, named SC-PA, to improve students’ motivation in peer assessment. In SC-PA, a peer assessment activity is modeled as a Stackelberg game, where the teacher acts as the leader and the students are followers. Submissions spot-checked and graded by teachers are treated as review resources, which are allocated among students based on their review reliabilities. In particular, to ensure the utilities of both of the teachers and students, an algorithm that computes the optimal allocation plan for review resources is carefully designed based on Stackelberg equilibrium. Then, different from the classical spot-checking model, the spot-checking probability status of each student which is determined by the allocation plan is shown to the student. Such a difference intuitively puts pressure on students to be spot-checked by teachers, so as to give full play to the role of spot-checking in enhancing students’ motivation in carefully grading their peers’ submissions. To the best of our knowledge, this is the first attempt to treat a spot-checking based peer assessment as a Stackelberg game and utilize the gaming confrontations between teachers and students to improve the students’ motivation in peer assessment. An empirical study involving 271 students from two computer science courses and lasting approximately one year was conducted to investigate the accuracy of peer assessment, as well as the incentive and learning perception of students in peer assessment. Significant findings from the empirical study and a comparison with baselines include: by showing students the spot-checking probabilities, the students’ motivation in carefully grading their peers’ submissions and the fairness of peer assessment are both apparently enhanced; compared with the best baseline, the accuracy of the SC-PA model for peer assessment is improved by 44.1%; although the accuracy of peer assessment can be improved by increasing the number of review resources, the performance gain may become insignificant when the number of resources exceeds a certain limit in real teaching practices; and most students believe that their assessment ability can be effectively improved after multiple peer assessment activities. In conclusion, the SC-PA model successfully enhances students’ motivation for peer assessment, and improves their performance in terms of the assessment accuracy, self-determination, and self-efficacy