Peer-assessment and grading of open answers in a web-based e-learning setting

Abstract

Grading the answers given to open ended questions (and questionnaires) is a rather heavy task for teachers and lecturers. Several techniques have been used to automatically analyze the student's essays, but natural language processing is not yet ready to completely solve the task. We propose a solution, for the automated support to answers grading, where the students' peer assessment is checked against the teacher's marking, while such teacher's marking has to be performed on only a subset of the answers (so relieving the teacher by at least a part of the marking task). Our aim is twofold: 1) to make grading more effective by propagating the teacher's assessments through the network of peer assessments (so to maintain a progressively improved evaluations of the peer assessments precision and make them more likely to be used for final grading); 2) to suggest iteratively the teacher with the next most informative essays to grade (respect to the network, that is one of the answers whose teacher's mark will inject most information into the network and make the system better able to deduce the rest of the grades. To this aim we have used a very simple Bayesian model of peers and peer assessment, and we have built a web-based system to support the teacher. We show the present state of the model and its implementation. Moreover, we show, through a simulation protocol, how the system can support the teacher, obtaining a reasonably good correction with just a subset of the whole grades.