Peer Assessment Rubric Analyzer: An NLP approach to analyzing rubric items for better peer-review

Abstract

Rubrics have long been used to provide a grading process that is fair and adherent to standards. Just as rubrics can help instructors assess a piece of work, they can also help students to do a more effective job of peer assessment. In a peer-review environment, reviewers provide formative feedback following the rubric criteria. High-quality feedback can greatly enhance the learning process. Rubric criteria need to be worded carefully to provide clear instruction and effective guidance. Heretofore, little research has been performed on how rubric text affects rubric feedback. This study focuses on analyzing rubric text to identify whether rubric criteria will induce peer-reviewers to write quality reviews. We have analyzed 408,104 formative feedback comments based on 3,164 rubric criteria using natural language processing techniques with advanced neural network methods. To our knowledge, this is the first attempt to analyze rubric text to improve review comments for the peer-review environment.