Calibrated Peer Reviews in Software Engineering Instruction: Experiences and Application

Abstract

One of the key aspects of software engineering is to ensure the developed software meets stakeholder expectations in terms of functional requirements as well as the apparent quality. But achieving high software quality is not just a question of conducting thorough requirements engineering and quality assurance, it is also a question of doing so meticulously and with the necessary level of attention to detail. Yet, software engineering education is mostly concerned with instructing technical knowledge (e.g., unit testing) and organizational soft skills (e.g., working in teams). Teaching the learner proper appreciation of quality, not just in terms of executing techniques, but also in terms of alternative solutions, design trade-offs, advantages, and limitations of engineering decisions is rarely an explicit learning outcome in specialized software engineering courses. We therefore propose the systematic application of calibrated peer reviews as a mechanism to expose the learner to as many different solutions as possible during skill-based knowledge acquisition. In this paper, we discuss in detail the application of calibrated peer reviews in a requirements engineering course and in a quality assurance course. Application results show that using calibrated peer reviews during skill acquisition (i.e., homework assignments) positively affects students’ ability to apply theory and skills in both specification and quality assurance tasks. Our results show further that while calibrated peer reviews have a positive significant effect to apply and retain course theory, the effect on their ability to produce high quality engineering documents is marginally positive and limited to specification tasks.