The use of Personal Response Systems (PRS) in multiple-choice assessment: benefits and pitfalls over traditional, paper-based approaches

Abstract

Abstract In this paper we discuss our experiences of using wireless keypads for multiple-choice (MCQ) assessment and evaluate the benefits over traditional, paper-based MCQ assessment methods. A Personal Response System (PRS), TurningPoint ®, was trialled for two years for a Geography undergraduate class test, designed to provide both formative and summative assessment. The benefits of PRS are considered in terms of i) student performance; ii) students’ experiences, which were assessed by questionnaires; iii) quality of feedback; and iv) staff time inputs. Of the students surveyed, 74% stated that they strongly preferred PRS over paper-based MCQ assessment, whilst 6% of students disagreed. Students praised the immediacy and graphical nature of feedback associated with PRS, but some criticised the inflexibility of PRS to change answers after a fixed ‘per question’ time interval. Staff time inputs were substantially reduced (ca. 75% less) for PRS than for the equivalent paper-based test. PRS was found to be a useful tool in MCQ assessment, but it has some disadvantages. We highly recommend PRS for formative assessment but caution its use for summative assessment in its current form and recommend that it only be used for assessments carrying a small mark-weighting.