Assessment of Conceptual Design Problems Comprising Design Rationale and Sketches

Abstract

Conventional forms of design assessment are time consuming for instructors. Crowdsourced assessment of students’ design concepts raises the need of efficient rubrics that facilitate novices to score similar to experts, in reduced time. We investigate rubrics in the context of conceptual design problems, that comprise open-ended questions, requiring students to express their design concepts and supporting rationale using text and sketches. We conducted exploratory post-hoc analysis on assessment data collected by instructors of a Design program at a Singaporean secondary school. Our results suggest that integrated rubrics — that consider both text and sketch component together — are better suited for the assessment of conceptual design problems, than task-specific rubrics, that consider textual and sketch components separately. Evidence from both novice assessors as well as experts suggests that the articulation of design rationale using text is crucial for the assessment of conceptual design problems as it provides assessors input into why design decisions were taken, thus aiding in the evaluation process. Our insights are relevant for developing frameworks that employ crowdsourcing for the assessment of conceptual design problems.