Stability of Assessment Using Creativity‐Related Indices

Abstract

The aim of this study is to determine how many raters are required for the design products proposed in this study to obtain stable ratings of creativity over time using the Consensual Assessment Technique (CAT) and to examine the creativity subscales, namely, originality and practicality, and their composite scores in a similar manner. The results revealed that creativity assessment using the CAT was stable over time if the assessment was conducted by about 10 to 15 raters. On the contrary, originality and practicality assessment was found to be stable over time with about three raters. However, when the sum of the originality and practicality scores was used as a composite score, the assessment was found to be unstable. This could be attributed to the negative correlation between the originality and practicality assessments of the products. Further, when the product of the originality and practicality scores or the Euclidean distance within these dimensions was used, stable scores were obtained over time by using about 15 to 20 raters.