Automated Indicators to Assess the Creativity of Solutions to Programming Exercises

Abstract

Computer programs are a specific type of knowledge artefacts that result from a creative process under strong formal restrictions. From an educational perspective, it has been argued that programming supports intellectual development and knowledge building. In this paper, we give a short overview of a system created to automatically detect the creativity of solutions to programming exercises that address general mathematical and algorithmic skills. A first step to make these artefacts susceptible to automatic analysis was the definition a descriptive feature set that captures both structural and procedural aspects of each solution. Secondly, machine learning techniques have been used to form higher-level metrics simulating expert judgments on a given set of solutions. It turned out that expert judgments of program creativity differ considerably and systematically. This also led to a classification of the experts.