An Improved Method for Label Matching in E-Assessment of Diagrams

Abstract

Abstract A challenging problem for e-assessment is automatic marking of diagrams. There are a number of difficulties not least that much of the meaning of a diagram resides in the labels and hence label matching is an important process in the e-assessment of diagrams. Previous research has shown that the labels used by the students in the diagrams can be diverse and imprecise which makes this problematic. In this paper we propose and evaluate a new method for label matching to support e-assessment of diagrams and address problems of synonyms, spelling errors and differing levels of decomposition. We have implemented the syntactic part of our method and evaluated it using 160 undergraduate assessments based upon a UML design task. We have found that our method performs better than the other syntax matching algorithms. This framework has significant implications for the ease in which we may develop future e-assessment systems. The results from this pilot study have been encouraging and motivate us to implement the semantic similarity part of our method and conduct further evaluations.