Comparative analysis of string similarity and corpus-based similarity for automatic essay scoring system on e-learning gamification

Abstract

Essay assessment within e-learning need to be conducted manually by human expert. This process takes time and costly. Hence, automatic essay scoring is needed. Since the scoring system will be integrated to the e-learning, we need a computationally lightweight method that still does not rule out the accuracy of the assessment. In this paper, we propose an automatic scoring system for essay examination using unsupervised approaches. We compare and analyze two similarity measure methods, cosine similarity and latent semantic analysis. The parameters that was used to measure the performance of the methods are the computational complexity — measured by the amount of CPU and memory usage, and page load time — and accuracy — measured by Pearson Correlation and Mean Absolute Error. The results showed that both algorithm consumed same amount of memory. For CPU usage, LSA consumption is 0.13% and cosine's is 0.06%. For page load time, cosine similarity is faster than LSA which is 0.2 second and 0.5 second consecutively. Based on the correlation measure with Pearson, LSA is more superior to the cosine similarity by 0.59 to 0.49. LSA also has less MAE than cosine similarity which is 5.69 compared to 5.33. From that result, LSA and Cosine Similarity has a very competitive result in accuracy. However, Cosine has a better server performance so that preferred to be implemented in e-learning automatic essay scoring system.