Statistical Learning Models for Japanese Essay Scoring Toward One-shot Learning

Abstract

A lot of studies of automatic essay scoring are conducted using machine learning models. The previous studies show high performance for scoring large scale essays with machine learning models, however, more than hundreds of scored answers are required to train the neural network models. In this paper we discuss the possibility of one-shot learning, that is, using only one model essay as a training sample of a highest score. For this purpose, we apply regression models to estimate essay scores with different embedding models, that are, BERT and bag-of-words based encoding models. In preliminary experiments, feature analyses of one-shot learning with UMAP for the two embedding models reveal that the bag-of-words based model has more potential to score the test essays comparing to the BERT encoding model. Thus, to clarify the performance of the bag-of-words based encoding model, we conduct two experiments: firstly, we evaluate the performance of models to estimate the scores of test essays using 80% of score essays are used as training data; secondly, one-shot learning is applied to the models. The experimental results show that the proposed bag-of-words based encoding model is promising.