Natural Language Processing based Automated Essay Scoring with Parameter-Efficient Transformer Approach

Abstract

Existing automated scoring models implement layers of traditional recurrent neural networks to achieve reasonable performance. However, the models provide limited performance due to the limited capacity to encode long-term dependencies. The paper proposed a novel architecture incorporating pioneering language models of the natural language processing community. We leverage pre-trained language models and integrate it with adapter modules, which use a bottle-neck architecture to reduce the number of trainable parameters while delivering excellent performance. We also propose a model by re-purposing the bidirectional attention flow model to detect adversarial essays. The model we put forward achieves state-of-the-art performance on most essay prompts in the Automated Student Assessment Prize data set. We outline the previous methods employed to attempt this task, and show how our model outperforms them.