A New Handwritten Essay Dataset for Automatic Essay Scoring with A New Benchmark

Abstract

The study of algorithms for Automatic Essay Scoring (AES) currently is motivated by textual essay-scoring datasets constructed by anonymous teachers from schools. We propose VisEssay, the first essay-scoring dataset containing handwriting images. VisEssay consists of over 13,000 visual essays originating from 25+ professional in-service teachers whose personal scoring accuracy are recorded by his/her scoring history, together with crowdsourced OCR result per handwriting image. VisEssay differs from the many existing AES datasets because 1) handwriting images are captured from non-native speakers with complementary essay types for existing datasets, 2) teachers scoring these essays are with personal profiles and score accuracy, and 3) corresponding text is checked to keep the consistency. Evaluation of modern algorithms for AES and text classification reveals that the proposed VisEssay is a challenging dataset. In the cause of encouraging a larger community to develop more generalized educational algorithms, we introduce three novel AES systems together with VisEssay and analysis the result as a new benchmark.