Autograding of Programming Skills

Abstract

To evaluate a learner’s knowledge of programming language skills, assessments are given. Grading of these is usually done manually which not only is tedious but prone to error due to repetition and fatigue. In this work, we employ pre-trained language models to perform automated grading of "C" programming language. Embeddings from different transformers on pre-assessed codes are used as feature vectors to train a wide range of regressors for the scoring task. Root-mean-square error (RMSE) is the metric utilized to compare the scores of these regressors. It’s observed that embeddings from T5-model with CatBoost regressor gives the least error around 15%.