Survey and Analysis for the Challenges in Computer Science to the Automation of Grading Systems

Assessment is essential to educational system. Automatic grading reduces the time and effort taken by tutors to assess the answers written by the students. To understand recent computational methods used for automatic grading, a review has been conducted. 4084 papers were initially identified using a keyword search. After filtering, the number was reduced to 57. It was found that statistical models are normally used in Automatic-Short-Answer-Grading (ASAG); vector-based similarity measures are the most popular among projects; pilot datasets are mostly used; standard datasets for evaluation are missing. Evidence shows that machine learning and deep learning are most popularly adopted methods and generative AI, e.g., LLMs and ChatGPT are also jump to the chance, which indicates that integrating AI in education is an inevitable trend. Also, most investigations prefer to adopt multiple approaches to improve computational quality, dataset analysis, and evaluation results. The identified research gaps will be a useful reference guide to users/researchers beneficial to formative/summative assessment. We concluded that the presented outcome, analysis and discussions are informative to academia and pedagogical practitioners who are interested in further developing/using ASAG systems. Although research into ASAG is still rudimentary, it is a promising area with impact on academic circles/commercially educational markets.