Cheating video description based on sequences of gestures

Abstract

Cheating during exams is a problem in the field of education. Cheating during exams undermine the efforts to evaluate the student's proficiency and growth. We propose a real-timecheating detection system using video feed that allows the ability to monitor students during written exams for any illegal behaviors and gestures, such as giving codes, looking at friends, using a cheatsheet, talking and exchanging papers between students. The gestures recognized during the runtime of the video from sequences of actions performed by the subjects which are then used to generate textual descriptions based on the detected cheating gestures. These textual descriptions help the process of documenting activities that transpired during the exams for later use. Our proposed system comprises two primary subsystems, a gesture recognition model based on 3DCNN and XGBoost and a language generation model based on an LSTM network. The gesture recognition model achieves recognition of the cheating gestures with 81.11% accuracy and Kappa statistic 0.760. The language generation model achieves 95.3 % word accuracy and average edit distance 1.076 on single subject description sentences, and 96.6% word accuracy and average edit distance 3.305 on interaction description sentences. The system runs at 32.54 fps on a mid-range laptop.