Automatic Problem Generation for CTF-Style Assessments in IT Forensics Courses

In this experience paper, we present an automated assessment and marking generation framework to create capture-the-flag (CTF) questions in the context of Information Technology (IT) Forensics. This allows educators to generate many randomised Virtual Hard Disk (VHD) and packet capture (PCAP) files with different forensic artefacts for each student suitable for assessment tasks in disk-based and network-based forensic courses, respectively. These files are then inscribed inside quizzes, which are constructively aligned to what students have learned in their lecture and tutorial classes. We replaced our invigilated closed-book end-of-semester exams with these open-book multiple-attempt non-invigilated in-semester quizzes. We also conducted a survey asking students about, how the designed quizzes (1) were aligned with (and covering) the promised course learning outcomes, (2) were run to address academic integrity concerns, and (3) helped students manage their stress once their final exams are replaced by the presented quizzes.