Train as you Fight: Evaluating Authentic Cybersecurity Training in Cyber Ranges

Humans can play a decisive role in detecting and mitigating cyber attacks if they possess sufficient cybersecurity skills and knowledge. Realizing this potential requires effective cybersecurity training. Cyber range exercises (CRXs) represent a novel form of cybersecurity training in which trainees can experience realistic cyber attacks in authentic environments. Although evaluation is undeniably essential for any learning environment, it has been widely neglected in CRX research. Addressing this issue, we propose a taxonomy-based framework to facilitate a comprehensive and structured evaluation of CRXs. To demonstrate the applicability and potential of the framework, we instantiate it to evaluate Iceberg CRX, a training we recently developed to improve cybersecurity education at our university. For this matter, we conducted a user study with 50 students to identify both strengths and weaknesses of the CRX.