Evaluation methodology for mission-centric cyber situational awareness capabilities

The emerging need for cyber defence capabilities able to bring closer cyberspace supremacy in joint military operations has led defence practitioners to begin a cyber race where academy, researchers, industry and military organizations work together. Because of the higher maturity of civilian technologies for cybersecurity, this often involves adapting capabilities not initially intended for military use to new dual-use requirements, where concepts like operations, missions or Courses of Action (CoAs) shall be presented and aligned with the military doctrine. One of the main 'battle horses' in this transformation is to develop supporting systems able to facilitate the mission-centric acquisition of Cyber Situational Awareness (CSA), where the observations in the cyberspace shall be properly correlated, propagated and understood in the scope of planned/ongoing mission. But despite these CSA needs, there is a wide methodological gap in the lack of suitable validation and verification frameworks, which not only relies on the raising need for capabilities able to verify if the existing solutions meet the requirements to operate on military actions, but also to support the thorough development life-cycle of brand new cyber defence technologies. In these grounds, this research introduces a novel evaluation framework able to guide the evaluation of CSA related tools, for which three core validation concepts are discussed: software, operational and application tests. They cover from the day-to-day implementation of the new capabilities, to their ability of facilitating that human decision-makers acquire a joint operational picture.