An integrated computer-aided cognitive task analysis method for tracing cyber-attack analysis processes

As cyber-attacks become more sophisticated, cyber-attack analysts are required to process large amounts of network data and to reason under uncertainty with the aim of detecting cyber-attacks. Capturing and studying the fine-grained analysts' cognitive processes helps researchers gain deep understanding of how they conduct analytical reasoning and elicit their procedure knowledge and experience to further improve their performance. However, it's very challenging to conduct cognitive task analysis studies in cyber-attack analysis. To address the problem, we propose an integrated computer-aided data collection method for cognitive task analysis (CTA) which has three building blocks: a trace representation of the fine-grained cyber-attack analysis process, a computer tool supporting process tracing and a laboratory experiment for collecting traces of analysts' cognitive processes in conducting a cyber-attack analysis task. This CTA method integrates automatic capture and situated self-reports in a novel way to avoiding distracting analysts from their work and adding much extra work load. With IRB approval, we recruited thirteen full-time professional analysts and seventeen doctoral students specialized in cyber security in our experiment. We mainly employ the qualitative data analysis method to analyze the collected traces and analysts' comments. The results of the preliminary trace analysis turn out highly promising.