Falsifiability of network security research: the good, the bad, and the ugly

Abstract

A falsifiability criterion helps us to distinguish between scientific and non-Scientific theories. One may try to raise a question whether this criterion is applicable to the information security research, especially to the intrusion detection and malware research fields. In fact, these research fields seems to fail to satisfy the falsifiability criterion, since they lack the practice of publishing raw experimental data which were used to prove the theories. Existing public datasets like the KDD Cup'99 dataset and VX Heavens virus dataset are outdated. Furthermore, most of current Scientific research projects tend to keep their datasets private. We suggest that the Scientific community should pay more attention to creating and maintaining public open datasets of malware and any kinds of computer attack-related data. But how can we bring this into reality, taking into account legal and privacy concerns?