Optimizing anomaly detector deployment under evolutionary black-box vulnerability testing

Abstract

This work focuses on testing anomaly detectors from the perspective of a Multi-objective Evolutionary Exploit Generator (EEG). Such a framework provides users of anomaly detection systems two capabilities. Firstly, no knowledge of protected data structures need to be assumed (i.e. the detector is a black-box), where the time, knowledge and availability of tools to perform such an analysis might not be generally available. Secondly, the evolved exploits are then able to demonstrate weaknesses in the ensuing detector parameterization. Therefore, the system administrator can identify the suitable parameters for the effective operation of the detector. EEG is employed against two second generation anomaly detectors, namely pH and pH with schema mask, on four UNIX applications in order to perform a vulnerability assessment and make a comparison between the two detectors.