Cross-level behavioral analysis for robust early intrusion detection

Abstract

We anticipate future attacks would evolve to become more sophisticated to outwit existing intrusion detection techniques. Existing anomaly analysis techniques and signature-based detection practices can no longer effective. We believe intrusion detection systems (IDSs) of the future will need to be capable to detect or infer attacks based on more valuable information from the network-related properties and characteristics. We observed that even though the signatures or traffic patterns of future stealthy attacks can be modified to outwit current IDSs, certain behavioral aspects of an attack are invariant. We propose a novel approach that jointly monitors network activities at three different levels: transport layer protocols, (vulnerable) network services, and invariant anomaly behaviors (called attack symptoms). Our system, SecMon, captures the network behaviors by simultaneously performing cross-level state correlation for effective detection of anomaly behaviors. For the most part, the invariant anomaly behavior has not been fully exploited in the past. A probabilistic attack inference model is also proposed for attack assessment by correlating the observed attack symptoms to achieve the low false alarm rate. The evaluations demonstrate our prototype system is efficient and effective for sophisticated attacks, including polymorphism, stealthy, and unknown attack.