A framework for detecting anomalies in HTTP traffic using instance-based learning and k-nearest neighbor classification

Attacks against web applications and web-based services that use HTTP as a communication protocol pose a serious threat to today's information technology infrastructures. A common countermeasure is to apply misuse detection and prevention systems that compare the contents of HTTP traffic against signatures of known attacks, as it is for example done by web application firewalls. A serious drawback of these systems is the fact that the used signatures often are not tailored for the individual web applications to be protected. Furthermore, signatures can often be circumvented by rewriting attacks into different forms, resulting in successful exploitation and circumvention of a misuse detection or prevention system. This paper presents the design and implementation of an anomaly detection framework for HTTP traffic that operates without signatures of known attacks. It rather learns normal usage patterns of web-based applications by inspecting full HTTP request and response contents. The results are then used for anomaly detection. The framework automatically adjusts to the applications to be monitored, derives normal usage patterns and compares subsequent HTTP traffic to the built knowledge base.