Malware detection using HTTP user-agent discrepancy identification

Abstract

Botnet detection systems that use Network Behavioral Analysis (NBA) principle struggle with performance and privacy issues on large-scale networks. Because of that many researchers focus on fast and simple bot detection methods that at the same time use as little information as possible to avoid privacy violations. Next, deep inspections, reverse engineering, clustering and other time consuming approaches are typically unfeasible in large-scale networks. In this paper we present a novel technique that uses User- Agent field contained in the HTTP header, that can be easily obtained from the web proxy logs, to identify malware that uses User-Agents discrepant with the ones actually used by the infected user. We are using statistical information about the usage of the User-Agent of each user together with the usage of particular User-Agent across the whole analyzed network and typically visited domains. Using those statistics we can identify anomalies, which we proved to be caused by malware-infected hosts in the network. Because of our simple and computationally inexpensive approach we can inspect data from extremely large networks with minimal computational costs.