Thwarting C2 Communication of DGA-Based Malware using Process-level DNS Traffic Tracking

Abstract

Many modern botnet malwares use Domain Generation Algorithms (DGAs) to dynamically generate the domain names that resolve to their command and control (C2) centers. This approach allows these malwares to subvert traditional detection systems which rely on blacklists of known domains associated with malicious activities to block malware communications. Since the advent of DGA-based malwares, the efforts to prevent the said malwares from contacting their command and control centers (C2) server have been centered around detecting Algorithmically Generated Domain Names through lexicographic analysis, isolating entire infected devices or both. Recent research has emerged, which more accurately identifies infected devices in a network, by monitoring the volumes of domain resolution failures. While effective, these techniques are slow to identify DGA generated domain names. Even after the delayed identification, the only preliminary mitigation known today is a complete shutdown of a device that is suspected to be infected. In this paper, we present a new method to counter DGA-based malwares by limiting the impact of mitigation. Instead of isolating the entire infected device from the network we limit network activity of the malicious process alone. Our objective is to prevent DGA-based malwares from communicating with their C2 centers while allowing an infected device to maintain its normal functionality. We achieve this by tracking Domain Name Service (DNS) responses of individual processes and blacklisting those processes for which DNS traffic have abnormally large numbers of domain resolution failures. The blacklisting at a process level ensures that non-malicious processes in the infected device can continue functioning.