Design and Implementation of System for URL Signature Construction and Impact Assessment

The attacker’s server plays an important role in sending attack orders and receiving stolen information, particularly in the more recent cyberattacks. Under these circumstances, it is important to use network-based signatures to block malicious communications in order to reduce the damage. However, in addition to blocking malicious communications, signatures are also required not to block benign communications during normal business operations. Therefore, the generation of signatures requires a high level of understanding of the business, and highly depends on individual skills. In addition, in actual operation, it is necessary to test whether the generated signatures do not interfere with benign communications, which results in high operational costs. In this paper, we propose SIGMA, a system that automatically generates signatures to block malicious communication without interfering with benign communication and then automatically evaluates the impact of the signatures. SIGMA automatically extracts the common parts of malware communication destinations by clustering them and generates multiple candidate signatures. After that, SIGMA automatically calculates the impact on normal communication based on business logs, etc., and presents the final signature to the analyst, which has the highest blockability of malicious communication and non-blockability of normal communication. Our objectives with this system are to reduce the human factor in generating the signatures, reduce the cost of the impact evaluation, and support the decision of whether to apply the signatures. In the preliminary evaluation, we showed that SIGMA can automatically generate a set of signatures that detect 100% of suspicious URLs with an over-detection rate of just 0.87%, using the results of 14,238 malware analyses and actual business logs. This result suggests that the cost for generation of signatures and the evaluation of their impact on business operations can be suppressed, which used to be a time-consuming and human-intensive process.