Discovering Cryptographic Algorithms in Binary Code Through Loop Enumeration

Abstract

In benign programs, encryption is used to prevent sensitive data from being exposed. Malware, on the other hand, uses encryption to hide from analysis or perform malicious activities, e.g. ransomware. The challenge in detecting the presence of these cryptographic algorithms lies in the fact that it is generally not possible to identify the entire functionality of binary programs through static analysis. In this paper we present a novel approach for detecting specific cryptographic algorithms through control flow analysis based on symbolic execution. The control flow graph generated and symbolic execution done by the angr framework is used to search for loops. Nodes that are executed a certain number of times and in a specific order let us point out possible cryptographic activities. In the proof-of-concept implementation we were able to identify and differentiate DES, TripleDES and several variants of the AES algorithm. Our solution is able to detect the presence of these algorithms without access to the source code of the program. It also eliminates the need for a skilled operator to perform the analysis.