RopSteg: program steganography with return oriented programming

Abstract

Many software obfuscation techniques have been proposed to hide program instructions or logic and to make reverse engineering hard. In this paper, we introduce a new property in software obfuscation, namely program steganography, where certain instructions are "diffused" in others in such a way that they are non-existent until program execution. Program steganography does not raise suspicion in program analysis, and conforms to the W⊕X and mandatory code signing security mechanisms. We further implement RopSteg, a novel software obfuscation system, to provide (to a certain degree) program steganography using return-oriented programming. We apply RopSteg to eight Windows executables and evaluate the program steganography property in the corresponding obfuscated programs. Results show that RopSteg achieves program steganography with a small overhead in program size and execution time. RopSteg is the first attempt of driving return-oriented programming from the "dark side", i.e., using return-oriented programming in a non-attack application. We further discuss limitations of RopSteg in achieving program steganography.