An Instruction-Set Randomization Using Length-Preserving Permutation

Abstract

Traditional instruction randomization using XOR operator or AES encryption is lack of instruction semantics after transforming and is not beneficial to the fast implementation on hardware, and its security only relies on a random key. This paper proposes an Instruction-Set Randomization using Length-preserving Permutation (LP-ISR). This approach can defend against code-injection attacks by transforming the instruction-set using multidimensional-hash instruction tables, instruction permutation tables and a random key. Simultaneously, the instruction semantics has been preserved after transformation, which is suitable to the fast implementation on hardware. In contrast with traditional approaches, the security of LP-ISR depends on the multidimensional-hash instruction tables, the size of instruction-set, the instruction permutation tables and the random key, which extends the security of LP-ISR from a simple key to efficient data structure. Finally, we have implemented LP-ISR using a dynamic binary instrumentation tool Pin and our experimental results demonstrate that our approach is feasible and acceptable.