Stack layout transformation: Towards diversity for securing binary programs

Despite protracted efforts by both researchers and practitioners, security vulnerabilities remain in modern software. Artificial diversity is an effective defense against many types of attack, and one form, address-space randomization, has been widely applied. Present artificial diversity implementations are either coarse-grained or require source code. Because of the widespread use of software of unknown provenance, e.g., libraries, where no source code is provided or available, building diversity into the source code is not always possible. I investigate an approach to stack layout transformation that operates on x86 binary programs, which would allow users to obfuscate vulnerabilities and increase their confidence in the software's dependability. The proposed approach is speculative: the stack frame layout for a function is inferred from the binary and assessed by executing the transformed program. Upon assessment failure, the inferred layout is refined in hopes to better reflect the actual function layout.