RCecker

Return-oriented programming (ROP) is the major exploitation technique to hijack control flow in the presence of non-executable page protections. ROP can be prohibited by ensuring that ret targets legal position. One method is to check whether the predecessor of the target of a ret is a call to identify the illegal use of return. Performing check at each ret with low performance overhead is challenging. To reduce the performance overhead, prior proposals check at critical API functions or system calls and rely on the OS to identify these events. The goal of this paper is to mitigate ROP attacks while incurring negligible storage and performance overheads, and without relying on OS support. This paper proposes a hardware mechanism RCecker (Return-Call pair checker) to enforce the backward CFI (control flow integrity). We propose RCecker-S checking at each ret when the target of the ret has been figured out at EX stage. We analyze the cause of the high performance overhead of RCecker-S. We further propose RCecker-R checking only when RAS (Return Address Stack) mispredicts the targets to reduce the performance overhead. However, the attacker can use Spectre-like attack to pollute RAS and bypass the check of RCecker-R. We propose RCecker-spec based on RCecker-R in addition to check at each speculative ret when the target of the ret has been predicted at the fetch stage. We implement RCecker on RISCV BOOM core and evaluate its security effectiveness and performance overhead. RCecker-spec can successfully detect the ROP attacks in RIPE benchmark. For the SPECINT CPU2006 benchmark, the average performance overhead is 0.69%.