FlexFilt: Towards Flexible Instruction Filtering for Security

As the complexity of software applications increases, there has been a growing demand for intra-process memory isolation. The commercially available intra-process memory isolation mechanisms in modern processors, e.g., Intel’s memory protection keys, trade-off between efficiency and security guarantees. Recently, researchers have tended to leverage the features with low security guarantees for intra-process memory isolation. Subsequently, they have relied on binary scanning and runtime binary rewriting to prevent the execution of unsafe instructions, which improves the security guarantees. Such intra-process memory isolation mechanisms are not the only security solutions that have to prevent the execution of unsafe instructions in untrusted parts of the code. In fact, we identify a similar requirement in a variety of other security solutions. Although binary scanning and runtime binary rewriting approaches can be leveraged to address this requirement, it is challenging to efficiently implement these approaches. In this paper, we propose an efficient and flexible hardware-assisted feature for runtime filtering of user-specified instructions. This flexible feature, called FlexFilt, assists with securing various isolation-based mechanisms. FlexFilt enables the software developer to create up to 16 instruction domains, where each instruction domain can be configured to filter the execution of user-specified instructions. In addition to filtering unprivileged instructions, FlexFilt is capable of filtering privileged instructions. To illustrate the effectiveness of FlexFilt compared to binary scanning approaches, we measure the overhead caused by scanning the JIT compiled code while browsing various webpages. We demonstrate the feasibility of FlexFilt by implementing our design on the RISC-V Rocket core, providing the Linux kernel support for it, and prototyping our full design on an FPGA.