Secure and efficient in-process monitor (and library) protection with Intel MPK

Abstract

The process reference monitor is a common technique to enforce security policies for application execution. Reference monitors can be used to detect attacks, enforce access control, check program integrity and even transform program state. Deciding where the monitor resides involves a trade-off between strong monitor isolation and low switching overheads. Running the monitor in the same address space as the protected/traced application (in-process monitors) allows for low overhead but raises isolation concerns. Thus, existing work place monitors in a separate address space, which leads to expensive monitor invocation latencies. We present MonGuard, a system in which a high-performance in-process monitor is efficiently isolated from the rest of the application. To that aim, we leverage the Intel Memory Protection Key (MPK) technology to enforce execute-only memory, combined with code randomization to protect and hide the monitor. MonGuard instruments around sensitive instructions to further prevent possible code reuse attacks. The carefully constructed monitor call gate switches the monitor memory permission in a context-sensitive way. We have built a prototype of MonGuard mostly as a loader extension and implemented a multi-variant execution (MVX) monitor. The evaluation shows MonGuard performs faster than the out-of-process monitor approach.