RUGRAT: Runtime Test Case Generation Using Dynamic Compilers

The testing of error handling and dynamic security mechanisms often depends on reproducing specific conditions outside the realm of an application's normal program state. We present RUGRAT, a novel technique to automatically generate tests for these challenging test situations. RUGRAT uses a dynamic compiler to add instructions to the program during execution, and thus dynamically generates tests to exercise code designed to handle uncommon situations during program execution. The RUGRAT testing approach is independent of the source language, requires no modification to the source orbinary program under test and generates runtime tests automatically based on a simple test specification. We demonstrate RUGRAT's capabilities by targeting two particular uncommon situations: handling errors from system and application calls, and testing security mechanisms that protect a program against attacks on function pointers. Both code coverage and failure detection results indicate that RUGRAT is a cost effective approach that reduces the number of required test inputs and need for vulnerable programs.