On the Selection of Error Model(s) for OS Robustness Evaluation

Abstract

The choice of error model used for robustness evaluation of operating systems (OSs) influences the evaluation run time, implementation complexity, as well as the evaluation precision. In order to find an "effective" error model for OS evaluation, this paper systematically compares the relative effectiveness of three prominent error models, namely bit-flips, data type errors and fuzzing errors using fault injection at the interface between device drivers OS. Bit-flips come with higher costs (time) than the other models, but allow for more detailed results. Fuzzing is cheaper to implement but is found to be less precise. A composite error model is presented where the low cost of fuzzing is combined with the higher level of details of bit-flips, resulting in high precision with moderate setup and execution costs.