Automated Testing of Timeliness : A Case Study

A problem with testing timeliness of real-time applications is the response-time dependency on the execution order of concurrent tasks. Conventional test methods ignore task interleaving and timing and thus do not help determine which execution orders need to be exercised to test temporal correctness. Model based mutation testing has been proposed to generate inputs and determine the execution orders that need to be verified to increase confidence in timeliness. This paper evaluate a mutation-based framework for automated testing of timeliness by applying it on a small control system running on Linux/RTAI. The experiments presented in this paper indicate that mutation-based test cases are more effective than random and stress tests in finding both naturally occurring and randomly seeded timeliness faults.