Achieving Appropriate Test Coverage for Reliable Measurement-Based Timing Analysis

Abstract

Establishing Worst Case Execution Times (WCET) using Measurement-Based Timing Analysis (MBTA) is only effective if we have reasonable confidence that we have fed the worst case execution trace into the analysis. Therefore for certification, the quality of these traces is of paramount importance. This paper aims to investigate how search algorithms can be used to automatically, and reliably, generate test cases so that appropriate execution traces are available to support MBTA. The work carried out in this paper uses a standard search algorithm and created a number of fitness functions to target the generation of 'good data'. The results are then input into a commercial measurement-based WCET analysis tool. The new fitness functions focus on achieving a combination of full branch coverage and a high number of loop counts, or partial path coverage, however are shown to achieve reliable approximations of the WCET particularly when combined with an MBTA tool. The code items used for the analysis included off the shelf benchmarks, as well as industrial safety-critical aircraft engine control software.