Profiting from Unit Tests for Integration Testing

In practice, integration testing typically focuses on a small selection of components or subsystems to integrate and test. This reduces the effort required to create test cases and test environments. However, many defects are only detected when performing integration testing on all possible integrations. These defects are typically only detected later in the development process and lead to increased testing and fault localization efforts. By describing and operationalizing knowledge of such defects, we are able to (semi-)automatically detect them in integration testing. Our OUTFIT tool targets superfluous or missing functionality and untested exception/fault handling in Matlab Simulink models and generated code. It re-uses existing, or automatically generates, high coverage test cases to measure coverage in an opportunistically assembled integration of components or subsystems. A manual inspection of the coverage results then reveals missing or potentially superfluous behavior and thus reveals defects of the targeted kind. Used in a bottom-up integration testing strategy, OUTFIT front loads the detection of such defects, and reduces the fault localization effort. We evaluate OUTFIT using three components of a real-world electrical engine control system of a hybrid car. We find that the results are reproducible, effective and efficiently produced. The achieved coverage of the results is reproducible for 10 executions within a small standard deviation. OUTFIT is effective in finding a potential defect, and efficiently analyzes all evaluated components within a worst case execution time of 110 minutes.