Improving configurable software testing with statistical test selection

Abstract

In software development of configurable systems, effective software testing is essential. As high configurability entails high testing effort, effective testing is required to meet cost constraints while ensuring satisfactory end product quality. In this paper, we present an initial study on the potential of using statistical testing techniques for improving the efficiency of test selection for configurable software. The study aims to answer whether statistical testing can reduce the effort of localizing the most critical software faults, seen from user perspective. We use probabilistic transition system as the formalism for modeling a probabilistic behavior of the system, and usage models to detect faults that would impact users the most. We present an exemplary case study used to assess the performance of statistical test selection compared to manual practice.