Industrial real-time regression testing and analysis using firewalls

Abstract

Industrial real-time systems are complex and need to be thoroughly tested before being released to the customer. We have found that last minute changes are often responsible for the introduction of defects, causing serious problems for the customer. We demonstrate that these defects can be introduced into real-time software in diverse ways, and there is no simple regression testing method that can deal with all of these defect sources. This paper describes the application of a testing firewall for regression testing whose form differs depending upon the defect. The idea of the testing firewall is to limit the regression testing to those potentially affected system elements directly dependent upon changed system elements, and then to thoroughly test these elements. This has resulted in substantial savings in regression testing costs, and yet has been effective in detecting critical defects with significant implication in terms of customer acceptance at ABB. Empirical studies are reported for these experiences in an industrial setting.