FITNESS: a framework for automatic testing of ASTERIX based software systems

As applications are developed, functional tests ensure they continue to function as expected. Nowadays, functional testing is mostly done manually, with human testers verifying a systems functionality themselves, following hand-written instructions: this make testing of software components one of the most expensive phases in the software development cycle, either in terms of time as well as human effort. Concerning in particular safety critical systems, such as the ones belonging to the Air Traffic Management field, for which it is always necessary to be taken complete and rigorous security test and evaluation among development team and/or by third-party security certification organization, performing automatic tests on such systems become a very tricky process considering that the goal is to verify not only the proper functioning of the SUT, but the system dependability too. However, such software testing is usually time consuming, cost consuming and boresome and thus technologies of software testing automation have alluring application foreground in that field: making the execution of test cases automatic allows to reduce costs and to improve software quality from a dependability point of view. In this paper we present FITNESS, a framework for the automation of testing procedures for complex software systems with strict safety and quality requirements, and in particular we have focused on Air Traffic Control (ATC) application who rely on ASTERIX standard as data exchange format with the intent to propose a flexible solution to automate testing procedure for a generic system that use such communication standard. We also present a quantitative study that analyze the effectiveness of the proposed approach using our framework to test a Secondary Surveillance Radar system and showing that most of manual test steps can be automatically converted to automated test steps with no human intervention.