Consistency of System Test Case Results for Software Capability Rating

Abstract

This paper discusses an approach for software capability rating for software under test by assessing the stability of features in the software. This is done by using the consistency of the results from the test cases executed based on test iterations imposed to the software. The focus is on how consistent any test case obtained PASS result consecutively. The approach is based on few parameters: total test iteration, result of test per iteration, total number of pass results per total iteration, result of last iteration and number of change in test result, which is from PASS to FAIL and vice-versa. The result produced consistency pass percentage per test case, which in total is translated into average consistency. The average consistency represented the capability rating for the software to deliver its intended features demonstrating specific degree of confidence of how stable the software features are, based on the rating obtained. Indirectly, it serves as early indicator on the capability of the software as a whole when it operates in real operating environment later.