Directed graph as a model for unit testing

Abstract

Unit testing is recognized as an important part of software development. There are several frameworks to facilitate unit testing. All these frameworks provide support for executing various functions in the unit under test. Our analysis of customer found defects indicated that most of the unit test escapes were a result of execution of sequences of API's. None of the frameworks we looked provided explicit support for execution of API sequences. We designed and implemented a unit testing framework where we provided direct support for specification of API sequences. Each unit test is specified as a directed graph. Each edge corresponds to execution of an API and nodes are used for validation. The framework generates various paths through the graph. Various traversal policies like random, all nodes/edges, every adjacent edge pair etc. are supported. We rewrote unit tests for four features using the new framework. Each unit under test is about 2000 lines of code. Directed graph model helped uncover about 6 new bugs/feature. Automatic path generation has helped explore various API sequences that are very difficult to test manually. Code coverage increased on the average by about 6%. The framework is in production use and is able to uncover various API sequence bugs that are not easily uncovered by traditional unit testing tools. After initial training, engineers are eager to model their tests as directed graphs.