Header-driven generation of sanity API tests for shared libraries

There are thousands of various software libraries being developed in the modern world - completely new libraries emerge as well as new versions of existing ones regularly appear. Unfortunately, developers of many libraries focus on developing functionality of the library itself but neglect ensuring high quality and backward compatibility of application programming interfaces (APIs) provided by their libraries. The best practice to address these aspects is having an automated regression test suite that can be regularly (e.g., nightly) run against the current development version of the library. Such a test suite would ensure early detection of any regressions in the quality or compatibility of the library. But developing a good test suite can cost significant amount of efforts, which becomes an inhibiting factor for library developers when deciding QA policy. That is why many libraries do not have a test suite at all. This paper discusses an approach for low cost automatic generation of basic tests for shared libraries based on the information automatically extracted from the library header files and additional information about semantics of some library data types. Such tests can call APIs of target libraries with some correct parameters and can detect typical problems like crashes “out-of-the-box”. Using this method significantly lowers the barrier for developing an initial version of library tests, which can be then gradually improved with a more powerful test development framework as resources appear. The method is based on analyzing API signatures and type definitions obtained from the library header files and creating parameter initialization sequences through comparison of target function parameter types with other functions' return values or out-parameters (usually, it is necessary to call some function to get a correct parameter value for another function and the initialization sequence of the necessary function calls can be quite long). The paper also describes the structure of a tool that implements the proposed method for automatic generation of basic tests for Linux shared libraries (for C and C++ languages). Results of practical usage of the tool are also presented.