Experiences with automated system testing and sleuth

Abstract

Software system testing can be time consuming and tedious. While tool support exists, it does not always improve matters. This paper presents an approach towards tool-assisted software system testing using a variety of tools for automating tedious and time consuming tasks. We also integrated a new test suite generation tool into the system testing process. The paper reports on the experiences with this testing environment from the professional testers' perspective. Testing encompasses a variety of different approaches, including detailed feature-oriented testing, suspicion based investigation, and random testing. It also includes volume, stress, and operational profile testing. We were particularly interested in how well the new test generation tool performed for such widely varying test objectives.