Do Exploratory Testers Need Formal Training? An Investigation Using HCI Techniques

Abstract

Exploratory software testing is an activity which can be carried out by both untrained and formally trained testers. We personify the former as Carmen and the latter as George. In this paper, we outline a joint research exercise between industry and academia that contributes to the body of knowledge by (1) proposing a data gathering and processing methodology which leverages HCI techniques to characterise the differences in strategies utilised by Carmen and George when approaching an exploratory testing task, and (2) present the findings of an initial study amongst twenty participants, ten formally trained testers and another ten with no formal training. Our results shed light on the types of strategies used by each type of tester, how they are used, the effectiveness of each type of strategy in terms of finding bugs, and the types of bugs each tester/strategy combination uncovers. We also demonstrate how our methodology can be used to help assemble and manage exploratory testing teams in the real world.