Usability evaluation of software debugging tools

Abstract

Testing and debugging are key tasks during the software development cycle. Spectrum-Based Fault Localization (SFL) is a consolidated debugging technique due to it is relative low execution cost. SFL pinpoints the most suspicious program elements by ranking lines, methods, classes and packages that are more likely to contain faults. Recently, SFL tools have been proposed to help developers during debugging. These tools use different metaphors to represent the suspiciousness of program elements. In this paper, we compare two SFL tools that utilize different metaphors: Jaguar and CodeForest. Jaguar uses a textual representation, presenting the most suspicious elements of a program as a list sorted by suspiciousness. CodeForest uses three-dimensional visualization metaphor, presenting a program as a cacti forest in which basic blocks are represented as thorns, methods as branches, and classes as cacti. We present the results of an evaluation with 76 students using both tools. The perception of usability of the tools was assessed using a questionnaire based on the Technology Acceptance Model (TAM). Three factors were considered to measure the impact of use of the tools in the debugging activity: intention of use, usefulness, and ease of use. The results suggest that there is not statistical difference in the perception of usability between CodeForest and Jaguar.