An empirical assessment of visual debugging tools effectiveness and efficiency

Debugging is the task of locating and fixing defects in a program. Spectrum-Based Fault Localization (SFL) is a promising debugging technique due to it is relative low execution cost. SFL pinpoints the most suspicious program elements by ranking lines, methods, classes and packages more likely to contain faults. We compare two SFL tools that utilize different metaphors. Jaguar uses a textual metaphor while CodeForest a three-dimensional one. The results of an evaluation with 76 students using both tools are presented. They were assessed regarding to the effectiveness and efficiency during fault localization. The effectiveness and efficiency were assessed, respectively, by the tool’s ability to direct the developer to the faulty method or line and the time spent to locate them. The results show that Jaguar is more effective than CodeForest. However, there is not difference between the tools for efficiency.