Visagreement: Visualizing and Exploring Explanations (Dis)Agreement.

Abstract

The emergence of distinct machine learning explanation methods has leveraged a number of new issues to be investigated. The disagreement problem is one such issue, as there may be scenarios where the output of different explanation methods disagree with each other. Although understanding how often, when, and where explanation methods agree or disagree is important to increase confidence in the explanations, few works have been dedicated to investigating such a problem. In this work, we proposed Visagreement, a visualization tool designed to assist practitioners in investigating the disagreement problem. Visagreement builds upon metrics to quantitatively compare and evaluate explanations, enabling visual resources to uncover where and why methods mostly agree or disagree. The tool is tailored for tabular data with binary classification and focuses on local feature importance methods. In the provided use cases, Visagreement turned out to be effective in revealing, among other phenomena, how disagreements relate to the quality of the explanations and machine learning model accuracy, thus assisting users in deciding where and when to trust explanations. To assess the effectiveness and practical utility of Visagreement, we conducted an evaluation involving four experts. These experts assessed the tool's Effectiveness, Usability, and Impact on Decision-Making. The experts confirm the Visagreement tool's effectiveness and user-friendliness, making it a valuable asset for analyzing and exploring (dis)agreements.