X-BaD: A Flexible Tool for Explanation-Based Bias Detection

Abstract

As widely known, machine learning has been thriving during the last two decades on the strength of two key factors: significant and continuous improvements in hardware performance and the possibility to produce large datasets through automated procedures. However, it has been shown that datasets often contain biases that can significantly affect the performance and resilience of machine learning models, e.g., when deployed to realize functionality for cyber-physical systems. For this reason, a lot of research has been devoted to methodologies and tools for detecting biases in the dataset.This paper presents X-BaD, a tool for bias detection designed to inject and discover biases in a neural network. It is implemented as an open-source Python library that extends the Spectral Relevance Analysis methodology. It allows data reusability and user customization by parameter configurations, and offers built-in functions to inject artificial biases into popular image datasets such as CIFAR-10, Pascal VOC, and ImageNet, for test purposes. This tool is compatible and extensible with features that are commonly used in machine learning frameworks, such as PyTorch and Pytorch Lightning datasets and models, Captum attributions, and Sci-kit Learn clustering algorithms and clustering performance evaluation methods. It also includes functions to interpret and assess the processed data. A set of experiments is finally presented to evaluate the effectiveness of the proposed tool.