Explaining Black-box Predictions by Generating Local Meaningful Perturbations

Generating explanations of predictions made by machine learning models is a difficult task, especially for black-box models. One possible way to explain an individual decision or recommendation for a given instance is to build an interpretable local surrogate for the underlying black-box model in the vicinity of the given instance. This approach has been adopted by many algorithms, for example LIME and LEAFAGE. These algorithms suffer from shortcomings, strict assumptions and prerequisites, which not only limit their applicability but also affect black-box fidelity of their local approximations. We present ways to overcome their shortcomings including the definition of neighborhood, removal of prerequisites and assumption of linearity in local model. The main contribution of this paper is a novel algorithm (LEMP) which provides explanation for the given instance by building a surrogate model using generated perturbations in the neighborhood of the given instance as training data. Experiments show that our approach is more widely applicable and generates interpretable models with better fidelity to the underlying black-box model than previous algorithms.