Explaining Machine Learning Predictions: A Case Study

Abstract

The growing trends and demands for Artificial Intelligence in various domains due to their excellent performance and generalization ability are known to all. These decisions affect the population in general as they usually deal with sensitive tasks in various fields such as healthcare, education, transportation, etc. Hence, understanding these learned representations would add more descriptive knowledge to better interpret the decisions with the ground truth. The European General Data Protection Regulation reserves the right to receive an explanation against a model producing an automated decision. Understanding the decisions would validate the model behavior, ensure trust, and deal with the risk associated with the model. Upon analyzing the relevant features, we can decide whether the model predictions could be trusted or not in the future. We can further try to reduce the misclassification rate by rectifying the features (of the misclassified instances) if needed. In this way, we can peek into the black-box and gain insight into a model’s prediction, thus understanding the learned representations. In pursuit of this objective, a common approach would be to devise an explanatory model that would explain the predictions made by a model and further analyze those predictions with the ground truth information. We initiated a case study on a diabetes risk prediction dataset by understanding local predictions made by five different Machine Learning models and trying to provide explanations for the misclassified instances.