Understanding prediction discrepancies in classification

Abstract

A multitude of classifiers can be trained on the same data to achieve similar performances during test time while having learned significantly different classification patterns. When selecting a classifier, the machine learning practitioner has no understanding on the differences between models, their limits, where they agree and where they don’t. But this choice will result in concrete consequences for instances to be classified in the discrepancy zone, since the final decision will be based on the selected classification pattern. Besides the arbitrary nature of the result, a bad choice could have further negative consequences such as loss of opportunity or lack of fairness. This paper proposes to address this question by analyzing the prediction discrepancies in a pool of best-performing models trained on the same data. A model-agnostic algorithm, DIG, is proposed to capture and explain discrepancies locally in tabular datasets, to enable the practitioner to make the best educated decision when selecting a model by anticipating its potential undesired consequences.