Exposing model bias in machine learning revisiting the boy who cried wolf in the context of phishing detection

ABSTRACT Grown out of the quest for artificial intelligence (AI), machine learning (ML) is today’s most active field across disciplines with a sharp increase in applications ranging from criminology to fraud detection and to biometrics. ML and statistics both emphasise model estimation/training and thus share the inescapable Type 1 and 2 errors. Extending the concept of statistical errors into the domain of ML, we devise a ground-breaking pH scale-like ratio and intend it as a litmus test indicator of ML model bias completely masked by the popular performance criterion of accuracy. Using publicly available phishing dataset, we conduct experiments on a series of classification models and consequently unravel the significant cost implications of models with varying levels of bias. Based on these results, we recommend practitioners exercise human judgement and match their own risk tolerance profile with the bias ratio associated with each ML model in order to guard against potential unintended adverse effects.