Customer Validation of Commercial Predictive Models

Abstract

A central need in the emerging business of model-based prediction is to enable customers to validate the accuracy of a predictive product. This paper discusses how analysts can evaluate data mining models and their inferences from the customer viewpoint, where the customer is not particularly knowledgeable in data mining. To date, academia has focused primarily on the validation of algorithms through mathematical metrics and benchmarking studies. This type of validation is not sufficient in the business context, where organizations must validate specific models in terms that customers can understand quickly and effortlessly. We describe our predictive business and our customer validation needs. To that end, we discuss examples of customer needs, review issues associated with model validation, and point out how academic research may help to address these business needs.