Approximate leave-one-out error estimation for learning with smooth, strictly convex margin loss functions

Abstract

Leave-one-out (LOO) error estimation is an important statistical tool for assessing generalization performance. A number of papers have focused on LOO error estimation for support vector machines, but little work has focused on LOO error estimation when learning with smooth, convex margin loss functions. We consider the problem of approximating the LOO error estimate in the context of sparse kernel machine learning. We first motivate a general framework for learning sparse kernel machines that involves minimizing a regularized, smooth, strictly convex margin loss. Then we present an approximation of the LOO error for the family of learning algorithms admissible in the general framework. We examine the implications of the approximation and review preliminary experimental results demonstrating the utility of the approach