Application of Cover's theorem to the evaluation of the performance of CI observers

Abstract

For any N points arbitrarily located in a d-dimensional space, Thomas Cover popularized and augmented a theorem that gives an expression for the number of the 2N possible two-class dichotomies of those points that are separable by a hyperplane. Since separation of two-class dichotomies in d dimensions is a common problem addressed by computational intelligence (CI) decision functions or “observers,” Cover's theorem provides a benchmark against which CI observer performance can be measured. We demonstrate that the performance of a simple perceptron approaches the ideal performance and how a single layer MLP and an SVM fare in comparison. We show how Cover's theorem can be used to develop a procedure for CI parameter optimization and to serve as a descriptor of CI complexity. Both simulated and micro-array genomic data are used.