Economical training sets for linear ID3 learning

Abstract

Our work is in machine learning, a subfield of artificial intelligence. We describe a variant of Quinlan's ID3 algorithm (1986) which is attuned to the situation that every feature's value-set is linearly ordered and finite. We then seek economical training sets, that is, ones which are small in size but result in learned decision trees of high accuracy. Our search focuses on geometric properties of the target concept, such as its extreme points, edges, faces, and surface. We categorize all concepts into three classes, from simplest to most general, and for each class we identify certain training sets, some quite small, others less so, which result in highly accurate learning of the concepts in that class. Some of our results are rigorously provable (but the proofs do not appear here), for other results our evidence is empirical.<>