Combining active learning and relevance vector machines for text classification

Abstract

Relevance vector machines (RVM) have proven successful in many learning tasks. However, in large applications, they scale poorly. In many settings there is a large amount of unlabeled data which could be actively chosen by a learner and integrated in the learning procedure. The idea is to improve performance meanwhile reducing costs from data categorization. In this paper we propose an active learning RVM method based on the kernel trick. The underpinning idea is to define a working space between the relevance vectors (RV) initially obtained in a small labeled data set and the new unlabeled examples, where the most informative instances are chosen. By using kernel distance metrics, such a space can be defined and more informative examples can be added to the training set, increasing performance even though the problem dimension is not significantly affected. We detail the proposed method giving illustrative examples in the Reuters-21578 benchmark. Results show performance improvement and scalability.