On Dynamic Selection of the Most Informative Samples in Classification Problems

In this paper, we propose a dynamic technique for selecting the most informative samples in classification problems as coming in two stages: the first stage conducts sample selection in batch off-line mode based on unsupervised criteria extracted from cluster partitions, the second phase proposes an active learning scheme during on-line adaptation of classifiers in non-stationary environments. This is based on the reliability of the classifiers in their output responses (confidences in their predictions). Both approaches contribute to a reduction of the annotation effort for operators, as operators only have to label/give feedback on a subset of the off-line/online. At the same time they are able to keep the accuracy on almost the same level as when the classifiers would have been trained on all samples. This will be verified based on real-world data sets from two image classification problems used in on-line surface inspection scenarios.