Searching for the Most Probable Combination of Class Labels Using Etcetera Abduction

Abstract

Many machine perception tasks require a trained model to assign class labels to multiple entities in the same context, e.g., labeling multiple objects in a single photograph. In these tasks, different combinations of labels may be more likely than others, e.g., when co-occurrence biases are considered, such that the most-confident label assigned to an individual object is not always the best choice. In this paper, we propose a new method for combining evidence from multiple class probability distributions to identify the most probable combination of labels in multi-entity contexts. Our method encodes discrete class probability distributions as literals in first-order logic, and uses probability-ranked logical abduction to identify the most likely label combination, incorporating the prior and conditional probabilities of each label. We evaluate our method on two computer vision benchmarks, first for labeling common objects in photographs of everyday contexts, and second for labeling actions of athletes in sports videos. Results indicate significant gains in classifier accuracy over systems that merely select the model's most confident class label.