Learning from Mistakes: Weakly Supervised Learning of Rocks

A standard method for teaching an object detector a new class is to fine-tune it with a fully-supervised image set. The issue with fully-supervised image sets are that they are tedious to create and rely on human annotators. In this work we demonstrate that a new class can be learned by leveraging the ‘mistakes’ of a pre-trained object detector under a weakly supervised learning (WSL) paradigm and removing the need for a fully-supervised image set. Our method iteratively cycles over four stages: observation, filtering, generation, and fine-tuning. The observation stage uses an object detector to gather inferences on an image set containing many instances of the new class. These observed inferences are passed to the filtering stage that keeps only the most frequently observed class inferences. These filtered inferences are used as object-level annotations and are passed to the generation stage. In the generation stage a training set is created by superimposing the image content of the annotations onto a set of background images. The generated training set is then used in the fine-tuning stage to create an updated object detector. We trained an object detector to recognize the novel object class, rock using this method. We compared the average precision obtained by an object detector trained using our method to an object detector trained using the fully-supervised method. We were able to achieve $\sim 78$% of the average precision obtained by the fully-supervised version.