Hand-Crafted and Learned Spatiotemporal Filters to Inform and Track Visual Saliency

This paper describes an event-tracking algorithm based on an unsupervised learning method to follow salient features. By learning spatiotemporal filters using computationally inexpensive distance metrics such as determinant comparisons, we show that salient features are captured by the learned activation prototypes, known as spatiotemporal templates. First, we discuss previous hand-crafted filter methods to capture spike-based data. While spatial and temporal filters are easily crafted for obvious features, hand-crafted filters are not robust and exhaustive templates for detecting events that may not be so obvious. It becomes clear that learning filters is a more diverse, rectifying method in identifying important features while remaining independent from human observations. We then show how spatiotemporal filters are learned through a series of prototype clustering. In order to handle information over time, we propose a series of decision trees in the form of a random forest inspired by lifelong learning. Finally, we conclude promising results on feature tracking, as well as the need for a ground-truth spike-based data-set to validate saliency algorithms.