Object retrieval in past video using bag-of-words model

Abstract

Together with the technology advancement, Computer Vision plays an important role in enhancing smart computing systems to help people overcome obstacles in their daily lives. One of the common troublesome problems is human memorization ability, especially memorizing things such as personal items. It is annoying for people to waste their time finding lost items manually by recall or notes. This motivates the authors to propose a solution that can help a user find an item that he or she already saw but vaguely remembers where and when it appeared in the past. The user simply provides our system a single image of that item, then the system retrieves a rank list of visual scenes that may contain the item from video recorded implicitly during user's daily activities. Our method is based on Bag-of-Words model, one of the most famous methods in image retrieval. We first conduct experiments to find the appropriate parameters and configurations of Bag-of-Words system for visual instance search. Then we perform experiments with 110 visual queries of 30 common objects in real video with 2837 shots recorded during daily activities of volunteers. Experimental results show that for all 30/30 categories of objects, our system can help users find their objects of interest just by looking into the top 10 video shots retrieved from recorded video with the balance accuracy from 50 to 80%. This demonstrates the potential use of our method to help people remind of their items in an easy and comfortable way.