Kinsight: Localizing and Tracking Household Objects Using Depth-Camera Sensors

Abstract

We solve the problem of localizing and tracking household objects using a depth-camera sensor network. We design and implement Kin sight that tracks household objects indirectly -- by tracking human figures, and detecting and recognizing objects from human-object interactions. We devise two novel algorithms: (1) Depth Sweep -- that uses depth information to efficiently extract objects from an image, and (2) Context Oriented Object Recognition -- that uses location history and activity context along with an RGB image to recognize object sat home. We thoroughly evaluate Kinsight's performance with a rich set of controlled experiments. We also deploy Kinsightin real-world scenarios and show that it achieves an average localization error of about 13 cm.