Sherlock: A Crowd-sourced System For Automatic Tagging Of Indoor Floor Plans

Having knowledge of the users’ indoor location and the semantics of their environment can facilitate the development of many indoor context-aware applications. For such applications, an accurate indoor map is often needed. While current techniques are capable of producing such maps, these maps are not labeled and hence are of limited utility for many applications. To address this shortcoming, we propose Sherlock, a crowdsourced system for automatically tagging indoor floor plans. Sherlock leverages the myriad of sensors embedded in modern smartphones to intelligently gather audio and visual data, and upload it to the Sherlock Server. At the Sherlock Server, acoustic monitoring and object recognition techniques are used to classify these data samples. The classification scores of current and past samples are then aggregated in a probabilistic framework to determine the confidence with which we can apply as label to a given space. We evaluate Sherlock on a dataset of more than 11,000 audio recordings and 1,200 images, that we collected in three different university campuses. In our evaluation, the confidence for the true label generally outstripped the confidence for all other labels and, in some cases, even reached as high as 100% with as little as 30 data samples.