Keynote talk #3: Quantifying and extracting visual information from mobile devices

Abstract

Due to advancement in minimization and mass-production, cameras are ubiquitously embedded in most of current mobile devices including phones, vehicles, robots, and augmented-reality displays. These mobile cameras are cheap and can gather in real-time large amounts of streaming data about the surrounding environment. Using an information-theoretic model of a streaming video captured by a mobile camera, we precisely characterize the information rates of this captured data. These results support a holistic approach that combines geometric reconstruction with semantic recognition for visual perception of dynamic environment. We then highlight several work in our group following this approach for extracting visual information from mobile devices including camera pose estimation, 3D environment mapping, and object localization and recognition.