BiDi screen: a thin, depth-sensing LCD for 3D interaction using light fields

Abstract

An LCD screen is transformed into a BiDi (bidirectional) screen to support 2D multi-touch and walk-up 3D gesture interaction. An image sensor placed a small distance behind an LCD forms a mask-based light-field camera, allowing passive depth estimation. The BiDi screen also supports novel mixed-reality rendering with external light-emitting widgets that light a virtual scene.