Single-shot super-resolved fringe projection profilometry (SSSR-FPP): 100,000 frames-per-second 3D imaging with deep learning

Abstract

To reveal the fundamental aspects hidden behind a variety of transient events in mechanics, physics, and biology, the highly desired ability to acquire three-dimensional (3D) images with ultrafast temporal resolution has been long sought. As one of the most commonly employed 3D sensing techniques, fringe projection profilometry (FPP) reconstructs the depth of a scene from stereo images taken with sequentially structured illuminations. However, the imaging speed of current FPP methods is generally capped at several kHz, which is limited by the projector-camera hardware and the number of fringe patterns required for phase retrieval and unwrapping. Here we report a novel learning-based ultrafast 3D imaging technique, termed single-shot super-resolved FPP (SSSR-FPP), which enables ultrafast 3D imaging at 100,000 Hz. SSSR-FPP uses only one pair of low signal-to-noise ratio (SNR), low-resolution, and pixelated fringe patterns as input, while the high-resolution unwrapped phase and fringe orders can be deciphered with a specific trained deep neural network. Our approach exploits the significant speed gain achieved by reducing the imaging window of conventional high-speed cameras, while “regenerating” the lost spatial resolution through deep learning. To demonstrate the high spatio-temporal resolution of SSSR-FPP, we present 3D videography of several transient scenes, including rotating turbofan blades, exploding building blocks, and the reciprocating motion of a steam engine, etc., which were previously challenging or even impossible to capture with conventional methods. Experimental results establish SSSR-FPP as a significant step forward in the field of 3D optical sensing, offering new insights into a broad spectrum of dynamic processes across various scientific disciplines.