Ultrafast Time-Compressive CMOS Image Sensors Based on Multitap Charge Modulators for Filming Light-In Flight

Abstract

Ultrafast time-compressive CMOS image sensors based on multitap charge modulators can capture light-in flight using coded exposure masks on the focal plane. Transient images can then be reconstructed using iterative methods or deep learning models. Although the image sensor is based on indirect time-of-flight (ToF) image sensors, the reconstructed images are equivalent to those captured by direct ToF (D-ToF) image sensors. Important design parameters of the image sensor include the pixel block size and the number of taps of the charge modulator. Several constraints regarding the charge transfer of the multitap charge modulator, the hamming distance between exposure codes at adjacent timings, and the minimal time window duration must be considered when designing exposure codes. The influence of these factors on the fidelity of the reconstructed images is analyzed numerically. The results show that a pixel block size of $4\times 4$ is optimal and that four or more taps are required for light detection and ranging (LiDAR) applications when 32 transient images of light-in flight are reconstructed. To demonstrate LiDAR in a scene with multipath interference, two objects were observed through a weakly diffusive sheet. The temporal resolution, as defined by the clock period of the exposure codes, was 1.65 ns. Multiple reflections were reconstructed using an iterative method (TVAL3) and a deep learning model (ADMM-Net). Although the waveforms of optical pulses reconstructed by TVAL3 are distorted, the amplitudes are more accurate. Conversely, although ADMM-Net reconstructs sharper optical pulses, the amplitudes are inaccurate. To achieve the shorter temporal resolution required for time-resolved diffuse optical tomography (DOT) and fluorescence lifetime imaging (FLIm), the feasibility of heterodyne compression was demonstrated through simulation.