Multiclass Object Classification Using Ultra-Low Resolution Time-of-Flight Sensors

Time-of-Flight (ToF) sensors are generally used in combination with red–blue–green sensors in image processing for adding the 3-D to 2-D scenes. Because of their low lateral resolution and contrast, they are scarcely used in object detection or classification. In this work, we demonstrate that ultra-low resolution (URL) ToF sensors with 8×8 pixels can be successfully used as stand-alone sensors for multiclass object detection even if combined with machine learning (ML) models, which can be implemented in a very compact and low-power custom circuit. Specifically, addressing an STMicroelectronics VL53L8CX 8×8 pixel ToF sensor, the designed ToF+ML system is capable to classify up to 10 classes with an overall mean accuracy of 90.21%. The resulting hardware architecture, prototyped on an AMD Xilinx Artix-7 field programmable gate array (FPGA), achieves an energy per inference consumption of 65.6 nJ and a power consumption of 1.095 $\mu \text{W}$ at the maximum output data rate of the sensor. These values are lower than the typical energy and power consumption of the sensor, enabling real-time postprocessing of depth images with significantly better performance than the state-of-the-art in the literature.