EQISP: efficient quantized image signal processing with multi-scale pyramid fusion for resource constrained embodied perception.

Abstract

Introduction: Resource-constrained environmental perception requires autonomous robots and embodied intelligent systems to process visual signals efficiently while preserving image fidelity in complex real-world environments. However, converting high dynamic range RAW sensor data into perceptually faithful RGB images remains computationally expensive, thereby limiting the deployment of neural image signal processors on edge platforms with restricted memory, energy, and computational budgets.

Methods: Consequently, this study proposes the enhanced quantized image signal processor (EQISP), comprising the quantized convolutional neural network (QCNN) and the unified pyramid fusion algorithm (UPFA). QCNN employs dynamic fixed-point hybrid quantization, which adjusts parameter ranges according to the linear relationship between threshold standard deviation and fractional length, thereby significantly reducing the computational load. Meanwhile, UPFA utilizes Gaussian pyramids to capture global illumination and Laplacian pyramids to preserve fine details, enabling multi-scale, multi-exposure fusion and iterative reconstruction to mitigate detail loss induced by quantization.

Results: Comprehensive comparative experiments demonstrated that EQISP achieved a PSNR of 22.90 dB, an SSIM of 0.9278, and 164.843 GFLOPs. Compared with the PyNET baseline, EQISP improved the PSNR by 1.71 dB while reducing the computational cost by a factor of 4.24. Furthermore, deployment experiments on an NVIDIA Jetson TX2 development board showed that EQISP achieved a model size of 57 MB, an inference latency of 189 ms, an inference speed of 6.1 FPS, and a peak memory usage of 2.2 GB.

Discussion: These results provide practical evidence that EQISP can serve as an efficient and scalable visual front end for resource-constrained embodied perception systems.