On-chip kernel-based spatial filter for CMOS image sensor

Abstract

The Internet of Things is transforming our interactions with the world. CMOS Image Sensors are crucial components in multimedia applications for the Internet of Things. They allow IoT devices to perceive, interpret, and respond to the physical world, improving efficiency, safety, and convenience across various industries. The images captured by these sensors are sent to an external digital device for processing. This data flow can result in high power consumption, limit the frame rate, and restrict the parallelization of algorithm data. This paper introduces a new CMOS Smart Imaging Sensor architecture that performs kernel-based spatial filtering at the pixel level, overcoming the drawbacks above. The design proposal enables spatial filtering using a single kernel, two different kernels on the same image, or two cascaded kernels. The circuit calculates the absolute value of the single convolution in parallel at each pixel using simple arithmetic operations within a neighborhood during photocurrent integration. This process is repeated when the filter requires more than one kernel. We designed a 128 × 128-pixel imager in a $0.35\mu m$ CMOS process and validated it through post-layout simulations. According to these simulations, the kernel-based spatial filter circuit is integrated into the imager, which processes images at frame rates ranging from 746 to 752 fps. Our CMOS implementation outperforms state-of-the-art circuits in denoising, with the best result: a Mean Squared Error of 0.90, Peak Signal Noise Ratio of 48.55 dB, and Structural Similarity Index Metric of 0.99 configured as the mean filter. Additionally, it demonstrates comparable performance in edge detection to state-of-the-art circuits. The best results were achieved with a Mean Squared Error of 4.68, a Peak Signal to Noise Ratio of 41.42 dB, and a Structural Similarity Index Metric of 0.97, using the y-direction Robert filter. Our proposed circuit demonstrated good edge-location accuracy, with a value of 0.99 for the Sobel filter, which is one of the best results according to the Pratt Figure of Merit.