High - quality decoding of RGB images from the neuronal signals of the pigeon optic tectum

Background

Decoding neural activity to reverse-engineer sensory inputs advances understanding of neural encoding and boosts brain-computer interface and visual prosthesis tech. A major challenge is high-quality RGB image reconstruction from natural scenes, which this study tackles using pigeon optic tectum neurons.

New method

We built a neural response dataset via microelectrode arrays capturing tectal neurons' ON-OFF responses to RGB images. A modular decoding algorithm, integrating a convolutional encoding network, linear decoder, and image enhancement network, enabled inverse RGB image reconstruction from neural signals.

Results

Experimental results confirmed high-quality RGB image reconstruction by the proposed algorithm. For all test set reconstructions, average metrics were: correlation coefficient (R) of 0.853, structural similarity index (SSIM) of 0.618, peak signal-to-noise ratio (PSNR) of 19.94 dB, and feature similarity index (FSIMc) of 0.801. These results confirm accurate recapitulation of both color and contour details of the original images.

Comparison with existing methods

In terms of key quantitative metrics, the proposed algorithm achieves a significant improvement over traditional linear reconstruction methods, with the correlation coefficient (R) increased by 12.65 %, the structural similarity index (SSIM) increased by 38.92 %, the peak signal-to-noise ratio (PSNR) increased by 12.65 %, and the feature similarity index (FSIMc) increased by 9.28 %.

Conclusions

This research provides a novel technical pathway for high-quality visual neural decoding, with robust experimental metrics validating its effectiveness. It also offers experimental evidence to support investigations into the information processing mechanisms of the avian visual pathway.