A bioinspired in-materia analog photoelectronic reservoir computing for human action processing

Abstract

Current computer vision is data-intensive and faces bottlenecks in shrinking computational costs. Incorporating physics into a bioinspired visual system is promising to offer unprecedented energy efficiency, while the mismatch between physical dynamics and bioinspired algorithms makes the processing of real-world samples rather challenging. Here, we report a bioinspired in-materia analogue photoelectronic reservoir computing for dynamic vision processing. Such system is built based on InGaZnO photoelectronic synaptic transistors as the reservoir and a TaO_X-based memristor array as the output layer. A receptive field inspired encoding scheme is implemented, simplifying the feature extraction process. High recognition accuracies (>90%) on four motion recognition datasets are achieved based on such system. Furthermore, falling behaviors recognition is also verified by our system with low energy consumption for processing per action (~45.78 μJ) which outperforms most previous reports on human action processing. Our results are of profound potential for advancing computer vision based on neuromorphic electronics.