Fully integrated multi-mode optoelectronic memristor array for diversified in-sensor computing

In-sensor computing, which integrates sensing, memory and processing functions, has shown substantial potential in artificial vision systems. However, large-scale monolithic integration of in-sensor computing based on emerging devices with complementary metal–oxide–semiconductor (CMOS) circuits remains challenging, lacking functional demonstrations at the hardware level. Here we report a fully integrated 1-kb array with 128 × 8 one-transistor one-optoelectronic memristor (OEM) cells and silicon CMOS circuits, which features configurable multi-mode functionality encompassing three different modes of electronic memristor, dynamic OEM and non-volatile OEM (NV-OEM). These modes are configured by modulating the charge density within the oxygen vacancies via synergistic optical and electrical operations, as confirmed by differential phase-contrast scanning transmission electron microscopy. Using this OEM system, three visual processing tasks are demonstrated: image sensory pre-processing with a recognition accuracy enhanced from 85.7% to 96.1% by the NV-OEM mode, more advanced object tracking with 96.1% accuracy using both dynamic OEM and NV-OEM modes and human motion recognition with a fully OEM-based in-sensor reservoir computing system achieving 91.2% accuracy. A system-level benchmark further shows that it consumes over 20 times less energy than graphics processing units. By monolithically integrating the multi-functional OEMs with Si CMOS, this work provides a cost-effective platform for diverse in-sensor computing applications.