A bioinspired in-materia analog photoelectronic reservoir computing for human action processing

• Published in: Nature communications Vol. 16; no. 1; pp. 2263 - 11
• Main Authors: Cui, Hangyuan, Xiao, Yu, Yang, Yang, Pei, Mengjiao, Ke, Shuo, Fang, Xiao, Qiao, Lesheng, Shi, Kailu, Long, Haotian, Xu, Weigao, Cai, Pingqiang, Lin, Peng, Shi, Yi, Wan, Qing, Wan, Changjin
• Format: Journal Article
• Language: English
• Published: London Nature Publishing Group UK 06.03.2025; Nature Publishing Group; Nature Portfolio
• Subjects: 639/301/1005/1007; 639/301/1005/1008; Algorithms; Arrays; Biomimetics - methods; Coding; Computer vision; Computing costs; Energy consumption; Energy efficiency; Feature extraction; Humanities and Social Sciences; Humans; Indium gallium zinc oxide; Memristors; Motion perception; multidisciplinary; Receptive field; Science; Science (multidisciplinary); Transistors; Transistors, Electronic; Visual system
• ISSN: 2041-1723; 2041-1723
• DOI: 10.1038/s41467-025-56899-3

Current computer vision is data-intensive and faces bottlenecks in shrinking computational costs. Incorporating physics into a bioinspired visual system is promising to offer unprecedented energy efficiency, while the mismatch between physical dynamics and bioinspired algorithms makes the processing of real-world samples rather challenging. Here, we report a bioinspired in-materia analogue photoelectronic reservoir computing for dynamic vision processing. Such system is built based on InGaZnO photoelectronic synaptic transistors as the reservoir and a TaO X -based memristor array as the output layer. A receptive field inspired encoding scheme is implemented, simplifying the feature extraction process. High recognition accuracies (>90%) on four motion recognition datasets are achieved based on such system. Furthermore, falling behaviors recognition is also verified by our system with low energy consumption for processing per action (~45.78 μJ) which outperforms most previous reports on human action processing. Our results are of profound potential for advancing computer vision based on neuromorphic electronics. Cui et al. report analogue photoelectronic reservoir computing with IGZO synaptic transistors as the reservoir and TaOX-based memristor array as the output layer. A receptive field inspired encoding scheme is implemented to simplify the feature extraction for dynamic vision processing.