Video Time Encoding Machines

• Published in: IEEE transactions on neural networks Vol. 22; no. 3; pp. 461 - 473
• Main Authors: Lazar, A A, Pnevmatikakis, E A
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.03.2011; Institute of Electrical and Electronics Engineers
• Subjects: Algorithms; Applied sciences; Architecture; Artificial Intelligence; Circuits; Computer science; control theory; systems; Connectionism. Neural networks; Decoding; Detection, estimation, filtering, equalization, prediction; Encoding; Exact sciences and technology; Faithful representation; Information, signal and communications theory; Integrated circuit modeling; Nerve Net - physiology; neural circuit architectures; Neural Networks (Computer); Neurons; Pattern Recognition, Automated - methods; Pattern recognition. Digital image processing. Computational geometry; Projection; Recovery; Retina; Signal and communications theory; Signal, noise; Silicon; Software Design; Spikes; spiking neurons; Streaming media; Streams; Telecommunications and information theory; time encoding; Video Recording - methods; Visual Cortex - physiology; Visual Perception - physiology; visual receptive fields; Streaming; Video coding; Input output; Spiking neuron; neural circuit architectures; time encoding; Video signal; visual receptive fields; Set computation; Decoding; Neural network; Faithful representation; Modeling; spiking neurons; Cascade circuit; Input circuit; Visual system; Fires; Filter bank; Movies; Security key
• ISSN: 1045-9227; 1941-0093; 1941-0093
• DOI: 10.1109/TNN.2010.2103323

We investigate architectures for time encoding and time decoding of visual stimuli such as natural and synthetic video streams (movies, animation). The architecture for time encoding is akin to models of the early visual system. It consists of a bank of filters in cascade with single-input multi-output neural circuits. Neuron firing is based on either a threshold-and-fire or an integrate-and-fire spiking mechanism with feedback. We show that analog information is represented by the neural circuits as projections on a set of band-limited functions determined by the spike sequence. Under Nyquist-type and frame conditions, the encoded signal can be recovered from these projections with arbitrary precision. For the video time encoding machine architecture, we demonstrate that band-limited video streams of finite energy can be faithfully recovered from the spike trains and provide a stable algorithm for perfect recovery. The key condition for recovery calls for the number of neurons in the population to be above a threshold value.