Hardware-Compliant Compressive Image Sensor Architecture Based on Random Modulations and Permutations for Embedded Inference

• Published in: IEEE transactions on circuits and systems. I, Regular papers Vol. 67; no. 4; pp. 1 - 14
• Main Authors: Benjilali, Wissam, Guicquero, William, Jacques, Laurent, Sicard, Gilles
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.04.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Analog circuits; Analog to digital converters; CMOS; Coherence; Coils; Complexity theory; compressive sensing; Computer architecture; Digital signal processing; Digital signal processors; Electronics; embedded object recognition; Embedding; Engineering Sciences; Hardware; Image coding; Image sensor; Incoherence; Inference; Machine Learning; Microprocessors; Mixer circuits; Neural networks; Object recognition; Permutations; Power consumption; Pseudorandom; random modulations; random permutations; Sensors; Sigma-Delta; Signal processing algorithms; Statistics; SVM; System-on-chip; Sigma-Delta; embedded object recognition; neural networks; random permutations; SVM; Image sensor; machine learning; random modulations; compressive sensing
• ISSN: 1549-8328; 1558-0806; 1558-0806
• DOI: 10.1109/TCSI.2020.2971565

This work presents a compact CMOS Image Sensor (CIS) architecture enabling embedded object recognition facilitated by a dedicated end-of-column Compressive Sensing (CS), reducing on-chip memory needs. Our sensing scheme is based on a combination of random modulations and permutations leading to an implementation with very limited hardware impacts. It is designed to meet both theoretical (i.e., stable embedding, measurements incoherence) and practical requirements (i.e., silicon footprint, power consumption). The only additional hardware compared to a standard CIS architecture using first order incremental Sigma-Delta (Σ Δ) Analog to Digital Converter (ADC) are a pseudo-random data mixing circuit, an in-Σ Δ ± 1 modulator and a small Digital Signal Processor (DSP). On the algorithmic side, three variants are presented to perform the inference on compressed measurements with a tunable complexity (i.e., one-vs.-all SVM, hierarchical SVM and small ANN with 1-D max-pooling). An object recognition accuracy of ~eq98.8% is reached on the COIL database (COIL, 100 classes) using our dedicated Neural Network classifier. We stress that the signal-independent dimensionality reduction performed by our dedicated CS scheme (1/480 in 480 x 640 VGA resolution case) allows to dramatically reduce memory requirements mainly related to the remotely learned coefficients used for the inference stage.