Beyond ℓ1 sparse coding in V1

• Published in: PLoS computational biology Vol. 19; no. 9; p. e1011459
• Main Authors: Rentzeperis, Ilias, Calatroni, Luca, Perrinet, Laurent U., Prandi, Dario
• Format: Journal Article
• Language: English
• Published: San Francisco, CA USA Public Library of Science 01.09.2023; Public Library of Science (PLoS)
• Subjects: Biology and Life Sciences; Medicine and Health Sciences; Physical Sciences; Research and Analysis Methods; Social Sciences
• ISSN: 1553-7358; 1553-734X; 1553-7358
• DOI: 10.1371/journal.pcbi.1011459

Growing evidence indicates that only a sparse subset from a pool of sensory neurons is active for the encoding of visual stimuli at any instant in time. Traditionally, to replicate such biological sparsity, generative models have been using the ℓ 1 norm as a penalty due to its convexity, which makes it amenable to fast and simple algorithmic solvers. In this work, we use biological vision as a test-bed and show that the soft thresholding operation associated to the use of the ℓ 1 norm is highly suboptimal compared to other functions suited to approximating ℓ p with 0 ≤ p < 1 (including recently proposed continuous exact relaxations), in terms of performance. We show that ℓ 1 sparsity employs a pool with more neurons, i.e. has a higher degree of overcompleteness, in order to maintain the same reconstruction error as the other methods considered. More specifically, at the same sparsity level, the thresholding algorithm using the ℓ 1 norm as a penalty requires a dictionary of ten times more units compared to the proposed approach, where a non-convex continuous relaxation of the ℓ 0 pseudo-norm is used, to reconstruct the external stimulus equally well. At a fixed sparsity level, both ℓ 0 - and ℓ 1 -based regularization develop units with receptive field (RF) shapes similar to biological neurons in V1 (and a subset of neurons in V2), but ℓ 0 -based regularization shows approximately five times better reconstruction of the stimulus. Our results in conjunction with recent metabolic findings indicate that for V1 to operate efficiently it should follow a coding regime which uses a regularization that is closer to the ℓ 0 pseudo-norm rather than the ℓ 1 one, and suggests a similar mode of operation for the sensory cortex in general.