Sparse coding and NMF

• Published in: 2004 IEEE International Joint Conference on Neural Networks Vol. 4; pp. 2529 - 2533 vol.4
• Main Authors: Eggert, J., Korner, E.
• Format: Conference Proceeding
• Language: English
• Published: Piscataway NJ IEEE 2004
• Subjects: Additives; Applied sciences; Artificial intelligence; Computer science; control theory; systems; Cost function; Encoding; Europe; Exact sciences and technology; Matrix decomposition; Pattern recognition. Digital image processing. Computational geometry; Principal component analysis; Sparse matrices; Time factors; Vector quantization; Visual system; Non negative matrix; Image processing; Visual field; Sparse representation; Matrix factorization; Neural network; Modeling
• ISBN: 0780383591; 9780780383593
• ISSN: 1098-7576
• DOI: 10.1109/IJCNN.2004.1381036

Non-negative matrix factorization (NMF) is a very efficient parameter-free method for decomposing multivariate data into strictly positive activations and basis vectors. However, the method is not suited for overcomplete representations, where usually sparse coding paradigms apply. We show how to merge the concepts of non-negative factorization with sparsity conditions. The result is a multiplicative algorithm that is comparable in efficiency to standard NMF, but that can be used to gain sensible solutions in the overcomplete cases. This is of interest e.g. for the case of learning and modeling of arrays of receptive fields arranged in a visual processing map, where an overcomplete representation is unavoidable.