Lossy Source Compression Using Low-Density Generator Matrix Codes: Analysis and Algorithms

• Published in: IEEE transactions on information theory Vol. 56; no. 3; pp. 1351 - 1368
• Main Authors: Wainwright, M.J., Maneva, E., Martinian, E.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.03.2010; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithm design and analysis; Algorithms; Applied sciences; Belief propagation; Bernoulli Hypothesis; Codes; Coding, codes; Compressing; Decoding; Distortion; Exact sciences and technology; Generators; graphical codes; Information theory; Information, signal and communications theory; Linear code; Lossy source coding; low-density generator matrix (LDGM) codes; Matrix; MAX-XORSAT; message-passing; Optimization; Parity check codes; Rate-distortion; satisfiability problems; Signal and communications theory; Source coding; Subroutines; sum-product; survey propagation; Symmetric matrices; Telecommunications and information theory; Tree graphs; Turbo codes; Upper bounds; Lossy compression; Signal compression; message-passing; Lossy source coding; belief propagation; survey propagation; Source coding; Subroutine; graphical codes; low-density generator matrix (LDGM) codes; Binary signal; Rate distortion theory; Lossy medium; Algorithm; Credal approach; Upper bound; Message passing; satisfiability problems; MAX-XORSAT; Minimal distance; Distance measurement; Decimation; Satisfiability; sum-product
• ISSN: 0018-9448; 1557-9654
• DOI: 10.1109/TIT.2009.2039160

We study the use of low-density generator matrix (LDGM) codes for lossy compression of the Bernoulli symmetric source. First, we establish rigorous upper bounds on the average distortion achieved by check-regular ensemble of LDGM codes under optimal minimum distance source encoding. These bounds establish that the average distortion using such bounded degree families rapidly approaches the Shannon limit as the degrees are increased. Second, we propose a family of message-passing algorithms, ranging from the standard belief propagation algorithm at one extreme to a variant of survey propagation algorithm at the other. When combined with a decimation subroutine and applied to LDGM codes with suitably irregular degree distributions, we show that such a message-passing/decimation algorithm yields distortion very close to the Shannon rate-distortion bound for the binary symmetric source.