On the role of mismatch in rate distortion theory

• Published in: IEEE transactions on information theory Vol. 43; no. 1; pp. 38 - 47
• Main Author: Lapidoth, A.
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.01.1997; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Additives; Channel capacity; Codes; Constraint optimization; Decoding; Distortion measurement; Encoding; Information technology; Rate distortion theory; Rate-distortion; Upper bound
• ISSN: 0018-9448; 1557-9654
• DOI: 10.1109/18.567644

Using a codebook C, a source sequence is described by the codeword that is closest to it according to the distortion measure d/sub 0/(x,x/spl circ//sub 0/). Based on this description, the source sequence is reconstructed to minimize the reconstruction distortion as measured by d/sub 1/(x,x/spl circ//sub 1/), where, in general, d/sub 1/(x,x/spl circ//sub 1/)/spl ne/d/sub 0/(x,x/spl circ//sub 0/). We study the minimum resulting d/sub 1/(x,x/spl circ//sub 1/)-distortion between the reconstructed sequence and the source sequence as we optimize over the codebook subject to a rate constraint. Using a random coding argument we derive an upper bound on the resulting distortion. Applying this bound to blocks of source symbols we construct a sequence of bounds which are shown to converge to the least distortion achievable in this setup. This solves the rate distortion dual of an open problem related to the capacity of channels with a given decoding rule-the mismatch capacity. Addressing a different kind of mismatch, we also study the mean-squared error description of non-Gaussian sources with random Gaussian codebooks. It is shown that the use of a Gaussian codebook to compress any ergodic source results in an average distortion which depends on the source via its second moment only. The source with a given second moment that is most difficult to describe is the memoryless zero-mean Gaussian source, and it is best described using a Gaussian codebook. Once a Gaussian codebook is used, we show that all sources of a given second moment become equally hard to describe.