Capacity and Random-Coding Exponents for Channel Coding With Side Information

• Published in: IEEE transactions on information theory Vol. 53; no. 4; pp. 1326 - 1347
• Main Authors: Moulin, P., Ying Wang
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.04.2007; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Alphabets; Applied sciences; Arbitrarily varying channels; Asymptotic properties; broadcast channels; Broadcasting; capacity; Channel coding; channel coding with side information; Channels; Coding; Coding, codes; Costs; Cryptography; Data encapsulation; data hiding; Decoders; Decoding; Digital transmission; Electrical engineering; Electronics; error exponents; Exact sciences and technology; Exponents; Information theory; Information, signal and communications theory; maximum a posteriori probability (MAP) decoding; Memoryless systems; method of types; Mutual information; Radiocommunications; random binning; randomized codes; reliability function; Signal and communications theory; Telecommunications; Telecommunications and information theory; Transmitters; Transmitters. Receivers; universal coding and decoding; Upper bound; Watermarking; Alphabet; Channel with memory; data hiding; Error probability; Image processing; Maximum mutual information; Transmitter; Random coding; capacity; Universal coding; Codec; Arbitrarily varying channels; Information protection; A posteriori estimation; Broadcast channels; random binning; maximum a posteriori probability (MAP) decoding; Data communication; Digital watermarking; Discrete channel; error exponents; Two channel system; randomized codes; Decoding; reliability function; Channel coding; watermarking; Random sequence; Upper bound; Memoryless channel; universal coding and decoding; Posterior probability; Noisy channel; channel coding with side information; method of types
• ISSN: 0018-9448; 1557-9654
• DOI: 10.1109/TIT.2007.892789

Capacity formulas and random-coding exponents are derived for a generalized family of Gel'fand-Pinsker coding problems. These exponents yield asymptotic upper bounds on the achievable log probability of error. In our model, information is to be reliably transmitted through a noisy channel with finite input and output alphabets and random state sequence, and the channel is selected by a hypothetical adversary. Partial information about the state sequence is available to the encoder, adversary, and decoder. The design of the transmitter is subject to a cost constraint. Two families of channels are considered: 1) compound discrete memoryless channels (CDMC), and 2) channels with arbitrary memory, subject to an additive cost constraint, or more generally, to a hard constraint on the conditional type of the channel output given the input. Both problems are closely connected. The random-coding exponent is achieved using a stacked binning scheme and a maximum penalized mutual information decoder, which may be thought of as an empirical generalized maximum a posteriori decoder. For channels with arbitrary memory, the random-coding exponents are larger than their CDMC counterparts. Applications of this study include watermarking, data hiding, communication in presence of partially known interferers, and problems such as broadcast channels, all of which involve the fundamental idea of binning