The Log-Volume of Optimal Codes for Memoryless Channels, Asymptotically Within a Few Nats

• Published in: IEEE transactions on information theory Vol. 63; no. 4; pp. 2278 - 2313
• Main Author: Moulin, Pierre
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.04.2017; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Asymptotic properties; binary symmetric channel; capacity; Channel coding; Channels; Codes; Coding; Communication channels; Decoding; Dispersion; Edgeworth expansion; Error probability; exponentially tilted distributions; Fisher information; large deviations; local limit theorem; Lower bounds; Monte Carlo methods; Neyman-Pearson testing; Normal distribution; random codes; Random variables; Shannon theory; Statistical analysis; Upper bound; Upper bounds; Z channel
• ISSN: 0018-9448; 1557-9654
• DOI: 10.1109/TIT.2016.2643681

Shannon's analysis of the fundamental capacity limits for memoryless communication channels has been refined over time. In this paper, the maximum volume M * avg (n, ∈) of length-n codes subject to an average decoding error probability ∈ is shown to satisfy the following tight asymptotic lower and upper bounds as n → ∞: A ∈ + o(1) ≤ log M* avg (n, ∈) - [nC - √nV ∈ Q -1 (∈)+ (1/2)log n] ≤ A ∈ + o(1), where C is the Shannon capacity, V ∈ is the ∈-channel dispersion, or secondorder coding rate, Q is the tail probability of the normal distribution, and the constants AE and AE are explicitly identified. This expression holds under mild regularity assumptions on the channel, including nonsingularity. The gap A ∈ - A ∈ is one nat for weakly symmetric channels in the Cover-Thomas sense, and typically a few nats for other symmetric channels, for the binary symmetric channel, and for the Z channel. The derivation is based on strong large-deviations analysis and refined central limit asymptotics. A random coding scheme that achieves the lower bound is presented. The codewords are drawn from a capacityachieving input distribution modified by an O(1/√n) correction term.