Lossless Coding for Distributed Streaming Sources

• Published in: IEEE transactions on information theory Vol. 60; no. 3; pp. 1447 - 1474
• Main Authors: Draper, Stark C., Cheng Chang, Sahai, Anant
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.03.2014; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Applied sciences; Codes; Coding, codes; Delays; Distributed source coding; Entropy; Exact sciences and technology; Information theory; Information, signal and communications theory; Joints; lossless source coding; Maximum likelihood decoding; Maximum likelihood method; Probability distribution; Signal and communications theory; Slepian-Wolf coding; Source coding; streaming data; Systems, networks and services of telecommunications; Telecommunications; Telecommunications and information theory; Transmission and modulation (techniques and equipments); universal decoding; Streaming; Lower bound; Block code; Lossless circuit; streaming data; Source coding; Coding circuit; Distributed source signal; Entropy; Slepian-Wolf coding; lossless source coding; Information transmission; Maximum likelihood decoding; Distributed source coding; Upper bound; Delay time; Maximum likelihood; Causality; universal decoding
• ISSN: 0018-9448; 1557-9654
• DOI: 10.1109/TIT.2013.2294368

Distributed source coding is traditionally viewed in a block coding context wherein all source symbols are known in advance by the encoders. However, many modern applications to which distributed source coding ideas are applied, are better modeled as having streaming data. In a streaming setting, source symbol pairs are revealed to separate encoders in real time and need to be reconstructed at the decoder with subject to some tolerable end-to-end delay. In this paper, a causal sequential random binning encoder is introduced and paired with maximum likelihood (ML) and universal decoders. The latter uses a novel weighted empirical suffix entropy decoding rule. We derive a lower bounds on the error exponent with delay for each decoder. We also provide upper bounds for the special case of streaming with decoder side information and discuss when upper and lower bounds match. We show that both ML and universal decoders achieve the same (positive) error exponents for all rate pairs inside the Slepian-Wolf achievable rate region. The dominant error events in streaming are different from those in block-coding and result in different exponents. Because the sequential random binning scheme is also universal over delays, the resulting code eventually reconstructs every source symbol correctly with probability one.