Cross-Domain Lossy Compression as Entropy Constrained Optimal Transport

• Published in: IEEE journal on selected areas in information theory Vol. 3; no. 3; p. 1
• Main Authors: Liu, Huan, Zhang, George, Chen, Jun, Khisti, Ashish
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.09.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Coders; Constraints; Decoding; Decoupling; Deep learning; Degradation; Distortion; Domains; Entropy; Image coding; Image Compression; Image reconstruction; Image restoration; Information Theory; Optimal Transport; Randomness; Rate-distortion Theory; Tradeoffs
• ISSN: 2641-8770; 2641-8770
• DOI: 10.1109/JSAIT.2022.3229670

We study an extension of lossy compression where the reconstruction is subject to a distribution constraint which can be different from the source distribution. We formulate our setting as a generalization of optimal transport with an entropy bottleneck to account for the rate constraint due to compression. We provide expressions for the tradeoff between compression rate and the achievable distortion with and without shared common randomness between the encoder and decoder. We study the examples of binary, uniform and Gaussian sources (in an asymptotic setting) in detail and demonstrate that shared randomness can strictly improve the tradeoff. For the case without common randomness and squared-Euclidean distortion, we show that the optimal solution partially decouples into the problem of optimal compression and transport and also characterize the penalty associated with fully decoupling them. We provide experimental results by training deep learning end-to-end compression systems for performing denoising on SVHN (The Street View House Numbers) and super-resolution on MNIST (Modified National Institute of Standards and Technology) datasets suggesting consistency with our theoretical results. Our code is available at https://github.com/liuh127/CrossdomainLC.