The Distributed Karhunen-Loève Transform

• Published in: IEEE transactions on information theory Vol. 52; no. 12; pp. 5177 - 5196
• Main Authors: Gastpar, M., Dragotti, P.L., Vetterli, M.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.12.2006; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Applied sciences; Approximation; Coding; Coding, codes; Communication; Compressing; Data transmission; Distributed source coding; distributed transforms; Exact sciences and technology; Image coding; Image reconstruction; Image sensors; Information theory; Information, signal and communications theory; Iterative algorithms; Iterative decoding; Karhunen-Loeve transforms; Mathematical analysis; Miscellaneous; Optimization; principal components analysis; rate-distortion function; Sensor fusion; side information; Signal and communications theory; Signal processing; Signaling; Source coding; Telecommunications and information theory; Terminals; Transform coding; Transforms; Performance evaluation; side information; Iterative method; Random vector; principal components analysis; Distributed processing; Distributed application; Karhunen Loeve transformation; Source coding; Rate distortion theory; Distributed source signal; Global optimum; Information access; rate-distortion function; Distributed system; Algorithm; Distributed source coding; Statistical method; Transform coding; Optimal solution; Information processing; Signal processing; Data fusion; distributed transforms; Optimal approximation; Image sensor; Sensor array; Principal component analysis
• ISSN: 0018-9448; 1557-9654; 1557-9654
• DOI: 10.1109/TIT.2006.885449

The Karhunen-Loeve transform (KLT) is a key element of many signal processing and communication tasks. Many recent applications involve distributed signal processing, where it is not generally possible to apply the KLT to the entire signal; rather, the KLT must be approximated in a distributed fashion. This paper investigates such distributed approaches to the KLT, where several distributed terminals observe disjoint subsets of a random vector. We introduce several versions of the distributed KLT. First, a local KLT is introduced, which is the optimal solution for a given terminal, assuming all else is fixed. This local KLT is different and in general improves upon the marginal KLT which simply ignores other terminals. Both optimal approximation and compression using this local KLT are derived. Two important special cases are studied in detail, namely, the partial observation KLT which has access to a subset of variables, but aims at reconstructing them all, and the conditional KLT which has access to side information at the decoder. We focus on the jointly Gaussian case, with known correlation structure, and on approximation and compression problems. Then, the distributed KLT is addressed by considering local KLTs in turn at the various terminals, leading to an iterative algorithm which is locally convergent, sometimes reaching a global optimum, depending on the overall correlation structure. For compression, it is shown that the classical distributed source coding techniques admit a natural transform coding interpretation, the transform being the distributed KLT. Examples throughout illustrate the performance of the proposed distributed KLT. This distributed transform has potential applications in sensor networks, distributed image databases, hyper-spectral imagery, and data fusion