Optimal Transport in Reproducing Kernel Hilbert Spaces: Theory and Applications

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 42; no. 7; pp. 1741 - 1754
• Main Authors: Zhang, Zhen, Wang, Mianzhi, Nehorai, Arye
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.07.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Covariance matrices; Covariance matrix; covariance operator; domain adaptation; Geometry; Hilbert space; Image classification; Kernel; kernel methods; Kernels; Matching; Mathematical analysis; Matrix methods; Modeling; Operators (mathematics); Optimal transport; optimal transport map; reproducing kernel hilbert spaces; Task analysis; Wasserstein distance; Wasserstein geometry
• ISSN: 0162-8828; 1939-3539; 2160-9292; 1939-3539
• DOI: 10.1109/TPAMI.2019.2903050

In this paper, we present a mathematical and computational framework for comparing and matching distributions in reproducing kernel Hilbert spaces (RKHS). This framework, called optimal transport in RKHS, is a generalization of the optimal transport problem in input spaces to (potentially) infinite-dimensional feature spaces. We provide a computable formulation of Kantorovich's optimal transport in RKHS. In particular, we explore the case in which data distributions in RKHS are Gaussian, obtaining closed-form expressions of both the estimated Wasserstein distance and optimal transport map via kernel matrices. Based on these expressions, we generalize the Bures metric on covariance matrices to infinite-dimensional settings, providing a new metric between covariance operators. Moreover, we extend the correlation alignment problem to Hilbert spaces, giving a new strategy for matching distributions in RKHS. Empirically, we apply the derived formulas under the Gaussianity assumption to image classification and domain adaptation. In both tasks, our algorithms yield state-of-the-art performances, demonstrating the effectiveness and potential of our framework.