Semi-Supervised Graph Regularized Deep NMF With Bi-Orthogonal Constraints for Data Representation

• Published in: IEEE transaction on neural networks and learning systems Vol. 31; no. 9; pp. 3245 - 3258
• Main Authors: Meng, Yang, Shang, Ronghua, Shang, Fanhua, Jiao, Licheng, Yang, Shuyuan, Stolkin, Rustam
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.09.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Bi-orthogonal constraints; Clustering; Data mining; Deep learning; deep non-negative matrix factorization (NMF); Dimensionality reduction; dual-hypergraph Laplacian regularization; Feature extraction; Graphical representations; Iterative methods; Laplace equations; Linear programming; Machine learning; Mapping; Matrix decomposition; Objective function; Reduction; Regularization; semi-supervised learning
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2019.2939637

Semi-supervised non-negative matrix factorization (NMF) exploits the strengths of NMF in effectively learning local information contained in data and is also able to achieve effective learning when only a small fraction of data is labeled. NMF is particularly useful for dimensionality reduction of high-dimensional data. However, the mapping between the low-dimensional representation, learned by semi-supervised NMF, and the original high-dimensional data contains complex hierarchical and structural information, which is hard to extract by using only single-layer clustering methods. Therefore, in this article, we propose a new deep learning method, called semi-supervised graph regularized deep NMF with bi-orthogonal constraints (SGDNMF). SGDNMF learns a representation from the hidden layers of a deep network for clustering, which contains varied and unknown attributes. Bi-orthogonal constraints on two factor matrices are introduced into our SGDNMF model, which can make the solution unique and improve clustering performance. This improves the effect of dimensionality reduction because it only requires a small fraction of data to be labeled. In addition, SGDNMF incorporates dual-hypergraph Laplacian regularization, which can reinforce high-order relationships in both data and feature spaces and fully retain the intrinsic geometric structure of the original data. This article presents the details of the SGDNMF algorithm, including the objective function and the iterative updating rules. Empirical experiments on four different data sets demonstrate state-of-the-art performance of SGDNMF in comparison with six other prominent algorithms.