Incremental Linear Discriminant Analysis: A Fast Algorithm and Comparisons

• Published in: IEEE transaction on neural networks and learning systems Vol. 26; no. 11; pp. 2716 - 2735
• Main Authors: Delin Chu, Li-Zhi Liao, Ng, Michael Kwok-Po, Xiaoyan Wang
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.11.2015; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Algorithm design and analysis; Algorithms; Classification; Classification accuracy; Complexity; Computation; Computational complexity; Discriminant analysis; Economics; Eigenvalues and eigenfunctions; incremental linear discriminant analysis (ILDA); Linear discriminant analysis; linear discriminant analysis (LDA); Linear systems; Neural networks; Training; linear discriminant analysis (LDA); Classification accuracy; incremental linear discriminant analysis (ILDA); computational complexity
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2015.2391201

It has always been a challenging task to develop a fast and an efficient incremental linear discriminant analysis (ILDA) algorithm. For this purpose, we conduct a new study for linear discriminant analysis (LDA) in this paper and develop a new ILDA algorithm. We propose a new batch LDA algorithm called LDA/QR. LDA/QR is a simple and fast LDA algorithm, which is obtained by computing the economic QR factorization of the data matrix followed by solving a lower triangular linear system. The relationship between LDA/QR and uncorrelated LDA (ULDA) is also revealed. Based on LDA/QR, we develop a new incremental LDA algorithm called ILDA/QR. The main features of our ILDA/QR include that: 1) it can easily handle the update from one new sample or a chunk of new samples; 2) it has efficient computational complexity and space complexity; and 3) it is very fast and always achieves competitive classification accuracy compared with ULDA algorithm and existing ILDA algorithms. Numerical experiments based on some real-world data sets demonstrate that our ILDA/QR is very efficient and competitive with the state-of-the-art ILDA algorithms in terms of classification accuracy, computational complexity, and space complexity.