Incremental Linear Discriminant Analysis Using Sufficient Spanning Sets and Its Applications

• Published in: International journal of computer vision Vol. 91; no. 2; pp. 216 - 232
• Main Authors: Kim, Tae-Kyun, Stenger, Björn, Kittler, Josef, Cipolla, Roberto
• Format: Journal Article
• Language: English
• Published: Boston Springer US 01.01.2011; Springer; Springer Nature B.V
• Subjects: Algorithms; Applied sciences; Artificial Intelligence; Computer Imaging; Computer Science; Computer science; control theory; systems; Datasets; Discriminant analysis; Distance learning; Exact sciences and technology; Image Processing and Computer Vision; Image retrieval; Learning; Learning and adaptive systems; Mathematical models; On-line systems; Online; Pattern Recognition; Pattern Recognition and Graphics; Pattern recognition. Digital image processing. Computational geometry; Principal components analysis; Scatter; Teaching methods; Vision; Incremental learning; Face authentication; Semi-supervised learning; Object categorisation; LDA; Linear discriminant analysis; Label propagation; Object recognition; Online learning; Face image retrieval; Computer vision; Discriminant analysis; Image processing; Face recognition; Standardization; Very large databases; Updating; Information retrieval; Pattern recognition; Computational complexity; Facies; Differential method; Categorization; Artificial intelligence
• ISSN: 0920-5691; 1573-1405
• DOI: 10.1007/s11263-010-0381-3

This paper presents an incremental learning solution for Linear Discriminant Analysis (LDA) and its applications to object recognition problems. We apply the sufficient spanning set approximation in three steps i.e. update for the total scatter matrix, between-class scatter matrix and the projected data matrix, which leads an online solution which closely agrees with the batch solution in accuracy while significantly reducing the computational complexity. The algorithm yields an efficient solution to incremental LDA even when the number of classes as well as the set size is large. The incremental LDA method has been also shown useful for semi-supervised online learning. Label propagation is done by integrating the incremental LDA into an EM framework. The method has been demonstrated in the task of merging large datasets which were collected during MPEG standardization for face image retrieval, face authentication using the BANCA dataset, and object categorisation using the Caltech101 dataset.