LANN-SVD: A Non-Iterative SVD-Based Learning Algorithm for One-Layer Neural Networks

• Published in: IEEE transaction on neural networks and learning systems Vol. 29; no. 8; pp. 3900 - 3905
• Main Authors: Fontenla-Romero, Oscar, Perez-Sanchez, Beatriz, Guijarro-Berdinas, Bertha
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.08.2018; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Analytics; Artificial neural networks; Biological neural networks; Complexity theory; Computer applications; Computing time; Data analysis; Data processing; Iterative methods; Large-scale data analytics; Learning algorithms; Learning systems; Linear programming; Machine learning; Mathematical model; Neural networks; Neurons; noniterative learning; one-layer (OL) neural networks; Singular value decomposition; singular value decomposition (SVD)
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2017.2738118

In the scope of data analytics, the volume of a data set can be defined as a product of instance size and dimensionality of the data. In many real problems, data sets are mainly large only on one of these aspects. Machine learning methods proposed in the literature are able to efficiently learn in only one of these two situations, when the number of variables is much greater than instances or vice versa. However, there is no proposal allowing to efficiently handle either circumstances in a large-scale scenario. In this brief, we present an approach to integrally address both situations, large dimensionality or large instance size, by using a singular value decomposition (SVD) within a learning algorithm for one-layer feedforward neural network. As a result, a noniterative solution is obtained, where the weights can be calculated in a closed-form manner, thereby avoiding low convergence rate and also hyperparameter tuning. The proposed learning method, LANN-SVD in short, presents a good computational efficiency for large-scale data analytic. Comprehensive comparisons were conducted to assess LANN-SVD against other state-of-the-art algorithms. The results of this brief exhibited the superior efficiency of the proposed method in any circumstance.