An Efficient Ridge Regression Algorithm with Parameter Estimation for Data Analysis in Machine Learning

• Published in: SN computer science Vol. 3; no. 2; p. 171
• Main Author: Rajan, M. P.
• Format: Journal Article
• Language: English
• Published: Singapore Springer Singapore 01.03.2022; Springer Nature B.V
• Subjects: Algorithms; Bias; Collinearity; Computer Imaging; Computer Science; Computer Systems Organization and Communication Networks; Data analysis; Data Structures and Information Theory; Elastic limit; Estimates; Information Systems and Communication Service; Iterative methods; Least squares; Linear algebra; Machine learning; Mathematical models; Original Research; Parameter estimation; Pattern Recognition and Graphics; Regression; Software Engineering/Programming and Operating Systems; Variables; Vision; Ridge regression; Conjugate gradient method; Data analysis; Parameter estimation; Machine learning
• ISSN: 2662-995X; 2661-8907
• DOI: 10.1007/s42979-022-01051-x

Multiple linear regression is a widely used supervisory machine learning technique that describes the distribution of a response variable with the help of a number of explanatory variables. The least-square approach is a widely accepted technique to solve this problem. However, in the presence of multi-collinearity, the least-square technique may show a poor performance as a solution methodology. Although least squares estimates are unbiased but their variances are large and they may be far from the true value. Ridge regression is a standard technique to tackle these kinds of problems. But the choice of the ridge parameter plays a crucial role in the performance of the ridge regression algorithm. The standard algorithms such as Ridge Trace or Ridge cross-validation or Lasso cross-validation or Elastic Net cross-validation have the limitation of giving a priori, an array of parameter values as an input to the algorithm and from which it choose that fit the model. But the actual parameter may be far away from one that is picked by the algorithm. However, if we have an algorithm that automatically computes this parameter without any prior knowledge of it and delivers the solution, then that could be very useful for practical purposes. In this paper, we propose an algorithm, based on an iterative approach, with a parameter choice strategy for solving ridge regression problems that circumvent such kind of limitations and automatically compute the parameter and the best-fitted model, which is the salient feature of this manuscript. The efficiency of the algorithm is illustrated through various examples and compared with standard methods. The experimental data analysis clearly demonstrates the supremacy of the proposed algorithm.