An Efficient Ridge Regression Algorithm with Parameter Estimation for Data Analysis in Machine Learning - Rajan, M. P. (SN computer science, 2022)
Multiple linear regression is a widely used supervisory machine learning technique that describes the distribution of a response variable with the help of a number of explanatory variables. The least-square approach is a widely accepted technique to solve this problem. However, in the presence of multi-collinearity, the least-square technique may show a poor performance as a solution methodology. Although least squares estimates are unbiased but their variances are large and they may be far from the true value. Ridge regression is a standard technique to tackle these kinds of problems. But the choice of the ridge parameter plays a crucial role in the performance of the ridge regression algorithm. The standard algorithms such as Ridge Trace or Ridge cross-validation or Lasso cross-validation or Elastic Net cross-validation have the limitation of giving a priori, an array of parameter values as an input to the algorithm and from which it choose that fit the model. But the actual parameter may be far away from one that is picked by the algorithm. However, if we have an algorithm that automatically computes this parameter without any prior knowledge of it and delivers the solution, then that could be very useful for practical purposes. In this paper, we propose an algorithm, based on an iterative approach, with a parameter choice strategy for solving ridge regression problems that circumvent such kind of limitations and automatically compute the parameter and the best-fitted model, which is the salient feature of this manuscript. The efficiency of the algorithm is illustrated through various examples and compared with standard methods. The experimental data analysis clearly demonstrates the supremacy of the proposed algorithm.