Penalized Linear Regression - Bowles, Michael (Machine Learning with Spark and Python, 2020)
This chapter discusses the performance of penalized linear models. It develops an extended family of methods for controlling the overfitting inherent in ordinary least squares. The chapter also explains how the penalty method determines the nature of the solution and the type of information that is available about the solution. It describes principles of operation for two modern algorithms for solving the penalized regression minimization problem and Python code implementing the main features of the algorithms in order to have a concrete instantiation of the core of the algorithms to make the principals of operation clear. The plain regression problem (numeric features and numeric targets) served as the exemplar for in‐depth coverage of algorithms. The chapter shows several extensions to broaden the use cases to include binary classification problems, multiclass classification problems, problems with nonlinear relationship between attributes and outcomes, and problems with non‐numeric features.