Convex optimization via inertial algorithms with vanishing Tikhonov regularization: fast convergence to the minimum norm solution

• Published in: Mathematical methods of operations research (Heidelberg, Germany) Vol. 99; no. 3; pp. 307 - 347
• Main Authors: Attouch, Hedy, László, Szilárd Csaba
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.06.2024; Springer Nature B.V; Springer Verlag
• Subjects: Algorithms; Business and Management; Calculus of Variations and Optimal Control; Optimization; Computer Science; Convergence; Convexity; Damping; First order algorithms; Mathematics; Mathematics and Statistics; Operations Research; Operations Research/Decision Theory; Optimization and Control; Original Article; Regularization; 65K05; Tikhonov approximation; 37N40; 65B99; 49M30; Convex optimization; Nesterov accelerated gradient method; 90C25; Minimum norm solution; Damped inertial dynamics; 65K10; Accelerated gradient methods; 90B50; 46N10; convex optimization; damped inertial dynamics; minimum norm solution
• ISSN: 1432-2994; 1432-5217; 1432-5217
• DOI: 10.1007/s00186-024-00867-y

In a Hilbertian framework, for the minimization of a general convex differentiable function f , we introduce new inertial dynamics and algorithms that generate trajectories and iterates that converge fastly towards the minimizer of f with minimum norm. Our study is based on the non-autonomous version of the Polyak heavy ball method, which, at time t , is associated with the strongly convex function obtained by adding to f a Tikhonov regularization term with vanishing coefficient ε ( t ) . In this dynamic, the damping coefficient is proportional to the square root of the Tikhonov regularization parameter ε ( t ) . By adjusting the speed of convergence of ε ( t ) towards zero, we will obtain both rapid convergence towards the infimal value of f , and the strong convergence of the trajectories towards the element of minimum norm of the set of minimizers of f . In particular, we obtain an improved version of the dynamic of Su-Boyd-Candès for the accelerated gradient method of Nesterov. This study naturally leads to corresponding first-order algorithms obtained by temporal discretization. In the case of a proper lower semicontinuous and convex function f , we study the proximal algorithms in detail, and show that they benefit from similar properties.