First-order inertial algorithms involving dry friction damping

• Published in: Mathematical programming Vol. 193; no. 1; pp. 405 - 445
• Main Authors: Adly, Samir, Attouch, Hedy
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.05.2022; Springer; Springer Nature B.V
• Subjects: Algorithms; Analysis; Asymptotic methods; Asymptotic properties; Calculus of Variations and Optimal Control; Optimization; Combinatorics; Convergence; Critical point; Damping; Dry friction; First order algorithms; Friction; Full Length Paper; Hilbert space; Mathematical analysis; Mathematical and Computational Physics; Mathematical Methods in Physics; Mathematics; Mathematics and Statistics; Mathematics of Computing; Numerical Analysis; Numerical methods; Operators (mathematics); Optimization; Splitting; Theoretical; Proximal-gradient algorithms; 70F40; Finite convergence; Inertial methods; Lasso problem; Differential inclusion; 34A60; 37N40; 34G25; Dry friction; 49K24
• ISSN: 0025-5610; 1436-4646
• DOI: 10.1007/s10107-020-01613-y

In a Hilbert space H , we introduce a new class of first-order algorithms which naturally occur as discrete temporal versions of an inertial differential inclusion jointly involving viscous friction and dry friction. The function f : H → R to be minimized is supposed to be differentiable (not necessarily convex), and enters the algorithm via its gradient. The dry friction damping function ϕ : H → R + is convex with a sharp minimum at the origin, (typically ϕ ( x ) = r ‖ x ‖ with r > 0 ). It enters the algorithm via its proximal mapping, which acts as a soft threshold operator on the velocities. As a result, we obtain a new class of splitting algorithms involving separately the proximal and gradient steps. The sequence of iterates has a finite length, and therefore strongly converges towards an approximate critical point x ∞ of f (typically ‖ ∇ f ( x ∞ ) ‖ ≤ r ). Under a geometric property satisfied by the limit point x ∞ , we obtain geometric and finite rates of convergence. The convergence results tolerate the presence of errors, under the sole assumption of their asymptotic convergence towards zero. By replacing the function f by its Moreau envelope, we extend the results to the case of nonsmooth convex functions. In this case, the algorithm involves the proximal operators of f and ϕ separately. Several variants of this algorithm are considered, including the case of the Nesterov accelerated gradient method. We then consider the extension in the case of additive composite optimization, thus leading to new splitting methods. Numerical experiments are given for Lasso-type problems. The performance profiles, as a comparison tool, demonstrate the efficiency of the Nesterov accelerated method with asymptotic vanishing damping combined with dry friction.