The Proximal Alternating Direction Method of Multipliers in the Nonconvex Setting: Convergence Analysis and Rates

• Published in: Mathematics of operations research Vol. 45; no. 2; pp. 682 - 712
• Main Authors: Boţ, Radu Ioan, Nguyen, Dang-Khoa
• Format: Journal Article
• Language: English
• Published: Linthicum INFORMS 01.05.2020; Institute for Operations Research and the Management Sciences
• Subjects: Algorithms; alternating direction method of multipliers; Analysis; Convergence; convergence analysis; convergence rates; Convex analysis; Iterative methods; Kurdyka–Łojasiewicz property; Linear operators; Linear programming; Mathematical analysis; Mathematical functions; Methods; Multipliers; nonconvex complexly structured optimization problems; Numerical analysis; Operations research; Optimization; Primary: 47H05, 65K05, 90C26; Primary: Mathematics: functions, matrices, sets; proximal splitting algorithms; Regularization; secondary: programming: algorithms; variable metric; Łojasiewicz exponent
• ISSN: 0364-765X; 1526-5471
• DOI: 10.1287/moor.2019.1008

We propose two numerical algorithms in the fully nonconvex setting for the minimization of the sum of a smooth function and the composition of a nonsmooth function with a linear operator. The iterative schemes are formulated in the spirit of the proximal alternating direction method of multipliers and its linearized variant, respectively. The proximal terms are introduced via variable metrics, a fact that allows us to derive new proximal splitting algorithms for nonconvex structured optimization problems, as particular instances of the general schemes. Under mild conditions on the sequence of variable metrics and by assuming that a regularization of the associated augmented Lagrangian has the Kurdyka–Łojasiewicz property, we prove that the iterates converge to a Karush–Kuhn–Tucker point of the objective function. By assuming that the augmented Lagrangian has the Łojasiewicz property, we also derive convergence rates for both the augmented Lagrangian and the iterates.