A random coordinate descent algorithm for optimization problems with composite objective function and linear coupled constraints

• Published in: Computational optimization and applications Vol. 57; no. 2; pp. 307 - 337
• Main Authors: Necoara, Ion, Patrascu, Andrei
• Format: Journal Article
• Language: English
• Published: Boston Springer US 01.03.2014; Springer Nature B.V
• Subjects: Algorithms; Computer science; Convergence; Convex analysis; Convex and Discrete Geometry; Derivatives; Descent; Management Science; Mathematical analysis; Mathematical models; Mathematics; Mathematics and Statistics; Methods; Operations Research; Operations Research/Decision Theory; Optimization; Statistics; Studies; Support vector machines; Texts; Convergence rate; Composite objective function; Linearly coupled constraints; Randomized algorithms; Coordinate descent
• ISSN: 0926-6003; 1573-2894
• DOI: 10.1007/s10589-013-9598-8

In this paper we propose a variant of the random coordinate descent method for solving linearly constrained convex optimization problems with composite objective functions. If the smooth part of the objective function has Lipschitz continuous gradient, then we prove that our method obtains an ϵ -optimal solution in iterations, where n is the number of blocks. For the class of problems with cheap coordinate derivatives we show that the new method is faster than methods based on full-gradient information. Analysis for the rate of convergence in probability is also provided. For strongly convex functions our method converges linearly. Extensive numerical tests confirm that on very large problems, our method is much more numerically efficient than methods based on full gradient information.