Variance reduced moving balls approximation method for smooth constrained minimization problems

• Published in: Optimization letters Vol. 18; no. 5; pp. 1253 - 1271
• Main Authors: Yang, Zhichun, Xia, Fu-quan, Tu, Kai
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.06.2024
• Subjects: Computational Intelligence; Mathematics; Mathematics and Statistics; Numerical and Computational Physics; Operations Research/Decision Theory; Optimization; Original Paper; Simulation; Convergence rate; Smooth constrained minimization; Moving balls approximation; Relaxed strong convexity condition; Variance reduction
• ISSN: 1862-4472; 1862-4480
• DOI: 10.1007/s11590-023-02049-x

In this paper, we consider the problem of minimizing the sum of a large number of smooth convex functions subject to a complicated constraint set defined by a smooth convex function. Such a problem has wide applications in many areas, such as machine learning and signal processing. By utilizing variance reduction and moving balls approximation techniques, we propose a new variance reduced moving balls approximation method. Compared with existing convergence rates of moving balls approximation-type methods that require the strong convexity of the objective function, a notable advantage of the proposed method is that the linear and sublinear convergence rates can be guaranteed under the quadratic gradient growth property and convexity condition, respectively. To demonstrate its effectiveness, numerical experiments for solving the smooth regularized logistic regression problem and the Neyman-Pearson classification problem are presented.