Optimal First-Order Algorithms as a Function of Inequalities

In this work, we present a novel algorithm design methodology that finds the optimal algorithm as a function of inequalities. Specifically, we restrict convergence analyses of algorithms to use a prespecified subset of inequalities, rather than utilizing all true inequalities, and find the optimal a...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Park, Chanwoo, Ryu, Ernest K
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 21.03.2024
Subjects
Algorithms
First order algorithms
Inequalities
Online AccessGet full text
ISSN2331-8422

Cover

More Information
Summary:In this work, we present a novel algorithm design methodology that finds the optimal algorithm as a function of inequalities. Specifically, we restrict convergence analyses of algorithms to use a prespecified subset of inequalities, rather than utilizing all true inequalities, and find the optimal algorithm subject to this restriction. This methodology allows us to design algorithms with certain desired characteristics. As concrete demonstrations of this methodology, we find new state-of-the-art accelerated first-order gradient methods using randomized coordinate updates and backtracking line searches.
Bibliography:content type line 50
SourceType-Working Papers-1
ObjectType-Working Paper/Pre-Print-1
ISSN:2331-8422