Scalable Parallel Nonlinear Optimization with PyNumero and Parapint

• Published in: INFORMS journal on computing Vol. 35; no. 2; pp. 509 - 517
• Main Authors: Rodriguez, Jose S., Parker, Robert B., Laird, Carl D., Nicholson, Bethany L., Siirola, John D., Bynum, Michael L.
• Format: Journal Article
• Language: English
• Published: Linthicum INFORMS 01.03.2023; Institute for Operations Research and the Management Sciences
• Subjects: Algorithms; Computer programming; Decomposition; decomposition algorithms; Linear equations; Linear systems; Mathematical analysis; Matrices (mathematics); Message passing; Natural language processing; Nonlinear programming; Object oriented programming; Object-oriented languages; open-source optimization software; Optimal control; Optimization; parallel computing; Programming languages; Python; Solvers
• ISSN: 1091-9856; 1526-5528; 1526-5528; 1091-9856
• DOI: 10.1287/ijoc.2023.1272

We describe PyNumero, an open-source, object-oriented programming framework in Python that supports rapid development of performant parallel algorithms for structured nonlinear programming problems (NLP’s) using the Message Passing Interface (MPI). PyNumero provides three fundamental building blocks for developing NLP algorithms: a fast interface for calculating first and second derivatives with the AMPL Solver Library (ASL), a number of interfaces to efficient linear solvers, and block-structured vectors and matrices based on NumPy, SciPy, and MPI that support distributed parallel storage and computation. PyNumero’s design enables efficient, parallel algorithm development using high-level Python syntax while keeping expensive numerical calculations in fast, compiled implementations based on languages like C and Fortran. To demonstrate the utility of PyNumero, we also present Parapint, a Python package built on PyNumero for parallel solution of dynamic optimization problems. Parapint includes a parallel interior-point solver based on Schur-Complement decomposition. We illustrate the effectiveness of PyNumero for developing parallel algorithms with both code examples and scalability analyses for parallel matrix-vector dot products, parallel solution of structured systems of linear equations using Schur-Complement decomposition, and the parallel solution of a two-dimensional PDE optimal control problem. Our numerical results show nearly perfect scaling to more than 1,000 cores for large matrix-vector dot products and structured linear systems. Moreover, we obtain more than 354 times speedup for the optimal control example. History: Accepted by Alice Smith, EIC/Ted Ralphs, Area Editor/Software Tools. Funding: This work was funded in part by the Institute for the Design of Advanced Energy Systems (IDAES) with funding from the Office of Fossil Energy, Cross-Cutting Research, U.S. Department of Energy. This work was also funded by Sandia National Laboratories Laboratory Directed Research and Development (LDRD) program. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2023.1272 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2021.0285 ) at ( http://dx.doi.org/10.5281/zenodo.7192328 ).