Large, Sparse Nonlinear Programming

• Published in: Practical Methods for Optimal Control and Estimation Using Nonlinear Programming p. 1
• Main Author: Betts, John T
• Format: Book Chapter
• Language: English
• Published: Society for Industrial and Applied Mathematics (SIAM) 2010; Society for Industrial and Applied Mathematics
• Edition: 2nd Edition
• Series: Advances in Design and Control
• Subjects: Control theory; Control Theory & Optimization; Mathematical optimization; Nonlinear programming; Process Design, Control & Automation
• ISBN: 0898716888; 9780898716887
• DOI: 10.1137/1.9780898718577.ch2

2.1 Overview: Large, Sparse NLP Issues Chapter 1 presents background on nonlinear programming (NLP) methods that are applicable to most problems. In this chapter, we will focus on NLP methods that are appropriate for problems that are both large and sparse. To set the stage for what follows, it is useful to define what we mean by large and sparse. The definition of “large” is closely tied with the tools available for solving linear systems of equations. For our purposes, we will consider a problem “small” if the underlying linear systems can be solved using a dense, direct matrix factorization. Solving a dense linear system using a direct factorization requires O ( n 3 ) arithmetic operations. Thus, for today's computers, this suggests that the problem size is probably limited to matrices of order n ≈ 1000. If the linear equations are solved using methods that exploit sparsity in the matrices but still use direct matrix factorizations, then with current computer hardware, the upper limit on the size of the problem is n ≈ 106. This problem is considered “large” and is the focus of methods in this book. Although some of the NLP methods can be extended, as a rule we will not address “huge” problems for which n > 106. Typically, linear systems of this size are solved by iterative (as opposed to direct) methods. The second discriminator of interest concerns matrix sparsity. A matrix is said to be “sparse“ when many of the elements are zero. For most applications of interest, the number of nonzero elements in the Hessian matrix H and Jacobian matrix G is less than 1%. Many of the techniques described in Chapter 1 extend, without change, to large, sparse problems. On the other hand, there are some techniques that cannot be used for large, sparse applications. In this chapter, we will focus on those issues that are unique to the large, sparse NLP problem. The first item concerns how to calculate Jacobian and Hessian information for large, sparse problems. All of the Newton-based methods described in Chapter 1 rely on the availability of first and second derivative information. For most practical applications, first derivatives are computed using finite difference approximations. Unfortunately, a single finite difference gradient evaluation can require n additional function evaluations, and when n is large, this computational cost can be excessive.