Payoff suboptimality and errors in value induced by approximation of the Hamiltonian

• Published in: 2008 47th IEEE Conference on Decision and Control pp. 3175 - 3180
• Main Authors: McEneaney, W.M., Deshpande, A.S.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.12.2008
• Subjects: Chromium; Computational efficiency; Dynamic programming; Error correction; Linearity; Nonlinear equations; Optimal control; Partial differential equations; Steady-state; Stochastic processes
• ISBN: 9781424431236; 1424431239
• ISSN: 0191-2216
• DOI: 10.1109/CDC.2008.4739382

Dynamic programming reduces the solution of optimal control problems to solution of the corresponding Hamilton-Jacobi-Bellman partial differential equations (HJB PDEs). In the case of nonlinear deterministic systems, the HJB PDEs are fully nonlinear, first-order PDEs. Standard, grid-based techniques to the solution of such PDEs are subject to the curse-of-dimensionality, where the computational costs grow exponentially with state-space dimension. Among the recently developed max-plus methods for solution of such PDEs, there is a curse-of-dimensionality-free algorithm. Such an algorithm can be applied in the case where the Hamiltonian takes the form of a pointwise maximum of a finite number of quadratic forms. In order to take advantage of this curse-of-dimensionality-free algorithm for more general HJB PDEs, we need to approximate the general Hamiltonian by a maximum of these quadratic forms. In doing so, one introduces some errors. In this work, we obtain a bound on the difference in solution of two HJB PDEs, as a function of a bound on the difference in the two Hamiltonians. Further, we obtain a bound on the suboptimality of the controller obtained from the solution of the approximate HJB PDE rather than from the original.