On the Out-of-Sample Performance of Stochastic Dynamic Programming and Model Predictive Control

• Published in: INFORMS journal on optimization
• Main Authors: Keehan, Dominic S. T., Philpott, Andrew B., Anderson, Edward J.
• Format: Journal Article
• Language: English
• Published: 01.08.2025
• ISSN: 2575-1484; 2575-1492; 2575-1492
• DOI: 10.1287/ijoo.2024.0060

This paper aims to understand when stochastic dynamic programming or model predictive control is the more appropriate method for multistage decision making under uncertainty. It reveals a connection to distributionally ambiguous optimization that depends on problem structure, and provides more specific conditions for an example revenue optimization problem. The research emerged from initial studies of the out-of-sample performance of distributionally robust optimization by the second and third authors, and observations of good performance of model predictive control applied to practical problems arising in the New Zealand dairy industry. This phenomenon is the subject of the Ph.D. study of the first author, under the supervision of the second. Sample average approximation-based stochastic dynamic programming (SDP) and model predictive control (MPC) are two different methods for approaching multistage stochastic optimization. In this paper we investigate the conditions under which SDP may be outperformed by MPC. We show that, depending on the presence of concavity or convexity, MPC can be interpreted as solving a mean-constrained distributionally ambiguous version of the problem that is solved by SDP. This furnishes performance guarantees when the true mean is known and provides intuition for why MPC performs better in some applications and worse in others. We then study a multistage stochastic optimization problem that is representative of the type for which MPC may be the better choice. We find that this can indeed be the case when the probability distribution of the underlying random variable is skewed or has enough weight in the right-hand tail. Funding: The first and second authors acknowledge support from UOCX2117 MBIE Catalyst Fund New Zealand–German Platform for Green Hydrogen Integration (HINT).