A Convex Duality Approach for Assigning Probability Distributions to the State of Nonlinear Stochastic Systems

• Published in: IEEE control systems letters Vol. 6; pp. 3080 - 3085
• Main Author: Pakniyat, Ali
• Format: Journal Article
• Language: English
• Published: IEEE 2022
• Subjects: Aerospace electronics; Costs; Nonlinear systems; Optimal control; Probability distribution; Stochastic optimal control; Stochastic systems; Time measurement
• ISSN: 2475-1456; 2475-1456
• DOI: 10.1109/LCSYS.2022.3181525

In order to optimally assign a desired probability distribution to the state of a nonlinear stochastic system, a convex duality approach is proposed to arrive at the associated optimality conditions. For a general class of stochastic systems governed by controlled Itô differential equations and subject to constraints on the probability distribution of the state at a fixed terminal time, a measure theoretic formulation is presented and it is shown that the original problem is embedded in a convex linear program on the space of Radon measures and that the embedding is tight, i.e., the optimal solution of both the original and the convex relaxation problems are equal. By exploiting the duality relationship between the space of continuous functions and that of measures, the associated optimality conditions are identified in the form of Hamilton-Jacobi problems where the optimization objective, in addition to the value function evaluation at the initial conditions, includes an extra term which is the integral of the product of the value function at the terminal time and the desired probability distribution. Numerical examples are provided to illustrate the results.