Stable iterative adaptive dynamic programming algorithm with approximation errors for discrete-time nonlinear systems

• Published in: Neural computing & applications Vol. 24; no. 6; pp. 1355 - 1367
• Main Authors: Wei, Qinglai, Liu, Derong
• Format: Journal Article
• Language: English
• Published: London Springer London 01.05.2014; Springer
• Subjects: Algorithmics. Computability. Computer arithmetics; Applied sciences; Artificial Intelligence; Computational Biology/Bioinformatics; Computational Science and Engineering; Computer Science; Computer science; control theory; systems; Control system synthesis; Control theory. Systems; Data Mining and Knowledge Discovery; Exact sciences and technology; Image Processing and Computer Vision; Learning and adaptive systems; Optimal control; Original Article; Probability and Statistics in Computer Science; Theoretical computing; Approximate dynamic programming; Adaptive dynamic programming; Neural networks; Nonlinear systems; Optimal control; Adaptive critic designs; Policy; On line; Stability; Reinforcement learning; Control program; Non linear control; Error bound; Iterative method; Neural network; Non linear system; Indirect method; Discrete time; Approximation error; Off line; Infinite horizon; Dynamic programming; Optimal control (mathematics)
• ISSN: 0941-0643; 1433-3058
• DOI: 10.1007/s00521-013-1361-7

In this paper, a novel iterative adaptive dynamic programming (ADP) algorithm is developed to solve infinite horizon optimal control problems for discrete-time nonlinear systems. When the iterative control law and iterative performance index function in each iteration cannot be accurately obtained, it is shown that the iterative controls can make the performance index function converge to within a finite error bound of the optimal performance index function. Stability properties are presented to show that the system can be stabilized under the iterative control law which makes the present iterative ADP algorithm feasible for implementation both on-line and off-line. Neural networks are used to approximate the iterative performance index function and compute the iterative control policy, respectively, to implement the iterative ADP algorithm. Finally, two simulation examples are given to illustrate the performance of the present method.