Adaptive Dynamic Programming for Stochastic Systems With State and Control Dependent Noise

In this technical note, the adaptive optimal control problem is investigated for a class of continuous-time stochastic systems subject to multiplicative noise. A novel non-model-based optimal control design methodology is employed to iteratively update the control policy on-line by using directly th...

Full description

Saved in:
Bibliographic Details
Published inIEEE transactions on automatic control Vol. 61; no. 12; pp. 4170 - 4175
Main Authors Bian, Tao, Jiang, Yu, Jiang, Zhong-Ping
Format Journal Article
LanguageEnglish
Published New York IEEE 01.12.2016
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects
Adaptive control
Adaptive dynamic programming
adaptive optimal control
Adaptive systems
Algorithm design and analysis
Algorithms
Convergence
Dynamic programming
Noise control
Optimal control
Robustness
Stability analysis
Stochastic processes
Stochastic systems
Online AccessGet full text
ISSN0018-9286
1558-2523
DOI10.1109/TAC.2016.2550518

Cover

Abstract In this technical note, the adaptive optimal control problem is investigated for a class of continuous-time stochastic systems subject to multiplicative noise. A novel non-model-based optimal control design methodology is employed to iteratively update the control policy on-line by using directly the data of the system state and input. Both adaptive dynamic programming (ADP) and robust ADP algorithms are developed, along with rigorous stability and convergence analysis. The effectiveness of the obtained methods is illustrated by an example arising from biological sensorimotor control.
AbstractList In this technical note, the adaptive optimal control problem is investigated for a class of continuous-time stochastic systems subject to multiplicative noise. A novel non-model-based optimal control design methodology is employed to iteratively update the control policy on-line by using directly the data of the system state and input. Both adaptive dynamic programming (ADP) and robust ADP algorithms are developed, along with rigorous stability and convergence analysis. The effectiveness of the obtained methods is illustrated by an example arising from biological sensorimotor control.
Author Tao Bian
Zhong-Ping Jiang
Yu Jiang
Author_xml – sequence: 1
  givenname: Tao
  surname: Bian
  fullname: Bian, Tao
– sequence: 2
  givenname: Yu
  surname: Jiang
  fullname: Jiang, Yu
– sequence: 3
  givenname: Zhong-Ping
  surname: Jiang
  fullname: Jiang, Zhong-Ping
BookMark eNp9kM1LAzEQxYNUsK3eBS8Bz1uTbJLdPZbWLygqtCJ4WbLZ2Talm9QkFfrfu6XFgwdPw8y8N8P7DVDPOgsIXVMyopQUd4vxZMQIlSMmBBE0P0N9KkSeMMHSHuoTQvOkYLm8QIMQ1l0rOad99Dmu1Taab8DTvVWt0fjNu6VXbWvsEjfO43l0eqVC7FbzfYjQBvxh4qqbqwhY2RpPnI3ebfAUtmBrsBG_OBPgEp03ahPg6lSH6P3hfjF5Smavj8-T8SzRrKAxURkFJYBXjJCUFIoCyyshlahA5qrhhdKSpbrICqpkTSUTDZFMSw0kFTWt0iG6Pd7deve1gxDLtdt5270sac6lSHnOWaciR5X2LgQPTbn1plV-X1JSHgiWHcHyQLA8Eews8o9Fmy60OcRVZvOf8eZoNADw-yfjPMtYmv4ARaJ_Ug
CODEN IETAA9
CitedBy_id crossref_primary_10_1109_TASE_2019_2948431
crossref_primary_10_1049_iet_cta_2019_0934
crossref_primary_10_1016_j_amc_2019_124568
crossref_primary_10_1109_JAS_2023_123186
crossref_primary_10_1007_s11424_022_1146_0
crossref_primary_10_1109_TSMC_2023_3284612
crossref_primary_10_3390_math12101533
crossref_primary_10_1016_j_arcontrol_2022_03_005
crossref_primary_10_1016_j_neucom_2025_129758
crossref_primary_10_1002_rnc_6191
crossref_primary_10_1080_00207721_2021_1929554
crossref_primary_10_1016_j_asoc_2024_112417
crossref_primary_10_1109_TCYB_2025_3530951
crossref_primary_10_1109_TCYB_2024_3468875
crossref_primary_10_1002_rnc_6432
crossref_primary_10_1109_TNNLS_2023_3347663
crossref_primary_10_1162_neco_a_01260
crossref_primary_10_3390_photonics11100927
crossref_primary_10_1007_s12190_023_01857_9
crossref_primary_10_1002_oca_2794
crossref_primary_10_1109_TCYB_2021_3050619
crossref_primary_10_1016_j_neucom_2024_127269
crossref_primary_10_1109_TCYB_2022_3203795
crossref_primary_10_1109_ACCESS_2023_3254879
crossref_primary_10_1080_00207721_2024_2392834
crossref_primary_10_1109_TNNLS_2021_3053269
crossref_primary_10_1007_s00422_022_00922_z
crossref_primary_10_1109_TCYB_2023_3320441
crossref_primary_10_1016_j_automatica_2024_111848
crossref_primary_10_1109_TNNLS_2021_3136939
crossref_primary_10_1109_TAC_2022_3172250
crossref_primary_10_1016_j_ast_2024_109446
crossref_primary_10_1109_TCNS_2021_3074256
crossref_primary_10_1109_TAC_2019_2905215
crossref_primary_10_1016_j_amc_2022_127763
crossref_primary_10_1109_TNNLS_2023_3244934
crossref_primary_10_1109_TAC_2022_3145632
crossref_primary_10_1109_TCYB_2024_3403680
crossref_primary_10_1016_j_neucom_2019_12_001
crossref_primary_10_1109_TETCI_2023_3301789
crossref_primary_10_1016_j_automatica_2018_09_028
crossref_primary_10_1016_j_ifacol_2023_10_880
crossref_primary_10_1002_acs_2862
crossref_primary_10_1007_s11424_025_4572_y
crossref_primary_10_1016_j_automatica_2016_05_003
crossref_primary_10_1007_s11424_024_2421_z
crossref_primary_10_1016_j_amc_2024_128803
crossref_primary_10_1007_s11768_021_00046_y
crossref_primary_10_1016_j_automatica_2025_112144
crossref_primary_10_1109_LCSYS_2020_2995547
crossref_primary_10_1109_TCYB_2021_3108034
crossref_primary_10_1016_j_isatra_2020_02_019
crossref_primary_10_1016_j_automatica_2023_111490
Cites_doi 10.1109/TSMCB.2010.2043839
10.1109/TNNLS.2013.2294968
10.1109/MCAS.2009.933854
10.1016/j.ejcon.2013.05.017
10.1109/TAC.1971.1099828
10.1109/TNN.2011.2165729
10.1016/j.automatica.2014.08.023
10.1109/TNNLS.2015.2453320
10.1137/S0363012996301336
10.1523/JNEUROSCI.05-07-01688.1985
10.1523/JNEUROSCI.1110-06.2007
10.1016/j.automatica.2012.06.096
10.1016/j.sysconle.2005.07.005
10.1007/978-1-4471-4757-2
10.1109/TAC.2003.821400
10.1016/j.automatica.2016.05.003
10.1007/BF01211469
10.1016/S0167-6911(97)00008-X
10.1016/j.automatica.2007.02.024
10.1109/TAC.1969.1099303
10.1007/s00422-014-0613-7
10.1007/s00221-003-1443-3
10.1109/TAC.2015.2414811
10.1016/j.automatica.2014.10.015
10.1109/TIE.2014.2345343
10.1109/TAC.2014.2317301
10.1109/MCS.2012.2214134
10.1007/s11768-016-5117-7
10.1109/TNNLS.2015.2424971
10.1016/j.automatica.2006.08.028
10.1016/0005-1098(76)90029-7
10.1016/j.automatica.2012.05.049
10.1007/978-1-4757-6577-9
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2016
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2016
DBID 97E
RIA
RIE
AAYXX
CITATION
7SC
7SP
7TB
8FD
FR3
JQ2
L7M
L~C
L~D
DOI 10.1109/TAC.2016.2550518
DatabaseName IEEE Xplore (IEEE)
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE Electronic Library (IEL)
CrossRef
Computer and Information Systems Abstracts
Electronics & Communications Abstracts
Mechanical & Transportation Engineering Abstracts
Technology Research Database
Engineering Research Database
ProQuest Computer Science Collection
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts – Academic
Computer and Information Systems Abstracts Professional
DatabaseTitle CrossRef
Technology Research Database
Computer and Information Systems Abstracts – Academic
Mechanical & Transportation Engineering Abstracts
Electronics & Communications Abstracts
ProQuest Computer Science Collection
Computer and Information Systems Abstracts
Engineering Research Database
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts Professional
DatabaseTitleList
Technology Research Database
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Xplore
  url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
EISSN 1558-2523
EndPage 4175
ExternalDocumentID 10_1109_TAC_2016_2550518
7447723
Genre orig-research
GrantInformation_xml – fundername: National Science Foundation
  grantid: ECCS-1101401; ECCS-1230040; ECCS-1501044
  funderid: 10.13039/100000001
GroupedDBID -~X
.DC
0R~
29I
3EH
4.4
5GY
5VS
6IK
97E
AAJGR
AARMG
AASAJ
AAWTH
ABAZT
ABQJQ
ABVLG
ACGFO
ACGFS
ACIWK
ACNCT
AENEX
AETIX
AGQYO
AGSQL
AHBIQ
AI.
AIBXA
AKJIK
AKQYR
ALLEH
ALMA_UNASSIGNED_HOLDINGS
ASUFR
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CS3
DU5
EBS
EJD
F5P
HZ~
H~9
IAAWW
IBMZZ
ICLAB
IDIHD
IFIPE
IFJZH
IPLJI
JAVBF
LAI
M43
MS~
O9-
OCL
P2P
RIA
RIE
RNS
TAE
TN5
VH1
VJK
~02
AAYXX
CITATION
7SC
7SP
7TB
8FD
FR3
JQ2
L7M
L~C
L~D
RIG
ID FETCH-LOGICAL-c291t-a71ea5e4b200309a1e28b56a5be68af49ac623c9791a6d1625f062c6ce035d1b3
IEDL.DBID RIE
ISSN 0018-9286
IngestDate Mon Jun 30 10:21:13 EDT 2025
Thu Apr 24 23:01:39 EDT 2025
Wed Oct 01 04:15:23 EDT 2025
Wed Aug 27 02:52:18 EDT 2025
IsDoiOpenAccess false
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue 12
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
https://doi.org/10.15223/policy-029
https://doi.org/10.15223/policy-037
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c291t-a71ea5e4b200309a1e28b56a5be68af49ac623c9791a6d1625f062c6ce035d1b3
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
PQID 1846534842
PQPubID 85475
PageCount 6
ParticipantIDs crossref_primary_10_1109_TAC_2016_2550518
crossref_citationtrail_10_1109_TAC_2016_2550518
proquest_journals_1846534842
ieee_primary_7447723
ProviderPackageCode CITATION
AAYXX
PublicationCentury 2000
PublicationDate 2016-Dec.
2016-12-00
20161201
PublicationDateYYYYMMDD 2016-12-01
PublicationDate_xml – month: 12
  year: 2016
  text: 2016-Dec.
PublicationDecade 2010
PublicationPlace New York
PublicationPlace_xml – name: New York
PublicationTitle IEEE transactions on automatic control
PublicationTitleAbbrev TAC
PublicationYear 2016
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References ref35
ref13
ref34
ref12
ref37
ref15
krsti? (ref24) 1998
ref31
ref30
ref33
ref11
ref32
sutton (ref36) 1998
ref17
ref38
ref16
ref19
ref18
lewis (ref25) 2013
krsti? (ref23) 1995
ref26
flash (ref10) 1985; 5
ref20
ref41
ref22
ref21
horn (ref14) 2013
ref28
ref27
åström (ref2) 1997
ref29
ref8
ref7
werbos (ref39) 1968; 3
ref9
ref4
ref3
ref6
arnold (ref1) 1974
ref5
ref40
References_xml – year: 2013
  ident: ref14
  publication-title: Matrix Analysis
– volume: 3
  start-page: 131
  year: 1968
  ident: ref39
  article-title: The elements of intelligence
  publication-title: Cybernetica (Namur)
– year: 1995
  ident: ref23
  publication-title: Nonlinear and Adaptive Control Design
– ident: ref26
  doi: 10.1109/TSMCB.2010.2043839
– ident: ref18
  doi: 10.1109/TNNLS.2013.2294968
– ident: ref27
  doi: 10.1109/MCAS.2009.933854
– ident: ref20
  doi: 10.1016/j.ejcon.2013.05.017
– ident: ref31
  doi: 10.1109/TAC.1971.1099828
– ident: ref15
  doi: 10.1109/TNN.2011.2165729
– ident: ref4
  doi: 10.1016/j.automatica.2014.08.023
– ident: ref34
  doi: 10.1109/TNNLS.2015.2453320
– ident: ref13
  doi: 10.1137/S0363012996301336
– volume: 5
  start-page: 1688
  year: 1985
  ident: ref10
  article-title: The coordination of arm movements: An experimentally confirmed mathematical model
  publication-title: J Neurosci
  doi: 10.1523/JNEUROSCI.05-07-01688.1985
– ident: ref29
  doi: 10.1523/JNEUROSCI.1110-06.2007
– ident: ref16
  doi: 10.1016/j.automatica.2012.06.096
– ident: ref3
  doi: 10.1016/j.sysconle.2005.07.005
– ident: ref41
  doi: 10.1007/978-1-4471-4757-2
– ident: ref8
  doi: 10.1109/TAC.2003.821400
– ident: ref7
  doi: 10.1016/j.automatica.2016.05.003
– ident: ref21
  doi: 10.1007/BF01211469
– year: 1998
  ident: ref36
  publication-title: Reinforcement Learning An Introduction
– ident: ref9
  doi: 10.1016/S0167-6911(97)00008-X
– ident: ref35
  doi: 10.1016/j.automatica.2007.02.024
– ident: ref22
  doi: 10.1109/TAC.1969.1099303
– ident: ref17
  doi: 10.1007/s00422-014-0613-7
– ident: ref11
  doi: 10.1007/s00221-003-1443-3
– ident: ref19
  doi: 10.1109/TAC.2015.2414811
– ident: ref37
  doi: 10.1016/j.automatica.2014.10.015
– ident: ref5
  doi: 10.1109/TIE.2014.2345343
– ident: ref32
  doi: 10.1109/TAC.2014.2317301
– ident: ref28
  doi: 10.1109/MCS.2012.2214134
– ident: ref6
  doi: 10.1007/s11768-016-5117-7
– ident: ref33
  doi: 10.1109/TNNLS.2015.2424971
– ident: ref30
  doi: 10.1016/j.automatica.2006.08.028
– year: 1974
  ident: ref1
  publication-title: Stochastic Differential Equations Theory and Applications
– year: 2013
  ident: ref25
  publication-title: Reinforcement Learning and Approximate Dynamic Programming for Feedback Control
– ident: ref40
  doi: 10.1016/0005-1098(76)90029-7
– ident: ref38
  doi: 10.1016/j.automatica.2012.05.049
– year: 1998
  ident: ref24
  publication-title: Stabilization of Nonlinear Uncertain Systems
– ident: ref12
  doi: 10.1007/978-1-4757-6577-9
– year: 1997
  ident: ref2
  publication-title: Adaptive Control
SSID ssj0016441
Score 2.4934123
Snippet In this technical note, the adaptive optimal control problem is investigated for a class of continuous-time stochastic systems subject to multiplicative noise....
SourceID proquest
crossref
ieee
SourceType Aggregation Database
Enrichment Source
Index Database
Publisher
StartPage 4170
SubjectTerms Adaptive control
Adaptive dynamic programming
adaptive optimal control
Adaptive systems
Algorithm design and analysis
Algorithms
Convergence
Dynamic programming
Noise control
Optimal control
Robustness
Stability analysis
Stochastic processes
Stochastic systems
Title Adaptive Dynamic Programming for Stochastic Systems With State and Control Dependent Noise
URI https://ieeexplore.ieee.org/document/7447723
https://www.proquest.com/docview/1846534842
Volume 61
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIEE
  databaseName: IEEE Xplore
  customDbUrl:
  eissn: 1558-2523
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0016441
  issn: 0018-9286
  databaseCode: RIE
  dateStart: 19630101
  isFulltext: true
  titleUrlDefault: https://ieeexplore.ieee.org/
  providerName: IEEE
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3PS8MwFH7oTnrwtzidkoMXwXZNljbNcWyKCBNBRfFSkjRFUTvR7uJfb36tiIp4CyQpIS8v-dL38n0AhyrJudaSRZxWKqJcmBJmOqJSpQbAUla5ZMzJRXZ2Q8_v0rsFOG7fwmitXfKZjm3RxfLLqZrZX2V9RqkBg4NFWGSM-7dabcTAnut-1zUOTPI2JJnw_vVwZHO4sphYOG7lPb4cQU5T5cdG7E6X01WYzMflk0qe4lkjY_XxjbLxvwNfg5UAM9HQr4t1WND1Bix_IR_chPthKV7tZofGXpUeXfpcrRdTjQyWRVfNVD0IS-SMArE5un1sHpADqEjUJRr5RHc0DlK6DbqYPr7rLbg5PbkenUVBaSFShOMmEgxrkWoqiQuNCqxJLtNMpFJnuaiMCZWBSYozjkVWYnNnqpKMqMyKjaUlloNt6NTTWu8AMj6NsbT1lhvN4EPJDCyQJBeSaMZUF_rzyS9UoCG3ahjPhbuOJLww5iqsuYpgri4ctT1ePQXHH2037ey37cLEd6E3t28RfPS9MHfbLB3QnJLd33vtwZL9tk9e6UGneZvpfQNBGnng1t4n9z_WRw
linkProvider IEEE
linkToHtml http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3PT8IwFH5BPKgHfxtR1B68mDigpVu3IwENKhATIRIvS9uVaFQgMi7-9fbHWIga461J26zp62u_7n19H8C5rIWRUoJ5ER1Jj0ZclzBTHhXS1wCWspElY3Z7QXtAb4f-sACX-VsYpZQln6mKKdpYfjKRc_OrrMoo1WCwvgKrvi4R91orjxmYk93tu9qFSZgHJWtRtd9oGhZXUCEGkBuBj6VDyKqq_NiK7flyvQXdxcgcreS1Mk9FRX5-S9r436Fvw2YGNFHDrYwdKKjxLmwspR_cg6dGwqdmu0Mtp0uP7h1b611XI41m0UM6kc_cpHJGWWpz9PiSPiMLUREfJ6jpqO6olYnppqg3eZmpfRhcX_WbbS_TWvAkiXDqcYYV9xUVxAZHOVYkFH7AfaGCkI-0EaUGSjJiEeZBgvWtaVQLiAyM3JifYFE_gOJ4MlaHgLRXYyxMvcmOphGiYBoYCBJyQRRjsgTVxeTHMktEbvQw3mJ7IalFsTZXbMwVZ-YqwUXeY-qScPzRds_Mft4um_gSlBf2jTMvncX6dhv4dRpScvR7rzNYa_e7nbhz07s7hnXzHUdlKUMx_ZirEw1IUnFq1-EXVbrZkg
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Adaptive+Dynamic+Programming+for+Stochastic+Systems+With+State+and+Control+Dependent+Noise&rft.jtitle=IEEE+transactions+on+automatic+control&rft.au=Bian%2C+Tao&rft.au=Jiang%2C+Yu&rft.au=Zhong-Ping%2C+Jiang&rft.date=2016-12-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.issn=0018-9286&rft.eissn=1558-2523&rft.volume=61&rft.issue=12&rft.spage=4170&rft_id=info:doi/10.1109%2FTAC.2016.2550518&rft.externalDBID=NO_FULL_TEXT
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9286&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9286&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9286&client=summon