UAV Path Planning in a Dynamic Environment via Partially Observable Markov Decision Process
A path-planning algorithm to guide unmanned aerial vehicles (UAVs) for tracking multiple ground targets based on the theory of partially observable Markov decision processes (POMDPs) is presented. A variety of features of interest are shown to be easy to incorporate into the framework by plugging in...
Saved in:
| Published in | IEEE transactions on aerospace and electronic systems Vol. 49; no. 4; pp. 2397 - 2412 |
|---|---|
| Main Authors | , |
| Format | Journal Article |
| Language | English |
| Published |
IEEE
01.10.2013
|
| Subjects | |
| Online Access | Get full text |
| ISSN | 0018-9251 |
| DOI | 10.1109/TAES.2013.6621824 |
Cover
| Abstract | A path-planning algorithm to guide unmanned aerial vehicles (UAVs) for tracking multiple ground targets based on the theory of partially observable Markov decision processes (POMDPs) is presented. A variety of features of interest are shown to be easy to incorporate into the framework by plugging in the appropriate models, which demonstrates the power and flexibility of the POMDP framework. Specifically, it is shown how to incorporate the following features by appropriately formulating the POMDP action space, transition law, and objective function: 1) control UAVs with both forward acceleration and bank angle subject to constraints; 2) account for the effect of wind disturbance on UAVs; 3) avoid collisions between UAVs and obstacles and among UAVs; 4) track targets while evading threats; 5) track evasive targets; and 6) mitigate track swaps. |
|---|---|
| AbstractList | A path-planning algorithm to guide unmanned aerial vehicles (UAVs) for tracking multiple ground targets based on the theory of partially observable Markov decision processes (POMDPs) is presented. A variety of features of interest are shown to be easy to incorporate into the framework by plugging in the appropriate models, which demonstrates the power and flexibility of the POMDP framework. Specifically, it is shown how to incorporate the following features by appropriately formulating the POMDP action space, transition law, and objective function: 1) control UAVs with both forward acceleration and bank angle subject to constraints; 2) account for the effect of wind disturbance on UAVs; 3) avoid collisions between UAVs and obstacles and among UAVs; 4) track targets while evading threats; 5) track evasive targets; and 6) mitigate track swaps. |
| Author | Ragi, Shankarachary Chong, Edwin K. P. |
| Author_xml | – sequence: 1 givenname: Shankarachary surname: Ragi fullname: Ragi, Shankarachary email: shankar.ragi@colostate.edu organization: Dept. of Electr. & Comput. Eng., Colorado State Univ., Fort Collins, CO, USA – sequence: 2 givenname: Edwin K. P. surname: Chong fullname: Chong, Edwin K. P. organization: Dept. of Electr. & Comput. Eng., Colorado State Univ., Fort Collins, CO, USA |
| BookMark | eNp9kL9OwzAQhz0UibbwAIjFL5DiP4njjFVbKFJRK9GyMES2cwFD6iDbitS3J1ULAwPT6XS_7-70jdDAtQ4QuqFkQikp7rbTxfOEEconQjAqWTpAQ0KoTAqW0Us0CuGjb1OZ8iF63U1f8EbFd7xplHPWvWHrsMLzg1N7a_DCdda3bg8u4s6qPuqjVU1zwGsdwHdKN4CflP9sOzwHY4NtHd741kAIV-iiVk2A63Mdo939YjtbJqv1w-NsukoME1lMuNCSm5RrQisjSQW5klAUWSFTmivOBakrrlOhK1YA4yLLM2kkqwiBvK614WNET3uNb0PwUJdf3u6VP5SUlEcj5dFIeTRSno30TP6HMTaq2H8fvbLNv-TtibQA8HvpZ_oNDftyfQ |
| CODEN | IEARAX |
| CitedBy_id | crossref_primary_10_1016_j_asoc_2016_06_046 crossref_primary_10_1109_ACCESS_2023_3326435 crossref_primary_10_1109_JSYST_2021_3116020 crossref_primary_10_1007_s11370_021_00354_x crossref_primary_10_1177_1729881418787061 crossref_primary_10_1007_s13369_024_09295_w crossref_primary_10_3390_s18124266 crossref_primary_10_3390_rs16030471 crossref_primary_10_1007_s10846_018_0806_8 crossref_primary_10_1002_asjc_2083 crossref_primary_10_1631_FITEE_2000632 crossref_primary_10_1109_TAES_2018_2812538 crossref_primary_10_1016_j_ast_2015_09_037 crossref_primary_10_1016_j_asoc_2021_107153 crossref_primary_10_3390_aerospace3010001 crossref_primary_10_1109_TITS_2024_3363716 crossref_primary_10_2514_1_G003472 crossref_primary_10_1007_s10846_021_01342_0 crossref_primary_10_1016_j_autcon_2023_105253 crossref_primary_10_1109_TSP_2019_2939076 crossref_primary_10_1007_s10458_019_09401_5 crossref_primary_10_3182_20130626_3_AU_2035_00042 crossref_primary_10_1109_ACCESS_2019_2952235 crossref_primary_10_1109_JSYST_2021_3134657 crossref_primary_10_3390_drones8110682 crossref_primary_10_1051_jnwpu_20183620323 crossref_primary_10_1109_TAES_2022_3162567 crossref_primary_10_1016_j_ijpe_2021_108131 crossref_primary_10_1016_j_engappai_2022_105152 crossref_primary_10_1360_SSI_2024_0050 crossref_primary_10_1109_ACCESS_2019_2925633 crossref_primary_10_1016_j_seta_2023_103032 crossref_primary_10_1007_s12555_020_0565_8 crossref_primary_10_1109_TGRS_2023_3239952 crossref_primary_10_1016_j_ast_2016_04_002 crossref_primary_10_1109_TAES_2024_3401094 crossref_primary_10_1109_TITS_2022_3140724 crossref_primary_10_1177_0954410015609361 crossref_primary_10_1061_JCEMD4_COENG_14787 crossref_primary_10_1016_j_ast_2015_11_001 crossref_primary_10_1049_iet_rsn_2019_0178 crossref_primary_10_1177_0954410019844434 crossref_primary_10_3390_app7010083 crossref_primary_10_3390_a14030091 crossref_primary_10_3390_app8112169 crossref_primary_10_1109_TAES_2024_3357436 crossref_primary_10_1109_TCDS_2018_2840971 crossref_primary_10_1016_j_ast_2016_05_016 crossref_primary_10_1109_ACCESS_2023_3273164 crossref_primary_10_1016_j_neucom_2018_11_072 crossref_primary_10_3390_drones8020051 crossref_primary_10_1109_ACCESS_2017_2746752 crossref_primary_10_1364_OE_27_025688 crossref_primary_10_3390_jmse11010208 crossref_primary_10_1016_j_neucom_2017_05_059 crossref_primary_10_3390_rs13214481 crossref_primary_10_1016_j_ast_2018_10_017 crossref_primary_10_1109_TAES_2016_150727 crossref_primary_10_1016_j_ijleo_2020_165213 crossref_primary_10_1016_j_procs_2014_05_115 crossref_primary_10_2514_1_G004249 crossref_primary_10_1007_s11831_022_09742_7 crossref_primary_10_1016_j_tre_2023_103172 crossref_primary_10_1109_TCBB_2015_2443789 crossref_primary_10_3390_drones7010047 crossref_primary_10_1002_rob_21756 crossref_primary_10_1049_joe_2019_0229 crossref_primary_10_1109_ACCESS_2019_2918867 crossref_primary_10_1109_MAES_2016_150166 crossref_primary_10_1109_TAES_2017_2677746 crossref_primary_10_1109_TAC_2020_2970153 crossref_primary_10_1109_ACCESS_2024_3385860 crossref_primary_10_1109_TAES_2020_2983532 crossref_primary_10_3390_rs12203386 crossref_primary_10_1109_ACCESS_2020_3001393 crossref_primary_10_1109_LCSYS_2020_3043991 crossref_primary_10_3390_drones8100574 crossref_primary_10_23919_JSEE_2024_000022 crossref_primary_10_3390_s21062233 crossref_primary_10_1109_ACCESS_2019_2892716 crossref_primary_10_1177_1550147719877610 crossref_primary_10_3390_drones6120411 crossref_primary_10_1142_S2301385017500029 crossref_primary_10_1109_TETC_2019_2930124 crossref_primary_10_1145_3560261 crossref_primary_10_1109_LRA_2018_2800085 crossref_primary_10_1109_TRO_2023_3299524 crossref_primary_10_3390_s16050666 crossref_primary_10_1145_3347713 crossref_primary_10_1007_s11042_018_5739_5 crossref_primary_10_1016_j_vehcom_2020_100313 crossref_primary_10_1049_iet_rsn_2018_5184 crossref_primary_10_1109_ACCESS_2023_3347500 crossref_primary_10_1109_ACCESS_2021_3117451 crossref_primary_10_1016_j_engappai_2024_108911 crossref_primary_10_1109_JIOT_2018_2885147 crossref_primary_10_1142_S0218001416590047 crossref_primary_10_3390_s140814131 crossref_primary_10_1007_s11119_023_10017_9 |
| Cites_doi | 10.1155/2009/724597 10.1109/CDC.2004.1428735 10.1109/CISDA.2009.5356551 10.1109/ROBOT.2008.4543567 10.1109/CDC.2006.376928 10.1109/ACC.2007.4282479 10.1002/0471221279 10.2514/6.2010-8040 10.1007/3-540-44862-4_24 10.1109/ACC.2005.1470775 10.1109/70.508439 10.1109/IROS.2010.5650130 10.1109/TSMCB.2002.804370 10.2514/6.2005-6950 10.1109/21.44033 10.1109/TAC.2005.861685 10.2514/6.2006-6199 10.1007/978-1-4612-1166-2 10.1177/0278364910386986 10.1115/IMECE2007-43728 10.1109/TAES.2011.5705663 10.1016/j.automatica.2007.07.023 10.1109/7.913669 10.1109/TSMCB.2011.2157493 10.1007/s10626-009-0071-x 10.1109/ACC.2006.1657388 10.1017/CBO9780511546877 10.1109/TAC.2002.1008362 10.1023/A:1009634810396 10.3166/ejc.11.310-334 10.1109/CDC.2000.912059 10.1002/env.753 10.1214/aoms/1177729694 |
| ContentType | Journal Article |
| DBID | 97E RIA RIE AAYXX CITATION |
| DOI | 10.1109/TAES.2013.6621824 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Xplore url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EndPage | 2412 |
| ExternalDocumentID | 10_1109_TAES_2013_6621824 6621824 |
| Genre | orig-research |
| GroupedDBID | -~X 0R~ 29I 4.4 41~ 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACIWK ACNCT AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD F5P H~9 IAAWW IBMZZ ICLAB IFIPE IFJZH IPLJI JAVBF LAI M43 OCL P2P RIA RIE RNS TN5 VH1 AAYXX CITATION |
| ID | FETCH-LOGICAL-c265t-36b83c43b01dc80de7a8e99598417a3360fd3b46bd29e2365758c82d00e7ffbc3 |
| IEDL.DBID | RIE |
| ISSN | 0018-9251 |
| IngestDate | Thu Apr 24 22:56:35 EDT 2025 Wed Oct 01 01:23:26 EDT 2025 Wed Aug 27 02:52:20 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 4 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c265t-36b83c43b01dc80de7a8e99598417a3360fd3b46bd29e2365758c82d00e7ffbc3 |
| PageCount | 16 |
| ParticipantIDs | crossref_primary_10_1109_TAES_2013_6621824 ieee_primary_6621824 crossref_citationtrail_10_1109_TAES_2013_6621824 |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | 2013-10-01 |
| PublicationDateYYYYMMDD | 2013-10-01 |
| PublicationDate_xml | – month: 10 year: 2013 text: 2013-10-01 day: 01 |
| PublicationDecade | 2010 |
| PublicationTitle | IEEE transactions on aerospace and electronic systems |
| PublicationTitleAbbrev | T-AES |
| PublicationYear | 2013 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| References | ref35 ref13 ref37 ref15 ref36 ref14 bertsekas (ref25) 1996 ref32 ref10 sutton (ref26) 1998 ref2 ref1 ref39 ref17 ref38 ref16 ref19 ref18 bar-shalom (ref33) 1988 bertsekas (ref30) 2007; 2 lu (ref11) 2011; 41 ref24 ref23 blackman (ref31) 1999 cassandras (ref9) 2003 bellman (ref34) 1957 ren (ref12) 2005; 7 ref20 ref41 ref22 ref21 ref43 bhattacharyya (ref42) 1943; 35 ref28 ref27 ref29 ref8 mahalanobis (ref40) 1936 ref7 ref4 ref3 he (ref6) 2010 ref5 |
| References_xml | – ident: ref1 doi: 10.1155/2009/724597 – ident: ref24 doi: 10.1109/CDC.2004.1428735 – ident: ref14 doi: 10.1109/CISDA.2009.5356551 – ident: ref4 doi: 10.1109/ROBOT.2008.4543567 – ident: ref10 doi: 10.1109/CDC.2006.376928 – ident: ref17 doi: 10.1109/ACC.2007.4282479 – ident: ref32 doi: 10.1002/0471221279 – start-page: 284 year: 2003 ident: ref9 article-title: A receding horizon approach for dynamic UAV mission management publication-title: Proceedings of the SPIE 17th Annual International Symposium – ident: ref23 doi: 10.2514/6.2010-8040 – ident: ref38 doi: 10.1007/3-540-44862-4_24 – year: 1996 ident: ref25 publication-title: Neuro-Dynamic Programming – volume: 7 start-page: 4923 year: 2005 ident: ref12 article-title: Nonlinear tracking control for nonholonomic mobile robots with input constraints: An experimental study publication-title: Proceedings of the 2005 American Control Conference doi: 10.1109/ACC.2005.1470775 – ident: ref19 doi: 10.1109/70.508439 – volume: 35 start-page: 99 year: 1943 ident: ref42 article-title: On a measure of divergence between two statistical populations defined by their probability distributions publication-title: Bulletin of the Calcutta Mathematical Society – ident: ref15 doi: 10.1109/IROS.2010.5650130 – ident: ref21 doi: 10.1109/TSMCB.2002.804370 – ident: ref8 doi: 10.2514/6.2005-6950 – ident: ref20 doi: 10.1109/21.44033 – ident: ref13 doi: 10.1109/TAC.2005.861685 – year: 1957 ident: ref34 publication-title: Dynamic Programming – ident: ref36 doi: 10.2514/6.2006-6199 – ident: ref43 doi: 10.1007/978-1-4612-1166-2 – start-page: 1 year: 2010 ident: ref6 article-title: Efficient planning under uncertainty for a target-tracking micro-aerial vehicle publication-title: Proceedings of the 2010 IEEE International Conference on Robotics and Automation (ICRA) – ident: ref16 doi: 10.1177/0278364910386986 – ident: ref5 doi: 10.1115/IMECE2007-43728 – ident: ref39 doi: 10.1109/TAES.2011.5705663 – ident: ref7 doi: 10.1016/j.automatica.2007.07.023 – ident: ref22 doi: 10.1109/7.913669 – year: 1998 ident: ref26 publication-title: Reinforcement Learning An Introduction – volume: 41 start-page: 1556 year: 2011 ident: ref11 article-title: Incremental multi-scale search algorithm for dynamic path planning with low worst-case complexity publication-title: IEEE Transactions on Systems Man and Cybernetics Part B Cybernetics doi: 10.1109/TSMCB.2011.2157493 – ident: ref2 doi: 10.1007/s10626-009-0071-x – ident: ref18 doi: 10.1109/ACC.2006.1657388 – ident: ref3 doi: 10.1017/CBO9780511546877 – ident: ref29 doi: 10.1109/TAC.2002.1008362 – volume: 2 year: 2007 ident: ref30 publication-title: Dynamic Programming and Optimal Control – ident: ref27 doi: 10.1023/A:1009634810396 – year: 1988 ident: ref33 publication-title: Tracking and Data Association – year: 1999 ident: ref31 publication-title: Design and Analysis of Modern Tracking Systems – ident: ref35 doi: 10.3166/ejc.11.310-334 – ident: ref28 doi: 10.1109/CDC.2000.912059 – start-page: 49 year: 1936 ident: ref40 article-title: On the generalised distance in statistics publication-title: Proceedings of the National Institute of Sciences of India – ident: ref37 doi: 10.1002/env.753 – ident: ref41 doi: 10.1214/aoms/1177729694 |
| SSID | ssj0014843 |
| Score | 2.5153513 |
| Snippet | A path-planning algorithm to guide unmanned aerial vehicles (UAVs) for tracking multiple ground targets based on the theory of partially observable Markov... |
| SourceID | crossref ieee |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 2397 |
| SubjectTerms | Acceleration Approximation methods Covariance matrices Mathematical model Sensors Target tracking |
| Title | UAV Path Planning in a Dynamic Environment via Partially Observable Markov Decision Process |
| URI | https://ieeexplore.ieee.org/document/6621824 |
| Volume | 49 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Xplore issn: 0018-9251 databaseCode: RIE dateStart: 19650101 customDbUrl: isFulltext: true dateEnd: 99991231 titleUrlDefault: https://ieeexplore.ieee.org/ omitProxy: false ssIdentifier: ssj0014843 providerName: IEEE |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3dS8MwEA_bnvTBrynOL_Lgk9gubbJ-PA63MYSp4CYDH0ouyWBYOpFuoH-9SZvVKSK-lXA5Qi7hrrn73Q-hy5AB9wiVTsx87jBfgQM-iZ1QCZ9KEWuna9DIo7tgOGG30860hq4rLIxSqig-U675LHL5ciGW5qmsHQSm3ziro3oYBSVWq8oYsMhWyHn6AmunbTOYHonb427_0RRxUdcq-OaDNkhVCp8y2EWj9WrKUpIXd5mDKz5-NGr873L30I4NLnG3PA37qKayA7S90XKwiZ4n3Sf8oMM-vKYrwvMMc9wrmelx_wv4hldzrkW1Mp6m7_geigdcSBU2AJ_FCvcsPw-2aINDNBn0xzdDxxIsOMIPOrlDA4ioYBSIJ0VEpAp5pEwDsoh5Iac0IDNJgQUg_Vj51ORoIhH5khAVzmYg6BFqZItMHSMc63suAJSOtiQDxoEy4SkhJaFaUQgtRNZbngjbfdyQYKRJ8RdC4sRYKTFWSuy-tdBVNeW1bL3xl3DTGKAStMMnvw-foi0zuSzKO0ON_G2pznVwkcNFcao-AeOIyrk |
| linkProvider | IEEE |
| linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LTwIxEJ4gHtSDLzTiswdPxoXutuzjSASDCmgiGBIPm522JEQCxgCJ_nrb3YJojPG2aaaTptNmZjsz3wdwHnBMXMqkE3Evcbin0EGPRk6ghMekiLTTNd3Irbbf6PLbXqWXg8tFL4xSKi0-UyXzmeby5VhMzVNZ2fcN3jhfgdUK57ySdWstcgY8tDVyrr7C2m3bHKZLo3KnWn80ZVysZFV880JLtCqpV7negtZ8PVkxyUtpOsGS-PgB1fjfBW_Dpg0vSTU7DzuQU6Nd2FgCHSzAc7f6RB504EfmhEVkMCIJqWXc9KT-1fpGZoNEi2plyXD4Tu4xfcLFoSKmxWc8IzXL0ENsv8EedK_rnauGYykWHOH5lYnDfAyZ4AypK0VIpQqSUBkIspC7QcKYT_uSIfdRepHymMnShCL0JKUq6PdRsH3Ij8YjdQAk0jddICodb0mOPEHGhauElJRpRQEWgc63PBYWf9zQYAzj9D-ERrGxUmysFNt9K8LFYsprBr7xl3DBGGAhaIcPfx8-g7VGp9WMmzftuyNYN4qyEr1jyE_epupEhxoTPE1P2Cet3s4G |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=UAV+Path+Planning+in+a+Dynamic+Environment+via+Partially+Observable+Markov+Decision+Process&rft.jtitle=IEEE+transactions+on+aerospace+and+electronic+systems&rft.au=Ragi%2C+Shankarachary&rft.au=Chong%2C+Edwin+K.+P.&rft.date=2013-10-01&rft.issn=0018-9251&rft.volume=49&rft.issue=4&rft.spage=2397&rft.epage=2412&rft_id=info:doi/10.1109%2FTAES.2013.6621824&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TAES_2013_6621824 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9251&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9251&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9251&client=summon |