DRL-UTPS: DRL-Based Trajectory Planning for Unmanned Aerial Vehicles for Data Collection in Dynamic IoT Network

• Published in: IEEE Transactions on Intelligent Vehicles Vol. 8; no. 2; pp. 1204 - 1218
• Main Authors: Liu, Run, Qu, Zhenzhe, Huang, Guosheng, Dong, Mianxiong, Wang, Tian, Zhang, Shaobo, Liu, Anfeng
• Format: Journal Article
• Language: English; Japanese
• Published: Piscataway IEEE 01.02.2023; Institute of Electrical and Electronics Engineers (IEEE); The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Autonomous aerial vehicles; Clusters; Data collection; deep reinforcement learning; Internet of Things; Path planning; Planning; Search algorithms; Sensors; Simulated annealing; Trajectory; Trajectory planning; UAV trajectory planning; unconnected network; Unmanned aerial vehicles; Wireless sensor networks
• ISSN: 2379-8858; 2379-8904
• DOI: 10.1109/TIV.2022.3213703

Using highly maneuverable Unmanned Aerial Vehicles (UAV) to collect data is a fast and efficient method that is widely studied. In most studies, they assume that the UAVs can obtain the location of the Cluster Head (CH) before take-off, allocate CHs, and optimize the trajectory in advance. However, in many real scenarios, many sensing devices are deployed in areas with no basic communication infrastructure or cannot communicate with the Internet due to emergencies such as disasters. In this kind of sensing network, the surviving devices often change, and the CHs cannot be known and allocated in advance, thus bringing new challenges to the efficient data collection of the networks by using UAVs. In this paper, a UAV path planning scheme for IoT networks based on reinforcement learning is proposed. It plans hover points for UAV by learning the historical location of CHs and maximizes the probability of meeting CHs and plans the shortest UAV path to visit all hover points by using the simulated annealing method. In addition, an algorithm to search for the location of CHs is proposed which is named Cluster-head Searching Algorithm with Autonomous Exploration Pattern (CHSA-AEP). By using CHSA-AEP, our scheme enables the UAV to respond to the position change of the CHs. Finally, we compare our scheme with other algorithms (area coverage algorithms and random algorithm). It is found that our proposed scheme is superior to other methods in energy efficiency and time utilization ratio.