Robot Navigation in Crowds Environment Base Deep Reinforcement Learning with POMDP

• Published in: Multimedia Technology and Enhanced Learning Vol. 446; pp. 675 - 685
• Main Authors: Li, Qinghua, Li, Haiming, Wang, Jiahui, Feng, Chao
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer 2022; Springer Nature Switzerland
• Series: Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
• Subjects: Deep reinforcement learning; Partially observable Markov decision process; Robot navigation
• ISBN: 3031181220; 9783031181221
• ISSN: 1867-8211; 1867-822X
• DOI: 10.1007/978-3-031-18123-8_53

With the development of deep learning technology, the navigation technology of mobile robot based on deep reinforcement learning is developing rapidly. But, navigation policy based on deep reinforcement learning still needs to be improved in crowds environment. The motion intention of pedestrians in crowds environment is variable, and the current motion intention information of pedestrian cannot be judged by only relying on a single frame of sensor sensing information. Therefore, in the case of only one frame of input, the pedestrian motion state information is partially observable. To dealing with this problem, we present the P-RL algorithm in this paper. The algorithm replaces traditional deep reinforcement learning Markov Decision Process model with a Partially Observable Markov Decision Process model, and introduces the LSTM neural network into the deep reinforcement learning algorithm. The LSTM neural network has the ability to process time series information, so that makes the algorithm has the ability to perceive the relationship between the observation data of each frame, which enhances the robustness of the algorithm. Experimental results show our algorithm is superior to other algorithms in time overhead and navigation success rate in crowds environment.