A multi-robot path-planning algorithm for autonomous navigation using meta-reinforcement learning based on transfer learning

• Published in: Applied soft computing Vol. 110; p. 107605
• Main Authors: Wen, Shuhuan, Wen, Zeteng, Zhang, Di, Zhang, Hong, Wang, Tao
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.10.2021
• Subjects: Deep reinforcement learning; Meta learning; Multi-robot system; Path planning; Transfer learning; Deep reinforcement learning; Multi-robot system; Path planning; Transfer learning; Meta learning
• ISSN: 1568-4946; 1872-9681
• DOI: 10.1016/j.asoc.2021.107605

The adaptability of multi-robot systems in complex environments is a hot topic. Aiming at static and dynamic obstacles in complex environments, this paper presents dynamic proximal meta policy optimization with covariance matrix adaptation evolutionary strategies (dynamic-PMPO-CMA) to avoid obstacles and realize autonomous navigation. Firstly, we propose dynamic proximal policy optimization with covariance matrix adaptation evolutionary strategies (dynamic-PPO-CMA) based on original proximal policy optimization (PPO) to obtain a valid policy of obstacles avoidance. The simulation results show that the proposed dynamic-PPO-CMA can avoid obstacles and reach the designated target position successfully. Secondly, in order to improve the adaptability of multi-robot systems in different environments, we integrate meta-learning with dynamic-PPO-CMA to form the dynamic-PMPO-CMA algorithm. In training process, we use the proposed dynamic-PMPO-CMA to train robots to learn multi-task policy. Finally, in testing process, transfer learning is introduced to the proposed dynamic-PMPO-CMA algorithm. The trained parameters of meta policy are transferred to new environments and regarded as the initial parameters. The simulation results show that the proposed algorithm can have faster convergence rate and arrive the destination more quickly than PPO, PMPO and dynamic-PPO-CMA. •Propose dynamic proximal policy optimization with covariance matrix adaptation evolutionary strategies (dynamic-PPO-CMA) to extend original proximal policy optimization (PPO) algorithm.•Propose a novel meta reinforcement learning framework for multi-robot path planning to improve the adaptation ability to new unknown environments.•Apply transfer learning to our framework for reducing on-board computation required to train a deep neural network.