Adaptive multi-robot team reconfiguration using a policy-reuse reinforcement learning approach

• Published in: Proceedings of the 10th international conference on Advanced Agent Technology pp. 330 - 345
• Main Authors: Dasgupta, Prithviraj, Cheng, Ke, Banerjee, Bikramjit
• Format: Conference Proceeding
• Language: English
• Published: Berlin, Heidelberg Springer-Verlag 02.05.2011
• Series: ACM Conferences
• Subjects: coalition game; multi-robot formation; Q-learning
• ISBN: 9783642272158; 3642272150
• DOI: 10.1007/978-3-642-27216-5_23

We consider the problem of dynamically adjusting the formation and size of robot teams performing distributed area coverage, when they encounter obstacles or occlusions along their path. Based on our earlier formulation of the robotic team formation problem as a coalitional game called a weighted voting game (WVG), we show that the robot team size can be dynamically adapted by adjusting the WVG's quota parameter. We use a Q-learning algorithm to learn the value of the quota parameter and a policy reuse mechanism to adapt the learning process to changes in the underlying environment. Experimental results using simulated e-puck robots within the Webots simulator show that our Q-learning algorithm converges within a finite number of steps in different types of environments. Using the learning algorithm also improves the performance of an area coverage application where multiple robot teams move in formation to explore an initially unknown environment by 5−10%.