An ARM-Based Q-Learning Algorithm

• Published in: Advanced Intelligent Computing Theories and Applications Vol. 2; pp. 11 - 20
• Main Authors: Hsu, Yuan-Pao, Hwang, Kao-Shing, Lin, Hsin-Yi
• Format: Book Chapter
• Language: English
• Published: The Netherlands Springer 2007; Springer Berlin / Heidelberg; Springer Berlin Heidelberg
• Series: Communications in Computer and Information Science
• Subjects: APPLICATIONS OF COMPUTING; Computer science
• ISBN: 3540742816; 9783540742814
• ISSN: 1865-0929; 1865-0937
• DOI: 10.1007/978-3-540-74282-1_2

This article presents an algorithm that combines a FAST-based algorithm (Flexible Adaptable-Size Topology), called ARM, and Q-learning algorithm. The ARM is a self organizing architecture. Dynamically adjusting the size of sensitivity regions of each neuron and adaptively pruning one of the redundant neurons, the ARM can preserve resources (available neurons) to accommodate more categories. The Q-learning is a dynamic programming-based reinforcement learning method, in which the learned action-value function, Q, directly approximates Q*, the optimal action-value function, independent of the policy being followed. In the proposed method, the ARM acts as a cluster to categorize input vectors from the outside world. Clustered results are then sent to the Q-learning architecture in order that it learns to present the best actions to the outside world. The effect of the algorithm is shown through computer simulations of the well-known control of balancing an inverted pendulum on a cart.