Smart Multi-RAT Access Based on Multiagent Reinforcement Learning

• Published in: IEEE transactions on vehicular technology Vol. 67; no. 5; pp. 4539 - 4551
• Main Authors: Yan, Mu, Feng, Gang, Zhou, Jianhong, Qin, Shuang
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.05.2018; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: access control; Cellular communication; Complexity; Data management; Learning (artificial intelligence); Licensed band; Long Term Evolution; Machine learning; Markov analysis; Monte-Carlo tree search; Multiagent systems; Optimization; Quality of service; Quality of service architectures; Radio access technologies; reinforcement learning; Resource allocation; Resource management; Throughput; unlicensed band; Wireless communications; Wireless fidelity; Wireless networks
• ISSN: 0018-9545; 1939-9359
• DOI: 10.1109/TVT.2018.2793186

The ongoing increasing traffic in the era of big data yields unprecedented demands in user experience and network capacity expansion. The users of next generation mobile networks (5 G) should be able to use 3GPP, IEEE, and other technologies simultaneously. The integration of multiple radio access technologies (RATs) of licensed or unlicensed bands has been widely deemed as a cost-efficient way to greatly increase the network capacity. In this paper, we propose a smart aggregated RAT access (SARA) strategy with aim of maximizing the long-term network throughput while meeting diverse traffic quality of service (QoS) requirements. We consider the scenario that users with different QoS requirements access to a heterogeneous network with coexisting cellular-WiFi. In order to maximize system throughput while meeting diverse traffic QoS requirements in such a complex and dynamic environment, we exploit multiagent reinforcement learning to perform RAT selection in conjunction with resource allocation for individual user access requests, through sensing dynamic channel states and traffic QoS requirements. In SARA, we first use Nash Q-learning to provide a set of feasible RAT selection strategies while decreasing the strategy space in learning process, and then employ Monte Carlo tree search (MCTS) based Q-learning to perform resource allocation. Numerical results reveal that the network throughput can be maximized while meeting various traffic QoS requirements with limited number of searches by using our proposed SARA algorithm. For bulk arrival access requests, a suboptimal solution can be obtained as high computational complexity is incurred for achieving global optimality. Another attractive feature of SARA is that a tradeoff between the solution optimality and learning time can be readily made by terminating the search of MCTS according to the time constraint. Compared with traditional WiFi offloading schemes, SARA can significantly improve network throughput while guaranteeing traffic QoS requirements.