Delay-Constrained Buffer-Aided Relay Selection in the Internet of Things With Decision-Assisted Reinforcement Learning

• Published in: IEEE internet of things journal Vol. 8; no. 12; pp. 10198 - 10208
• Main Authors: Huang, Chong, Chen, Gaojie, Gong, Yu
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 15.06.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: <italic xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">Q -learning; Algorithms; Buffer-aided relay selection; Buffers; Communications systems; Constraints; Convergence; Data models; Data transmission; deep reinforcement learning; Delay; delay constrained; Delays; Internet of Things; Learning; Network latency; Reinforcement learning; Relay networks; Relay networks (telecommunication); Relay systems; Sarsa learning; Service introduction; Throughput
• ISSN: 2327-4662; 2327-4662
• DOI: 10.1109/JIOT.2021.3051239

This article investigates the reinforcement learning for the relay selection in the delay-constrained buffer-aided networks. The buffer-aided relay selection significantly improves the outage performance but often at the price of higher latency. On the other hand, modern communication systems such as the Internet of Things often have strict requirement on latency. It is thus necessary to find relay selection policies to achieve good throughput performance in the buffer-aided relay network while stratifying the delay constraint. With the buffers employed at the relays and delay constraints imposed on the data transmission, obtaining the best relay selection becomes a complicated high-dimensional problem, making it hard for the reinforcement learning to converge. In this article, we propose the novel decision-assisted deep reinforcement learning to improve the convergence. This is achieved by exploring the a priori information from the buffer-aided relay system. The proposed approaches can achieve high throughput subject to delay constraints. Extensive simulation results are provided to verify the proposed algorithms.