Multi-Armed Bandit Based Learning Algorithms for Offloading in Queueing Systems

• Published in: IEEE Vehicular Technology Conference pp. 1 - 6
• Main Authors: Sushma, M., Naveen, K. P.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 24.06.2024
• Subjects: Complexity theory; Convex functions; convex optimization; Guidelines; M/M/1 queues; Measurement; Multi-Armed Bandits; Queueing analysis; Servers; UCB and Thompson Sampling; Vehicular and wireless technologies
• ISSN: 2577-2465
• DOI: 10.1109/VTC2024-Spring62846.2024.10683365

We propose a queueing theoretic based model to address the problem of offloading (packets or tasks) arising in multi-server systems. Using the framework of convex optimization we characterize the solution in terms of optimal offloading probabilities. We propose a low-complexity algorithm for identifying the optimal offloading probabilities; our algorithm is based on ordering the servers in terms of a proposed \sigma- \mathbf{metric} that takes into account the residual service as well as expected queue-lengths of the servers. Using the structure of the optimal policy as a guideline, we design multi-armed bandit based learning algorithms for offloading packets using only estimates of the service rates. Finally we conduct a detailed simulation study to understand the efficacy of the proposed learning algorithms in terms of queue-length regret metric.