Multi-Armed Bandit Based Learning Algorithms for Offloading in Queueing Systems - Sushma, M.; Naveen, K. P. (IEEE Vehicular Technology Conference, 2024)
We propose a queueing theoretic based model to address the problem of offloading (packets or tasks) arising in multi-server systems. Using the framework of convex optimization we characterize the solution in terms of optimal offloading probabilities. We propose a low-complexity algorithm for identifying the optimal offloading probabilities; our algorithm is based on ordering the servers in terms of a proposed σ- metric that takes into account the residual service as well as expected queue-lengths of the servers. Using the structure of the optimal policy as a guideline, we design multi-armed bandit based learning algorithms for offloading packets using only estimates of the service rates. Finally we conduct a detailed simulation study to understand the efficacy of the proposed learning algorithms in terms of queue-length regret metric.