Accuracy-Guaranteed Collaborative DNN Inference in Industrial IoT via Deep Reinforcement Learning

• Published in: IEEE transactions on industrial informatics Vol. 17; no. 7; pp. 4988 - 4998
• Main Authors: Wu, Wen, Yang, Peng, Zhang, Weiting, Zhou, Conghao, Shen, Xuemin
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.07.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Algorithms; Artificial neural networks; Collaboration; Collaborative deep neural network (DNN) inference; Computation offloading; Constraints; deep reinforcement learning (RL); Delay; Delays; Edge computing; Electronic devices; Industrial applications; Industrial Internet of Things; Inference; inference accuracy; Inference algorithms; Internet of Things; Machine learning; Markov processes; Optimization; Optimization techniques; Resource allocation; Resource management; Sampling; sampling rate adaption; Sensors; Task analysis
• ISSN: 1551-3203; 1941-0050
• DOI: 10.1109/TII.2020.3017573

Collaboration among industrial Internet of Things (IoT) devices and edge networks is essential to support computation-intensive deep neural network (DNN) inference services, which require low delay and high accuracy. Sampling rate adaption, which dynamically configures the sampling rates of industrial IoT devices according to network conditions, is the key in minimizing the service delay. In this article, we investigate the collaborative DNN inference problem in industrial IoT networks. To capture the channel variation and task arrival randomness, we formulate the problem as a constrained Markov decision process (CMDP). Specifically, sampling rate adaption, inference task offloading, and edge computing resource allocation are jointly considered to minimize the average service delay while guaranteeing the long-term accuracy requirements of different inference services. Since CMDP cannot be directly solved by general reinforcement learning (RL) algorithms due to the intractable long-term constraints, we first transform the CMDP into an MDP by leveraging the Lyapunov optimization technique. Then, a deep RL-based algorithm is proposed to solve the MDP. To expedite the training process, an optimization subroutine is embedded in the proposed algorithm to directly obtain the optimal edge computing resource allocation. Extensive simulation results are provided to demonstrate that the proposed RL-based algorithm can significantly reduce the average service delay while preserving long-term inference accuracy with a high probability.