Environment Aware Adaptive Q-Learning to Deploy SFC on Edge Computing

• Published in: International Conference on Network and Service Management (Print) pp. 1 - 5
• Main Authors: Pandey, Suman, Hong, James W., Yoo, Jae-Hyung
• Format: Conference Proceeding
• Language: English
• Published: IFIP 02.11.2020
• Subjects: Computational modeling; Edge computing; Heuristic algorithms; Machine learning algorithms; Network topology; OpenStack; Q-Learning; Reinforcement Learning; SDN; Servers; SFC; Topology
• ISSN: 2165-963X
• DOI: 10.23919/CNSM50824.2020.9269046

Biggest challenge in deploying Service Function Chain (SFC) in the Edge Computing environment is the lack of resources at the edge. Hence while finding the optimum path for SFC deployment, the resource constraint environment should be observed and incorporated well in deployment scenarios. In this paper, we developed an environment aware adaptive Q-Learning algorithm to find an optimal SFC deployment path in edge computing environment. The available servers are divided into hierarchical network structure with local, neighbor, and datacenter servers to model an edge computing environment. The resource dynamics in the environment is modeled as a state transition probability. We compared the new algorithm with our base case algorithm that solely depends on Q-Learning and doesn't incorporate the state transition probabilities. An intuitive reward function is designed to give maximum reward to complex deployment with minimum delays. We integrated our algorithm with physical testbeds using OpenStack and open source REST APIs. We evaluated SFC deployment on physical testbed using 42 different scenarios by measuring RTT.