Multi-Objective Workflow Scheduling With Deep-Q-Network-Based Multi-Agent Reinforcement Learning

• Published in: IEEE access Vol. 7; pp. 39974 - 39982
• Main Authors: Wang, Yuandou, Liu, Hang, Zheng, Wanbo, Xia, Yunni, Li, Yawen, Chen, Peng, Guo, Kunyin, Xie, Hong
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Cloud computing; Completion time; Computer networks; deep-Q-network (DQN); Game theory; Games; Genetic algorithms; Greedy algorithms; infrastructure-as-a-service (IaaS) cloud; Job shops; Learning; Markov processes; multi-agent reinforcement learning (MARL); Multi-objective workflow scheduling; Multiagent systems; Multiple objective analysis; Optimal scheduling; Particle swarm optimization; quality-of-service (QoS); Real time; Reinforcement learning; Scheduling; Sorting algorithms; Task analysis; Virtual environments; Workflow software
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2019.2902846

Cloud Computing provides an effective platform for executing large-scale and complex workflow applications with a pay-as-you-go model. Nevertheless, various challenges, especially its optimal scheduling for multiple conflicting objectives, are yet to be addressed properly. The existing multi-objective workflow scheduling approaches are still limited in many ways, e.g., encoding is restricted by prior experts' knowledge when handling a dynamic real-time problem, which strongly influences the performance of scheduling. In this paper, we apply a deep-Q-network model in a multi-agent reinforcement learning setting to guide the scheduling of multi-workflows over infrastructure-as-a-service clouds. To optimize multi-workflow completion time and user's cost, we consider a Markov game model, which takes the number of workflow applications and heterogeneous virtual machines as state input and the maximum completion time and cost as rewards. The game model is capable of seeking for correlated equilibrium between make-span and cost criteria without prior experts' knowledge and converges to the correlated equilibrium policy in a dynamic real-time environment. To validate our proposed approach, we conduct extensive case studies based on multiple well-known scientific workflow templates and Amazon EC2 cloud. The experimental results clearly suggest that our proposed approach outperforms traditional ones, e.g., non-dominated sorting genetic algorithm-II, multi-objective particle swarm optimization, and game-theoretic-based greedy algorithms, in terms of optimality of scheduling plans generated.