Deep reinforcement learning for data-driven scheduling in multi-variety and small-batch flexible job shops: Integrating fluid models for enhanced optimization

• Published in: Computers & industrial engineering Vol. 208; p. 111342
• Main Authors: Ding, Linshan, Luo, Dan, Rauf, Mudassar, Yue, Lei
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.10.2025
• Subjects: Data-driven; Data-driven scheduling; Deep reinforcement learning; Flexible job shop scheduling; Fluid model; Multi-variety and small-batch; Data-driven scheduling; Deep reinforcement learning; Data-driven; Multi-variety and small-batch; Fluid model; Flexible job shop scheduling
• ISSN: 0360-8352
• DOI: 10.1016/j.cie.2025.111342

The advancement of cloud computing, the internet of things (IoT), and big data facilitated the implementation of data-driven scheduling through the use of real-time monitoring and data processing in workshops. However, the implementation of data-driven scheduling in multi-variety and small-batch flexible job shops remains challenging due to the flexible and changeable systems caused by production patterns. Deep reinforcement learning (DRL) can handle complex state spaces and make prompt decisions, offering a potential method to enhance precision in scheduling and decision-making. However, DRL is subject to limitations, including the potential for local optima and low learning efficacy in workshop scheduling problems. To address the scheduling problem in multi-variety and small-batch flexible job shops (MVSB-FJSP), this study develops a data-driven integrated scheduling framework that incorporates a fluid model into DRL to enhance its state recognition capability and the quality of decision-making actions. The proposed framework exploits the intrinsic benefits of the fluid model in continuous system modeling to approximate the continuous flow of a diverse and multi-operation production process, thereby enhancing DRL’s comprehension and decision-making for the production process. This, in turn, improves DRL’s global optimization capability. Furthermore, a double deep Q-network (DDQN) based on the integrated scheduling framework is proposed. The DDQN integrates nine state features, five machine assignments, and four operation selection rules with the fluid model for intelligent decision-making. To enhance the robustness of the learned strategies across different problem sizes, the maximum completion time derived from the fluid model, serves as a normalization factor in defining the reward function. Comparative experiments with the DDQN, CPLEX, and other algorithms on a public dataset demonstrate the effectiveness and superiority of the proposed method. •Proposed a novel optimization method for the scheduling problem in multi-variety and small-batch flexible job shops (MVSB-FJSP).•Incorporated fluid models into deep reinforcement learning (DRL) based scheduling frameworks.•Outperform state-of-the-art algorithms on the specific public dataset.•Designed novel state features and scheduling rules linked to the fluid model.