Reinforcement learning and digital twin-driven optimization of production scheduling with the digital model playground

• Published in: Discover Internet of things Vol. 4; no. 1; pp. 34 - 19
• Main Authors: Seipolt, Arne, Buschermöhle, Ralf, Haag, Vladislav, Hasselbring, Wilhelm, Höfinghoff, Maximilian, Schumacher, Marcel, Wilbers, Henrik
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 01.12.2024; Springer Nature B.V; Springer
• Subjects: Algorithms; Artificial intelligence; Computer Science; Cyber-physical systems; Digital technology; Digital twins; Digitizing; Discrete event simulation; Electrical Engineering; Hybrid simulation; Industrial Internet of Things; Information Systems Applications (incl.Internet); IoT; Job shops; Machine learning; Manufacturing; Optimization; Optimization algorithms; Playgrounds; Production planning; Production scheduling; Python; Reinforcement learning; Simulation; Traveling salesman problem; Production scheduling; Discrete event simulation; Digital twins; Reinforcement learning; Hybrid simulation
• ISSN: 2730-7239; 2730-7239
• DOI: 10.1007/s43926-024-00087-0

The significance of digital technologies in the context of digitizing production processes, such as Artificial Intelligence (AI) and Digital Twins, is on the rise. A promising avenue of research is the optimization of digital twins through Reinforcement Learning (RL). This necessitates a simulation environment that can be integrated with RL. One is introduced in this paper as the Digital Model Playground (DMPG). The paper outlines the implementation of the DMPG, followed by demonstrating its application in optimizing production scheduling through RL within a sample process. Although there is potential for further development, the DMPG already enables the modeling and optimization of production processes using RL and is comparable to commercial discrete event simulation software regarding the simulation-speed. Furthermore, it is highly flexible and adaptable, as shown by two projects, which distribute the DMPG to a high-performance cluster or generate 2D/3D-Visualization of the simulation model with Unreal. This establishes the DMPG as a valuable tool for advancing the digital transformation of manufacturing systems, affirming its potential impact on the future of production optimization. Currently, planned extensions include the integration of more optimization algorithms and Process Mining techniques, to further enhance the usability of the framework. Article Highlights Open-Source Flexibility: As a user-friendly, adaptable framework, DMPG is comparable to commercial simulation tools regarding the simulation speed. It can be used to distribute simulations on high-performance clusters or to generate 2D/3D-Visualization of processes with Unreal. Enhanced Production Scheduling: DMPG streamlines production scheduling using reinforcement learning. The extendable code structure allows the implementation of further simulation algorithms. Ongoing Development: Future enhancements include detailed transport and process mining, broadening its application.