A New Deep Reinforcement Learning Run-to-Run Control Algorithm for Mixed-Product Production Mode in Semiconductor Manufacturing

• Published in: IEEE transactions on automation science and engineering p. 1
• Main Authors: Fan, Shu-Kai S., Chen, Tzu-Jung
• Format: Journal Article
• Language: English
• Published: IEEE 2025
• Subjects: Deep Deterministic Policy Gradient (DDPG; Deep reinforcement learning; Heuristic algorithms; Indexes; Mathematical models; mixed-product Run-to-Run production; Process control; Production; Run-to-Run (R2R) control; Schedules; Semiconductor device manufacture; Semiconductor device modeling; Training
• ISSN: 1545-5955; 1558-3783
• DOI: 10.1109/TASE.2025.3526675

This paper proposes a new Run-to-Run (R2R) control framework based on deep deterministic policy gradient (DDPG) for the mixed-product production mode in semiconductor manufacturing. The DDPG algorithm is particularly developed to configure a deep reinforcement learning environment well suited to mixed-product production modes. To address the challenges posed in deep reinforcement learning, three enhanced mechanisms have been developed to improve the training of the proposed DDPG model for mixed-product R2R applications. These mechanisms include a piece-wise reward function, training with dynamic targets, and the new recall principle. It is demonstrated from the comprehensive simulation results that the proposed R2R control framework outperforms five noted mixed-product R2R control algorithms in the literature. The research outcome of this paper signifies a promising viability of deep reinforcement learning for highly complex and dynamic environments with continuous action spaces in the mixed-product R2R practice.