Recurrent Reinforcement Learning Strategy with a Parameterized Agent for Online Scheduling of a State Task Network Under Uncertainty

• Published in: Industrial & engineering chemistry research Vol. 64; no. 13; pp. 7126 - 7140
• Main Authors: Rangel-Martinez, Daniel, Ricardez-Sandoval, Luis A.
• Format: Journal Article
• Language: English
• Published: American Chemical Society 02.04.2025
• Subjects: chemistry; hybrids; Process Systems Engineering; uncertainty
• ISSN: 0888-5885; 1520-5045; 1520-5045
• DOI: 10.1021/acs.iecr.4c04900

This study presents a framework for developing reinforcement learning hybrid agents that can build online schedules for state task networks under epistemic and aleatoric uncertainty. The hybrid agent can perform multiple discrete or continuous decisions at every time interval. To approach the uncertainty in the scheduling process, the hybrid agent is augmented with a set of LSTM layers that integrate a sequence of observations. This feature allows for the consideration of previous information to make decisions in view of the realization and propagation of uncertainties throughout the plant. Moreover, the techniques required for an efficient training oriented toward the objective function are described. The method is implemented in two case studies for validation and testing of the agent subject to epistemic and aleatoric uncertainty. A similar hybrid agent without recurrence is used as a benchmark. The proposed hybrid agent accumulated larger rewards while minimizing the number of constraint violations in the process under uncertainty, thus, making this online scheduling agent attractive for industrial-scale applications.