Deep learning based simulators for the phosphorus removal process control in wastewater treatment via deep reinforcement learning algorithms

• Published in: Engineering applications of artificial intelligence Vol. 133; p. 107992
• Main Authors: Mohammadi, Esmaeel, Stokholm-Bjerregaard, Mikkel, Hansen, Aviaja Anna, Nielsen, Per Halkjær, Ortiz-Arroyo, Daniel, Durdevic, Petar
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.07.2024
• Subjects: Attention; Deep reinforcement learning; Dynamic model; Phosphorus; Sequence to sequence; Simulator; Sequence to sequence; Deep reinforcement learning; Phosphorus; Dynamic model; Simulator; Attention
• ISSN: 0952-1976; 1873-6769; 1873-6769
• DOI: 10.1016/j.engappai.2024.107992

Phosphorus removal is vital in wastewater treatment to reduce reliance on limited resources. Deep reinforcement learning (DRL) can be used to optimize the processes in wastewater treatment plants by learning control policies through trial and error. However, applying DRL to chemical and biological processes is challenging due to the need for accurate simulators. This study trained six models to identify the phosphorus removal process and used them to create a simulator for the DRL environment. While achieving high accuracy (>97%) in one-step ahead prediction of the test dataset, these models struggled as simulators over longer horizons, showing uncertainty and incorrect predictions when using their own outputs for multi-step simulations. Compounding errors in the models’ predictions were identified as one of the causes of this problem. This approach for improving process control involves creating simulation environments for DRL algorithms, using data from supervisory control and data acquisition (SCADA) systems with a sufficient historical horizon without complex system modeling or parameter estimation. [Display omitted] •Facilitating the training of deep reinforcement learning algorithms for wastewater treatment.•Investigating the state-of-the-art models’ potential to create simulation environments.•Stating the challenges of creating simulators for highly dynamic systems in control engineering.