Tags

Type your tag names separated by a space and hit enter

An influent responsive control strategy with machine learning: Q-learning based optimization method for a biological phosphorus removal system.
Chemosphere 2019; 234:893-901C

Abstract

Biological phosphorus removal (BPR) is an economical and sustainable processes for the removal of phosphorus (P) from wastewater, achieved by recirculating activated sludge through anaerobic and aerobic (An/Ae) processes. However, few studies have systematically analyzed the optimal hydraulic retention times (HRTs) in anaerobic and aerobic reactions, or whether these are the most appropriate control strategies. In this study, a novel optimization methodology using an improved Q-learning (QL) algorithm was developed, to optimize An/Ae HRTs in a BPR system. A framework for QL-based BPR control strategies was established and the improved Q function, Qt+1(st,st+1)=Qt(st,st+1)+k·[R(st,st+1)+γ·maxatQt(st,st+1)-Qt(st,st+1)] was derived. Based on the improved Q function and the state transition matrices obtained under different HRT step-lengths, the optimum combinations of HRTs in An/Ae processes in any BPR system could be obtained, in terms of the ordered pair combinations of the <current state-transition state>. Model verification was performed by applying six different influent chemical oxygen demand (COD) concentrations, varying from 150 to 600 mg L-1 and influent P concentrations, varying from 12 to 30 mg L-1. Superior and stable effluent qualities were observed with the optimal control strategies. This indicates that the proposed novel QL-based BPR model performed properly and the derived Q functions successfully realized real-time modelling, with stable optimal control strategies under fluctuant influent loads during wastewater treatment processes.

Authors+Show Affiliations

State Key Laboratory of Urban Water Resource and Environment, Harbin Institute of Technology, Harbin 150000, PR China.State Key Laboratory of Urban Water Resource and Environment, Harbin Institute of Technology, Harbin 150000, PR China. Electronic address: shanshanyang@hit.edu.cn.State Key Laboratory of Urban Water Resource and Environment, Harbin Institute of Technology, Harbin 150000, PR China.State Key Laboratory of Urban Water Resource and Environment, Harbin Institute of Technology, Harbin 150000, PR China.State Key Laboratory of Urban Water Resource and Environment, Harbin Institute of Technology, Harbin 150000, PR China.State Key Laboratory of Urban Water Resource and Environment, Harbin Institute of Technology, Harbin 150000, PR China.State Key Laboratory of Urban Water Resource and Environment, Harbin Institute of Technology, Harbin 150000, PR China.State Key Laboratory of Urban Water Resource and Environment, Harbin Institute of Technology, Harbin 150000, PR China.

Pub Type(s)

Journal Article

Language

eng

PubMed ID

31252361

Citation

Pang, Ji-Wei, et al. "An Influent Responsive Control Strategy With Machine Learning: Q-learning Based Optimization Method for a Biological Phosphorus Removal System." Chemosphere, vol. 234, 2019, pp. 893-901.
Pang JW, Yang SS, He L, et al. An influent responsive control strategy with machine learning: Q-learning based optimization method for a biological phosphorus removal system. Chemosphere. 2019;234:893-901.
Pang, J. W., Yang, S. S., He, L., Chen, Y. D., Cao, G. L., Zhao, L., ... Ren, N. Q. (2019). An influent responsive control strategy with machine learning: Q-learning based optimization method for a biological phosphorus removal system. Chemosphere, 234, pp. 893-901. doi:10.1016/j.chemosphere.2019.06.103.
Pang JW, et al. An Influent Responsive Control Strategy With Machine Learning: Q-learning Based Optimization Method for a Biological Phosphorus Removal System. Chemosphere. 2019 Jun 14;234:893-901. PubMed PMID: 31252361.
* Article titles in AMA citation format should be in sentence-case
TY - JOUR T1 - An influent responsive control strategy with machine learning: Q-learning based optimization method for a biological phosphorus removal system. AU - Pang,Ji-Wei, AU - Yang,Shan-Shan, AU - He,Lei, AU - Chen,Yi-Di, AU - Cao,Guang-Li, AU - Zhao,Lei, AU - Wang,Xin-Yu, AU - Ren,Nan-Qi, Y1 - 2019/06/14/ PY - 2019/02/17/received PY - 2019/06/11/revised PY - 2019/06/13/accepted PY - 2019/6/30/pubmed PY - 2019/6/30/medline PY - 2019/6/29/entrez KW - ASM2d KW - Biological phosphorus removal KW - Fluctuant influent loads KW - Improved QL algorithm KW - Machine learning KW - Real-time control strategy SP - 893 EP - 901 JF - Chemosphere JO - Chemosphere VL - 234 N2 - Biological phosphorus removal (BPR) is an economical and sustainable processes for the removal of phosphorus (P) from wastewater, achieved by recirculating activated sludge through anaerobic and aerobic (An/Ae) processes. However, few studies have systematically analyzed the optimal hydraulic retention times (HRTs) in anaerobic and aerobic reactions, or whether these are the most appropriate control strategies. In this study, a novel optimization methodology using an improved Q-learning (QL) algorithm was developed, to optimize An/Ae HRTs in a BPR system. A framework for QL-based BPR control strategies was established and the improved Q function, Qt+1(st,st+1)=Qt(st,st+1)+k·[R(st,st+1)+γ·maxatQt(st,st+1)-Qt(st,st+1)] was derived. Based on the improved Q function and the state transition matrices obtained under different HRT step-lengths, the optimum combinations of HRTs in An/Ae processes in any BPR system could be obtained, in terms of the ordered pair combinations of the <current state-transition state>. Model verification was performed by applying six different influent chemical oxygen demand (COD) concentrations, varying from 150 to 600 mg L-1 and influent P concentrations, varying from 12 to 30 mg L-1. Superior and stable effluent qualities were observed with the optimal control strategies. This indicates that the proposed novel QL-based BPR model performed properly and the derived Q functions successfully realized real-time modelling, with stable optimal control strategies under fluctuant influent loads during wastewater treatment processes. SN - 1879-1298 UR - https://www.unboundmedicine.com/medline/citation/31252361/An_influent_responsive_control_strategy_with_machine_learning:_Q-learning_based_optimization_method_for_a_biological_phosphorus_removal_system L2 - https://linkinghub.elsevier.com/retrieve/pii/S0045-6535(19)31344-X DB - PRIME DP - Unbound Medicine ER -