The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Reinforcement Learning – Overview of recent progress and implications for process control
Abstract
This paper provides an introduction to Reinforcement Learning (RL) technology, summarizes recent developments in this area, and discusses their potential implications for the field of process control, and more generally, of operational decision-making. The paper begins with an introduction to RL that allows an agent to learn, through trial and error, the best way to accomplish a task. We then highlight new developments in RL that have led to the recent wave of applications and media interest. A comparison of the key features of RL and mathematical programming based methods (e.g., model predictive control) is then presented to clarify their similarities and differences. This is followed by an assessment of several ways that RL technology can potentially be used in process control and operational decision applications. A final section summarizes our conclusions and lists directions for future RL research that may improve its relevance for the process systems engineering field.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
I.—COMPUTING MACHINERY AND INTELLIGENCE
TURING, A. M. (1950) · Mind · doi:10.1093/mind/lix.236.433 · Cited by 10,290
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Reinforcement Learning: An Introduction
Johnson, J. D., Li, J., & Chen, Z. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00324-6 · Cited by 8,414
(2006) · Springer eBooks · Cited by 8,123
Markov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Computing Machinery and Intelligence (1950)
Turing, A. (2004) · The Essential Turing · doi:10.1093/oso/9780198250791.003.0017 · Cited by 6,118
A survey of industrial model predictive control technology
Qin, S. J., & Badgwell, T. A. (2003) · Control Engineering Practice · doi:10.1016/s0967-0661(02)00186-7 · Cited by 5,099
Self-citationNeuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,233
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Baxter, L. A. (1995) · Technometrics · doi:10.1080/00401706.1995.10484354 · Cited by 2,415
Optimization under uncertainty: state-of-the-art and opportunities
Sahinidis, N. V. (2003) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2003.09.017 · Cited by 1,182
Optimization of Chemical Processes
Edgar, T. F., Himmelblau, D. M., & Lasdon, L. S. (1987) · Cited by 922
Zhang, H., Cui, L., Zhang, X., & Luo, Y. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2168538 · Cited by 643
Reinforcement learning is direct adaptive optimal control
(1992) · IEEE Control Systems · doi:10.1109/37.126844 · Cited by 524
A Survey on Explicit Model Predictive Control
Alessio, A., & Bemporad, A. (2009) · Lecture Notes in Control and Information Sciences · doi:10.1007/978-3-642-01094-1_29 · Cited by 511
Reinforcement learning for control: Performance, stability, and deep approximators
Buşoniu, L., de Bruin, T., Tolić, D., Kober, J., & Palunko, I. (2018) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2018.09.005 · Cited by 463
Lee, J. H., Shin, J., & Realff, M. J. (2018) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2017.10.008 · Cited by 387
Self-citationDearden, R., Friedman, N., & Russell, S. (1998) · Cited by 363
Value iteration and adaptive dynamic programming for data-driven adaptive optimal control design
Bian, T., & Jiang, Z.-P. (2016) · Automatica · doi:10.1016/j.automatica.2016.05.003 · Cited by 348
Policy evaluation with temporal differences
Dann, C., Neumann, G. M., & Peters, J. (2014) · Journal of Machine Learning Research · doi:10.5555/2627435.2638563 · Cited by 253
Bayesian Reinforcement Learning: A Survey
Ghavamzadeh, M., Mannor, S., Pineau, J., & Tamar, A. (2015) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000049 · Cited by 234
(2009) · Control Theory · doi:10.1109/9780470544334.ch10 · Cited by 207
Data-Efficient Reinforcement Learning with Probabilistic Model Predictive ControlPreprint
Kamthe, S., & Deisenroth, M. P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.06491 · Cited by 199
Mu, C., Ni, Z., Sun, C., & He, H. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2548941 · Cited by 181
An adaptive optimal controller for discrete-time Markov environments
Witten, I. H. (1977) · Information and Control · doi:10.1016/s0019-9958(77)90354-0 · Cited by 180
Optimization in Chemical Processes
Coker, A. K. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-30114-8.00011-0 · Cited by 179
Design and planning under uncertainty: issues on problem formulation and solution
Cheng, L., Subrahmanian, E., & Westerberg, A. (2003) · Computers & Chemical Engineering · doi:10.1016/s0098-1354(02)00264-8 · Cited by 129
Operational planning and optimal sizing of microgrid considering multi-scale wind uncertainty
Shin, J., Lee, J. H., & Realff, M. J. (2017) · Applied Energy · doi:10.1016/j.apenergy.2017.03.081 · Cited by 106
Self-citationLee, J. M., & Lee, J. H. (2004) · International Journal of Control Automation and Systems · Cited by 103
Self-citationYang, X., Liu, D., & Wei, Q. (2014) · IET Control Theory and Applications · doi:10.1049/iet-cta.2014.0186 · Cited by 89
Multitime scale Markov decision processes
Chang, H. S., Fard, P. J., Marcus, S. I., & Shayman, M. A. (2003) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2003.812782 · Cited by 83
Approximate dynamic programming approach for process control
Lee, J. H., & Wong, W. (2010) · Journal of Process Control · doi:10.1016/j.jprocont.2010.06.007 · Cited by 65
Self-citationEnergy supply planning and supply chain optimization under uncertainty
Lee, J. H. (2013) · Journal of Process Control · doi:10.1016/j.jprocont.2013.09.025 · Cited by 48
Self-citationInformation Collection on a Graph
Ryzhov, I. O., & Powell, W. B. (2011) · Operations Research · doi:10.1287/opre.1100.0873 · Cited by 44
Dual MPC with Reinforcement Learning
Morinelly, J. E., & Ydstie, B. E. (2016) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2016.07.276 · Cited by 23
Cited by
A review On reinforcement learning: Introduction and applications in industrial process control
Nian, R., Liu, J., & Huang, B. (2020) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2020.106886 · Cited by 711
Reinforcement learning algorithms: A brief survey
Shakya, A. K., Pillai, G., & Chakrabarty, S. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120495 · Cited by 656
Advances and opportunities in machine learning for process data analytics
Qin, S. J., & Chiang, L. H. (2019) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2019.04.003 · Cited by 351
Hu, Y., Miao, X., Si, Y., Pan, E., & Zio, E. (2022) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2021.108063 · Cited by 333
Process systems engineering – The generation next?
Pistikopoulos, E. N., Barbosa-Povoa, A., Lee, J. H., Misener, R., Mitsos, A., Reklaitis, G. V., Venkatasubramanian, V, You, F., & Gani, R. (2021) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2021.107252 · Cited by 260
Self-citation
Metadata
- Authors
- Shin, J., Badgwell, T. A., Liu, K.-H., & Lee, J. H.
- Venue
- Computers & Chemical Engineering
- Publisher
- Elsevier BV
- Published
- 2019-08
- Volume
- 127
- Pages
- 282-294
- ISSN
- 0098-1354 (ISSN-L)
- OpenAlex
- W2885282028
- Cited by
- 310
- References
- 38
- FWCI
- 13.01
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.