Reinforcement Learning: An Introduction

Genetic algorithms in search, optimization, and machine learning

(1989) · Choice Reviews Online · doi:10.5860/choice.27-0936 · Cited by 49,010

Learning Internal Representations by Error Propagation

Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,172

Dynamic Programming

Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588

Untitled work

(1973) · CERN Document Server (European Organization for Nuclear Research) · Cited by 12,895

Matrix Iterative Analysis

Varga, R. S. (2000) · Springer Series in Computational Mathematics · doi:10.1007/978-3-642-05156-2 · Cited by 5,811

Learning from Delayed Rewards

Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,297

Parallel and Distributed Computation: Numerical Methods

Bertsekas, D. P., & Tsitsiklis, J. N. (1989) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 5,293

Learning to predict by the methods of temporal differences

Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148

Some Studies in Machine Learning Using the Game of Checkers

Samuel, A. L. (1959) · IBM Journal of Research and Development · doi:10.1147/rd.33.0210 · Cited by 4,443

Adaptive switching circuits

WIDROW, B., & HOFF, M. E. (1960) · doi:10.21236/ad0241531 · Cited by 4,009

DeSMP: Differential Privacy-exploited Stealthy Model Poisoning Attacks in Federated Learning

Hossain, M. T., Islam, S., Badsha, S., & Shen, H. (2021) · 2021 17th International Conference on Mobility, Sensing and Networking (MSN) · doi:10.1109/msn53354.2021.00038 · Cited by 44

Minimalistic Attacks: How Little It Takes to Fool Deep Reinforcement Learning Policies

Qu, X., Sun, Z., Ong, Y.-S., Gupta, A., & Wei, P. (2021) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2020.2974509 · Cited by 40

Protein–DNA/RNA interactions: machine intelligence tools and approaches in the era of artificial intelligence and big data

Cui, F., Zhang, Z., Cao, C., Zou, Q., Chen, D., & Su, X. (2022) · PROTEOMICS · doi:10.1002/pmic.202100197 · Cited by 40

A description–experience gap in statistical intuitions: Of smart babies, risk-savvy chimps, intuitive statisticians, and stupid grown-ups

Schulze, C., & Hertwig, R. (2021) · Cognition · doi:10.1016/j.cognition.2020.104580 · Cited by 36

Learning-based approach for layered adaptive video streaming over SDN

Uzakgider, T., Cetinkaya, C., & Sayit, M. (2015) · Computer Networks · doi:10.1016/j.comnet.2015.09.027 · Cited by 37

Between the primate and 'reptilian' brain: rodent models demonstrate the role of corticostriatal circuits in decision making

Lee, A. M., Tai, L.-H., Zador, A., & Wilbrecht, L. (2015) · Neuroscience · doi:10.1016/j.neuroscience.2014.12.042 · Cited by 37

Valuation of uncertain and delayed rewards in primate prefrontal cortex

Kim, S., Hwang, J., Seo, H., & Lee, D. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.03.010 · Cited by 37

Safe Reinforcement Learning via Shielding

Alshiekh, M., Bloem, R., Ehlers, R., Könighofer, B., Niekum, S., & Topcu, U. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11797 · Cited by 37

Cognitive prostheses for goal achievement

Lieder, F., Chen, O. X., Krueger, P. M., & Griffiths, T. L. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0672-9 · Cited by 32

Evidence for model-based action planning in a sequential finger movement task

Fermin, A., Yoshida, T., Ito, M., Yoshimoto, J., & Doya, K. (2010) · Journal of Motor Behavior · doi:10.1080/00222895.2010.526467 · Cited by 36

Conditioned associations and economic decision biases.

Guitart-Masip, M., Talmi, D., & Dolan, R. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.06.021 · Cited by 36

A reinforcement learning framework for relevance feedback

Montazeralghaem, A., Zamani, H., & Allan, J. (2020) · Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3397271.3401099 · Cited by 36

Idiosyncratic representation of peripersonal space depends on the success of one's own motor actions, but also the successful actions of others!

Coello, Y., Quesque, F., Gigliotti, M.-F., Ott, L., & Bruyelle, J.-L. (2018) · PLOS ONE · doi:10.1371/journal.pone.0196874 · Cited by 35

The principles of goal-directed decision-making: from neural mechanisms to computation and robotics

Pezzulo, G., Verschure, P. F. M. J., Balkenius, C., & Pennartz, C. M. A. (2014) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2013.0470 · Cited by 35

Designing a virtual patient dialogue system based on terminology-rich resources: Challenges and evaluation

Campillos-Llanos, L., Thomas, C., Bilinski, É., Zweigenbaum, P., & Rosset, S. (2020) · Natural Language Engineering · doi:10.1017/s1351324919000329 · Cited by 34

Neural Batch Sampling with Reinforcement Learning for Semi-supervised Anomaly Detection

Chu, W.-H., & Kitani, K. M. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58574-7_45 · Cited by 34

Reinforcement learning of linking and tracing contours in recurrent neural networks

Brosch, T., Neumann, H., & Roelfsema, P. R. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004489 · Cited by 33

Robot Online Learning Through Digital Twin Experiments: A Weightlifting Project

Verner, I., Cuperman, D., Fang, A., Reitman, M., Romm, T., & Balikin, G. (2018) · Lecture Notes in Networks and Systems · doi:10.1007/978-3-319-64352-6_29 · Cited by 34

ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulation

Xia, F., Li, C., Martín-Martín, R., Litany, O., Toshev, A., & Savarese, S. (2020) · arXiv · doi:10.48550/arxiv.2008.07792 · Cited by 33

Expectancy, ambiguity, and behavioral flexibility: separable and complementary roles of the orbital frontal cortex and amygdala in processing reward expectancies

Pauli, W. M., Hazy, T. E., & O'Reilly, R. C. (2012) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00155 · Cited by 32

A simple approach to ignoring irrelevant variables by population decoding based on multisensory neurons

Kim, H. R., Pitkow, X., Angelaki, D. E., & DeAngelis, G. C. (2016) · Journal of Neurophysiology · doi:10.1152/jn.00005.2016 · Cited by 32

A Stochastic Game Approach for Modeling Wholesale Energy Bidding in Deregulated Power Markets

Ragupathi, R., & Das, T. K. (2004) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2004.825910 · Cited by 31

Q-FDBA: improving QoE fairness for video streaming

Jiang, J., Hu, L., Hao, P., Sun, R., Hu, J., & Li, H. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-017-4917-1 · Cited by 31

Deep Reinforcement Active Learning for Medical Image Classification

Wang, J., Yan, Y., Zhang, Y., Cao, G., Yang, M., & Ng, M. K. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-59710-8_4 · Cited by 31

Protecting Semantic Trajectory Privacy for VANET with Reinforcement Learning

Wang, W., Min, M., Xiao, L., Chen, Y., & Dai, H. (2019) · ICC 2019 - 2019 IEEE International Conference on Communications (ICC) · doi:10.1109/icc.2019.8761415 · Cited by 31

Learning Robust Options

Mankowitz, D., Mann, T., Bacon, P.-L., Precup, D., & Mannor, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12115 · Cited by 30

Learning-Based and Data-Driven TCP Design for Memory-Constrained IoT

Li, W., Zhou, F., Meleis, W., & Chowdhury, K. (2016) · 2016 International Conference on Distributed Computing in Sensor Systems (DCOSS) · doi:10.1109/dcoss.2016.8 · Cited by 29

Model-Based Policy Search Using Monte Carlo Gradient Estimation With Real Systems Application

Amadio, F., Dalla Libera, A., Antonello, R., Nikovski, D., Carli, R., & Romeres, D. (2022) · IEEE Transactions on Robotics · doi:10.1109/tro.2022.3184837 · Cited by 29

FinRL-podracer: high performance and scalable deep reinforcement learning for quantitative finance

Li, Z., Liu, X.-Y., Zheng, J., Wang, Z., Walid, A., & Guo, J. (2021) · Proceedings of the Second ACM International Conference on AI in Finance · doi:10.1145/3490354.3494413 · Cited by 28

Organizational knowledge, learning and memory – A perspective of an immune system

Huang, J.-J. (2013) · Knowledge Management Research & Practice · doi:10.1057/kmrp.2011.48 · Cited by 28

Sensitivity to value-driven attention is predicted by how we learn from value

Jahfari, S., & Theeuwes, J. (2017) · Psychonomic Bulletin & Review · doi:10.3758/s13423-016-1106-6 · Cited by 28

Artificial intelligence as structural estimation: Deep Blue, Bonanza, and AlphaGo

Igami, M. (2020) · The Econometrics Journal · doi:10.1093/ectj/utaa005 · Cited by 28

Rational thoughts in neural codes

Wu, Z., Kwon, M., Daptardar, S., Schrater, P., & Pitkow, X. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1912336117 · Cited by 27

Computational psychopharmacology: a translational and pragmatic approach

Robbins, T. W., & Cardinal, R. N. (2019) · Psychopharmacology · doi:10.1007/s00213-019-05302-3 · Cited by 25

Differential reinforcement learning responses to positive and negative information in unmedicated individuals with depression

Reinen, J. M., Whitton, A. E., Pizzagalli, D. A., Slifstein, M., Abi-Dargham, A., McGrath, P. J., Iosifescu, D. V., & Schneier, F. R. (2021) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2021.08.002 · Cited by 27

Counterfactual Data Augmentation using Locally Factored Dynamics

Pitis, S., Creager, E., & Garg, A. (2020) · arXiv · doi:10.48550/arxiv.2007.02863 · Cited by 26

Untitled work

(2015) · International Conference on Artificial Intelligence · Cited by 25

Thinking While Moving: Deep Reinforcement Learning with Concurrent Control

Xiao, T., Jang, E., Kalashnikov, D., Levine, S., Ibarz, J., Hausman, K., & Herzog, A. (2020) · arXiv · doi:10.48550/arxiv.2004.06089 · Cited by 25

Goedel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements

Schmidhuber, J. (2003) · arXiv · doi:10.48550/arxiv.cs/0309048 · Cited by 24

Surprise beyond prediction error

Chumbley, J. R., Burke, C. J., Stephan, K. E., Friston, K. J., Tobler, P. N., & Fehr, E. (2014) · Human Brain Mapping · doi:10.1002/hbm.22513 · Cited by 23

Experimental Quantum Embedding for Machine Learning

Gianani, I., Mastroserio, I., Buffoni, L., Bruno, N., Donati, L., Cimini, V., Barbieri, M., Cataliotti, F. S., & Caruso, F. (2022) · Advanced Quantum Technologies · doi:10.1002/qute.202100140 · Cited by 23

Towards Finding Longer Proofs

Zombori, Z., Csiszárik, A., Michalewski, H., Kaliszyk, C., & Urban, J. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-86059-2_10 · Cited by 22

Learning adversarial attack policies through multi-objective reinforcement learning

García, J., Majadas, R., & Fernández, F. (2020) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2020.104021 · Cited by 22

Reinforcement-Learning-Based Variational Quantum Circuits Optimization for Combinatorial Problems

Khairy, S., Shaydulin, R., Cincio, L., Alexeev, Y., & Balaprakash, P. (2019) · arXiv · doi:10.48550/arxiv.1911.04574 · Cited by 21

A neural circuit architecture for rapid behavioral flexibility in goal-directed navigation

Dan, C., Hulse, B. K., Kappagantula, R., Jayaraman, V., & Hermundstad, A. M. (2021) · doi:10.1101/2021.08.18.456004 · Cited by 51

Optimization of Spirulina sp. cultivation using reinforcement learning with state prediction based on LSTM neural network

Doan, Y. T.-T., Ho, M.-T., Nguyen, H.-K., & Han, H.-D. (2021) · Journal of Applied Phycology · doi:10.1007/s10811-021-02488-y · Cited by 20

Trajectory-wise Multiple Choice Learning for Dynamics Generalization in Reinforcement Learning

Seo, Y., Lee, K., Clavera, I., Kurutach, T., Shin, J., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2010.13303 · Cited by 20

Count-based exploration with the successor representation

Machado, M. C., Bellemare, M. G., & Bowling, M. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5955 · Cited by 19

Adaptive Traffic Signal Control Using Distributed MARL and Federated Learning

Wang, T., Liang, T., Li, J., Zhang, W., Zhang, Y., & Lin, Y. (2020) · 2020 IEEE 20th International Conference on Communication Technology (ICCT) · doi:10.1109/icct50939.2020.9295660 · Cited by 19

Artificial intelligence–guided precision treatment of chronic kidney disease–mineral bone disorder

Gaweda, A. E., Lederer, E. D., & Brier, M. E. (2022) · CPT: Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.12843 · Cited by 18

Deep Blue: A Fuzzy Q-Learning Enhanced Active Queue Management Scheme

Masoumzadeh, S. S., Taghizadeh, G., Meshgi, K., & Shiry, S. (2009) · 2009 International Conference on Adaptive and Intelligent Systems · doi:10.1109/icais.2009.17 · Cited by 17

An adaptive AQM algorithm based on neuron reinforcement learning

Zhou, C., Di, D., Chen, Q., & Guo, J. (2009) · 2009 IEEE International Conference on Control and Automation · doi:10.1109/icca.2009.5410198 · Cited by 17

Transient Non-Stationarity and Generalisation in Deep Reinforcement Learning

Igl, M., Farquhar, G., Luketina, J., Boehmer, W., & Whiteson, S. (2020) · arXiv · doi:10.48550/arxiv.2006.05826 · Cited by 17

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I. (2018) · arXiv · doi:10.48550/arxiv.1810.07862 · Cited by 17

What's to Know? Uncertainty as a Guide to Asking Goal-Oriented Questions

Abbasnejad, E., Wu, Q., Shi, Q., & van den Hengel, A. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00428 · Cited by 16

Dopamine-Signaled Reward Predictions Generated by Competitive Excitation and Inhibition in a Spiking Neural Network Model

Chorley, P., & Seth, A. K. (2011) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2011.00021 · Cited by 16

Predictive Representations: Building Blocks of Intelligence

Carvalho, W., Tomov, M. S., de Cothi, W., Barry, C., & Gershman, S. J. (2024) · Neural Computation · doi:10.1162/neco_a_01705 · Cited by 15

The Difficulty of Passive Learning in Deep Reinforcement Learning

Ostrovski, G., Castro, P. S., & Dabney, W. (2021) · arXiv · doi:10.48550/arxiv.2110.14020 · Cited by 15

Interference and Generalization in Temporal Difference Learning

Bengio, E., Pineau, J., & Precup, D. (2020) · arXiv · doi:10.48550/arxiv.2003.06350 · Cited by 14

Hierarchical control using networks trained with higher-level forward models

Wayne, G., & Abbott, L. F. (2014) · Neural Computation · doi:10.1162/neco_a_00639 · Cited by 14

Explore, Approach, and Terminate: Evaluating Subtasks in Active Visual Object Search Based on Deep Reinforcement Learning

Schmid, J. F., Lauri, M., & Frintrop, S. (2019) · 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967805 · Cited by 12

A reinforcement-learning approach to efficient communication

Kågebäck, M., Carlsson, E., Dubhashi, D., & Sayeed, A. (2020) · PLOS ONE · doi:10.1371/journal.pone.0234894 · Cited by 13

Sim-to-Real in Reinforcement Learning for Everyone

Vacaro, J., Marques, G., Oliveira, B., Paz, G., Paula, T., Staehler, W., & Murphy, D. (2019) · 2019 Latin American Robotics Symposium (LARS), 2019 Brazilian Symposium on Robotics (SBR) and 2019 Workshop on Robotics in Education (WRE) · doi:10.1109/lars-sbr-wre48964.2019.00060 · Cited by 13

Quantum Observables for continuous control of the Quantum Approximate Optimization Algorithm via Reinforcement Learning

Garcia-Saez, A., & Riu, J. (2019) · arXiv · doi:10.48550/arxiv.1911.09682 · Cited by 13

Synthetic Returns for Long-Term Credit Assignment

Raposo, D., Ritter, S., Santoro, A., Wayne, G., Weber, T., Botvinick, M., van Hasselt, H., & Song, F. (2021) · arXiv · doi:10.48550/arxiv.2102.12425 · Cited by 12

Reward-based plasticity of spatial priority maps: Exploiting inter-subject variability to probe the underlying neurobiology

Della Libera, C., Calletti, R., Eštočinová, J., Chelazzi, L., & Santandrea, E. (2017) · Cognitive Neuroscience · doi:10.1080/17588928.2016.1213226 · Cited by 11

Exploration and preference satisfaction trade-off in reward-free learning

Sajid, N., Tigas, P., Zakharov, A., Fountas, Z., & Friston, K. (2021) · arXiv · doi:10.48550/arxiv.2106.04316 · Cited by 11

Untitled work

(2003) · doi:10.71910/supsi.7059 · Cited by 11

Using dialogue acts to learn better repair strategies for spoken dialogue systems

Frampton, M., & Lemon, O. (2008) · 2008 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2008.4518792 · Cited by 9

Interpretable Goal-based Prediction and Planning for Autonomous Driving

Albrecht, S. V., Brewitt, C., Wilhelm, J., Gyevnar, B., Eiras, F., Dobre, M., & Ramamoorthy, S. (2021) · 2021 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48506.2021.9560849 · Cited by 12

Resource-Efficient Sensor Data Management for Autonomous Systems Using Deep Reinforcement Learning

Jeong, S., Yoo, G., Yoo, M., Yeom, I., & Woo, H. (2019) · Sensors · doi:10.3390/s19204410 · Cited by 9

Podracer architectures for scalable Reinforcement Learning

Hessel, M., Kroiss, M., Clark, A., Kemaev, I., Quan, J., Keck, T., Viola, F., & van Hasselt, H. (2021) · arXiv · doi:10.48550/arxiv.2104.06272 · Cited by 8

Learning dynamic polynomial proofs

Fawzi, A., Malinowski, M., Fawzi, H., & Fawzi, O. (2019) · arXiv · doi:10.48550/arxiv.1906.01681 · Cited by 8

Playtesting: What is Beyond Personas

Ariyurek, S., Surer, E., & Betin-Can, A. (2023) · IEEE Transactions on Games · doi:10.1109/tg.2022.3165882 · Cited by 7

A two‐stage searching method for continuous time‐delay systems identification

Bai, M., & Li, M. (2021) · Asian Journal of Control · doi:10.1002/asjc.2199 · Cited by 7

Measuring Progress in Deep Reinforcement Learning Sample Efficiency

Dorner, F. E. (2021) · arXiv · doi:10.48550/arxiv.2102.04881 · Cited by 6

Sublinear regret for learning POMDPs

Xiong, Y., Chen, N., Gao, X., & Zhou, X. (2022) · Production and Operations Management · doi:10.1111/poms.13778 · Cited by 6

Using Unity to Help Solve Intelligence

Ward, T., Bolt, A., Hemmings, N., Carter, S., Sanchez, M., Barreira, R., Noury, S., Anderson, K., Lemmon, J., Coe, J., et al. (2020) · arXiv · doi:10.48550/arxiv.2011.09294 · Cited by 6

On Value Functions and the Agent-Environment Boundary

Jiang, N. (2019) · arXiv · doi:10.48550/arxiv.1905.13341 · Cited by 5

Multi-level feature driven storage management of surveillance videos

Tsai, M.-H., Venkatasubramanian, N., & Hsu, C.-H. (2021) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2021.101441 · Cited by 4

Towards Practical Credit Assignment for Deep Reinforcement Learning

Alipov, V., Simmons-Edler, R., Putintsev, N., Kalinin, P., & Vetrov, D. (2021) · arXiv · doi:10.48550/arxiv.2106.04499 · Cited by 3

Monte-Carlo Graph Search for AlphaZero

Czech, J., Korus, P., & Kersting, K. (2020) · arXiv · doi:10.48550/arxiv.2012.11045 · Cited by 3

Intrinsic Robotic Introspection: Learning Internal States From Neuron Activations

Pitsillos, N., Pore, A., Jensen, B. S., & Aragon-Camarasa, G. (2021) · 2021 IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/icdl49984.2021.9515672 · Cited by 2

Connecting Context-specific Adaptation in Humans to Meta-learning

Dubey, R., Grant, E., Luo, M., Narasimhan, K., & Griffiths, T. (2020) · arXiv · doi:10.48550/arxiv.2011.13782 · Cited by 1

Neural Episodic Control

Pritzel, A., Uria, B., Srinivasan, S., Puigdomènech, A., Vinyals, O., Hassabis, D., Wierstra, D., & Blundell, C. (2017) · arXiv · doi:10.48550/arxiv.1703.01988

Sequence Modeling of Temporal Credit Assignment for Episodic Reinforcement Learning

Liu, Y., Luo, Y., Zhong, Y., Chen, X., Liu, Q., & Peng, J. (2019) · arXiv · doi:10.48550/arxiv.1905.13420

Fast Task Inference with Variational Intrinsic Successor Features

Hansen, S., Dabney, W., Barreto, A., Van de Wiele, T., Warde-Farley, D., & Mnih, V. (2019) · arXiv · doi:10.48550/arxiv.1906.05030