Evidence for model-based action planning in a sequential finger movement task

Reinforcement Learning: An Introduction

Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works

Reinforcement Learning: An Introduction

Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works

The Psychophysics Toolbox

Brainard, D. H. (1997) · Spatial Vision · doi:10.1163/156856897x00357 · Cited by 19,308 · see citing works

Q-learning

Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083 · see citing works

Parallel organization of functionally segregated circuits linking basal ganglia and cortex

Alexander, G. E., DeLong, M. R., & Strick, P. L. (1986) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.09.030186.002041 · Cited by 8,527 · see citing works

Internal models for motor control and trajectory planning.

Kawato, M. (1999) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(99)00028-8 · Cited by 2,685 · see citing works

Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control

Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · see citing works

A Closed-Loop Theory of Motor Learning

Adams, J. A. (1971) · Journal of Motor Behavior · doi:10.1080/00222895.1971.10734898 · Cited by 1,636 · see citing works

Anatomical evidence for cerebellar and basal ganglia involvement in higher cognitive function

Middleton, F. A., & Strick, P. L. (1994) · Science · doi:10.1126/science.7939688 · Cited by 1,356 · see citing works

Control of mental activities by internal models in the cerebellum

Ito, M. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2332 · Cited by 1,356 · see citing works

Interacting adaptive processes with different timescales underlie short-term motor learning

Smith, M. A., Ghazizadeh, A., & Shadmehr, R. (2006) · PLoS Biology · doi:10.1371/journal.pbio.0040179 · Cited by 1,113 · see citing works

Central mechanisms of motor skill learning

Hikosaka, O., Nakamura, K., Sakai, K., & Nakahara, H. (2002) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(02)00307-0 · Cited by 1,023 · see citing works

Complementary roles of basal ganglia and cerebellum in learning and motor control

Doya, K. (2000) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(00)00153-7 · Cited by 964 · see citing works

Determinants of individual differences during skill acquisition: Cognitive abilities and information processing.

Ackerman, P. L. (1988) · Journal of Experimental Psychology General · doi:10.1037/0096-3445.117.3.288 · Cited by 899 · see citing works

Parallel neural networks for learning sequential procedures

Hikosaka, O., Nakahara, H., Rand, M. K., Sakai, K., Lu, X., Nakamura, K., Miyachi, S., & Doya, K. (1999) · Trends in Neurosciences · doi:10.1016/s0166-2236(99)01439-3 · Cited by 799 · see citing works

What are the computations of the cerebellum, the basal ganglia and the cerebral cortex?

Doya, K. (1999) · Neural Networks · doi:10.1016/s0893-6080(99)00046-5 · Cited by 739 · see citing works

Anatomy of motor learning. I. Frontal cortex and attention to action

Jueptner, M., Stephan, K. M., Frith, C. D., Brooks, D. J., Frackowiak, R. S. J., & Passingham, R. E. (1997) · Journal of Neurophysiology · doi:10.1152/jn.1997.77.3.1313 · Cited by 624 · see citing works

The cognitive and neural architecture of sequence representation.

Keele, S. W., Ivry, R. B., Mayr, U., Hazeltine, E., & Heuer, H. (2003) · Psychological Review · doi:10.1037/0033-295x.110.2.316 · Cited by 503 · see citing works

Anatomy of Motor Learning. II. Subcortical Structures and Learning by Trial and Error

Jueptner, M., Frith, C., Brooks, D. J., Frackowiak, R. S. J., & Passingham, R. E. (1997) · Journal of Neurophysiology · doi:10.1152/jn.1997.77.3.1325 · Cited by 466 · see citing works

“Coordination, control and skill,”

Newell, K. M. (1985) · Advances in Psychology · doi:10.1016/s0166-4115(08)62541-8 · Cited by 423 · see citing works

Attention to action

Passingham, R. E. (1996) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.1996.0132 · Cited by 236 · see citing works

Dual Adaptation Supports a Parallel Architecture of Motor Memory

Lee, J.-Y., & Schweighofer, N. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.1294-09.2009 · Cited by 203 · see citing works

Parallel Cortico-Basal Ganglia Mechanisms for Acquisition and Execution of Visuomotor Sequences—A Computational Approach

Nakahara, H., Doya, K., & Hikosaka, O. (2001) · Journal of Cognitive Neuroscience · doi:10.1162/089892901750363208 · Cited by 192 · see citing works

Learning of sequential movements in the monkey: process of learning and retention of memory

Hikosaka, O., Rand, M. K., Miyachi, S., & Miyashita, K. (1995) · Journal of Neurophysiology · doi:10.1152/jn.1995.74.4.1652 · Cited by 181 · see citing works

Evidence for effector independent and dependent representations and their differential time course of acquisition during motor sequence learning

Bapi, R. S., Doya, K., & Harner, A. M. (2000) · Experimental Brain Research · doi:10.1007/s002219900332 · Cited by 176 · see citing works

Reinforcement learning: Computational theory and biological mechanisms

Doya, K. (2007) · HFSP journal · doi:10.2976/1.2732246 · Cited by 150 · see citing works

Differing perspectives in motor learning, memory and control

Shaffer, L. H. (1986) · Biological Psychology · doi:10.1016/0301-0511(86)90070-0 · Cited by 119 · see citing works

Reinforcement learning: Computational theory and biological mechanisms

Doya, K. (2007) · HFSP journal · doi:10.2976/1.2732246/10.2976/1 · Cited by 117 · see citing works

Sequence Learning - Paradigms, Algorithms, and Applications

Sun, R., & Giles, C. L. (2001) · Digital Access to Libraries (Université catholique de Louvain (UCL), l'Université de Namur (UNamur) and the Université Saint-Louis (USL-B)) · doi:10.1007/3-540-44565-x · Cited by 89 · see citing works

Effect on movement selection of an evolving sensory representation: A multiple controller model of skill acquisition

Shah, A., & Barto, A. G. (2009) · Brain Research · doi:10.1016/j.brainres.2009.07.006 · Cited by 16 · see citing works

Multiple Forward Model Architecture for Sequence Processing

Bapi, R. S., & Doya, K. (2000) · Lecture notes in computer science · doi:10.1007/3-540-44565-x_14 · Cited by 6 · see citing works

Goals and Habits in the Brain

Dolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977 · see citing works

Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models

Huang, V. S., Haith, A., Mazzoni, P., & Krakauer, J. W. (2011) · Neuron · doi:10.1016/j.neuron.2011.04.012 · Cited by 472 · see citing works

The ubiquity of model-based reinforcement learning

Doll, B. B., Simon, D. A., & Daw, N. D. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.08.003 · Cited by 380 · see citing works

Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation

Dayan, P., & Berridge, K. C. (2014) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-014-0277-8 · Cited by 357 · see citing works

Model-based choices involve prospective neural activity

Doll, B. B., Duncan, K., Simon, D. A., Shohamy, D., & Daw, N. (2015) · Nature Neuroscience · doi:10.1038/nn.3981 · Cited by 284 · see citing works

Model-Based and Model-Free Mechanisms of Human Motor Learning

Haith, A. M., & Krakauer, J. W. (2013) · Advances in Experimental Medicine and Biology · doi:10.1007/978-1-4614-5465-6_1 · Cited by 263 · see citing works

Actions, Action Sequences and Habits: Evidence That Goal-Directed and Habitual Action Control Are Hierarchically Organized

Dezfouli, A., & Balleine, B. W. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003364 · Cited by 221 · see citing works

The algorithmic anatomy of model-based evaluation

Daw, N. D., & Dayan, P. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0478 · Cited by 195 · see citing works

Twenty-Five Lessons from Computational Neuromodulation

Dayan, P. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.027 · Cited by 190 · see citing works

Dissociating hippocampal and striatal contributions to sequential prediction learning

Bornstein, A. M., & Daw, N. D. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07920.x · Cited by 120 · see citing works

Multiplicity of control in the basal ganglia: computational roles of striatal subregions

Bornstein, A. M., & Daw, N. D. (2011) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2011.02.009 · Cited by 97 · see citing works

Model-based action planning involves cortico-cerebellar and basal ganglia networks

Fermin, A. S. R., Yoshida, T., Yoshimoto, J., Ito, M., Tanaka, S., & Doya, K. (2016) · Scientific Reports · doi:10.1038/srep31378 · Cited by 65 · see citing works

The Singularity and Machine Ethics

Muehlhauser, L., & Helm, L. (2012) · The frontiers collection · doi:10.1007/978-3-642-32560-1_6 · Cited by 62 · see citing works

Compositional clustering in task structure learning

Franklin, N., & Frank, M. J. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006116 · Cited by 48 · see citing works

Humans can navigate complex graph structures acquired during latent learning

Rmus, M., Ritz, H., Hunter, L. E., Bornstein, A. M., & Shenhav, A. (2019) · Cognition · doi:10.1016/j.cognition.2022.105103 · Cited by 40 · see citing works

Separate encoding of model-based and model-free valuations in the human brain

Beierholm, U., Anen, C., Quartz, S. R., & Bossaerts, P. L. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.06.071 · Cited by 41 · see citing works

Navigating complex decision spaces: Problems and paradigms in sequential choice.

Walsh, M. M., & Anderson, J. R. (2013) · Psychological Bulletin · doi:10.1037/a0033455 · Cited by 39 · see citing works

Don’t Let Me Do That! – Models of Precommitment

Kurth-Nelson, Z., & Redish, A. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00138 · Cited by 37 · see citing works

Learning End-to-end Autonomous Driving using Guided Auxiliary Supervision

Mehta, A., Subramanian, A., & Subramanian, A. (2018) · Indian Conference on Computer Vision, Graphics and Image Processing · doi:10.1145/3293353.3293364 · Cited by 38 · see citing works

Generalizing to generalize: Humans flexibly switch between compositional and conjunctive structures during reinforcement learning

Franklin, N. T., & Frank, M. J. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007720 · Cited by 30 · see citing works

Reinforcement learning establishes a minimal metacognitive process to monitor and control motor learning performance

Sugiyama, T., Schweighofer, N., & Izawa, J. (2023) · Nature Communications · doi:10.1038/s41467-023-39536-9 · Cited by 27 · see citing works

Rationalizable Irrationalities of Choice

Dayan, P. (2014) · Topics in Cognitive Science · doi:10.1111/tops.12082 · Cited by 27 · see citing works

Revisiting the Role of the Medial Temporal Lobe in Motor Learning

McDougle, S. D., Wilterson, S. A., Turk‐Browne, N. B., & Taylor, J. A. (2021) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01809 · Cited by 24 · see citing works

Motor Chunking in Internally Guided Sequencing

Bera, K., Shukla, A. K., & Bapi, R. S. (2021) · Brain Sciences · doi:10.3390/brainsci11030292 · Cited by 23 · see citing works

Toward an Understanding of Dynamic Moral Decision Making: Model-Free and Model-Based Learning

Christopoulos, G. I., Liu, X., & Hong, Y. (2016) · Journal of Business Ethics · doi:10.1007/s10551-016-3058-1 · Cited by 20 · see citing works

Hierarchical clustering optimizes the tradeoff between compositionality and expressivity of task structures for flexible reinforcement learning

Liu, R. G., & Frank, M. J. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2022.103770 · Cited by 13 · see citing works

Cognitive and Motor Learning in Internally-Guided Motor Skills

Bera, K., Shukla, A. K., & Bapi, R. S. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.604323 · Cited by 15 · see citing works

The role of training variability for model-based and model-free learning of an arbitrary visuomotor mapping

Velázquez-Vargas, C. A., Daw, N. D., & Taylor, J. A. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1012471 · Cited by 9 · see citing works

Prefrontal neuronal ensembles link prior knowledge with novel actions during flexible action selection

Jarovi, J., Pilkiw, M., & Takehara‐Nishiuchi, K. (2023) · Cell Reports · doi:10.1016/j.celrep.2023.113492 · Cited by 8 · see citing works

Cognitive computational model reveals repetition bias in a sequential decision-making task

Legler, E., Rivera, D. C., Schwöbel, S., Wagner, B., & Kiebel, S. J. (2025) · Communications Psychology · doi:10.1038/s44271-025-00271-0 · Cited by 7 · see citing works

Learning to Move and Plan like the Knight: Sequential Decision Making with a Novel Motor Mapping

Velázquez-Vargas, C. A., & Taylor, J. A. (2025) · Computational Brain & Behavior · doi:10.1007/s42113-025-00245-9 · Cited by 3 · see citing works

Generalizing to generalize: when (and when not) to be compositional in task structure learningPreprint

Franklin, N. T., & Frank, M. J. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/547406 · Cited by 3 · see citing works

Learning generalizable visuomotor mappings fo r de novo skillsPreprint

Velázquez-Vargas, C. A., Daw, N. D., & Taylor, J. A. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.07.18.549179 · Cited by 3 · see citing works

Model-free decision-making underlies motor errors in rapid sequential movements under threat

Sankhe, P., & Haruno, M. (2024) · Communications Psychology · doi:10.1038/s44271-024-00123-3 · Cited by 2 · see citing works

Modelling Grid Navigation Using Reinforcement Learning Linear Ballistic Accumulators

Venugopal, G., & Bapi, R. S. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10192038 · Cited by 1 · see citing works

On the limit: Working memory capacity constrains learning of complex visuomotor mappingsPreprint

Bejjanki, V. R., & Taylor, J. A. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.01.30.702906 · Cited by 2 · see citing works

Learning to Move and Plan like the Knight: Sequential Decision Making with a Novel Motor Mapping

Velázquez-Vargas, C. A., & Taylor, J. A. (2025) · Computational Brain & Behavior · doi:10.1007/s42113-025-00245-9 · Cited by 3 · see citing works

Learning Expands the Preplanning Horizon in Finger Sequence Tasks

Kordjazi, N. (2018) · Scholarship@Western (Western University)

Sequence learning is driven by improvements in motor planning

Ariani, G., & Diedrichsen, J. (2019) · Journal of Neurophysiology · doi:10.1152/jn.00041.2019 · Cited by 82 · see citing works

Predictive Movements and Human Reinforcement Learning of Sequential Action

de Kleijn, R., Kachergis, G., & Hommel, B. (2018) · Cognitive Sciences · doi:10.1111/cogs.12599 · Cited by 9 · see citing works

Model-free decision-making underlies motor errors in rapid sequential movements under threat

Sankhe, P., & Haruno, M. (2024) · Communications Psychology · doi:10.1038/s44271-024-00123-3 · Cited by 2 · see citing works