Time representation in reinforcement learning models of the basal ganglia

Reinforcement Learning: An Introduction

Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works

Reinforcement Learning: An Introduction

Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works

A neural substrate of prediction and reward

Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works

A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement

Rescorla, R. A. (1972) · Medical Entomology and Zoology · Cited by 7,268 · see citing works

Scalar expectancy theory and Weber's law in animal timing.

Gibbon, J. (1977) · Psychological Review · doi:10.1037/0033-295x.84.3.279 · Cited by 2,224 · see citing works

What makes us tick? Functional and neural mechanisms of interval timing

Buhusi, C. V., & Meck, W. H. (2005) · Nature reviews. Neuroscience · doi:10.1038/nrn1764 · Cited by 2,074 · see citing works

A framework for mesencephalic dopamine systems based on predictive Hebbian learning

Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · see citing works

Scalar Timing in Memory

Gibbon, J., Church, R. M., & Meck, W. H. (1984) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1984.tb23417.x · Cited by 1,806 · see citing works

Midbrain dopamine neurons encode a quantitative reward prediction error signal

Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304 · see citing works

Dopamine neurons report an error in the temporal prediction of reward during learning

Hollerman, J. R., & Schultz, W. (1998) · Nature Neuroscience · doi:10.1038/1124 · Cited by 1,103 · see citing works

The neostriatal mosaic: multiple levels of compartmental organization in the basal ganglia

Gerfen, C. R. (1992) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.15.030192.001441 · Cited by 1,004 · see citing works

A causal link between prediction errors, dopamine neurons and learning

Steinberg, E. E., Keiflin, R., Boivin, J. R., Witten, I. B., Deisseroth, K. A., & Janak, P. H. (2013) · Nature Neuroscience · doi:10.1038/nn.3413 · Cited by 879 · see citing works

Cortico-striatal circuits and interval timing: coincidence detection of oscillatory processes

Matell, M. S., & Meck, W. H. (2004) · Cognitive Brain Research · doi:10.1016/j.cogbrainres.2004.06.012 · Cited by 877 · see citing works

Opponent interactions between serotonin and dopamine

Daw, N. D., Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00052-7 · Cited by 844 · see citing works

Neuroanatomical and Neurochemical Substrates of Timing

Coull, J. T., Cheng, R., & Meck, W. H. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2010.113 · Cited by 810 · see citing works

Toward a neurobiology of temporal cognition: advances and challenges

Gibbon, J., Malapani, C., Dale, C. L., & Gallistel, C. R. (1997) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(97)80005-0 · Cited by 766 · see citing works

Reinforcement learning in the brain

Niv, Y. (2009) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2008.12.005 · Cited by 724 · see citing works

Neural Basis of the Perception and Estimation of Time

Merchant, H., Harrington, D. L., & Meck, W. H. (2013) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-062012-170349 · Cited by 710 · see citing works

Isolation of an internal clock.

Roberts, S. (1981) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037/0097-7403.7.3.242 · Cited by 635 · see citing works

Representation of Time by Neurons in the Posterior Parietal Cortex of the Macaque

Leon, M. I., & Shadlen, M. N. (2003) · Neuron · doi:10.1016/s0896-6273(03)00185-5 · Cited by 601 · see citing works

A Model of How the Basal Ganglia Generate and Use Neural Signals That Predict Reinforcement

Houk, J. C., Adams, J. L., & Barto, A. G. (1994) · The MIT Press eBooks · doi:10.7551/mitpress/4708.003.0020 · Cited by 555 · see citing works

Dopamine-dependent plasticity of corticostriatal synapses

Reynolds, J. N. J., & Wickens, J. R. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00045-x · Cited by 528 · see citing works

Actor–critic models of the basal ganglia: new anatomical and computational perspectives

Joel, D., Niv, Y., & Ruppin, E. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00047-3 · Cited by 500 · see citing works

Time-Derivative Models of Pavlovian Reinforcement

Sutton, R. S., & Barto, A. G. (1990) · Cited by 496 · see citing works

Bisection of temporal intervals.

Church, R. M., & Deluty, M. Z. (1977) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037//0097-7403.3.3.216 · Cited by 475 · see citing works

Influence of Reward Delays on Responses of Dopamine Neurons

Kobayashi, S., & Schultz, W. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.1600-08.2008 · Cited by 431 · see citing works

Coupled Temporal Memories in Parkinson's Disease: A Dopamine-Related Dysfunction

Malapani, C., Rakitin, B. C., Lévy, R., Meck, W. H., Deweer, B., Dubois, B., & Gibbon, J. (1998) · Journal of Cognitive Neuroscience · doi:10.1162/089892998562762 · Cited by 424 · see citing works

TEMPORAL DISCRIMINATION IS ABNORMAL IN PARKINSON'S DISEASE

Artieda, J., Pastor, M. A., Lacruz, F. A., & Obeso, J. A. (1992) · Brain · doi:10.1093/brain/115.1.199 · Cited by 416 · see citing works

TIME ESTIMATION AND REPRODUCTION IS ABNORMAL IN PARKINSON'S DISEASE

Pastor, M. A., Artieda, J., Jahanshahi, M., & Obeso, J. A. (1992) · Brain · doi:10.1093/brain/115.1.211 · Cited by 379 · see citing works

TIME AND MEMORY: TOWARDS A PACEMAKER‐FREE THEORY OF INTERVAL TIMING

Staddon, J. E. R., & Higa, J. J. (1999) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1999.71-215 · Cited by 375 · see citing works

Learning the temporal dynamics of behavior.

Machado, A. D. (1997) · Psychological Review · doi:10.1037/0033-295x.104.2.241 · Cited by 373 · see citing works

Bisection of temporal intervals.

Church, R. M., & Deluty, M. Z. (1977) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037/0097-7403.3.3.216 · Cited by 368 · see citing works

Neural dynamics of adaptive timing and temporal discrimination during associative learning

Grossberg, S., & Schmajuk, N. A. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90026-9 · Cited by 367 · see citing works

A Model of Interval Timing by Neural Integration

Simen, P., Balcı, F., deSouza, L., Cohen, J. D., & Holmes, P. J. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.3121-10.2011 · Cited by 312 · see citing works

The temporal precision of reward prediction in dopamine neurons

Fiorillo, C. D., Newsome, W. T., & Schultz, W. (2008) · Nature Neuroscience · doi:10.1038/nn.2159 · Cited by 310 · see citing works

Methamphetamine and time estimation.

Maricq, A. V., Roberts, S., & Church, R. M. (1981) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037//0097-7403.7.1.18 · Cited by 304 · see citing works

A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task

Suri, R. E., & Schultz, W. (1999) · Neuroscience · doi:10.1016/s0306-4522(98)00697-6 · Cited by 294 · see citing works

The differential effects of haloperidol and methamphetamine on time estimation in the rat

Maricq, A. V., & Church, R. M. (1983) · Psychopharmacology · doi:10.1007/bf00433008 · Cited by 286 · see citing works

Affinity for the dopamine D2 receptor predicts neuroleptic potency in decreasing the speed of an internal clock

Meck, W. H. (1986) · Pharmacology Biochemistry and Behavior · doi:10.1016/0091-3057(86)90109-7 · Cited by 272 · see citing works

The Storage of Time Intervals Using Oscillating Neurons

Miall, C. (1989) · Neural Computation · doi:10.1162/neco.1989.1.3.359 · Cited by 254 · see citing works

What is reinforced by phasic dopamine signals?

Redgrave, P., GURNEY, K. N., & Reynolds, J. N. J. (2007) · Brain Research Reviews · doi:10.1016/j.brainresrev.2007.10.007 · Cited by 254 · see citing works

Neural representation of time in cortico-basal ganglia circuits

Jin, D. Z., Fujii, N., & Graybiel, A. M. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0909881106 · Cited by 246 · see citing works

Neurocomputational models of basal ganglia function in learning, memory and choice

Cohen, M. X., & Frank, M. J. (2008) · Behavioural Brain Research · doi:10.1016/j.bbr.2008.09.029 · Cited by 238 · see citing works

Adaptively timed conditioned responses and the cerebellum: A neural network approach

Moore, J. W., Desmond, J. E., & Berthier, N. E. (1989) · Biological Cybernetics · doi:10.1007/bf00217657 · Cited by 188 · see citing works

Statistics of Midbrain Dopamine Neuron Spike Trains in the Awake Primate

Bayer, H. M., Lau, B., & Glimcher, P. W. (2007) · Journal of Neurophysiology · doi:10.1152/jn.01140.2006 · Cited by 184 · see citing works

Reinforcement learning, conditioning, and the brain: Successes and challenges

Maia, T. V. (2009) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/cabn.9.4.343 · Cited by 184 · see citing works

Population clocks: motor timing with neural dynamics

Buonomano, D. V., & Laje, R. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.09.002 · Cited by 183 · see citing works

Representation and timing in theories of the dopamine system

Daw, N. D., Courville, A. C., & Touretzky, D. S. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1637 · Cited by 173 · see citing works

Single-trials analyses demonstrate that increases in clock speed contribute to the methamphetamine-induced horizontal shifts in peak-interval timing functions

Matell, M. S., Bateson, M., & Meck, W. H. (2006) · Psychopharmacology · doi:10.1007/s00213-006-0489-x · Cited by 166 · see citing works

Differential Modulation of Clock Speed by the Administration of Intermittent Versus Continuous Cocaine.

Matell, M. S., King, G., & Meck, W. H. (2004) · Behavioral Neuroscience · doi:10.1037/0735-7044.118.1.150 · Cited by 155 · see citing works

Stimulus representation and the timing of reward-prediction errors in models of the dopamine system

Ludvig, E. A., Sutton, R. S., & Kehoe, E. J. (2008) · Neural Computation · doi:10.1162/neco.2008.11-07-654 · Cited by 148 · see citing works

A Role for Dopamine in Temporal Decision Making and Reward Maximization in Parkinsonism

Moustafa, A. A., Cohen, M. X., Sherman, S. J., & Frank, M. J. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.3116-08.2008 · Cited by 147 · see citing works

On dopaminergic modulation of temporal information processing

Rammsayer, T. H. (1993) · Biological Psychology · doi:10.1016/0301-0511(93)90018-4 · Cited by 139 · see citing works

Effects of dopamine antagonists on the timing of two intervals

Drew, M. R., Fairhurst, S., Malapani, C., Horvitz, J. C., & Balsam, P. D. (2003) · Pharmacology Biochemistry and Behavior · doi:10.1016/s0091-3057(03)00036-4 · Cited by 137 · see citing works

Comparison of patients with Parkinson’s disease or cerebellar lesions in the production of periodic movements involving event-based or emergent timing

Spencer, R. M. C., & Ivry, R. B. (2004) · Brain and Cognition · doi:10.1016/j.bandc.2004.09.010 · Cited by 136 · see citing works

A Scale-Invariant Internal Representation of Time

Shankar, K. H., & Howard, M. W. (2011) · Neural Computation · doi:10.1162/neco_a_00212 · Cited by 135 · see citing works

Dissociating hippocampal and striatal contributions to sequential prediction learning

Bornstein, A. M., & Daw, N. D. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07920.x · Cited by 120 · see citing works

A Role for the Medial Temporal Lobe in Feedback-Driven Learning: Evidence from Amnesia

Foerde, K., Race, E., Verfaellie, M. H., & Daphna Shohamy, A. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.5217-12.2013 · Cited by 113 · see citing works

Neurocomputational models of motor and cognitive deficits in Parkinson’s disease

Wiecki, T. V., & Frank, M. J. (2010) · Progress in brain research · doi:10.1016/s0079-6123(10)83014-6 · Cited by 108 · see citing works

Evaluating the TD model of classical conditioning

Ludvig, E. A., Sutton, R. S., & Kehoe, E. J. (2012) · Learning & Behavior · doi:10.3758/s13420-012-0082-6 · Cited by 106 · see citing works

Differential effects of clozapine and haloperidol on interval timing in the supraseconds range

MacDonald, C. J., & Meck, W. H. (2005) · Psychopharmacology · doi:10.1007/s00213-005-0074-8 · Cited by 102 · see citing works

Ketamine “unlocks” the reduced clock-speed effects of cocaine following extended training: Evidence for dopamine–glutamate interactions in timing and time perception

Cheng, R., Ali, Y. M., & Meck, W. H. (2007) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2007.04.005 · Cited by 98 · see citing works

Temporal Convergence of Dynamic Cell Assemblies in the Striato-Pallidal Network

Adler, A., Katabi, S., Finkes, I., Israel, Z. H., Prut, Y., & Bergman, H. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.4830-11.2012 · Cited by 89 · see citing works

Stimulus timing by people with Parkinson’s disease

Wearden, J. H., Smith-Spark, J. H., Cousins, R., Edelstyn, N. M. J., Cody, F. W. J., & O'Boyle, D. J. (2008) · Brain and Cognition · doi:10.1016/j.bandc.2008.01.010 · Cited by 83 · see citing works

Timescale Invariance in the Pacemaker-Accumulator Family of Timing Models

Simen, P., Rivest, F., Ludvig, E. A., Balcı, F., & Killeen, P. R. (2013) · Timing & Time Perception · doi:10.1163/22134468-00002018 · Cited by 83 · see citing works

Motivational effects on interval timing in dopamine transporter (DAT) knockdown mice

Balcı, F., Ludvig, E. A., Abner, R. C., Zhuang, X., Poon, P., & Brunner, D. (2010) · Brain Research · doi:10.1016/j.brainres.2010.02.034 · Cited by 68 · see citing works

Pharmacological manipulations of interval timing using the peak procedure in male C3H mice

Balcı, F., Ludvig, E. A., Gibson, J. M., Allen, B. D., Frank, K. M., Kapustinski, B. J., Fedolak, T. E., & Brunner, D. (2008) · Psychopharmacology · doi:10.1007/s00213-008-1248-y · Cited by 61 · see citing works

EFFECTS OF D‐AMPHETAMINE IN A TEMPORAL DISCRIMINATION PROCEDURE: SELECTIVE CHANGES IN TIMING OR RATE DEPENDENCY?

Odum, A. L., Lieving, L. M., & Schaal, D. W. (2002) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.2002.78-195 · Cited by 60 · see citing works

Midbrain Dopamine Neurons

Schultz, W. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374176-9.00021-x · Cited by 54 · see citing works

Subjective time estimation in Parkinson's disease.

Lange, K. W., Tucha, O. M., Steup, A., Gsell, W., & Naumann, M. (1995) · PubMed · Cited by 54 · see citing works

Effects of D-amphetamine on temporal discrimination in pigeons

McClure, E. A., Saulsgiver, K. A., & Wynne, C. D. L. (2005) · Behavioural Pharmacology · doi:10.1097/01.fbp.0000171773.69292.bd · Cited by 47 · see citing works

A Primer on Reinforcement Learning in the Brain

Ludvig, E. A., Bellemare, M. G., & Pearson, K. G. (2011) · IGI Global eBooks · doi:10.4018/978-1-60960-021-1.ch006 · Cited by 31 · see citing works

The Substantia Nigra, the Basal Ganglia, Dopamine and Temporal Processing

Jones, C. R. G., & Jahanshahi, M. (2009) · Springer eBooks · doi:10.1007/978-3-211-92660-4_13 · Cited by 31 · see citing works

Alternative time representation in dopamine models

Rivest, F., Kalaska, J. F., & Bengio, Y. (2009) · Journal of Computational Neuroscience · doi:10.1007/s10827-009-0191-1 · Cited by 23 · see citing works

Internal-Time Temporal Difference Model for Neural Value-Based Decision Making

Nakahara, H., & Kaveri, S. R. (2010) · Neural Computation · doi:10.1162/neco_a_00049 · Cited by 23 · see citing works

A computational model of hippocampal function in trace conditioning

Ludvig, E. A., Sutton, R. S., Verbeek, E. L., & Kehoe, E. J. (2008) · University of Warwick · doi:10.82444/warw.33635548 · Cited by 23 · see citing works

Interval timing

Staddon, J. E. R., & Higa, J. J. (2006) · Nature reviews. Neuroscience · doi:10.1038/nrn1764-c1 · Cited by 7 · see citing works

[Neural representation of time].

Tanaka, M., Kunimatsu, J., & Ohmae, S. (2013) · PubMed · Cited by 5 · see citing works

A Unified Framework for Dopamine Signals across Timescales

Kim, H. R., Malik, A. N., Mikhael, J. G., Bech, P., Tsutsui‐Kimura, I., Sun, F., Zhang, Y., Li, Y., Watabe‐Uchida, M., Gershman, S. J., et al. (2020) · Cell · doi:10.1016/j.cell.2020.11.013 · Cited by 297 · see citing works

Believing in dopamine

Gershman, S. J., & Uchida, N. (2019) · Nature Reviews Neuroscience · doi:10.1038/s41583-019-0220-7 · Cited by 236 · see citing works

Dopamine reward prediction errors reflect hidden-state inference across time

Starkweather, C. K., Babayan, B. M., Uchida, N., & Gershman, S. J. (2017) · Nature Neuroscience · doi:10.1038/nn.4520 · Cited by 206 · see citing works

Striatal dynamics explain duration judgments

Gouvêa, T. S., Monteiro, T., Motiwala, A., Soares, S., Machens, C. K., & Paton, J. J. (2015) · eLife · doi:10.7554/elife.11386 · Cited by 203 · see citing works

Model-based predictions for dopamine

Langdon, A. J., Sharpe, M. J., Schoenbaum, G., & Niv, Y. (2018) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.10.006 · Cited by 161 · see citing works

A Unifying Probabilistic View of Associative Learning

Gershman, S. J. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004567 · Cited by 158 · see citing works

Toward an integration of deep learning and neuroscience

Marblestone, A. H., Wayne, G., & Kording, K. P. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00094 · Cited by 119 · see citing works

The neural bases for timing of durations

Tsao, A., Yousefzadeh, S. A., Meck, W. H., Moser, M., & Moser, E. I. (2022) · Nature reviews. Neuroscience · doi:10.1038/s41583-022-00623-3 · Cited by 109 · see citing works

Time perception mechanisms at central nervous system

Fontes, R. M., Ribeiro, J. A., Gupta, D. S., Machado, D. D. C. D., Lopes-Júnior, F., Magalhães, F. E. X., Bastos, V. H., Rocha, K. D. M., Marinho, V., Lima, G., et al. (2016) · Neurology International · doi:10.4081/ni.2016.5939 · Cited by 104 · see citing works

Dual credit assignment processes underlie dopamine signals in a complex spatial environment

Krausz, T. A., Comrie, A. E., Kahn, A. E., Frank, L. M., Daw, N. D., & Berke, J. D. (2023) · Neuron · doi:10.1016/j.neuron.2023.07.017 · Cited by 82 · see citing works

Dopamine and the interdependency of time perception and reward

Fung, B. J., Sutlief, E., & Shuler, M. G. H. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.02.030 · Cited by 79 · see citing works

A cerebellar mechanism for learning prior distributions of time intervals

Narain, D., Remington, E. D., Zeeuw, C. I. D., & Jazayeri, M. (2018) · Nature Communications · doi:10.1038/s41467-017-02516-x · Cited by 66 · see citing works

A review of psychological and neuroscientific research on musical groove

Etani, T., Miura, A., Kawase, S., Fujii, S., Keller, P. E., Vuust, P., & Kudo, K. (2023) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105522 · Cited by 62 · see citing works

Adapting the flow of time with dopamine

Mikhael, J. G., & Gershman, S. J. (2019) · Journal of Neurophysiology · doi:10.1152/jn.00817.2018 · Cited by 58 · see citing works

Integrating Models of Interval Timing and Reinforcement Learning

Petter, E. A., Gershman, S. J., & Meck, W. H. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.08.004 · Cited by 58 · see citing works

Forgetting in Reinforcement Learning Links Sustained Dopamine Signals to Motivation

Kato, A., & Morita, K. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005145 · Cited by 57 · see citing works

Tamping Ramping: Algorithmic, Implementational, and Computational Explanations of Phasic Dopamine Signals in the Accumbens

Lloyd, K., & Dayan, P. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004622 · Cited by 49 · see citing works

Reinforcement learning with Marr

Niv, Y., & Langdon, A. (2016) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2016.04.005 · Cited by 49 · see citing works

Interval Timing, Dopamine, and Motivation

Balcı, F. (2014) · Timing & Time Perception · doi:10.1163/22134468-00002035 · Cited by 46 · see citing works

Temporal Processing in Audition: Insights from Music

Rajendran, V. G., Teki, S., & Schnupp, J. W. H. (2017) · Neuroscience · doi:10.1016/j.neuroscience.2017.10.041 · Cited by 44 · see citing works

Emergence of belief-like representations through reinforcement learning

Hennig, J. A., Romero Pinto, S. A., Yamaguchi, T., Linderman, S. W., Uchida, N., & Gershman, S. J. (2023) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011067 · Cited by 41 · see citing works

Dopamine, inference, and uncertainty

Gershman, S. J. (2017) · Neural Computation · doi:10.1162/neco_a_01023 · Cited by 40 · see citing works

Teaching Robots to Perceive Time: A Twofold Learning Approach

Lourenço, I., Ventura, R., & Wahlberg, B. (2020) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/icdl-epirob48136.2020.9278033 · Cited by 38 · see citing works

Choice-selective sequences dominate in cortical relative to thalamic inputs to NAc to support reinforcement learning

Parker, N. F., Baidya, A., Cox, J., Haetzel, L. M., Zhukovskaya, A., Murugan, M., Engelhard, B., Goldman, M. S., & Witten, I. B. (2022) · Cell Reports · doi:10.1016/j.celrep.2022.110756 · Cited by 38 · see citing works

Frontal Norepinephrine Represents a Threat Prediction Error Under Uncertainty

Basu, A., Yang, J.-H., Yu, A. L., Glaeser-Khan, S., Rondeau, J. A., Feng, J., Krystal, J. H., Li, Y., & Kaye, A. P. (2024) · Biological Psychiatry · doi:10.1016/j.biopsych.2024.01.025 · Cited by 33 · see citing works

Rational inattention and tonic dopamine

Mikhael, J. G., Lai, L., & Gershman, S. J. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008659 · Cited by 30 · see citing works

Using computational theory to constrain statistical models of neural data

Linderman, S. W., & Gershman, S. J. (2017) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.06.004 · Cited by 28 · see citing works

“Liking” as an early and editable draft of long-run affective value

Dayan, P. (2022) · PLoS Biology · doi:10.1371/journal.pbio.3001476 · Cited by 26 · see citing works

How do real animals account for the passage of time during associative learning?

Namboodiri, V. M. K. (2022) · Behavioral Neuroscience · doi:10.1037/bne0000516 · Cited by 26 · see citing works

A unified neurocomputational model of prospective and retrospective timing.

de Jong, J., Voelker, A. R., Stewart, T. C., Akyürek, E. G., Eliasmith, C., & van Rijn, H. (2025) · Psychological Review · doi:10.1037/rev0000519 · Cited by 25 · see citing works

Temporal updating, behavioral learning, and the phenomenology of time-consciousness

Hayman, G., & Huebner, B. (2019) · Behavioral and Brain Sciences · doi:10.1017/s0140525x19000517 · Cited by 23 · see citing works

Rapid encoding of task regularities in the human hippocampus guides sensorimotor timing

Polti, I., Nau, M., Kaplan, R., van Wassenhove, V., & Doeller, C. F. (2022) · eLife · doi:10.7554/elife.79027 · Cited by 23 · see citing works

Thunderstruck: The ACDC model of flexible sequences and rhythms in recurrent neural circuits

Calderon, C. B., Verguts, T., & Frank, M. J. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1009854 · Cited by 21 · see citing works

A Simple Network Architecture Accounts for Diverse Reward Time Responses in Primary Visual Cortex

Huertas, M. A., Shuler, M. G. H., & Shouval, H. Z. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.0871-15.2015 · Cited by 21 · see citing works

Dopamine mediates the bidirectional update of interval timing.

Jakob, A. M. V., Mikhael, J. G., Hamilos, A. E., Assad, J. A., & Gershman, S. J. (2022) · Behavioral Neuroscience · doi:10.1037/bne0000529 · Cited by 20 · see citing works

Information Theory, Memory, Prediction, and Timing in Associative Learning

Wilkes, J., & Gallistel, C. R. (2017) · Wiley eBooks · doi:10.1002/9781119159193.ch35 · Cited by 20 · see citing works

Robert Rescorla: Time, Information and Contingency.

Gallistel, C. R. (2021) · Revista de historia de la psicología · doi:10.5093/rhp2021a3 · Cited by 19 · see citing works

Functional Relevance of Different Basal Ganglia Pathways Investigated in a Spiking Model with Reward Dependent Plasticity

Berthet, P., Lindahl, M., Tully, P. J., Kotaleski, J. H., & Lansner, A. (2016) · Frontiers in Neural Circuits · doi:10.3389/fncir.2016.00053 · Cited by 18 · see citing works

A computational formulation of the behavior systems account of the temporal organization of motivated behavior

Sanabria, F., Daniels, C. W., Gupta, T. A., & Santos, C. (2019) · Behavioural Processes · doi:10.1016/j.beproc.2019.103952 · Cited by 17 · see citing works

Time-scale-invariant information-theoretic contingencies in discrimination learning.

Kalmbach, A., Chun, E. K., Taylor, K. M., Gallistel, C. R., & Balsam, P. D. (2019) · Journal of Experimental Psychology Animal Learning and Cognition · doi:10.1037/xan0000205 · Cited by 17 · see citing works

Proposal for using time estimation training for the treatment of Parkinson’s disease

Teixeira, S. S., Magalhães, F. E. X., Marinho, V., Velasques, B., & Ribeiro, P. F. M. (2016) · Medical Hypotheses · doi:10.1016/j.mehy.2016.08.012 · Cited by 14 · see citing works

Underlying interactive neural mechanism of motor learning governed by the cerebellum, the basal ganglia, and motor/sensory cortex: a review from theoretical perspective

Torbati, A. H. M., Jami, S., Kobravi, H.-R., Davoudi, N., Gholibeigi, M., & Ashkzari, A. (2024) · Neuroscience and Behavioral Physiology · doi:10.1007/s11055-024-01583-0 · Cited by 13 · see citing works

Computing reward‐prediction error: an integrated account of cortical timing and basal‐ganglia pathways for appetitive and aversive learning

Morita, K., & Kawaguchi, Y. (2015) · European Journal of Neuroscience · doi:10.1111/ejn.12994 · Cited by 12 · see citing works

Cilia in the Striatum Mediate Timing-Dependent Functions

Alhassen, W., Alhassen, S., Chen, J., Monfared, R. V., & Alachkar, A. (2022) · Molecular Neurobiology · doi:10.1007/s12035-022-03095-9 · Cited by 11 · see citing works

Basal ganglia: physiological, behavioral, and computational studies

Moustafa, A. A., Bar‐Gad, I., Korngreen, A., & Bergman, H. (2014) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2014.00150 · Cited by 11 · see citing works

Ramping activity in the striatum

Ponzi, A. P. D., & Wickens, J. R. (2022) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2022.902741 · Cited by 9 · see citing works

Behavioral plasticity through the modulation of switch neurons

Vassiliades, V., & Christodoulou, C. C. (2015) · Neural Networks · doi:10.1016/j.neunet.2015.11.001 · Cited by 9 · see citing works

Number and time in acquisition, extinction and recovery

Gallistel, C. R., & Papachristos, E. B. (2019) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.571 · Cited by 9 · see citing works

Choice-selective sequences dominate in cortical relative to thalamic inputs to nucleus accumbens, providing a potential substrate for credit assignmentPreprint

Parker, N. F., Baidya, A., Cox, J., Haetzel, L. M., Zhukovskaya, A., Murugan, M., Engelhard, B., Goldman, M. S., & Witten, I. B. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/725382 · Cited by 9 · see citing works

Quantitative properties of the creation and activation of a cell-intrinsic duration-encoding engram

Gallistel, C. R., Johansson, F., Jirenhed, D.-A., Rasmussen, A., Ricci, M. R., & Hesslow, G. (2022) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2022.1019812 · Cited by 8 · see citing works

Representation learning with reward prediction errors

Alexander, W. H., & Gershman, S. J. (2022) · Neurons Behavior Data analysis and Theory · doi:10.51628/001c.37270 · Cited by 7 · see citing works

Delay of reinforcement versus rate of reinforcement in Pavlovian conditioning.

Austen, J. M., & Sanderson, D. J. (2019) · Journal of Experimental Psychology Animal Learning and Cognition · doi:10.1037/xan0000199 · Cited by 7 · see citing works

Information Theory, Memory, Prediction, and Timing in Associative LearningPreprint

Wilkes, J., & Gallistel, C. R. (2016) · doi:10.31219/osf.io/y9sg2 · Cited by 6 · see citing works

Time elapsed between choices in a probabilistic task correlates with repeating the same decision

Jabłońska, J., Szumiec, Ł., Zieliński, P., & Parkitna, J. M. R. (2021) · European Journal of Neuroscience · doi:10.1111/ejn.15144 · Cited by 5 · see citing works

What is the state space of the world for real animals?Preprint

Namboodiri, V. M. K. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.02.07.430001 · Cited by 4 · see citing works

A Biologically Inspired Computational Model of Time Perception

Lourenço, I., Mattila, R., Ventura, R., & Wahlberg, B. (2021) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2021.3120301 · Cited by 3 · see citing works

Reinforcement learning with dopamine: A convergence of natural and artificial intelligence

Masset, P., & Gershman, S. J. (2025) · Handbook of behavioral neuroscience · doi:10.1016/b978-0-443-29867-7.00019-0 · Cited by 3 · see citing works

Tread softly and carry a clock's tick

Paton, J. J., & Lau, B. (2015) · Nature Neuroscience · doi:10.1038/nn.3959 · Cited by 3 · see citing works

Insights into the interaction between time and reward prediction on the activity of striatal tonically active neurons: A pilot study in rhesus monkeys

Martel, A., & Apicella, P. (2024) · Physiological Reports · doi:10.14814/phy2.70037 · Cited by 2 · see citing works

Expected reward value and reward prediction errors reinforce but also interfere with human time perception

DiMarco, E. K., Shipp, A. R., & Kishida, K. T. (2026) · Scientific Reports · doi:10.1038/s41598-026-62991-5 · Cited by 1 · see citing works

Classical Computational Approaches to Modeling the Basal Ganglia

Moustafa, A. A., & Chakravarthy, V. S. (2018) · Cognitive science and technology · doi:10.1007/978-981-10-8494-2_4 · Cited by 1 · see citing works

Teaching robots to perceive time -- A reinforcement learning approach (Extended version)Preprint

Lourenço, I., Wahlberg, B., & Ventura, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.10113 · Cited by 1 · see citing works

Thunderstruck: The ACDC model of flexible sequences and rhythms in recurrent neural circuitsPreprint

Calderon, C. B., Verguts, T., & Frank, M. J. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.04.07.438842 · Cited by 1 · see citing works

On the relationship among different motor processes: a computational modeling approach

Moustafa, A. A. (2015) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2015.00034

Computational Modeling of the Basal Ganglia : Functional Pathways and Reinforcement Learning

Berthet, P. (2015) · KTH Publication Database DiVA (KTH Royal Institute of Technology)

Experimental and Computational Investigation of the Effect of Caffeine on Human Time Perception.

Sankar, R., Shukla, A. K., & Bapi, R. S. (2017) · Cognitive Science

Neural and Behavioral Mechanisms of Interval Timing in the Striatum

Mello, G. B. M. E. (2016) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT)

Hidden State Inference in the Midbrain Dopamine System

Starkweather, C. K. (2018) · Digital Access to Scholarship at Harvard (DASH) (Harvard University)

The Role of the Direct and Indirect Basal Ganglia Pathways in the Learning, Performance, and Goal-Directed Control of Action Sequences

Garr, E. (2019) · CUNY Academic Works (City University of New York)

التفاعل بین نمطى التعزیز (فاصل زمنی / نسبی) بمحفزات الألعاب الرقمیة وطریقتى تقدیمهما (ثابت / متغیر) فی بیئة تعلم الکترونی وأثره على الاندماج فی التعلم وتنمیة الطمأنینة النفسیة ومهارات الاستخدام الآمن للإنترنت لدى تلامیذ الحلقة الابتدائیة

الرحمن, إ. ا. م. أ. ع. (2020) · تکنولوجيا التعليم سلسلة دراسات وبحوث · doi:10.21608/tesr.2020.160500

Faculty Opinions recommendation of Dopamine reward prediction errors reflect hidden-state inference across time.

Schultz, W. (2019) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.727374683.793565153

What’s Next if Reward is Enough? Insights for AGI from Animal Reinforcement Learning

Rajagopal, S. (2023) · Journal of Artificial General Intelligence · doi:10.2478/jagi-2023-0002

A window on Subjective Time

de Jong, J. (2024) · doi:10.33612/diss.1004106212

A perspective on dopamine

Dayan, P. (2025) · Handbook of behavioral neuroscience · doi:10.1016/b978-0-443-29867-7.09992-8

Dynamic belief representation and updating through learned attractor-like dynamics in the frontal cortexPreprint

Pinto, S. R., Hennig, J. A., Burrell, M., Okada, D., Benquet, C., Regester, D., Isogai, Y., Linderman, S. W., Gershman, S. J., & Uchida, N. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.09.17.752495

A multilayer drift‐diffusion broadcast model: Converting temporal evidence into response timing

Balcı, F. (2026) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.70150

Integrating Models of Interval Timing and Reinforcement Learning

Petter, E. A., Gershman, S. J., & Meck, W. H. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.08.004 · Cited by 58 · see citing works

Biological Models of Reinforcement Learning

Vitay, J., Fix, J., Beuth, F., Schroll, H., & Hamker, F. H. (2009) · HAL · Cited by 4 · see citing works

Computational Model of the Cerebellum and the Basal Ganglia for Interval Timing Learning

Katakura, O., & Yamazaki, T. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46681-1_30

Time in the basal ganglia

Soares, S. (2017)

Biological Models of Reinforcement

Vitay, J., Fix, J., Beuth, F., Schroll, H., & Hamker, F. H. (2009)