Action chunking as conditional policy compression
Open accessAbstract
Many skills in our everyday lives are learned by sequencing actions towards a desired goal. The action sequence can become a ``chunk'' when individual actions are grouped together and executed as one unit, making them more efficient to store and execute. While chunking has been studied extensively across various domains, a puzzle remains as to why and under what conditions action chunking occurs. To tackle these questions, we develop a model of conditional policy compression—the reduction in cognitive cost by conditioning on an additional source of information—to explain the origin of chunking. We argue that chunking is a result of optimizing the trade-off between reward and conditional policy complexity. Chunking compresses policies when there is temporal structure in the environment that can be leveraged for action selection, reducing the amount of memory necessary to encode the policy. We experimentally confirm our model's predictions, showing that chunking reduces conditional policy complexity and reaction times. Chunking also increases with working memory load, consistent with the hypothesis that the degree of policy compression scales with the scarcity of cognitive resources. Finally, chunking also reduces overall working memory load, freeing cognitive resources for the benefit of other, not-chunked information.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Action chunking has recently emerged as the de facto standard in imitation learning for visuomotor control.
“Inspired in part by human motor control [33], action chunking has recently emerged as the de facto standard in imitation learning for visuomotor control [68, 11].”
From Real-Time Execution of Action Chunking Flow Policies (2025) · cites this paper as [33]
Action chunking has enabled temporal consistency in high-frequency control tasks.
“While action chunking has enabled temporal consistency in high-frequency control tasks, it does not fully address the latency problem, leading to pauses or out-of-distribution jerky movements at chunk boundaries.”
From Real-Time Execution of Action Chunking Flow Policies (2025) · cites this paper as [33]
The Introduction and §4.1 discuss action chunking in human sequence-learning tasks, including faster and more accurate responses to patterned stimuli. Our copy does not address temporal consistency in high-frequency control tasks; searches for that phrase and for high-frequency control, latency, and chunk-boundary movements found no supporting passage. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The magical number seven, plus or minus two: Some limits on our capacity for processing information.
Miller, G. A. (1994) · Psychological Review · doi:10.1037/0033-295x.101.2.343 · Cited by 17,469 · see citing works
Reinforcement Learning: An Introduction
Johnson, J. D., Li, J., & Chen, Z. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00324-6 · Cited by 8,821 · see citing works
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Sutton, R. S., Precup, D., & Singh, S. (1999) · Artificial Intelligence · doi:10.1016/s0004-3702(99)00052-1 · Cited by 3,119 · see citing works
Computation of channel capacity and rate-distortion functions
Blahut, R. (1972) · IEEE Transactions on Information Theory · doi:10.1109/tit.1972.1054855 · Cited by 1,381 · see citing works
Chunking mechanisms in human learning
GOBET, F., LANE, P., CROKER, S., CHENG, P., JONES, G., OLIVER, I., & PINE, J. (2001) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01662-4 · Cited by 1,157 · see citing works
Changing concepts of working memory
Ma, W. J., Masud Husain, B., & Bays, P. M. (2014) · Nature Neuroscience · doi:10.1038/nn.3655 · Cited by 1,128 · see citing works
The Basal Ganglia and Chunking of Action Repertoires
Graybiel, A. M. (1998) · Neurobiology of Learning and Memory · doi:10.1006/nlme.1998.3843 · Cited by 1,042 · see citing works
Bayesian model selection for group studies — Revisited
Rigoux, L., Stephan, K. E., Friston, K. J., & Jean Daunizeau, A. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.08.065 · Cited by 675 · see citing works
Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective
Botvinick, M. M., Niv, Y., & Barto, A. G. (2009) · Cognition · doi:10.1016/j.cognition.2008.08.011 · Cited by 646 · see citing works
Sequential Sampling Models in Cognitive Neuroscience: Advantages, Applications, and Extensions
Forstmann, B. U., Ratcliff, R., & Wagenmakers, E. (2015) · Annual Review of Psychology · doi:10.1146/annurev-psych-122414-033645 · Cited by 613 · see citing works
Start/stop signals emerge in nigrostriatal circuits during sequence learning
Jin, X., & Costa, R. M. (2010) · Nature · doi:10.1038/nature09263 · Cited by 611 · see citing works
Decision theory, reinforcement learning, and the brain
Dayan, P., & Daw, N. D. (2008) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.8.4.429 · Cited by 580 · see citing works
A Bayesian framework for word segmentation: exploring the effects of context.
Goldwater, S., Griffiths, T., & Johnson, M. A. (2009) · Cognition · doi:10.1016/j.cognition.2009.03.008 · Cited by 506 · see citing works
Basal ganglia subcircuits distinctively encode the parsing and concatenation of action sequences
Jin, X., Tecuapetla, F., & Costa, R. M. (2014) · Nature Neuroscience · doi:10.1038/nn.3632 · Cited by 499 · see citing works
PARSER: A Model for Word Segmentation
Perruchet, P., & Vinter, A. (1998) · Journal of Memory and Language · doi:10.1006/jmla.1998.2576 · Cited by 423 · see citing works
Cognitive control over learning: Creating, clustering, and generalizing task-set structure.
Collins, A. G. E., & Frank, M. J. (2013) · Psychological Review · doi:10.1037/a0030852 · Cited by 421 · see citing works
Collins, A. G. E., & Frank, M. J. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07980.x · Cited by 419 · see citing works
Learning artificial grammars with competitive chunking.
Servan-Schreiber, E., & Anderson, J. R. (1990) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/0278-7393.16.4.592 · Cited by 400 · see citing works
Brady, T. F., Konkle, T., & Alvarez, G. A. (2009) · Journal of Experimental Psychology General · doi:10.1037/a0016797 · Cited by 325 · see citing works
Expert Chess Memory: Revisiting the Chunking Hypothesis
Gobet, F., & Simon, H. A. (1998) · Memory · doi:10.1080/741942359 · Cited by 320 · see citing works
Chunking during human visuomotor sequence learning
Sakai, K., Kitaguchi, K., & Hikosaka, O. (2003) · Experimental Brain Research · doi:10.1007/s00221-003-1548-8 · Cited by 306 · see citing works
Theories of artificial grammar learning.
Pothos, E. M. (2007) · Psychological Bulletin · doi:10.1037/0033-2909.133.2.227 · Cited by 302 · see citing works
Habits, action sequences and reinforcement learning
Dezfouli, A., & Balleine, B. W. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08050.x · Cited by 297 · see citing works
Bayesian learning of visual chunks by human observers
Orbán, G., Fiser, J., Aslin, R., & Lengyel, M. (2008) · Proceedings of the National Academy of Sciences of the United States of America · doi:10.1073/pnas.0708424105 · Cited by 291 · see citing works
Information Theory of Decisions and Actions
Tishby, N., & Polani, D. (2011) · Perception-Action Cycle · doi:10.1007/978-1-4419-1452-1_19 · Cited by 267 · see citing works
What’s magic about magic numbers? Chunking and data compression in short-term memory
Mathy, F., & Feldman, J. (2011) · Cognition · doi:10.1016/j.cognition.2011.11.003 · Cited by 259 · see citing works
How does chunking help working memory?
Thalmann, M., Souza, A. S., & Oberauer, K. (2018) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/xlm0000578 · Cited by 238 · see citing works
Modeling human performance in statistical word segmentation.
Frank, M. C., Goldwater, S., Mansinghka, V. K., Griffiths, T., & Tenenbaum, J. (2010) · Cognition · doi:10.1016/j.cognition.2010.07.005 · Cited by 183 · see citing works
Visuospatial Working Memory Capacity Predicts the Organization of Acquired Explicit Motor Sequences
Bo, J., & Seidler, R. D. (2009) · Journal of Neurophysiology · doi:10.1152/jn.00006.2009 · Cited by 183 · see citing works
Buffer loading and chunking in sequential keypressing.
Verwey, W. B. (1996) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/0096-1523.22.3.544 · Cited by 176 · see citing works
An information-theoretic approach to curiosity-driven reinforcement learning
Still, S., & Precup, D. (2012) · Theory in Biosciences · doi:10.1007/s12064-011-0142-z · Cited by 165 · cited 2× · see citing works
“6 leads to the optimal policy, π ∗ (Parush et al., 2011; Tishby and Polani, 2011; Still and Precup, 2012): The optimal policy is a softmax function, with an added frequency-dependent bias (persever-ation) term.”
An Ideal Observer Analysis of Visual Working Memory
Sims, C., Jacobs, R., & Knill, D. (2012) · Psychology Review · doi:10.1037/a0029856 · Cited by 158 · see citing works
Motor Sequences and the Basal Ganglia: Kinematics, Not Habits
Desmurget, M., & Turner, R. S. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.0163-10.2010 · Cited by 157 · see citing works
Skill representation in the primary motor cortex after long-term practice
Matsuzaka, Y., Picard, N., & Strick, P. L. (2007) · Journal of Neurophysiology · doi:10.1152/jn.00784.2006 · Cited by 156 · see citing works
Collins, A. G. E., & Frank, M. J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1720963115 · Cited by 146 · see citing works
Chunk Limits and Length Limits in Immediate Recall: A Reconciliation.
Chen, Z., & Cowan, N. (2005) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/0278-7393.31.6.1235 · Cited by 145 · see citing works
Neurocomputational mechanisms of prior-informed perceptual decision-making in humans
Kelly, S. P., Corbett, E. A., & O’Connell, R. G. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-00967-9 · Cited by 145 · see citing works
Evidence for a multistage model of practice in a sequential movement task.
Verwey, W. B. (1999) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.25.6.1693 · Cited by 140 · see citing works
Chunking as the result of an efficiency computation trade-off
Ramkumar, P., Acuna, D. E., Berniker, M., Grafton, S. T., Turner, R. S., & Kording, K. P. (2016) · Nature Communications · doi:10.1038/ncomms12176 · Cited by 133 · see citing works
Working Memory Load Strengthens Reward Prediction Errors
Collins, A. G. E., Ciullo, B., Frank, M. J., & Badre, D. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.2700-16.2017 · Cited by 129 · see citing works
Rate–distortion theory and human perception
Sims, C. R. (2016) · Cognition · doi:10.1016/j.cognition.2016.03.020 · Cited by 125 · see citing works
Chunking as a rational strategy for lossy data compression in visual working memory.
Nassar, M. R., Helmers, J. C., & Frank, M. J. (2018) · Psychological Review · doi:10.1037/rev0000101 · Cited by 123 · see citing works
The Tortoise and the Hare: Interactions between Reinforcement Learning and Working Memory
Collins, A. G. E. (2018) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01238 · Cited by 97 · see citing works
Working memory capacity correlates with implicit serial reaction time task performance
Bo, J., Jennett, S., & Seidler, R. D. (2011) · Experimental Brain Research · doi:10.1007/s00221-011-2807-8 · Cited by 97 · see citing works
Origin of perseveration in the trade-off between reward and complexity
Gershman, S. J. (2020) · Cognition · doi:10.1016/j.cognition.2020.104394 · Cited by 97 · see citing works
Self-citationA probabilistic clustering theory of the organization of visual short-term memory.
Orhan, A. E., & Jacobs, R. A. (2013) · Psychological Review · doi:10.1037/a0031541 · Cited by 88 · see citing works
Chunking during serial learning by a pigeon: I. Basic evidence.
Terrace, H. S. (1991) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037//0097-7403.17.1.81 · Cited by 78 · see citing works
Policy compression: An information bottleneck in action selection
Lai, L., & Gershman, S. (2021) · The psychology of learning and motivation · doi:10.1016/bs.plm.2021.02.004 · Cited by 77 · see citing works
Self-citationMeasuring habit formation through goal-directed response switching.
Luque, D., Molinero, S., Watson, P., López, F. J., & Pelley, M. E. L. (2019) · Journal of Experimental Psychology General · doi:10.1037/xge0000722 · Cited by 75 · see citing works
Distribution of mutual information
Abarbanel, H. D. I., Masuda, N., Rabinovich, M. I., & Tumer, E. (2001) · Physics Letters A · doi:10.1016/s0375-9601(01)00128-1 · Cited by 70 · see citing works
Efficient data compression in perception and perceptual memory.
Bates, C., & Jacobs, R. A. (2020) · Psychological Review · doi:10.1037/rev0000197 · Cited by 69 · see citing works
Chunking and data compression in verbal short-term memory
Norris, D. G., & Kalm, K. (2020) · Cognition · doi:10.1016/j.cognition.2020.104534 · Cited by 68 · see citing works
Distribution of Mutual Information
Hütter, M. (2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.003.0056 · Cited by 61 · see citing works
Multifaceted aspects of chunking enable robust algorithms
Acuna, D. E., Wymbs, N. F., Reynolds, C. A., Picard, N., Turner, R. S., Strick, P. L., Grafton, S. T., & Kording, K. P. (2014) · Journal of Neurophysiology · doi:10.1152/jn.00028.2014 · Cited by 54 · see citing works
Bates, C., Lerch, R., Sims, C., & Jacobs, R. (2019) · Journal of Vision · doi:10.1167/19.2.11 · Cited by 49 · see citing works
Chunking or not chunking? How do we find words in artificial language learning?
Franco, A., & Destrebecqz, A. (2012) · Advances in Cognitive Psychology · doi:10.5709/acp-0111-3 · Cited by 46 · see citing works
Task-relevant chunking in sequence learning.
Perlman, A., Pothos, E. M., Edwards, D. J., & Tzelgov, J. (2010) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/a0017178 · Cited by 42 · see citing works
Dopaminergic Balance between Reward Maximization and Policy Complexity
Parush, N., Tishby, N., & Bergman, H. (2011) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2011.00022 · Cited by 37 · see citing works
Temporal and state abstractions for efficient learning, transfer, and composition in humans.
Xia, L., & Collins, A. G. E. (2021) · Psychological Review · doi:10.1037/rev0000295 · Cited by 34 · see citing works
Chunking improves symbolic sequence processing and relies on working memory gating mechanismsPreprint
Solopchuk, O., Alamia, A., Olivier, E., & Zénon, A. (2016) · Learning & Memory · doi:10.1101/lm.041277.115 · Cited by 27 · see citing works
Temporal chunking as a mechanism for unsupervised learning of task-sets
Bouchacourt, F., Palminteri, S., Koechlin, É., & Ostojic, S. (2020) · eLife · doi:10.7554/elife.50469 · Cited by 20 · see citing works
The Reward-Complexity Trade-off in Schizophrenia
Gershman, S. J., & Lai, L. (2021) · Computational Psychiatry · doi:10.5334/cpsy.71 · Cited by 18 · see citing works
Self-citationA Probabilistic Clustering Theory of the Organization of Visual Short-Term Memory
Jacobs, R. A., & Orhan, A. E. (2013) · Journal of Vision · doi:10.1167/13.9.1368 · Cited by 12 · see citing works
Hierarchical Chunking during Learning of Visuomotor Sequences
Miyapuram, K. P., Bapi, R. S., Pammi, C., Ahmed, A., & Doya, K. (2006) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2006.1716099 · Cited by 10 · see citing works
E pluribus unum but how? Chunking as a rational solution to the speed-accuracy trade-offPreprint
Wu, S., Éltető, N., Dasgupta, I., & Schulz, E. (2022) · doi:10.31234/osf.io/sjh27 · Cited by 2 · see citing works
Flagged first, then by citations
2 of the 67 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
Zhao, T., Kumar, V., Levine, S., & Finn, C. (2023) · Robotics: Science and Systems XIX · doi:10.15607/rss.2023.xix.016 · Cited by 674 · see citing works
Undermatching Is a Consequence of Policy Compression
Bari, B. A., & Gershman, S. J. (2022) · Journal of Neuroscience · doi:10.1523/jneurosci.1003-22.2022 · Cited by 18 · see citing works
Self-citationReal-Time Execution of Action Chunking Flow Policies
Black, K., Galliker, M., & Levine, S. (2025) · Advances in Neural Information Processing Systems 38 · doi:10.52202/085713-1122 · Cited by 12 · see citing works
Colas, J. T., O’Doherty, J. P., & Grafton, S. T. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011950 · Cited by 6 · see citing works
The successor representation subserves hierarchical abstraction for goal-directed behavior
Wientjes, S., & Holroyd, C. B. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011312 · Cited by 6 · see citing works
Multi-Plasticities: Distinguishing Context-Specific Habits from Complex Perseverations
Banavar, N. V., & Bornstein, A. M. (2024) · Springer eBooks · doi:10.1007/978-3-031-55889-4_5 · Cited by 4 · see citing works
Furukawa, J.-I., & Morimoto, J. (2025) · npj Robotics · doi:10.1038/s44182-025-00033-4 · Cited by 3 · see citing works
A computational principle of habit formationPreprint
Lakshminarasimhan, K. J. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.10.12.618033 · Cited by 3 · see citing works
Li, J., & Collins, A. G. E. (2024) · Cognition · doi:10.1016/j.cognition.2024.105967 · Cited by 2 · see citing works
Separable neurocomputational mechanisms underlying multisensory learningPreprint
Bedi, S., Casimiro, E., de Hollander, G., Raduner, N., Helmchen, F., Brem, S., Konovalov, A., & Ruff, C. C. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.11.18.688925
Suboptimal human inference reflects an efficient and flexible information bottleneckPreprint
Parker, J. A., Filipowicz, A. L. S., Li, K., Balasubramanian, V., Kable, J. W., & Gold, J. I. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.06.10.731461
- Authors
- Lai, L., Huang, A. Z., & Gershman, S. J.
- Affiliations
- Harvard University (US)
- McGill University (CA)
- Corresponding author
- Lai, Lucy
- Publisher
- Center for Open Science
- Published
- 2022-09-16
- OpenAlex
- W4296182512
- Cited by
- 9
- References
- 67
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 5 · 2022: 1
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Locations
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- PsyArXiv (OSF Preprints) (submittedVersion)
- Funders
- National Science Foundation (ONR/DoD N00014-17-1-2961, 1231216)
- Office of Naval Research (N00014-17-1-2961, N00014-17-1, N00014)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Life Sciences › Neuroscience › Cognitive Neuroscience › Motor Control and Adaptation
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- action chunking, policy complexity, working memory load, cognitive resource allocation, action sequencing, reaction time, skill acquisition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
