Internal models in biological control
Open accessAbstract
Rationality principles such as optimal feedback control and Bayesian inference underpin a probabilistic framework that has accounted for a range of empirical phenomena in biological sensorimotor control. To facilitate the optimization of flexible and robust behaviors consistent with these theories, the ability to construct internal models of the motor system and environmental dynamics can be crucial. In the context of this theoretic formalism, we review the computational roles played by such internal models and the neural and behavioral evidence for their implementation in the brain.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Forward models can help embodied agents to estimate and adjust the positions of their effectors in the course of action, by providing a representation of the expected position at each moment.
“In particular, forward models can help embodied agents to estimate and adjust the positions of their effectors in the course of action, by providing a representation of the expected position at each moment.”
From Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023) · cites this paper as McNamee & Wolpert 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 140 · Cited by 225
Research works printed but not linked: 116
- [1]Conant Roger Ashby Ross Every good regulator of a system must be a model of that system International Journal of Systems Science 1970 1 2 89 97
- [2]Miall RC Wolpert DM Forward models for physiological motor control Neural Networks 1996 9 1265 1279 12662535
- [3]Kawato M Internal models for motor control and trajectory planning Current Opinion in Neurobiology 1999 9 6 718 727 10607637
- [4]Craik Kenneth James Williams The Nature of Explanation Cambridge University Press 1943 445
- [5]Popper Karl The Poverty of Historicism The Beacon Press Boston 1957
- [7]MacKay David Information Theory, Inference, and Learning Algorithms Cambridge University Press 2003
- [9]Todorov Emanuel General duality between optimal control and estimation Proceedings of the IEEE Conference on Decision and Control 2008 4286 4292
- [11]Todorov Emanuel Optimality principles in sensorimotor control Nature Neuroscience 2004 7 907 915 15332089 PMC1488877
- [12]Hinton GE Dayan P Frey BJ Neal RM The “wake-sleep” algorithm for unsupervised neural networks Science 1995 268 1158 1161 7761831
- [13]Faisal Aldo Selen Luc PJ Wolpert Daniel M Noise in the nervous system Nature Reviews. Neuroscience 2008 9 292 303 18319728 PMC2631351
- [14]Kording KP Ku SP Wolpert DM Bayesian integration in force estimation J Neurophysiol 2004 92 5 3161 3165 15190091
- [15]Kording KP Wolpert DM Bayesian integration in sensorimotor learning Nature 2004 427 6971 244 247 14724638
- [16]Jazayeri M Shadlen MN Temporal context calibrates interval timing Nat Neurosci 2010 13 8 1020 1026 20581842 PMC2916084
- [17]Tassinari H Hudson TE Landy MS Combining priors and noisy visual cues in a rapid pointing task J Neurosci 2006 26 40 10154 10163 17021171 PMC6674625
- [20]Stone J Kerrigan I Porrill J Where is the light? bayesian perceptual priors for lighting direction Proceedings of the Royal Society B 2009 PMC2674484 19324801
- [22]Tenenbaum JB Griffiths TL Generalization, similarity, and bayesian inference Behavioral and Brain Sciences 2001 24 629 640 12048947
- [23]Griffiths Thomas L Tenenbaum Joshua B Theory-based causal induction Psychological Review 2009 10 116 4 661 716 19839681
- [25]Houlsby Neil MT Huszár Ferenc Ghassemi Mohammad M Orbán Gergő Wolpert Daniel M Lengyel Máté Cognitive tomography reveals complex, task-independent mental representations Current Biology 2013 23 21 2169 2175 24354016 PMC3898796
- [26]Kim JS Greene MJ Zlateski A Lee K Richardson M Turaga SC Purcaro M Balkam M Robinson A Behabadi BF Campos M Space-time wiring specificity supports direction selectivity in the retina Nature 2014 509 7500 331 336 24805243 PMC4074887
- [27]Berry Michael J II Brivanlou Iman H Jordan Thomas A Meister Markus Anticipation of moving stimuli by the retina Nature 1999 398 6725 334 10192333
- [29]Jazayeri Mehrdad Movshon J Anthony Optimal representation of sensory information by neural populations Nature Neuroscience 2006 9 690 696 16617339
- [30]Beck JM Latham PE Pouget A Marginalization in Neural Circuits with Divisive Normalization Journal of Neuroscience 2011 10 31 43 15310 15319 22031877 PMC3230133
- [31]Ganguli D Simoncelli EP Efficient sensory encoding and bayesian inference with heterogeneous neural populations Neural Comput 2014 26 10 2103 2134 25058702 PMC4167880
- [35]Beck Jeffrey M Ma Wei Ji Kiani Roozbeh Hanks Tim Churchland Anne K Roitman Jamie Shadlen Michael N Latham Peter E Pouget Alexandre Probabilistic population codes for bayesian decision making Neuron 2008 60 1142 1152 19109917 PMC2742921
- [36]Yang JF Scholz JP Latash ML The role of kinematic redundancy in adaptation of reaching Exp Brain Res 2007 176 1 54 69 16874517 PMC1945250
- [37]Vogels TP Rajan K Abbott LF Neural network dynamics Annu Rev Neurosci 2005 28 357 376 16022600
- [38]Murphy BK Miller KD Balanced amplification: a new mechanism of selective amplification of neural activity patterns Neuron 2009 61 4 635 648 19249282 PMC2667957
- [40]Fiser József Berkes Pietro Orbán Gergo Lengyel Máté Statistically optimal perception and learning: from behavior to neural representations Trends in Cognitive Sciences 2010 14 119 130 20153683 PMC2939867
- [42]Todorov E Li W Optimal Control Methods Suitable for Biomechanical Systems IEEE Engineering in Medicine and Biology Society 2003 2 5 17271336
- [43]Todorov Emanuel Stochastic optimal control and estimation methods adapted to the noise characteristics of the sensorimotor system Neural Computation 2005 17 1084 1108 15829101 PMC1550971
- [44]Flash T Hogan N The coordination of arm movements: an experimentally confirmed mathematical model Journal of Neuroscience 1985 5 1688 1703 4020415 PMC6565116
- [45]Uno Y Kawato M Suzuki R Formation and control of optimal trajectory in human multijoint arm movement Biological Cybernetics 1989 61 2 89 101 2742921
- [46]Bernstein NI The Coordination and Regulation of Movement Pergamon Oxford 1967
- [47]Latash Mark L Scholz John P Schölner Gregor Motor control strategies revealed in the structure of motor variability Exercise and sport sciences reviews 2002 30 26 31 11800496
- [48]Diedrichsen J Optimal task-dependent changes of bimanual feedback control and adaptation Curr Biol 2007 17 19 1675 1679 17900901 PMC2230536
- [49]Liu Dan Todorov Emanuel Evidence for the flexible sensorimotor strategies predicted by optimal feedback control Journal of Neuroscience 2007 27 35 9354 9368 17728449 PMC6673117
- [50]Kutch Jason J Kuo Arthur D Bloch Anthony M Rymer William Z Endpoint force fluctuations reveal flexible rather than synergistic patterns of muscle cooperation Journal of Neurophysiology 2008 100 2455 2471 18799603 PMC2585402
- [52]Scott Stephen H The computational and neural basis of voluntary motor control and planning Trends in Cognitive Sciences 2012 16 541 549 23031541
- [53]Diedrichsen Jorn Optimal task-dependent changes of bimanual feedback control and adaptation Current Biology 2007 17 1675 1679 17900901 PMC2230536
- [54]Jones KE Hamilton AF Wolpert DM Sources of signal-dependent noise during isometric force production J Neurophysiol 2002 88 3 1533 1544 12205173
- [55]Hamilton AF Jones KE Wolpert DM The scaling of motor noise with muscle strength and motor unit number in humans Exp Brain Res 2004 157 4 417 430 15014922
- [56]Franklin David W Burdet Etienne Tee Keng Peng Osu Rieko Chew Chee-meng Milner Theodore E Kawato Mitsuo Cns learns stable, accurate, and efficient movements using a simple algorithm Journal of Neuroscience 2008 28 11165 11173 18971459 PMC6671516
- [57]Valero-Cuevas Francisco J Venkadesan Madhusudhan Todorov Emanuel Structured Variability of Muscle Activations Supports the Minimal Intervention Principle of Motor Control Journal of Neurophysiology 2009 102 59 68 19369362 PMC2712269
- [58]Nagengast Arne J Braun Daniel a Wolpert Daniel M Risk-sensitive optimal feedback control accounts for sensorimotor behavior under uncertainty PLoS Computational Biology 2010 6 7 15 PMC2904762 20657657
- [59]Dimitriou Michael Wolpert Daniel M Franklin David W The temporal evolution of feedback gains rapidly update to task demands Journal of Neuroscience 2013 33 26 10898 909 23804109 PMC3724995
- [60]Yeo Sang-Hoon Franklin David W Wolpert Daniel M When optimal feedback control is not enough: Feedforward strategies are required for optimal control with active sensing PLOS Computational Biology 2016 12 e1005190 27973566 PMC5156370
- [61]Scott Stephen H A functional taxonomy of bottom-up sensory feedback processing for motor actions Trends in Neurosciences 2016 39 8 1 15 27378546
- [62]Rothwell JC Traub MM Marsden CD Influence of voluntary intent on the human long-latency stretch reflex Nature 1980 286 496 498 7402329
- [63]Pruszynski J Andrew Kurtzer Isaac Nashed Joseph Y Omrani Mohsen Brouwer Brenda Scott Stephen H Primary motor cortex underlies multi-joint integration for fast feedback control Nature 2012 478 387 390 PMC4974074 21964335
- [64]Shen L Alexander GE Neural correlates of a spatial sensory-to-motor transformation in primary motor cortex Journal of Neurophysiology 1997 77 1171 1194 9084589
- [65]Scott S Kalaska J Reaching movements with similar hand paths but different arm orientations. i. activity of individual cells in motor cortex Journal of Neurophysiology 1997 77 826 852 9065853
- [66]Scott Stephen H Optimal feedback control and the neural basis of volitional motor control Nature Reviews. Neuroscience 2004 5 7 532 546 15208695
- [67]Pruszynski JA Scott SH Optimal feedback control and the long-latency stretch response Exp Brain Res 2012 218 3 341 359 22370742
- [68]Hatze H Buys JD Energy-optimal controls in the mammalian neuromuscular system Biological Cybernetics 1977 27 9 20 889959
- [69]Anderson Frank C Pandy Marcus G Dynamic optimization of human walking J Biomech Eng 2001 123 5 381 11601721
- [70]Srinivasan Manoj Ruina Andy Computer optimization of a minimal biped model discovers walking and running Nature 2006 439 72 75 16155564
- [71]Shadmehr Reza Huang Helen J Ahmed Alaa A A representation of effort in decision-making and motor control Current Biology 2016 1 6 PMC7912535 27374338
- [72]Wolpert DM Landy MS Motor control is decision-making Curr Opin Neurobiol 2012 22 6 996 1003 22647641 PMC3434279
- [73]Trommershauser J Maloney LT Landy MS Decision making, movement planning and statistical decision theory Trends Cogn Sci 2008 12 8 291 297 18614390 PMC2678412
- [74]Diamond JS Wolpert DM Flanagan JR Rapid target foraging with reach or gaze: The hand looks further ahead than the eye PLoS Comput Biol 2017 13 7 e1005504 28683138 PMC5500014
- [76]Kalman RE A New Approach to Linear Filtering and Prediction Problems Journal of Basic Engineering 1960 82 1 35
- [77]Mitter Sanjoy Newton Nigel A variational approach to nonlinear estimation SIAM Journal on Control and Optimization 2003 42 5 1 21
- [78]Todorov Emanuel Compositionality of optimal control laws Advances in Neural Information Processing Systems 2009 3 1856 1864
- [79]Todorov Emanuel Linearly-solvable markov decision problems Advances in Neural Information Processing Systems 2007 19 1369 1376
- [80]Rawlik Konrad Toussaint Marc Vijayakumar Sethu On stochastic optimal control and reinforcement learning by approximate inference (extended abstract) International Joint Conference on Artificial Intelligence (IJCAI) 2013
- [82]van Beers Robert J Motor learning is optimally tuned to the properties of motor noise Neuron 2009 63 406 417 19679079
- [83]Braun DA Nagengast AJ Wolpert DM Risk-sensitivity in sensorimotor control Front Hum Neurosci 2011 5 1 21283556 PMC3028548
- [84]Kawato M Furukawa Kazunori Suzuki R A hierarchical neural-network model for control and learning of voluntary movement Biological Cybernetics 1987 57 169 185 3676355
- [87]Crapse Trinity B Sommer Marc A Corollary discharge across the animal kingdom Nature Reviews. Neuroscience 2008 9 8 587 18641666 PMC5153363
- [88]Haarmeier T Bunjes F Lindner A Berret E Thier P Optimizing visual motion perception during eye movements Neuron 2001 32 527 535 11709162
- [89]Sommer MA Wurtz RH Influence of the thalamus on spatial visual processing in frontal cortex Nature 2006 444 374 377 17093408
- [90]Bell C Memory-based expectations in electrosensory systems Current Opinion In Neurobiology 2001 11 4 481 487 11502396
- [91]Sawtell Nathaniel B Neural mechanisms for predicting the sensory consequences of behavior: Insights from electrosensory systems Annual Review of Physiology 2017 79 381 399 27813831
- [92]Rothwell JC Traub MM Day BL Obeso JA Thomas PK Marsden CD Manual motor performance in a deaffertated man Brain 1982 105 515 542 6286035
- [93]Gordon J Ghilardi MF Ghez C Impairments of reaching movements in patients without proprioception. ii. effects of visual information on accuracy J Neurophysiol 1995 73 361 372 7714578
- [94]Wolpert DM Ghahramani Z Jordan MI An internal model for sensorimotor integration Science 1995 269 5232 1880 1882 7569931
- [95]Diedrichsen Jörn Verstynen Timothy Hon Andrew Lehman Steven L Ivry Richard B Anticipatory adjustments in the unloading task: Is an efference copy necessary for learning? Experimental Brain Research 2003 148 272 276 12520418
- [96]Flanagan J Randall Wing Alan M Modulation of grip force with load force during point-to-point arm movements Experimental Brain Research 1993 95 131 143 8405245
- [97]Kawato Mitsuo Kuroda Tomoe Imamizu Hiroshi Nakano Eri Miyauchi Satoru Yoshioka Toshinori Internal forward models in the cerebellum: fmri study on grip force and load force coupling Progress in Brain Research 2003 142 171 188 12693261
- [98]Johansson Roland S Riso Ronald Häger Charlotte Bäckström Lars Somatosensory control of precision grip during unpredictable pulling loads - i. changes in load force amplitude Experimental Brain Research 1992 89 181 191 1601096
- [99]Crevecoeur F Scott SH Priors engaged in long-latency responses to mechanical perturbations suggest a rapid update in state estimation PLoS Comput Biol 2013 9 8 e1003177 23966846 PMC3744400
- [100]Crevecoeur Frédéric Munoz Douglas P Scott Stephen H Dynamic multisensory integration: Somatosensory speed trumps visual accuracy during feedback control Journal of Neuroscience 2016 36 8598 8611 27535908 PMC6601898
- [101]Nasseroleslami B Hasson CJ Sternad D Rhythmic manipulation of objects with complex dynamics: predictability over chaos PLoS Comput Biol 2014 10 e1003900 25340581 PMC4207605
- [103]Therrien Amanda S Bastian Amy J Cerebellar damage impairs internal predictions for sensory and motor function Current Opinion in Neurobiology 2015 33 127 133 25863011 PMC4786071
- [104]Ebner Timothy J Pasalar Siavash Cerebellum predicts the future motor state Cerebellum 2008 7 583 588 18850258 PMC2754147
- [106]Brooks Jessica X Carriot Jerome Cullen Kathleen E Learning to expect the unexpected: rapid updating in primate cerebellum during voluntary self-motion Nature Neuroscience 2015 18 1310 1317 26237366 PMC6102711
- [107]Müller F Dichgans J Dyscoordination of pinch and lift forces during grasp in patients with cerebellar lesions Experimental Brain Research 1994 101 485 492 7851515
- [109]Bhanpuri Nasir H Okamura Allison M Bastian Amy J Predicting and correcting ataxia using a model of cerebellar function Brain 2014 137 1931 1944 24812203 PMC4065021
- [110]Mulliken Grant H Musallam Sam Andersen Richard A Forward estimation of movement state in posterior parietal cortex Proceedings of the National Academy of Sciences of the United States of America 2008 6 105 24 8170 7 18499800 PMC2448809
- [111]Buneo Christopher A Andersen Richard A The posterior parietal cortex: sensorimotor interface for the planning and online control of visually guided movements Neuropsychologia 2006 1 44 13 2594 606 16300804
- [112]Desmurget M Epstein CM Turner RS Prablanc C Alexander GE Grafton ST Role of the posterior parietal cortex in updating reaching movements to a visual target Nature Neuroscience 1999 2 563 567 10448222
- [114]Rushworth MFS Nixon PD Passingham RE Parietal cortex and movement. i. movement selection and reaching Experimental Brain Research 1997 117 292 310 9419075
- [117]Dolan Ray J Dayan Peter Goals and habits in the brain Neuron 2013 80 312 325 24139036 PMC3807793
- [118]Schultz Wolfram Dayan Peter Montague P Read A neural substrate of prediction and reward Science 1997 3 275 5306 1593 9 9054347
- [120]Kappen HJ Ruiz HC Adaptive importance sampling for control and inference Journal of Statistical Physics 2016 162 1244 1266
- [121]Theodorou Evangelos Buchli Jonas Schaal Stefan A generalized path integral control approach to reinforcement learning Journal of Machine Learning Research 2010 11 3137 3181
- [122]Silver David Veness Joel Monte-carlo planning in large pomdps Advances in Neural Information Processing Systems 2010 1 9
- [123]Battaglia PW Hamrick JB Tenenbaum JB Simulation as an engine of physical scene understanding Proceedings of the National Academy of Sciences 2013 110 18327 18332 PMC3831455 24145417
- [124]Bellman Richard Dynamic Programming Princeton University Press Princeton, NJ, USA 1957
- [125]D’Avella Andrea Saltiel Philippe Bizzi Emilio Combinations of muscle synergies in the construction of a natural motor behavior Nature Neuroscience 2003 6 3 300 308 12563264
- [126]Huys Quentin JM Lally Níall Faulkner Paul Eshel Neir Seifritz Erich Gershman Samuel J Dayan Peter Roiser Jonathan P Interplay of approximate planning strategies Proceedings of the National Academy of Sciences 2015 112 10 3098 3103 PMC4364207 25675480
- [127]Botvinick MM Niv Y Barto AC Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective Cognition 2009 113 3 262 280 18926527 PMC2783353
- [128]Newell Allen Rosenbloom Paul S Mechanisms of skill acquisition and the law of practice Anderson John Robert Cognitive Skills and their Acquisition MIT Press 1981 1 56
- [129]McNamee D Rangel A O’Doherty JP Category-dependent and category-independent goal-value codes in human ventromedial prefrontal cortex Nat Neurosci 2013 16 4 479 485 23416449 PMC3665508
- [130]Padoa-Schioppa Camillo Assad John A Neurons in the orbitofrontal cortex encode economic value Nature 2006 441 223 226 16633341 PMC2630027
- [131]Barron Helen C Dolan Raymond J Behrens Timothy EJ Online evaluation of novel choices by simultaneous representation of multiple memories Nature Neuroscience 2013 16 1492 1498 24013592 PMC4001211
- [132]Bogacz Rafal Brown Eri Moehlis Jeff Holmes Philip Cohen Jonathan D The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks Psychological Review 2006 113 4 700 17014301
- [133]Solway Alec Botvinick Matthew M Goal-directed decision making as probabilistic inference Psychological Review 2012 119 1 PMC3767755 22229491
- [134]Resulaj Arbora Kiani Roozbeh Wolpert Daniel M Shadlen Michael N Changes of mind in decision-making Nature 2009 461 263 266 19693010 PMC2875179
- [135]Solway Alec Botvinick Matthew M Evidence integration in model-based tree search Proceedings of the National Academy of Sciences 2015 112 11708 11713 PMC4577209 26324932
- [136]Daw Nathaniel D Niv Yael Dayan Peter Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control Nature Neuroscience 2005 12 8 12 1704 11 16286932
- [137]Wilson Robert C Takahashi Yuji K Schoenbaum Geoffrey Niv Yael Orbitofrontal cortex as a cognitive map of task space Neuron 2014 81 267 279 24462094 PMC4001869
- [138]Bornstein Aaron M Norman Kenneth A Reinstated episodic context guides sampling-based decisions for reward Nature Neuroscience 2017 20 997 1003 28581478
- [139]Jax SA Rosenbaum DA Hand path priming in manual obstacle avoidance: evidence that the dorsal stream does not only control visually guided actions in real time J Exp Psychol Hum Percept Perform 2007 33 2 425 441 17469977
- [140]Ghahramani Zoubin Probabilistic machine learning and artificial intelligence Nature 2015 521 7553 452 459 26017444
- [141]Esposito MS Capelli P Arber S Brainstem nucleus mdv mediates skilled forelimb motor tasks Nature 2014 508 7496 351 356 24487621
- Authors
- McNamee, D., & Wolpert, D. M.
- Affiliations
- University of Cambridge (GB)
- Corresponding author
- McNamee, Daniel
- Wolpert, Daniel M.
- Venue
- Annual Review of Control, Robotics, and Autonomous Systems
- Publisher
- Annual Reviews
- Published
- 2019-05-03
- Volume
- 2
- Issue
- 1
- Pages
- 339-364
- ISSN
- 2573-5144 (ISSN-L)
- PMC
- PMC6520231
- OpenAlex
- W2906340543
- Cited by
- 225
- References
- 140
- Citations by year
- 2026: 23 · 2025: 34 · 2024: 45 · 2023: 40 · 2022: 42 · 2021: 30 · 2020: 10 · 2019: 1
- FWCI
- 2.63
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Annual Review of Control Robotics and Autonomous Systems (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Cambridge University Engineering Department Publications Database (submittedVersion)
- Funders
- Wellcome Trust (097803, 110257/A/15/Z, 097803/Z/11/Z)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Motor Control and Adaptation
- Life Sciences › Neuroscience › Cognitive Neuroscience › Embodied and Extended Cognition
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- internal models, biological control, optimal feedback control, Bayesian inference, sensorimotor control, motor system, environmental dynamics, neural implementation, computational neuroscience
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review