CoChat
Sign Up
Cited by 212Neural dynamics and brain functionEmbodied and Extended CognitionAdvanced Thermodynamics and Statistical Mechanics

Information and efficiency in the nervous system-a synthesis.

Open access
Sengupta, Biswa, Stemmler, Martin B., Friston, Karl J.·2013·PLoS Computational Biology
DOI 10.1371/journal.pcbi.1003157PMC PMC3723496
Open this paper in CoChat — free accountOpen PDF

Abstract

In systems biology, questions concerning the molecular and cellular makeup of an organism are of utmost importance, especially when trying to understand how unreliable components--like genetic circuits, biochemical cascades, and ion channels, among others--enable reliable and adaptive behaviour. The repertoire and speed of biological computations are limited by thermodynamic or metabolic constraints: an example can be found in neurons, where fluctuations in biophysical states limit the information they can encode--with almost 20-60% of the total energy allocated for the brain used for signalling purposes, either via action potentials or by synaptic transmission. Here, we consider the imperatives for neurons to optimise computational and metabolic efficiency, wherein benefits and costs trade-off against each other in the context of self-organised and adaptive behaviour. In particular, we try to link information theoretic (variational) and thermodynamic (Helmholtz) free-energy formulations of neuronal processing and show how they are related in a fundamental way through a complexity minimisation lemma.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pcbi.1003157

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Minimizing complexity implicitly maximizes the thermodynamic efficiency of information processing in the brain.

    “The premise is that minimizing complexity implicitly maximizes the thermodynamic efficiency of information processing in the brain.”

    From Virtual reality and consciousness inference in dreaming (2014) · cites this paper as Sengupta et al., 2013

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 78

  1. [1]Sutherland WJ (2005) The best solution. Nature 435: 569. 15931199
  2. [2]Alexander RM (1996) Optima for animals. Princeton/Chichester: Princeton University Press.
  3. [3]Niven JE, Laughlin SB (2008) Energy limitation as a selective pressure on the evolution of sensory systems. J Exp Biol 211: 1792– 1804. 18490395
  4. [4]Feynman RP, Hey AJG, Allen RW (1996) Feynman lectures on computation. Reading (Massachusetts): Addison-Wesley.
  5. [5]Landauer R (1961) Irreversibility and heat generation in the computing process. IBM Journal of Research and Development 5: 183– 191.
  6. [6]Landauer R (1996) Minimal energy requirements in communciation. Science 272: 1914– 1918. 8662490
  7. [7]Stemmler M, Sengupta B, Laughlin SB, Niven JE (2011) Energetically optimal action potentials. In: Shawe-Taylor J, Zemel RS, Bartlett P, Pereira F, Weinberger KQ, editors. Advances in neural information processing systems 24. pp. 1566–1574.
  8. [9]Ashby WR (1947) Principles of the self-organising dynamic system. J Gen Psychol 37: 125– 128. 20270223
  9. [10]Nicolis G, Prigogine I (1977) Self-organisation in non-equilibrium systems. New York: Wiley.
  10. [11]Birkhoff GD, Koopman BO (1932) Recent contributions to the ergodic theory. Proc Natl Acad Sci U S A 18: 279– 282. 16587677 PMC1076207
  11. [12]Friston K, Ao P (2012) Free energy, value, and attractors. Comput Math Methods Med 2012: 937860. 22229042 PMC3249597
  12. [13]Friston K (2009) The free-energy principle: a rough guide to the brain? TICS 13: 293– 301. 19559644
  13. [15]Feynman RP (1998) Statistical mechanics: a set of lectures. Boulder (Colorado): Westview Press.
  14. [16]Beal M (2003) Variational algorithms for approximate bayesian inference [PhD thesis]. London: University College London.
  15. [17]Hinton GE, Sejnowski TJ (1983) Analyzing cooperative computation. In: Proceedings of the 5th Annual Congress of the Cognitive Science Society; 1983; Rochester, New York, United States.
  16. [18]Hinton GE, Sejnowski TJ (1983) Optimal perceptual inference. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 1983; Washington, D.C., United States. pp. 448–453.
  17. [20]Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci U S A 79: 2554– 2558. 6953413 PMC346238
  18. [21]Friston K, Kilner J, Harrison L (2006) A free energy principle for the brain. J Physiol Paris 100: 70– 87. 17097864
  19. [22]Friston K, Mattout J, Trujillo-Barreto N, Ashburner J, Penny W (2007) Variational free energy and the Laplace approximation. Neuroimage 34: 220– 234. 17055746
  20. [23]Harrison LM, Stephan KE, Rees G, Friston KJ (2007) Extra-classical receptive field effects measured in striate cortex with fMRI. Neuroimage 34: 1199– 1208. 17169579 PMC2640483
  21. [24]Garrido MI, Kilner JM, Kiebel SJ, Friston KJ (2007) Evoked brain responses are generated by feedback loops. Proc Natl Acad Sci U S A 104: 20961– 20966. 18087046 PMC2409249
  22. [29]Mumford D (1992) On the computational architecture of the neocortex. II. The role of cortico-cortical loops. Biol Cybern 66: 241– 251. 1540675
  23. [34]Abbott LF, Varela JA, Sen K, Nelson SB (1997) Synaptic depression and cortical gain control. Science 275: 220– 224. 8985017
  24. [36]Elias P (1955) Predictive coding-I. IRE Transactions on Information Theory 1: 16– 24.
  25. [37]Elias P (1955) Predictive coding-II. IRE Transactions on Information Theory 1: 24– 33.
  26. [38]Stemmler M, Koch C (1999) How voltage-dependent conductances can adapt to maximize the information encoded by neuronal firing rate. Nat Neurosci 2: 521– 527. 10448216
  27. [40]Friston K (2011) What is optimal about motor control? Neuron 72: 488– 498. 22078508
  28. [41]Peleg Y, Efraim H, Shental O, Kanter I (2010) Mutual information via thermodynamics: three different approaches. Journal of Statistical Mechanics: Theory and Experiments Jan: P01014.
  29. [43]Shannon C (1948) A mathematical theory of communication. Bell System Technical Journal 27: 379– 423.
  30. [44]Jaynes ET (2003) Probability theory: the logic of science. Cambridge (United Kingdom): Cambridge University Press.
  31. [45]Atick JJ (1992) Could information theory provide an ecological theory of sensory processing? Network 3: 213– 251. 22149669
  32. [48]Barlow HB (1959) Sensory mechanisms, the reduction of redundancy, and intelligence. National Physical Laboratory Symposium. Teddington (United Kingdom): H.M. Stationery Office.
  33. [49]Laughlin S (1981) A simple coding procedure enhances a neuron's information capacity. Z Naturforsch C 36: 910– 912. 7303823
  34. [50]Johnson D (2002) Four top reasons mutual information does not quantify neural information processing. In: Proceedings of the Annual Computational Neuroscience Meeting; 2002; Chicago, Illinois, United States.
  35. [51]Papoulis A (1984) Probability, random variables, and stochastic processes. New York: McGraw-Hill.
  36. [52]Benda J, Gollisch T, Machens CK, Herz AV (2007) From response to stimulus: adaptive sampling in sensory physiology. Curr Opin Neurobiol 17: 430– 436. 17689952
  37. [54]Paninski L (2005) Asymptotic theory of information-theoretic experimental design. Neural Comput 17: 1480– 1507. 15901405
  38. [56]Ames A (2000) CNS energy metabolism as related to function. Brain Res Brain Res Rev 34: 42– 68. 11086186
  39. [57]Rolfe DF, Brown GC (1997) Cellular energy utilization and molecular origin of standard metabolic rate in mammals. Physiol Rev 77: 731– 758. 9234964
  40. [58]Clarke D, Sokoloff L (1999) Circulation and energy metabolism of the brain. In: Siegel GJ, Agranoff BW, Albers RW, Fisher SK, Uhler MD, editors. Basic neurochemistry: molecular, cellular and medical aspects. Philadelphia: Lippincott-Raven. pp. 637–669.
  41. [59]Attwell D, Laughlin SB (2001) An energy budget for signaling in the grey matter of the brain. J Cereb Blood Flow Metab 21: 1133– 1145. 11598490
  42. [60]Skou JC (1989) The identification of the sodium-pump as the membrane-bound Na+/K+-ATPase: a commentary on ‘The Influence of Some Cations on an Adenosine Triphosphatase from Peripheral Nerves’. Biochim Biophys Acta 1000: 435– 438. 2550073
  43. [61]Skou JC (1957) The influence of some cations on an adenosine triphosphatase from peripheral nerves. Biochim Biophys Acta 23: 394– 401. 13412736
  44. [62]Post RL, Hegyvary C, Kume S (1972) Activation by adenosine triphosphate in the phosphorylation kinetics of sodium and potassium ion transport adenosine triphosphatase. J Biol Chem 247: 6530– 6540. 4263199
  45. [63]Sen AK, Post RL (1964) Stoichiometry and localization of adenosine triphosphate-dependent sodium and potassium transport in the erythrocyte. J Biol Chem 239: 345– 352. 14114864
  46. [64]Pangrsic T, Stusek P, Belusic G, Zupancic G (2005) Light dependence of oxygen consumption by blowfly eyes recorded with a magnetic diver balance. J Comp Physiol A Neuroethol Sens Neural Behav Physiol 191: 75– 84. 15558288
  47. [66]Laughlin SB (2001) Energy as a constraint on the coding and processing of sensory information. Curr Opin Neurobiol 11: 475– 480. 11502395
  48. [67]Laughlin SB, Sejnowski TJ (2003) Communication in neuronal networks. Science 301: 1870– 1874. 14512617 PMC2930149
  49. [68]Levy WB, Baxter RA (1996) Energy efficient neural codes. Neural Comput 8: 531– 543. 8868566
  50. [69]von der Malsburg C, Willshaw DJ (1977) How to label nerve cells so that they can interconnect in an ordered fashion. Proc Natl Acad Sci U S A 74: 5176– 5178. 270757 PMC432124
  51. [70]Bullmore E, Sporns O (2012) The economy of brain network organization. Nat Rev Neurosci 13: 336– 349. 22498897
  52. [71]Bullmore E, Sporns O (2009) Complex brain networks: graph theoretical analysis of structural and functional systems. Nat Rev Neurosci 10: 186– 198. 19190637
  53. [72]Sarpeshkar R (1998) Analog versus digital: extrapolating from electronics to neurobiology. Neural Comp 10: 1601– 1638. 9744889
  54. [73]Sarpeshkar R (2010) Ultra low power bioelectronics: fundamentals, biomedical applications, and bio-inspired systems. Cambridge (United Kingdom): Cambridge University Press.
  55. [74]Laughlin SB, de Ruyter van Steveninck RR, Anderson JC (1998) The metabolic cost of neural information. Nat Neurosci 1: 36– 41. 10195106
  56. [75]Niven JE, Vähäsöyrinki M, Kauranen M, Hardie RC, Juusola M, et al (2003) The contribution of Shaker K+ channels to the information capacity of Drosophila photoreceptors. Nature 421: 630– 634. 12571596
  57. [76]Niven JE, Vähäsöyrinki M, Juusola M (2003) Shaker K+ channels are predicted to reduce the metabolic cost of neural information in Drosophila photoreceptors. Proc Biol Sci 270 Suppl 1: S58– S61. 12952637 PMC1698034
  58. [77]Attwell D, Gibb A (2005) Neuroenergetics and the kinetic design of excitatory synapses. Nat Rev Neurosci 6: 841– 849. 16261178
  59. [78]Carter BC, Bean BP (2009) Sodium entry during action potentials of mammalian neurons: incomplete inactivation and reduced metabolic efficiency in fast-spiking neurons. Neuron 64: 898– 909. 20064395 PMC2810867
  60. [80]Sengupta B, Stemmler M, Laughlin SB, Niven JE (2010) Action potential energy efficiency varies among neuron types in vertebrates and invertebrates. PLoS Comput Biol 6: e1000840 doi: 10.1371/journal.pcbi.1000840 20617202 PMC2895638 · DOI
  61. [81]Alle H, Roth A, Geiger JR (2009) Energy-efficient action potentials in hippocampal mossy fibers. Science 325: 1405– 1408. 19745156
  62. [82]Lennie P (2003) The cost of cortical computation. Curr Biol 13: 493– 497. 12646132
  63. [83]Nawroth JC, Greer CA, Chen WR, Laughlin SB, Shepherd GM (2007) An energy budget for the olfactory glomerulus. J Neurosci 27: 9790– 9800. 17804639 PMC6672954
  64. [84]Okawa H, Sampath AP, Laughlin SB, Fain GL (2008) ATP consumption by mammalian rod photoreceptors in darkness and in light. Curr Biol 18: 1917– 1921. 19084410 PMC2615811
  65. [85]Howarth C, Peppiatt-Wildman CM, Attwell D (2010) The energy use associated with neural computation in the cerebellum. J Cereb Blood Flow Metab 30: 403– 414. 19888288 PMC2859342
  66. [87]Attneave F (1954) Some informational aspects of visual perception. Psychol Rev 61: 183– 193. 13167245
  67. [89]Wen Q, Chklovskii DB (2008) A cost-benefit analysis of neuronal morphology. J Neurophysiol 99: 2320– 2328. 18305091
  68. [90]Wen Q, Stepanyants A, Elston GN, Grosberg AY, Chklovskii DB (2009) Maximization of the connectivity repertoire as a statistical principle governing the shapes of dendritic arbors. Proc Natl Acad Sci U S A 106: 12536– 12541. 19622738 PMC2713752
  69. [91]Chen BL, Hall DH, Chklovskii DB (2006) Wiring optimization can relate neuronal structure and function. Proc Natl Acad Sci U S A 103: 4723– 4728. 16537428 PMC1550972
  70. [92]Crotty P, Sangrey T, Levy WB (2006) Metabolic energy cost of action potential velocity. J Neurophysiol 96: 1237– 1246. 16554507
  71. [93]Ortega PA, Brau DA (2012) Thermodynamics as a theory of decision-making with information processing costs. ArXiv 1204.6481v1. · arXiv
  72. [95]Still S, Sivak DA, Bell AJ, Crooks GE (2012) Thermodynamics of prediction. Phys Rev Lett 109: 120604. 23005932
  73. [97]Linsker R (1990) Perceptual neural organization: some approaches based on network models and information theory. Annu Rev Neurosci 13: 257– 281. 2183677
  74. [99]Paus T, Keshavan M, Giedd JN (2008) Why do many psychiatric disorders emerge during adolescence? Nat Rev Neurosci 9: 947– 957. 19002191 PMC2762785
  75. [101]Sengupta B, Faisal AA, Laughlin SB, Niven JE (2013) The effect of cell size and channel density on neuronal information encoding and energy efficiency. J Cereb Blood Flow Metab In press. PMC3764378 23778164
  76. [102]Christie P, Stroobandt D (2000) The interpretation and application of Rent's rule. IEEE Journal of VLSI 8: 639– 648.
  77. [103]Bassett DS, Greenfield DL, Meyer-Lindenberg A, Weinberger DR, Moore SW, et al (2010) Efficient physical embedding of topologically complex information processing networks in brains and computer circuits. PLoS Comput Biol 6: e1000748 doi: 10.1371/journal.pcbi.1000748 20421990 PMC2858671 · DOI
  78. [104]Sengupta B (2011) Information encoding and energy consumption in single neuron models [PhD thesis]. Cambridge (United Kingdom): University of Cambridge.

Cited by

Sorted by citations

Authors
Sengupta, B., Stemmler, M. B., & Friston, K. J.
Affiliations
  • Wellcome Centre for Human Neuroimaging (GB)
  • Ludwig-Maximilians-Universität München (DE)
Corresponding author
Sengupta, Biswa
ORCID
Venue
PLoS Computational Biology
Publisher
Public Library of Science (PLoS)
Published
2013-07-25
Volume
9
Issue
7
Pages
e1003157
ISSN
  • 1553-734X (ISSN-L)
  • 1553-7358
DOI
10.1371/journal.pcbi.1003157
PMC
PMC3723496
OpenAlex
W2073820635
Cited by
212
References
18
Citations by year
2026: 13 · 2025: 6 · 2024: 6 · 2023: 27 · 2022: 24 · 2021: 17 · 2020: 18 · 2019: 40 · 2018: 11 · 2017: 15 · 2016: 15 · 2015: 7 · 2014: 10 · 2013: 2
FWCI
0.96
Type
journal article
Language
en
Open access
Licence
Locations
  • PLoS Computational Biology (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • NOT FOUND REPOSITORY (Indian Institute of Science Bangalore) (acceptedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • UCL Discovery (University College London) (submittedVersion)
  • Open access LMU (Ludwid Maxmilian's Universitat Munchen) (submittedVersion)
Funders
  • Wellcome Trust (091593, 088130)
  • Department of Biotechnology, Ministry of Science and Technology, India
  • Boehringer Ingelheim Fonds
  • Bundesministerium für Bildung und Forschung
  • European Social Fund
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Embodied and Extended Cognition
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Advanced Thermodynamics and Statistical Mechanics
Keywords
metabolic efficiency, Helmholtz free energy, adaptive behavior, synaptic transmission, action potential, neural computation
MeSH terms
Action Potentials, Humans, Nervous System Physiological Phenomena, Thermodynamics, Signal Transduction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review