Principles of Neural Design

“Proponents of basal cognition remind us that neural networks evolved from far more ancient signalling pathways; neurons mainly optimised existing mechanisms for speed (Sterling and Laughlin 2015).”

“As discussed above, if an organism is at rest and everything is as expected it might want not only to integrate and update its ‘self’/’world’ model less frequently, but also to update its lower-level components less frequently to conserve energy as firing action potentials is energetically expensive [47].”

“In biology, the sparseness of both connectivity and activity is thought to be a key design principle for energy-efficient information processing (Sterling & Laughlin, 2017).”

“These phenomena can indeed be observed in real visual systems [61].”

“Within the realm of neuroscience and psychology, PP offers an account of brain function that is metabolically efficient (Friston, 2010; Rao & Ballard, 1999; Sterling & Laughlin, 2015) and neurobiologically plausible (Bastos et al., 2012).”

“The primary ganglion, or brain, is located in the head, with sense organs clustered there for efficiency [47].”

“Such an approach provides a natural unsupervised learning objective for the brain (Rao & Ballard, 1999), while also minimizing redundancy and maximizing information transmission by transmitting only unpredicted information (Barlow, 1961; Sterling & Laughlin, 2015).”

“…input, assuming Poisson statistics of release (Ashmore and Copenhagen, 1983; Freed, 2000; Freed and Liang, 2010), would be proportional to the square root of the release rate; therefore, the high baseline rate, although expensive metabolically, would improve the SNR (Sterling and Laughlin, 2015).”

“In general, however, asking a neuron to perform two jobs simultaneously has been said to ensure that both are done poorly (Sterling and Laughlin, 2017).”

“While both metabolic and electrical signaling supported more efficient communication, electrical signaling was more efficient than metabolic signaling (Deco et al., 2014; Sterling & Laughlin, 2015).”

“Expanding on these initial ideas, many researchers have described how conceptual change in childhood is fuelled by increases in general capacities 48,68,70,230–232 .”

“Tract-tracing studies demonstrate that different PAG columns project to different autonomic nuclei in the brainstem (Cameron et al., 1995) that are critical for implementing allostasis (Sterling and Laughlin, 2015).”

“13,14).”

“Finally, Jeffort penalizes 198 control effort in the form of input magnitude through199 out the whole trial, thus promoting energy-efficient con200 trol solutions amongst a typically infinite set of possi201 bilities (Kao et al., 2021; Sterling and Laughlin, 2015).”

“Because of evolutionary pressure, we can assume measurements from wild-type animals or neurotypical people will be located near this peak (blue circles) [25].”

“As circuit wiring principles are highly conserved among mammalian brains (Sterling and Laughlin, 2015), our results may have wider implications in the developmental basis of neural circuitry evolution and individuality.”

“…references with additional details and complementary views; see the following and references therein: Barrett, 2017; Barrett & Simmons, 2015; Chanes & Barrett, 2016; Hutchinson & Barrett, 2019; Katsumi et al., 2022, 2023; Kleckner et al., 2017; Schulkin & Sterling, 2019; Sterling & Laughlin, 2015).”

“It has been increasingly recognized that evolution has optimized neurons and their circuits for best simultaneous performance in terms of functional effectiveness and economy [34,40,48].”

“Physiological states are insulated from hazardous fluctuations in the external environment by the operational and computational processes known as homeostasis [1-3].”

“Using biological brains as an inspiration for designing novel computing paradigms can lead to massively distributed computing technologies that operate on extremely tight power budgets [92], while being robust to ambiguities in real-world sensory information and resilient to component failures [39].”

“For example the human brain performs myriad computations, including complex object recognition in 150 ms [1] while consuming less than 20 watts [2].”

“Other cell-adhesion molecules that regulate neuron selforganization through homotypic binding include the MEGF proteins and the γ-protocadherin complex (48, 88), while semaphorins and plexins restrict axon and dendrite arbors across the vertical plane of the retina (i.e., photoreceptors to RGCs) (70, 71, 89, 90).”

“Thus, diameter scaling may be a consequence of the metabolic needs of the cells, supporting the view that energetics is an important design consideration in the nervous system (27).”

“The details of how the brain deals with energy and transmission efficiency are brilliantly exposed in the books by Sterling and Laughlin (2017) and by both, Niven and Laughlin, 2008 and Harris and collaborators (Harris et al., 2012).”

“(Solovei et al., 2009; Sterling and Laughlin, 2015; Werner and Chalupa, 2004; Williams and Moody, 2004) and thereby the risk of scattering induced veil and loss of image contrast.”

“A “long” RF thus averages out high frequency stimuli, whereas a “short” one is sensitive to high frequency stimuli, consistent with the main known distinguishing characteristics of RGC receptive fields (Sanes and Masland, 2015; Sterling and Laughlin, 2015).”

“…in conscious experience and perception, specifically the idea that affect arises from metabolic energy regulation (i.e., allostasis; Sterling, 2012; Sterling & Laughlin, 2015) and the resulting internal sensations from the body (i.e., interoception; Craig, 2015) (Barrett, 2017; Barrett & Simmons,…”

“A person can be motivated to move, think, pay attention, and learn, which impacts the capacity to survive and reproduce (James, 1890; Sterling and Laughlin, 2015).”

“However, contrast encoding and heterosynapticity are both ubiquitous mechanisms that can approximate L2 normalization for the mossy fiber activations and granule dendrites, respectively (Sterling & Laughlin, 2015; Rumelhart & Zipser, 1985; Tononi & Cirelli, 2014).”

“Neuromorphic engineering mimics the way in which the human brain processes information, which is a hierarchy of layers of neurons connected typically in a feed-forward way with some recurrent connections [8,9].”

“On the one hand, a large body of literature studies information transmission starting from a spiking neuron [7], including all sorts of additional constraints such as energy or size [8–12].”

“, the statistics associated to spatial information in our input environment) to maximize computational efficiency (Simoncelli and Olshausen, 2001; Sterling and Laughlin, 2015; Tesileanu et al., 2020).”

“Allostasis is a brain-wide phenomenon (Barrett & Simmons, 2015; Kaplan & Zimmer, 2020; Kleckner et al., 2017; Sennesh et al., 2022; Sterling & Laughlin, 2015) and all experiences (e.g., perceptions, cognitions, emotions) are conditioned on it (Barrett, 2017; Ganzel et al., 2010).”

“This architectural variability further complicates the understanding of their internal mechanisms — akin to assuming that different species, each sensing the world through distinct systems, would nonetheless process these signals through identical neural pathways [54].”

“…in his later work, even if one thinks that sensory compression is one thing the brain does, it is not obvious that it ³See Simoncelli and Olshausen ( ); Sterling and Laughlin ( ); Stone ( ) for reviews of the contemporary literature on efficient coding in the sensory system. is the only thing.”

“Accordingly, texture images have been extensively used in experimental studies that have examined the contribution of different visual areas to the processing of texture statistics.”

“Current neuroscientific perspectives argue that the brain’s core function is allostasis, the process of monitoring, managing, and coordinating physiology to support an organism’s movement, growth, reproduction, and behavior (11,12).”

“Neural circuits in the periphery of the visual [2, 3, 4, 5, 6, 7, 8, 9, 10], auditory [11, 12, 13, 14], and perhaps also olfactory [15] systems use limited resources efficiently to represent sensory information by adapting to the statistical structure of the environment [16].”

“Many lines of evidence show that light-adaptive changes in the visual systems of both vertebrates and invertebrates adhere to these principles (Atick 2011; Sterling and Laughlin 2015; van Hateren 1992a).”

“[9]) largely ignores delay but dominates theory in the hardware layers [2], [1].”

“As Sterling and Laughlin (2015) develop, organisms take advantage of local processing as much as possible.”

“…understanding of mental representations as structural models (Cummins 1989; Gładziejewski 2015; Grush 2004; Ryder 2004; Shea 2014; Williams and Colling 2017); and an increased focus on regulation and predictive control as principles of brain functioning (Barrett 2017; Sterling and Laughlin 2015).”

“In this regard, Sterling and Laughlin write (Sterling and Laughlin 2015, p.124), "These advantages—compactness, energy efficiency, and ability to adapt and match—all suggest the principle compute with chemistry.”

“…with each other so that each point in visual space is sampled by each type of RGC. Dividing the responsibility for different types of visual information to different RGC types allows RGCs to more efficiently convey visual information to the brain, ultimately saving energy (11, 151, 179, 184).”

“In general, it is cheaper to transmit spikes at low average rates on parallel channels than at higher average rates using fewer neurons (Balasubramanian 2015; Laughlin et al. 1998; Niven 2016; Niven and Laughlin 2008; Sengupta et al. 2010; Sterling and Laughlin 2015).”

“For example, insect brains employ many computation-saving shortcuts for perception and decision making, and use a small number of “command” neurons to control complex behavioral patterns (Sterling & Laughlin, 2015).”

“However, neural tissue is costly to build, maintain, and power (Sterling & Laughlin, 2015), putting an upper bound on the number of systems likely to evolve (Milli et al., 2017).”

“[1] describes biologists’ tasks as reverse-engineering studies: asking both how an animal trait”

“In the equation above, spike probability then becomes the product of recorded firing rate r and time bin size 1 t , so that p = r 1 t (Rieke et al., 1999; Arcas et al., 2003; Sterling and Laughlin, 2015).”

“Like all signal processing “devices,” also neuronal networks suffer from noise (Sterling & Laughlin, 2016).”

“70 Thus, keeping firing rates low can be a way to maximize the information transmitted per spike.”

“The evolutionary history of animals is reflected in the structure of their brains, which process sensory inputs and mediate adaptive behaviours (Sterling & Laughlin, 2015).”

“Neuronal activity occurs on a background of synaptic and intrinsic excitability governed by the biophysical properties of the component cells, synaptic plasticity, and the control of these parameters by internal homeostasis and brainstem modulatory systems (Sterling and Laughlin, 2015).”

“This approach is advocated at length by neuroscientists Sterling and Laughlin (2015), but I refer to this text from 1961 to reinforce the point that these ideas in neuroscience and biology predate functionalism in the philosophy of mind: “The modern student finds it difficult to see the wood for the trees.”

“Sterling and Laughlin (2017) discuss the unicellular eukaryote Paramecium, which ingests other, smaller microorganisms, and uses cilia for locomotion.”

“ed energy is that the whole neuronal function can be susceptible to noise. Conventional neuroscience treats noise as an undesirable nuisance. For example, in their book “Principles of Neural Design”, Stirling and Laughlin (2017) comment: “Where noise is inevitable, it should be minimized before transmission, so most n…”

“Itiscommonly believed that the energy consumption for the information transmission between neurons in human brain is usually very light due to the selective and adaptive regulation mechanism [22].”

“Considering the costs of transmission with high rates (Sterling and Laughlin, 2015), these limitations in early sensory processing disfavor fast VNS dynamics in the brain.”

“Spikeling can be excited and its activity recorded 66 so as to design a variety of classical experiments similar to those that might be 67 carried out on a biological neuron and which students learn about in textbooks [12], 68 [13].”

“Natural vision operates under strict resource constraints by filtering out redundant background signals at the sensory front-end and prioritizing dynamic motion [13–15].”

“21 Maybe it reduces metabolic costs (cf. Sterling & Laughlin, 2015).”

“While nervous systems are as diverse as the behavioural strategies they create, key principles of CNS organization and integration remain constant across phylogeny [1].”

“Insect nervous systems are compact, suggesting that they have been selected to minimize space and energy requirements (Sterling and Laughlin, 2015).”

“sign principles that are based on those of biological nervous systems [4].”

“Consequently, the rate of meaningful information transmission is proportional to the system’s signal-to-noise ratio (Shannon, 1948; Borst and Theunissen, 1999; Sterling and Laughlin, 2015).”