Principles of Neural Design

“des use neurons to July 16, 2020 1/20 arXiv:2007.07346v1 [cs.NE] 14 Jul 2020 implement periodic motor patterns and to make simple decisions such as retreating in response to a toxic chemical stimulus [7]. This topic has been extensively studied in the robotic context, but most applications construct periodic motions f…”

“the proposed digital retina system. Besdies, the energy cost of the information transmission between neurons in human brain is usually very low due to the selective and adaptive regulation mechanism [6], this inspires us to propose a low transmission cost strategy that can deliver learning models incrementally and ada…”

““Updating” in this case means that the sensitivity of the deafferented cells is increased [17,18], which enables a lower level of neuronal activity to update the model and thus affect perception.”

“Beyond a certain point in spectral narrowing, it is clearly no longer energetically valuable to maintain such complex color vision (Sterling and Laughlin 2015).”

“A third example comes from a very recent book—“Principles of Neural Design” by Sterling and Laughlin (2015).”

“[1] The intrinsic characteristic is nonlinear dynamics: intracellular integration (analog computation) such that the summation of inputs from synaptically connected presynaptic neurons reaches the threshold of the cytosolic membrane potential, generating millisecond pulsed events (analog digital…”

“See also Figure S6. reflect stimulus duration rather than intrinsic properties.”

“…on the central control of autonomic nervous system function (7–9) all suggest the existence of a unified, distributed brain system that anticipates the metabolic needs of the body and prepares to meet those needs before they arise, a process called allostasis (10; for recent reviews, see 11, 12).”

“, 2022) peripheries are also adapted to the statistical structure of the environment so that they use limited resources efficiently to represent sensory information (Sterling and Laughlin, 2015).”

“It is widely believed that the tuning of sensory neurons is adapted to the statistical structure of the signals they must encode (Sterling and Laughlin 2015).”

“Our results suggest that the temporally regular multivesicular release strategy employed by rods provides an efficient means of encoding small voltage changes to meet both constraints on high release rates and requirements for reliable signaling at rod ribbon synapses (Sterling and Laughlin, 2015).”

“Slight differences would have little effect on our estimates, because ion transport is by far the largest utilizer of ATP in photoreceptors [85] as in neurons in the brain [7, 108].”

“e interneurons that innervate axon terminals of multiple photoreceptors. The photoreceptors, in turn, receive lateral feedback from the amacrine cells as well as feedback from LMCs such as L2 neurons [18, 19, 20]. A circuit diagram of the photoreceptor-amacrine cell layer is shown in Figure 1. For the sake of clarity…”

“Optimization, and its focus on system performance, can help to answer this sort of question (Biswas et al. 2020, Richards et al. 2019, Sterling & Laughlin 2015).”

“18 Branch diameters, another important geometric property of neurons, may optimize the distribution of materials for growth and homeostasis 19–21 and the transfer of electrical currents.”

“The neurally endowed nematode Caenorhabditis elegans (with 302 neurons) also engages in its version of tumbles, called turns or pirouettes (Pierce-Shimomura et al. 1999; Srivastava et al. 2009; Sterling and Laughlin 2015; Tanimoto and Kimura 2019).”

“The serious costs of neural transmission [43,44] argue against the unwanted broadcasting of information.”

“The brain is continuously monitoring the needs of the body and predictively regulating it (Sterling and Laughlin 2015).”

“Optimising neuronal systems for efficient processing and representation of information is a key principle for both understanding and designing neuronal circuits (Sterling & Laughlin, 2015), but recognising that a particular neuronal phenomena reflects an optimisation process is often difficult.”

“Work from the field of evolutionary neuroscience suggests a primary role of the brain is to maintain an efficient energetic state of the body via predictive regulation – a process termed allostasis (Schulkin and Ster-ling, 2019; Sterling, 2012; Sterling and Laughlin, 2015; Table 1).”

“It is energetically expensive to build and maintain muscles and neurons that operate with low delay (Sterling and Laughlin, 2015).”

“However, since these metabolically costly additions must pay-for-themselves in terms of fitness, this raises the question of their adaptive value (Sterling & Laughlin,2015) .”

“Nevertheless, real scenarios in neuronal systems employ laminar, column-like, and other designing principles for neural architectures [82].”

“…in the mammalian cortex (though perhaps not in the squid giant axon, where Hodgkin and Huxley first described them) these mechanisms might be optimized to consume minimal energy while still reliably maintaining the shape of the action potential (Alle et al., 2009; Sterling & Laughlin, 2015).”

“…role of these features in relation to language processing and integrate the implementational level with the algorithmic and functional levels of description. neurophysiology (Kandel et al., 2012; Luo, 2015; Sterling & Laughlin, 2015), and biophysics (Koch, 1998) that inform model construction.”

“Due to energy constraints, these components often face speedaccuracy tradeoffs (SATs); they are either fast and inaccurate or slow and accurate but not both [4].”

“The speed/accuracy tradeoff (4) for spiking neurons is given by R ∝ d in [4], [1] if we assume the space and energy for those neurons are fixed.”

“Research progress is increasingly elucidating these hardware speed, accuracy, and efficiency tradeoffs in familiar engineering terms [1].”

“…account (e.g., Clark, 2013; Friston et al., 2017; Hutchinson and Barrett, 2019; Hohwy, 2013), which posits that the brain is an active entity that is continuously making predictions about the future (e.g., Bastos et al., 2012; Friston, 2010; Rao and Ballard, 1999; Sterling and Laughlin, 2015).”

“At 20% contrast, for instance, many cycles failed to elicit a significant response, here considered to be two standard deviations above the baseline noise (Figure 7B).”

“Reducing the learning rate may reduce the metabolic cost of making the updates, since less synaptic plasticity is required and the brain has been heavily optimized by evolution to minimize energy expenditure (Sterling & Laughlin, 2015).”

“The fundamental role of the brain is to regulate the body which has the necessary effectors to enable the brain to support itself metabolically (see Sterling, 2004, 2012; Sterling & Eyer, 1988; Sterling & Laughlin, 2015).”

“…that is applied efficiently, according to behavioural requirements and constraints imposed by875 biophysics and the properties of natural signals (Sterling and Laughlin, 2015, Chapter 8).876 Conclusion and outlook877 Our study of the optimal allocation of photoreceptor length for opponent…”

“Energy efficiency is a fundamental organizational principle in brains that influences numerous aspects of signal generation, processing and transmission [1, 2, 3, 4, 5].”

“15), such as the visual system famously studied by Hubel and Wiesel (1962), are best understood by neuroscientists; but everything currently known about the way in which neurons encode and process information (Quian Quiroga et al., 2005; Sterling and Laughlin, 2015), as well as how even comparatively simple nervous sy…”

“…empirical values we extract from published data lie within or close to the boundaries defined by our theoretical curves (Figure 5 & 6), and the broad high-efficiency zones in perfor-723 mance surfaces allow values to vary considerably, while retaining 95% of maximum information capacity (Figure 3).”

“In this way, anatomically relevant microcircuits, such as the cortical-hippocampal projection, can be established, thereby providing a robust methodology for studying neural network function and dysfunction in a controlled setting (11, 18).”

“We also achieve the desired neural operation through inspection without a training algorithm, in the spirit of biological vision systems that “hard-code” basic computational primitives, such as Gabor filters matched to common spatial frequencies in natural scenes [34].”

“Affective realism may seem maladaptive, but it likely evolved in service of maintaining the biological needs of the body (i.e., allostasis; Sterling, 2004, 2012; Sterling & Laughlin, 2015).”

“Another, and I think more promising, idea is that such symbols are actually realized at the sub-cellular level using various biochemical messengers and the states of such messengers (Gallistel and King 2011; Sterling and Laughlin 2015).”

“For sufficiently large myelinated axons, the propagation speed 1/Ti and firing rate φ are both approximately proportional to the axon radius ρ [6], i.”

“For sufficiently large myelinated axons, we assume that the propagation speed 1/Ts is proportional to the axon radius ρ (6), i.”

“Organisms need energy to drive essential organs including the brain [12, 13], heart [14, 15] and muscles [16].”

“20 One approach to assessing the information efficiency of neurons has been to calculate the number of bits of information carried by spikes, 21,22 but, in actuality, excitatory synaptic transmission has a greater cost.”

“The amount of current or information conveyed by a link depends on wires’ physical characteristics, such as their diameter and length but also their mechanical and conduction properties (Sterling and Laughlin, 2015).”

“From a metabolic standpoint, the brain is one of the most expensive systems within an organism: the human brain is 2 % of the body weight but 20 % of its metabolic load [131, 132].”

“In their great book on the principles of neural design Sterlin and Laughlin work out 528 three main principles that define the way nervous systems should be designed [59]: (i) 529 Only the absolutely required information should be sent down axons, (ii) sending 530 information should employ as few action potentials as…”

“…that brain networks evolved to solve the trade-off between wiring cost and processing speed can be traced back to Ramón y Cajal ’ s time and space conservation laws (Bullmore & Sporns, 2012; Chklovskii & Koulakov, 2004; Kaiser & Hilgetag, 2006; Sporns, 2016; Sterling & Laughlin, 2015, Chapter 13).”

“Why change? The answer must involve the intense competition for resources—including time, space, and energy— under which brains operate (Sterling and Laughlin, 2015).”

“Energy efficiency stands as a fundamental constraint in neural computation and a cornerstone principle in computational neuroscience (Lennie, 2003; Sterling & Laughlin, 2015).”

“Thisisimportantbecausetheretina likely employs a noise-robust coding strategy, given the ubiquity of noise in the retina and the visual environment 74 .”

“This adaptation is no disorder or dysregulation but a standard feature of all neural circuits; in fact, it is a key principle of neural design (Sterling and Laughlin 2015).”

“This could be exemplified by what Sterling and Laughlin (2015) call “prin-ciples of neural design”.”

“Such a perspective introduces a more socially-oriented interpretation of the original definition of “allostasis,” according to which the brain is constantly engaged in regulating the organism’s internal milieu by anticipating needs and preparing to fulfill them before they arise (Sterling, 2012; Sterling and Laughlin,…”

“A conventional “vanilla” VAE model will capture the properties of encoded sensory features but would be devoid of any of the spatial properties of cortical neurons that are critical for determining how sensory stimuli are represented across the cortical surface.”

“…transmission of signals between brain cells, and thus affords an opportunity for new information to be derived with reference to the underlying geometry, in turn enabling specific computations which facilitate survival (Wilson and Bednar, 2015; Bednar and Wilson, 2016; Sterling and Laughlin, 2015).”

“Because processing more (complex) information is expensive, both in terms of metabolism and the required neural substrate [2, 44], the ability to extract the relevant information also results in decreased costs to the animal.”

“One possible answer to this question is that the sustained energy that is required to keep a memory alive (Fuster & Alexander, 1971) induces a metabolic cost (Attwell & Laughlin, 2001; Laughlin, 2001; Sterling & Laughlin, 2015).”

“2) Layered architecture in sensorimotor control: The human sensorimotor control system is extremely robust, although the sensing is distributed, sparse, quantized, noisy and delayed; the computing in the central nervous system is slow [5]; and the muscle actuation fatigues and saturates [16].”

“Critically, action and perception have to proceed in the face of substantial informational deficiency (Brooks, 1990; Sterling and Laughlin, 2015).”

“The connectome comparison in this study was initially motivated by activation efficiency, which relates to efficient coding and metabolic-cost perspectives in neuroscience [1, 3, 14, 16, 25, 27, 36, 42, 44, 46].”

“Evolvability and functional adaptation to external signals [5] both underpin the resilience of life and individuality [6, 7], and can be considered together within the broad context of tunable physical systems.”

“This splitting has been understood as an efficiency: splitting the stimulus reduces the range of each neuron, saving energy (Sterling & Laughlin, 2015; Gjorgjieva et al., 2014).”

“Having a brain like ours, then, is a significant metabolic investment, which requires some sort of justification (see Sterling & Laughlin 2015).”

“Extra neuronal machinery increases energy usage at the cost of diminishing returns and is not advantageous (99).”

“Both the spot of daylight and the light from a faint star in our example above are compressed down to a single spike [2, 3].”

“The visual system optimizes contrast sensitivity by dedicating the limited dynamic range of ON and OFF RGCs to signaling increments and decrements, respectively (Gjorgjieva et al., 2014; Sterling and Laughlin, 2017).”

“Being related to internal states (Sterling & Laughlin, 2015), needing generates the value attributed to rewards “ from within ” and is thus separate from the external prediction of reward.”

“Since increasing turning frequency without monitoring the path trajectory is less energy-intensive, its initiation and duration might be less temporally restricted than path integration-based search (Bell 1990; Sterling and Laughlin 2015).”

“…Pennsylvania) and Simon Laughlin (University of Cambridge) suggested that storing memory and performing computations using molecular chemistry inside the neuron would be energetically cheaper in comparison to using neural spikes and synapses (Hebb’s synaptic hypothesis) (Sterling & Laughlin, 2015).”

“The reason that brains are likely to adopt this arrangement — we suggest — is the raw cost of broadcasting unnecessary action potentials [14 ,15].”

“Modern neuroscientists know that the wiring cost of connections in the brain has multiple origins: it arises from volume, metabolic requirements, signal delay and attenuation, and guidance defects in development, among other factors [29,36].”