A Tutorial for Information Theory in Neuroscience
Open accessThe DOI matches its Crossref record; 100 of the 100 references it cites were found in a registry or an index.
Abstract
Understanding how neural systems integrate, encode, and compute information is central to understanding brain function. Frequently, data from neuroscience experiments are multivariate, the interactions between the variables are nonlinear, and the landscape of hypothesized or possible interactions between variables is extremely broad. Information theory is well suited to address these types of data, as it possesses multivariate analysis tools, it can be applied to many different types of data, it can capture nonlinear interactions, and it does not require assumptions about the structure of the underlying data (i.e., it is model independent). In this article, we walk through the mathematics of information theory along with common logistical problems associated with data type, data binning, data quantity requirements, bias, and significance testing. Next, we analyze models inspired by canonical neuroscience experiments to improve understanding and demonstrate the strengths of information theory analyses. To facilitate the use of information theory analyses, and an understanding of how these analyses are implemented, we also provide a free MATLAB software package that can be applied to a wide range of data from neuroscience experiments, as well as from other fields of study.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · cited 1×
“First, it is vital to consider the number of significance tests performed and to control for multiple comparisons (e.g., via Bonferroni correction or false discovery rate control; Benjamini and Hochberg, 1995; Benjamini and Yekutieli, 2001).”
BackgroundA Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Investigating causal relations by econometric models and cross-spectral methods
Granger, C. W. J. (1969) · Econometrica · doi:10.2307/1912791 · Cited by 22,989
Complex brain networks: graph theoretical analysis of structural and functional systems
Bullmore, E., & Sporns, O. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2575 · Cited by 11,820 · cited 3×
“Due to the widespread interest in neural connectivity (Bullmore and Sporns, 2009; Friston, 2011), transfer entropy has been widely used in the literature (for example, Honey et al.”
BackgroundMethodThe control of the false discovery rate in multiple testing under dependency
Benjamini, Y., & Yekutieli, D. (2001) · The Annals of Statistics · doi:10.1214/aos/1013699998 · Cited by 10,490 · cited 1×
“First, it is vital to consider the number of significance tests performed and to control for multiple comparisons (e.g., via Bonferroni correction or false discovery rate control; Benjamini and Hochberg, 1995; Benjamini and Yekutieli, 2001).”
BackgroundThe hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat
O'Keefe, J., & Dostrovsky, J. (1971) · Brain Research · doi:10.1016/0006-8993(71)90358-1 · Cited by 6,151 · cited 3×
“Place cells For our last canonical experiment, we examined a simulation of hippocampal place cells (O’Keefe and Dostrovsky, 1971; Bear et al., 2007).”
MethodSimple model of spiking neurons
Izhikevich, E. M. (2003) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2003.820440 · Cited by 4,828 · cited 6×
“To move beyond simple circuits of a few neurons, we examined a large 1000-neuron network model (Izhikevich, 2003; Fig.”
BackgroundMethodInfluentialFriston, K. J., Harrison, L., & Penny, W. (2003) · NeuroImage · doi:10.1016/s1053-8119(03)00202-7 · Cited by 4,583
Measuring Information Transfer
Schreiber, T. (2000) · Physical Review Letters · doi:10.1103/physrevlett.85.461 · Cited by 4,530
Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting
Izhikevich, E. M. (2006) · The MIT Press eBooks · doi:10.7551/mitpress/2526.001.0001 · Cited by 3,765
Kraskov, A., Stögbauer, H., & Grassberger, P. (2004) · Physical Review E · doi:10.1103/physreve.69.066138 · Cited by 3,512 · cited 1×
“For instance, kernel-based or binless strategies exist for several information theory measures (Victor, 2002; Kraskov et al., 2004; Vicente et al., 2011).”
BackgroundFunctional and Effective Connectivity: A Review
Friston, K. J. (2011) · Brain Connectivity · doi:10.1089/brain.2011.0008 · Cited by 3,415 · cited 3×
“Due to the widespread interest in neural connectivity (Bullmore and Sporns, 2009; Friston, 2011), transfer entropy has been widely used in the literature (for example, Honey et al.”
BackgroundMethodDISCHARGE PATTERNS AND FUNCTIONAL ORGANIZATION OF MAMMALIAN RETINA
Kuffler, S. W. (1953) · Journal of Neurophysiology · doi:10.1152/jn.1953.16.1.37 · Cited by 2,609 · cited 4×
“Center-surround retinal ganglion cells The second canonical neuroscience experiment we simulated was the center-surround receptive fields of retinal ganglion cells (Kuffler, 1953; Bear et al., 2007).”
MethodInfluentialGeorgopoulos, A., Kalaska, J., Caminiti, R., & Massey, J. (1982) · The Journal of Neuroscience · doi:10.1523/jneurosci.02-11-01527.1982 · Cited by 2,130 · cited 4×
“Movement direction and motor cortex neurons The third canonical neuroscience system we examined was movement direction encoding by motor cortex neurons (Georgopoulos et al., 1982; Bear et al., 2007).”
BackgroundMethodInfluentialTutorial on maximum likelihood estimation
Myung, I. J. (2003) · Journal of Mathematical Psychology · doi:10.1016/s0022-2496(02)00028-7 · Cited by 1,802 · cited 1×
“It should be noted that this is a form of maximum likelihood estimation (Myung, 2003), in that we assume that the data observed represent the most likely outcome of the underlying probability distribution.”
BackgroundNetwork structure of cerebral cortex shapes functional connectivity on multiple time scales
Honey, C. J., Kötter, R., Breakspear, M., & Sporns, O. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0701519104 · Cited by 1,711 · cited 3×
“Two particular areas of interest include studies of connectivity (Honey et al., 2007; Ito et al., 2011; Timme et al., 2014b, 2016; Nigam et al., 2016; Wollstadt et al., 2017) and sensory encoding (Bialek et al., 1991; DeWeese and Meister, 1999; Brenner et al., 2000; Panzeri et al., 2001; Butts,…”
BackgroundMethodEstimation of entropy and mutual information
Paninski, L. (2003) · Neural Computation · doi:10.1162/089976603321780272 · Cited by 1,442
Dimensionality reduction for large-scale neural recordings
Cunningham, J. P., & Yu, B. M. (2014) · Nature Neuroscience · doi:10.1038/nn.3776 · Cited by 1,219 · cited 1×
“Furthermore, the original data can be first processed via a wavelet transform (Hramov et al., 2015) or a dimensionality reduction technique (Cunningham and Yu, 2014), for instance, and then fed into an information theory analysis.”
MethodInformation theory and neural coding
Borst, A., & Theunissen, F. E. (1999) · Nature Neuroscience · doi:10.1038/14731 · Cited by 1,113 · cited 2×
“Because of its general applicability, information theory has been widely used in neuroscience (Rieke et al., 1997; Borst and Theunissen, 1999; Victor, 2006; Quiroga and Panzeri, 2009; Dimitrov et al., 2011; Timme et al., 2014a; Wibral et al., 2014a, b).”
BackgroundMethodTransfer entropy—a model-free measure of effective connectivity for the neurosciences
Vicente, R., Wibral, M., Lindner, M., & Pipa, G. (2011) · Journal of Computational Neuroscience · doi:10.1007/s10827-010-0262-3 · Cited by 1,014 · cited 4×
“Due to the widespread interest in neural connectivity (Bullmore and Sporns, 2009; Friston, 2011), transfer entropy has been widely used in the literature (for example, Honey et al., 2007; Lizier et al., 2008; Ito et al., 2011; Vicente et al., 2011; Timme et al., 2014b, 2016; Wibral et al., 2014b; Nigam et al., 2016; B…”
BackgroundMethodInfluentialBialek, W., Rieke, F., de Ruyter van Steveninck, R. R., & Warland, D. (1991) · Science · doi:10.1126/science.2063199 · Cited by 992 · cited 3×
“Often, these sorts of analyses are valuable because they can quantify phenomena like encoding (e.g., how much information a neuron provides; Bialek et al., 1991; Brenner et al., 2000) and complex encoding relationships (e.”
Structure and function of complex brain networks
Sporns, O. (2013) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2013.15.3/osporns · Cited by 932 · cited 2×
“The crucial difference between these two types of connectivity is of vital importance and has been widely discussed in the literature (Sporns, 2007, 2013; Bullmore and Sporns, 2009; Friston, 2011; Schölvinck et al., 2013).”
BackgroundExtracting information from neuronal populations: information theory and decoding approaches
Quian Quiroga, R., & Panzeri, S. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2578 · Cited by 811 · cited 1×
“Because of its general applicability, information theory has been widely used in neuroscience (Rieke et al., 1997; Borst and Theunissen, 1999; Victor, 2006; Quiroga and Panzeri, 2009; Dimitrov et al., 2011; Timme et al., 2014a; Wibral et al., 2014a, b).”
MethodGABAergic Hub Neurons Orchestrate Synchrony in Developing Hippocampal Networks
Bonifazi, P., Goldin, M., Picardo, M. A., Jorquera, I., Cattani, A., Bianconi, G., Represa, A., Ben-Ari, Y., & Cossart, R. (2009) · Science · doi:10.1126/science.1175509 · Cited by 695 · cited 2×
“Cross-correlation is widely used in the analysis of neural spiking data (Bonifazi et al., 2009; Ito et al., 2014) to assess functional connections between pairs of neurons (i.e., neuron A tends to spike just before neuron B spikes).”
BackgroundGranger Causality: Basic Theory and Application to Neuroscience
Ding, M., Chen, Y., & Bressler, S. L. (2006) · Handbook of Time Series Analysis · doi:10.1002/9783527609970.ch17 · Cited by 653 · cited 2×
“Granger causality has been widely used to assess causal interactions between both continuous and discrete neural signals (Granger, 1969; Ding et al., 2006; Nakhnikian et al., 2014), though underlying assumptions about the analysis (e.”
MethodNature and precision of temporal coding in visual cortex: a metric-space analysis
Victor, J. D., & Purpura, K. P. (1996) · Journal of Neurophysiology · doi:10.1152/jn.1996.76.2.1310 · Cited by 611 · cited 1×
“For instance, with neurons the distinction between rate coding and spike timing coding has frequently been discussed (Victor and Purpura, 1996; Borst and Theunissen, 1999; Panzeri and Schultz, 2001; Van Rullen and Thorpe, 2001; Stanley, 2013).”
BackgroundStaniek, M., & Lehnertz, K. (2008) · Physical Review Letters · doi:10.1103/physrevlett.100.158101 · Cited by 595 · cited 1×
“Numerous methods have been employed to define past and future state (Staniek and Lehnertz, 2008; Ito et al., 2011; Wibral et al., 2013; Timme et al., 2014b).”
MethodA review of symbolic analysis of experimental data
Daw, C. S., Finney, C. E. A., & Tracy, E. R. (2003) · Review of Scientific Instruments · doi:10.1063/1.1531823 · Cited by 591 · cited 2×
“In other words, using three bins instead of four will produce different results, especially in terms of information values (see Daw et al. (2003) for a review of discretization methods and their impacts on data analyses from a more general perspective).”
BackgroundMethodNeuronal Mechanisms of Habituation and Dishabituation of the Gill-Withdrawal Reflex in Aplysia
Castellucci, V., Pinsker, H., Kupfermann, I., & Kandel, E. R. (1970) · Science · doi:10.1126/science.167.3926.1745 · Cited by 501 · cited 5×
“First, we examined a simple model of sensory habituation in Aplysia (Castellucci et al., 1970; Bear et al., 2007).”
BackgroundMethodInfluentialSynergy, Redundancy, and Independence in Population Codes
Schneidman, E., Bialek, W., & Berry, M. J. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-37-11539.2003 · Cited by 497 · cited 1×
“In fact, this area of information theory is currently undergoing rapid development, with numerous previous methods for analyzing synergy and redundancy in neuro-science contexts (Brenner et al., 2000; Schneidman et al., 2003a,b) being improved on.”
MethodRate Coding Versus Temporal Order Coding: What the Retinal Ganglion Cells Tell the Visual Cortex
Rullen, R. V., & Thorpe, S. J. (2001) · Neural Computation · doi:10.1162/08997660152002852 · Cited by 487 · cited 1×
“For instance, with neurons the distinction between rate coding and spike timing coding has frequently been discussed (Victor and Purpura, 1996; Borst and Theunissen, 1999; Panzeri and Schultz, 2001; Van Rullen and Thorpe, 2001; Stanley, 2013).”
BackgroundThe Role of Spike Timing in the Coding of Stimulus Location in Rat Somatosensory Cortex
Panzeri, S., Petersen, R. S., Schultz, S. R., Lebedev, M., & Diamond, M. E. (2001) · Neuron · doi:10.1016/s0896-6273(01)00251-3 · Cited by 436
Temporal precision in the neural code and the timescales of natural vision
Butts, D. A., Weng, C., Jin, J., Yeh, C.-I., Lesica, N. A., Alonso, J.-M., & Stanley, G. B. (2007) · Nature · doi:10.1038/nature06105 · Cited by 415 · cited 1×
“…interest include studies of connectivity (Honey et al., 2007; Ito et al., 2011; Timme et al., 2014b, 2016; Nigam et al., 2016; Wollstadt et al., 2017) and sensory encoding (Bialek et al., 1991; DeWeese and Meister, 1999; Brenner et al., 2000; Panzeri et al., 2001; Butts, 2003; Butts et al., 2007).”
BackgroundMutual information analysis of the EEG in patients with Alzheimer's disease
Jeong, J., Gore, J. C., & Peterson, B. S. (2001) · Clinical Neurophysiology · doi:10.1016/s1388-2457(01)00513-2 · Cited by 397 · cited 1×
“For instance, research has focused on analyses of electroencephalography (EEG), magnetoencephalography (MEG), and functional MRI (fMRI) data (Jeong et al., 2001; Lizier et al., 2011; Vicente et al., 2011).”
BackgroundJIDT: An Information-Theoretic Toolkit for Studying the Dynamics of Complex Systems
Lizier, J. T. (2014) · Frontiers in Robotics and AI · doi:10.3389/frobt.2014.00011 · Cited by 390 · cited 8×
“…fills a valuable role in the field, we wish to emphasize that other excellent information theory software packages exist and that these packages may be more useful depending on the desired application (Ince et al., 2010; Quiroga and Panzeri, 2013; Lizier, 2014; Szabo, 2014; Moore et al., 2017).”
BackgroundMethodInfluentialCorrecting for the sampling bias problem in spike train information measures
Panzeri, S., Senatore, R., Montemurro, M. A., & Petersen, R. S. (2007) · Journal of Neurophysiology · doi:10.1152/jn.00559.2007 · Cited by 378
Neuronal Correlates of Habituation and Dishabituation of the Gill-Withdrawal Reflex in Aplysia
Kupfermann, I., Castellucci, V., Pinsker, H., & Kandel, E. (1970) · Science · doi:10.1126/science.167.3926.1743 · Cited by 366
Brenner, N., Strong, S. P., Koberle, R., Bialek, W., & Steveninck, R. R. D. R. V. (2000) · Neural Computation · doi:10.1162/089976600300015259 · Cited by 346 · cited 4×
“Often, these sorts of analyses are valuable because they can quantify phenomena like encoding (e.g., how much information a neuron provides; Bialek et al., 1991; Brenner et al., 2000) and complex encoding relationships (e.”
BackgroundMethodInfluentialNetwork Information and Connected Correlations
Schneidman, E., Still, S., Berry, M. J., & Bialek, W. (2003) · Physical Review Letters · doi:10.1103/physrevlett.91.238701 · Cited by 317
An Introduction to Transfer Entropy
Bossomaier, T. R. J., Barnett, L., Harré, M., & Lizier, J. T. (2016) · Springer eBooks · doi:10.1007/978-3-319-43222-9 · Cited by 307 · cited 1×
“…(Bullmore and Sporns, 2009; Friston, 2011), transfer entropy has been widely used in the literature (for example, Honey et al., 2007; Lizier et al., 2008; Ito et al., 2011; Vicente et al., 2011; Timme et al., 2014b, 2016; Wibral et al., 2014b; Nigam et al., 2016; Bossomaier et al., 2016).”
MethodMeasuring Information-Transfer Delays
Wibral, M., Pampu, N., Priesemann, V., Siebenhühner, F., Seiwert, H., Lindner, M., Lizier, J. T., & Vicente, R. (2013) · PLoS ONE · doi:10.1371/journal.pone.0055809 · Cited by 286 · cited 4×
“Research has also focused on trial-based data (Wollstadt et al., 2014; Gomez-Herraro et al., 2015; Asaad et al., 2017) and single-trial time-averaged analyses (Wibral et al., 2013; Timme et al., 2014b, 2016; Nigam et al., 2016).”
BackgroundMethodInfluentialLocal information transfer as a spatiotemporal filter for complex systems
Lizier, J. T., Prokopenko, M., & Zomaya, A. Y. (2008) · Physical Review E · doi:10.1103/physreve.77.026110 · Cited by 277 · cited 3×
“Due to the widespread interest in neural connectivity (Bullmore and Sporns, 2009; Friston, 2011), transfer entropy has been widely used in the literature (for example, Honey et al., 2007; Lizier et al., 2008; Ito et al., 2011; Vicente et al., 2011; Timme et al., 2014b, 2016; Wibral et al., 2014b; Nigam et al., 2016; B…”
MethodQuinn, C. J., Coleman, T. P., Kiyavash, N., & Hatsopoulos, N. G. (2011) · Journal of Computational Neuroscience · doi:10.1007/s10827-010-0247-2 · Cited by 273 · cited 1×
“An interesting alternative measure called the directed information has also been used in the literature (Quinn et al., 2011).”
BackgroundQuantifying unique information
Bertschinger, N., Rauh, J., Olbrich, E., Jost, J., & Ay, N. (2014) · Entropy · doi:10.3390/e16042161 · Cited by 253
Binless strategies for estimation of information from neural data
Victor, J. D. (2002) · Physical Review E · doi:10.1103/physreve.66.051903 · Cited by 238 · cited 1×
“For instance, kernel-based or binless strategies exist for several information theory measures (Victor, 2002; Kraskov et al., 2004; Vicente et al., 2011).”
BackgroundIto, S., Hansen, M. E., Heiland, R., Lumsdaine, A., Litke, A. M., & Beggs, J. M. (2011) · PLoS ONE · doi:10.1371/journal.pone.0027431 · Cited by 236 · cited 12×
“Therefore, it is necessary to conduct measurements at appropriate delays and/or to conduct some type of delay scanning procedure (Ito et al., 2011; Wibral et al., 2013).”
BackgroundMethodResultInfluentialLindner, M., Vicente, R., Priesemann, V., & Wibral, M. (2011) · BMC Neuroscience · doi:10.1186/1471-2202-12-119 · Cited by 234 · cited 11×
“Several more advanced methods have been developed, some of which are implemented in other software packages, to overcome these problems in certain cases (Nemenman et al., 2002; Goldberg et al., 2009; Ince et al., 2009; Magri et al., 2009; Ito et al., 2011; Lindner et al., 2011; Pastore et al., 2016).”
BackgroundMethodInfluentialA toolbox for the fast information analysis of multiple-site LFP, EEG and spike train recordings
Magri, C., Whittingstall, K., Singh, V., Logothetis, N. K., & Panzeri, S. (2009) · BMC Neuroscience · doi:10.1186/1471-2202-10-81 · Cited by 226 · cited 5×
“Second, several software packages could not be made to run on our machines or required programming languages with which we are not familiar (Goldberg et al., 2009; Ince et al., 2009; Magri et al., 2009; Pastore et al., 2016; Moore et al., 2017; dit-contributors, 2018).”
MethodInfluentialSynergy, redundancy, and multivariate information measures: an experimentalist’s perspective
Timme, N., Alford, W., Flecker, B., & Beggs, J. M. (2014) · Journal of Computational Neuroscience · doi:10.1007/s10827-013-0458-4 · Cited by 226 · cited 8×
“The randomization procedures necessary to generate surrogate null model data can be performed before creating probability distributions (e.g., spike jittering (Rolston et al., 2007; Timme et al., 2014b)) or after creating probability distributions.”
BackgroundMethodResultInfluentialSelf-citationDirected Information Measures in Neuroscience
(2014) · Understanding Complex Systems · doi:10.1007/978-3-642-54474-3 · Cited by 218 · cited 6×
“Such an analysis will produce an instantaneous information value at a given time point in the trial (Lizier et al., 2008; Wibral et al., 2014a for other interpretations of instantaneous information values).”
BackgroundMethodInfluentialHramov, A. E., Koronovskii, A. A., Makarov, V. A., Pavlov, A. N., & Sitnikova, E. (2015) · Springer Series in Synergetics · doi:10.1007/978-3-662-43850-3 · Cited by 213
Entropy and Inference, Revisited
Nemenman, I., Shafee, F., & Bialek, W. (2002) · Advances in Neural Information Processing Systems 14 · doi:10.7551/mitpress/1120.003.0065 · Cited by 211 · cited 2×
“…more advanced methods have been developed, some of which are implemented in other software packages, to overcome these problems in certain cases (Nemenman et al., 2002; Goldberg et al., 2009; Ince et al., 2009; Magri et al., 2009; Ito et al., 2011; Lindner et al., 2011; Pastore et al., 2016).”
MethodLizier, J. T., Heinzle, J., Horstmann, A., Haynes, J.-D., & Prokopenko, M. (2011) · Journal of Computational Neuroscience · doi:10.1007/s10827-010-0271-2 · Cited by 210 · cited 1×
“For instance, research has focused on analyses of electroencephalography (EEG), magnetoencephalography (MEG), and functional MRI (fMRI) data (Jeong et al., 2001; Lizier et al., 2011; Vicente et al., 2011).”
BackgroundBivariate measure of redundant information
Harder, M., Salge, C., & Polani, D. (2013) · Physical Review E · doi:10.1103/physreve.87.012130 · Cited by 209 · cited 5×
“While the minimum information possesses distinct advantages over other methods, it has been shown that it measures the redundant amount of information as opposed to the redundant content, which several authors find counterintuitive (Harder et al., 2013; Bertschinger et al., 2014; Griffith et al., 2014; Ince, 2017).”
BackgroundMethodInfluentialEntropy and information in neural spike trains: Progress on the sampling problem
Nemenman, I., Bialek, W., & de Ruyter van Steveninck, R. (2004) · Physical Review E · doi:10.1103/physreve.69.056111 · Cited by 196
Montalto, A., Faes, L., & Marinazzo, D. (2014) · PLoS ONE · doi:10.1371/journal.pone.0109462 · Cited by 195 · cited 6×
“First, several packages require the use of more advanced probability estimation techniques that are not available in the Neuroscience Information Theory Toolbox (Lindner et al., 2011; Montalto et al., 2014; Szabo, 2014).”
BackgroundMethodInfluentialEstimating functions of probability distributions from a finite set of samples
Wolpert, D. H., & Wolf, D. R. (1995) · Physical Review E · doi:10.1103/physreve.52.6841 · Cited by 191 · cited 2×
“Bias in entropy and mutual information Several researchers have previously discussed biases associated with different methods for estimating information values from continuous data and/or small data sets (Treves and Panzeri, 1995; Wolpert and Wolf, 1995; Panzeri and Treves, 1996; Paninski, 2003; Nemenman et al., 2004;…”
MethodRich-Club Organization in Effective Connectivity among Cortical Neurons
Nigam, S., Shimono, M., Ito, S., Yeh, F.-C., Timme, N., Myroshnychenko, M., Lapish, C. C., Tosi, Z., Hottowy, P., Smith, W. C., et al. (2016) · The Journal of Neuroscience · doi:10.1523/jneurosci.2177-15.2016 · Cited by 179 · cited 3×
“Research has also focused on trial-based data (Wollstadt et al., 2014; Gomez-Herraro et al., 2015; Asaad et al., 2017) and single-trial time-averaged analyses (Wibral et al., 2013; Timme et al., 2014b, 2016; Nigam et al., 2016).”
BackgroundMethodSelf-citationPrecisely timed spatiotemporal patterns of neural activity in dissociated cortical cultures
Rolston, J. D., Wagenaar, D. A., & Potter, S. M. (2007) · Neuroscience · doi:10.1016/j.neuroscience.2007.05.025 · Cited by 170
How to measure the information gained from one symbol
Deweese, M., & Meister, M. (1999) · Network: Computation in Neural Systems · doi:10.1088/0954-898x/10/4/303 · Cited by 167
(2013) · CRC Press eBooks · doi:10.1201/b14756 · Cited by 163
Information Processing and Dynamics in Minimally Cognitive Agents
Beer, R. D., & Williams, P. L. (2015) · Cognitive Science · doi:10.1111/cogs.12142 · Cited by 149 · cited 2×
“The partial information decomposition has also been adapted to transfer entropy measures between three variables (Williams and Beer, 2011), as well as measures of information transfer and information gain (Beer and Williams, 2014).”
BackgroundMethodInformation theory applications for biological sequence analysis
Vinga, S. (2013) · Briefings in Bioinformatics · doi:10.1093/bib/bbt068 · Cited by 147 · cited 1×
“Information theory is currently used in genetics (Vinga, 2013; Ignac et al., 2014; Smouse et al., 2015), but studies could be performed linking genetics and neuroscience.”
MethodMeasuring Multivariate Redundant Information with Pointwise Common Change in Surprisal
Ince, R. (2017) · Entropy · doi:10.3390/e19070318 · Cited by 146 · cited 6×
“While the minimum information possesses distinct advantages over other methods, it has been shown that it measures the redundant amount of information as opposed to the redundant content, which several authors find counterintuitive (Harder et al., 2013; Bertschinger et al., 2014; Griffith et al., 2014; Ince, 2017).”
BackgroundMethodInfluentialA Unified Approach to the Study of Temporal, Correlational, and Rate Coding
Panzeri, S., & Schultz, S. R. (2001) · Neural Computation · doi:10.1162/08997660152002870 · Cited by 141 · cited 2×
“For instance, with neurons the distinction between rate coding and spike timing coding has frequently been discussed (Victor and Purpura, 1996; Borst and Theunissen, 1999; Panzeri and Schultz, 2001; Van Rullen and Thorpe, 2001; Stanley, 2013).”
BackgroundAnalytical estimates of limited sampling biases in different information measures
Panzeri, S., & Treves, A. (1996) · Network: Computation in Neural Systems · doi:10.1080/0954898x.1996.11978656 · Cited by 138
Handbook of time series analysis : recent theoretical developments and applications
Schelter, B. O., Matthias Winterhalder, A., & Timmer, J. (2006) · Cited by 135
Efficient Transfer Entropy Analysis of Non-Stationary Neural Time Series
Wollstadt, P., Martínez-Zarzuela, M., Vicente, R., Díaz-Pernas, F. J., & Wibral, M. (2014) · PLoS ONE · doi:10.1371/journal.pone.0102833 · Cited by 133 · cited 2×
“Research has also focused on trial-based data (Wollstadt et al., 2014; Gomez-Herraro et al., 2015; Asaad et al., 2017) and single-trial time-averaged analyses (Wibral et al., 2013; Timme et al., 2014b, 2016; Nigam et al., 2016).”
MethodApproaches to Information-Theoretic Analysis of Neural Activity
Victor, J. D. (2006) · Biological Theory · doi:10.1162/biot.2006.1.3.302 · Cited by 132 · cited 1×
“Because of its general applicability, information theory has been widely used in neuroscience (Rieke et al., 1997; Borst and Theunissen, 1999; Victor, 2006; Quiroga and Panzeri, 2009; Dimitrov et al., 2011; Timme et al., 2014a; Wibral et al., 2014a, b).”
MethodSporns, O. (2007) · Scholarpedia · doi:10.4249/scholarpedia.4695 · Cited by 129
Bits from Brains for Biologically Inspired Computing
Wibral, M., Lizier, J. T., & Priesemann, V. (2015) · Frontiers in Robotics and AI · doi:10.3389/frobt.2015.00005 · Cited by 128 · cited 6×
“Such an analysis will produce an instantaneous information value at a given time point in the trial (Lizier et al., 2008; Wibral et al., 2014a for other interpretations of instantaneous information values).”
BackgroundMethodInfluentialReading and writing the neural code
Stanley, G. B. (2013) · Nature Neuroscience · doi:10.1038/nn.3330 · Cited by 126 · cited 1×
“For instance, with neurons the distinction between rate coding and spike timing coding has frequently been discussed (Victor and Purpura, 1996; Borst and Theunissen, 1999; Panzeri and Schultz, 2001; Van Rullen and Thorpe, 2001; Stanley, 2013).”
BackgroundInformation theory in neuroscience
Dimitrov, A. G., Lazar, A. A., & Victor, J. D. (2011) · Journal of Computational Neuroscience · doi:10.1007/s10827-011-0314-3 · Cited by 118 · cited 1×
“Because of its general applicability, information theory has been widely used in neuroscience (Rieke et al., 1997; Borst and Theunissen, 1999; Victor, 2006; Quiroga and Panzeri, 2009; Dimitrov et al., 2011; Timme et al., 2014a; Wibral et al., 2014a, b).”
MethodEntropy estimates of small data sets
Bonachela, J. A., Haye Hinrichsen, B., & Muñoz, M. A. (2008) · Journal of Physics A Mathematical and Theoretical · doi:10.1088/1751-8113/41/20/202001 · Cited by 106 · cited 2×
“Bias in entropy and mutual information Several researchers have previously discussed biases associated with different methods for estimating information values from continuous data and/or small data sets (Treves and Panzeri, 1995; Wolpert and Wolf, 1995; Panzeri and Treves, 1996; Paninski, 2003; Nemenman et al., 2004;…”
BackgroundMethodInformation Theoretical Estimators ToolboxPreprint
Szabó, Z. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.2106 · Cited by 99 · cited 9×
“…fills a valuable role in the field, we wish to emphasize that other excellent information theory software packages exist and that these packages may be more useful depending on the desired application (Ince et al., 2010; Quiroga and Panzeri, 2013; Lizier, 2014; Szabo, 2014; Moore et al., 2017).”
BackgroundMethodInfluentialThe contribution of electrophysiology to functional connectivity mapping
Schölvinck, M. L., Leopold, D. A., Brookes, M. J., & Khader, P. H. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.04.010 · Cited by 95
Assessing Coupling Dynamics from an Ensemble of Time Series
Gómez-Herrero, G., Wu, W. L., Rutanen, K., Soriano, M. C., Pipa, G., & Raúl, V. (2015) · Entropy · doi:10.3390/e17041958 · Cited by 91 · cited 2×
“Research has also focused on trial-based data (Wollstadt et al., 2014; Gomez-Herraro et al., 2015; Asaad et al., 2017) and single-trial time-averaged analyses (Wibral et al., 2013; Timme et al., 2014b, 2016; Nigam et al., 2016).”
MethodHigh-Degree Neurons Feed Cortical Computations
Timme, N. M., Ito, S., Myroshnychenko, M., Nigam, S., Shimono, M., Yeh, F.-C., Hottowy, P., Litke, A. M., & Beggs, J. M. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004858 · Cited by 86 · cited 5×
“Research has also focused on trial-based data (Wollstadt et al., 2014; Gomez-Herraro et al., 2015; Asaad et al., 2017) and single-trial time-averaged analyses (Wibral et al., 2013; Timme et al., 2014b, 2016; Nigam et al., 2016).”
BackgroundMethodInfluentialSelf-citationMultivariate Dependence beyond Shannon Information
James, R. G., & Crutchfield, J. P. (2017) · Entropy · doi:10.3390/e19100531 · Cited by 84
Intersection Information Based on Common Randomness
Griffith, V., Chong, E., James, R., Ellison, C., & Crutchfield, J. (2014) · Entropy · doi:10.3390/e16041985 · Cited by 84 · cited 5×
“While the minimum information possesses distinct advantages over other methods, it has been shown that it measures the redundant amount of information as opposed to the redundant content, which several authors find counterintuitive (Harder et al., 2013; Bertschinger et al., 2014; Griffith et al., 2014; Ince, 2017).”
BackgroundMethodInfluentialSmouse, P. E., Whitehead, M. R., & Peakall, R. (2015) · Molecular Ecology Resources · doi:10.1111/1755-0998.12422 · Cited by 80
Pointwise Partial Information Decomposition Using the Specificity and Ambiguity Lattices
Finn, C., & Lizier, J. (2018) · Entropy · doi:10.3390/e20040297 · Cited by 80
Goldberg, D. H., Victor, J. D., Gardner, E. P., & Gardner, D. (2009) · Neuroinformatics · doi:10.1007/s12021-009-9049-y · Cited by 78
How much information is associated with a particular stimulus?
Butts, D. (2003) · Network: Computation in Neural Systems · doi:10.1088/0954-898x/14/2/301 · Cited by 76
Ito, S., Yeh, F.-C., Hiolski, E. M., Rydygier, P., Gunning, D. E., Hottowy, P., Timme, N. M., Litke, A. M., & Beggs, J. M. (2014) · PLoS ONE · doi:10.1371/journal.pone.0105324 · Cited by 73 · cited 2×
“Cross-correlation is widely used in the analysis of neural spiking data (Bonifazi et al., 2009; Ito et al., 2014) to assess functional connections between pairs of neurons (i.e., neuron A tends to spike just before neuron B spikes).”
BackgroundSelf-citationIdentification of Functional Information Subgraphs in Complex Networks
Bettencourt, L. M. A., Gintautas, V., & Ham, M. (2008) · Physical Review Letters · doi:10.1103/physrevlett.100.238701 · Cited by 65 · cited 2×
“…are valuable because they can quantify phenomena like encoding (e.g., how much information a neuron provides; Bialek et al., 1991; Brenner et al., 2000) and complex encoding relationships (e.g., how much information neurons A and B provide together; Bettencourt et al., 2008; Timme et al., 2016).”
BackgroundPython for information theoretic analysis of neural data
Ince, R. A. A. (2009) · Frontiers in Neuroinformatics · doi:10.3389/neuro.11.004.2009 · Cited by 61 · cited 6×
“Second, several software packages could not be made to run on our machines or required programming languages with which we are not familiar (Goldberg et al., 2009; Ince et al., 2009; Magri et al., 2009; Pastore et al., 2016; Moore et al., 2017; dit-contributors, 2018).”
MethodInfluentialInvariant Components of Synergy, Redundancy, and Unique Information among Three Variables
Pica, G., Piasini, E., Chicharro, D., & Panzeri, S. (2017) · Entropy · doi:10.3390/e19090451 · Cited by 60 · cited 3×
“Second, several software packages could not be made to run on our machines or required programming languages with which we are not familiar (Goldberg et al., 2009; Ince et al., 2009; Magri et al., 2009; Pastore et al., 2016; Moore et al., 2017; dit-contributors, 2018).”
MethodTracing the Flow of Perceptual Features in an Algorithmic Brain Network
Ince, R. A. A., van Rijsbergen, N. J., Thut, G., Rousselet, G. A., Groß, J., Panzeri, S., & Schyns, P. G. (2015) · Scientific Reports · doi:10.1038/srep17681 · Cited by 56 · cited 1×
“An alternative method for measuring information transmission about some other variable can be found in Ince et al. (2015).”
Multiplex Networks of Cortical and Hippocampal Neurons Revealed at Different Timescales
Timme, N. M., Ito, S., Myroshnychenko, M., Yeh, F.-C., Hiolski, E. M., Hottowy, P., & Beggs, J. M. (2014) · PLoS ONE · doi:10.1371/journal.pone.0115764 · Cited by 55
Self-citationBreakdown of local information processing may underlie isoflurane anesthesia effects
Wollstadt, P., Sellers, K. K., Rudelt, L., Priesemann, V., Hutt, A., Fröhlich, F., & Wibral, M. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005511 · Cited by 49 · cited 2×
“…particular areas of interest include studies of connectivity (Honey et al., 2007; Ito et al., 2011; Timme et al., 2014b, 2016; Nigam et al., 2016; Wollstadt et al., 2017) and sensory encoding (Bialek et al., 1991; DeWeese and Meister, 1999; Brenner et al., 2000; Panzeri et al., 2001; Butts,…”
BackgroundPrefrontal Neurons Encode a Solution to the Credit-Assignment Problem
Asaad, W. F., Lauro, P. M., Perge, J. A., & Eskandar, E. N. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.3311-16.2017 · Cited by 48 · cited 3×
“Research has also focused on trial-based data (Wollstadt et al., 2014; Gomez-Herraro et al., 2015; Asaad et al., 2017) and single-trial time-averaged analyses (Wibral et al., 2013; Timme et al., 2014b, 2016; Nigam et al., 2016).”
MethodQuantifying Synergistic Information Using Intermediate Stochastic Variables
Quax, R., Har-Shemesh, O., & Sloot, P. M. A. (2017) · Entropy · doi:10.3390/e19020085 · Cited by 43 · cited 1×
“…measures for redundancy or unique information to address this problem (Williams and Beer, 2010; Harder et al., 2013; Bertschinger et al., 2014; Griffith et al., 2014; Ince, 2017; Finn and Lizier, 2018; see Quax et al., 2017 for an alternative information theory approach to measuring synergy).”
BackgroundOpen source tools for the information theoretic analysis of neural data
Ince, R. A. A., Mazzoni, A., Petersen, R. S., & Panzeri, S. (2010) · Frontiers in Neuroscience · doi:10.3389/neuro.01.011.2010 · Cited by 41 · cited 2×
“While we feel our software package fills a valuable role in the field, we wish to emphasize that other excellent information theory software packages exist and that these packages may be more useful depending on the desired application (Ince et al., 2010; Quiroga and Panzeri, 2013; Lizier, 2014; Szabo, 2014; Moore et…”
BackgroundMethodMultiplex networks of cortical and hippocampal neurons revealed at different timescales
Timme, N. M., Ito, S., Myroshnychenko, M., Yeh, F.-C., Hiolski, E. M., Litke, A. M., & Beggs, J. M. (2014) · BMC Neuroscience · doi:10.1186/1471-2202-15-s1-p212 · Cited by 37 · cited 8×
“The randomization procedures necessary to generate surrogate null model data can be performed before creating probability distributions (e.g., spike jittering (Rolston et al., 2007; Timme et al., 2014b)) or after creating probability distributions.”
BackgroundMethodResultInfluentialSelf-citationBehavior Modulates Effective Connectivity between Cortex and Striatum
Nakhnikian, A., Rebec, G. V., Grasse, L. M., Dwiel, L. L., Shimono, M., & Beggs, J. M. (2014) · PLoS ONE · doi:10.1371/journal.pone.0089443 · Cited by 35 · cited 2×
“Granger causality has been widely used to assess causal interactions between both continuous and discrete neural signals (Granger, 1969; Ding et al., 2006; Nakhnikian et al., 2014), though underlying assumptions about the analysis (e.”
MethodToolConnect: A Functional Connectivity Toolbox for In vitro Networks
Pastore, V. P., Poli, D., Godjoski, A., Sergio Martinoia, A., & Massobrio, P. (2016) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2016.00013 · Cited by 25
Discovering Pair-Wise Genetic Interactions: An Information Theory-Based Approach
Ignac, T. M., Skupin, A., Sakhanenko, N. A., & Galas, D. J. (2014) · PLoS ONE · doi:10.1371/journal.pone.0092310 · Cited by 24 · cited 1×
“Information theory is currently used in genetics (Vinga, 2013; Ignac et al., 2014; Smouse et al., 2015), but studies could be performed linking genetics and neuroscience.”
MethodMinimum Sample Size for Reliable Causal Inference Using Transfer Entropy
Ramos, A. M. T., & Macau, E. E. N. (2017) · Entropy · doi:10.3390/e19040150 · Cited by 19 · cited 1×
“The number of observations necessary to estimate a probability distribution has been explored to some extent in the literature (Ramos and Macau, 2017), but a great deal of attention has been paid to other methods to assess bias and estimate probability distributions (see Handling Continuous Data…”
MethodInform: A toolkit for information-theoretic analysis of complex systems
Moore, D. G., Valentini, G., Walker, S. I., & Levin, M. G. (2017) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2017.8285197 · Cited by 13 · cited 4×
“Second, several software packages could not be made to run on our machines or required programming languages with which we are not familiar (Goldberg et al., 2009; Ince et al., 2009; Magri et al., 2009; Pastore et al., 2016; Moore et al., 2017; dit-contributors, 2018).”
BackgroundMethodInfluential
Cited by
PLD3 affects axonal spheroids and network defects in Alzheimer’s disease
Yuan, P., Zhang, M., Tong, L., Morse, T. M., McDougal, R. A., Ding, H., Chan, D., Cai, Y., & Grutzendler, J. (2022) · Nature · doi:10.1038/s41586-022-05491-6 · Cited by 155
Information decomposition and the informational architecture of the brain
Luppi, A. I., Rosas, F. E., Mediano, P. A. M., Menon, D. K., & Stamatakis, E. A. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2023.11.005 · Cited by 140
Neural and behavioural state switching during hippocampal dentate spikes
Farrell, J. S., Hwaun, E., Dudok, B., & Soltesz, I. (2024) · Nature · doi:10.1038/s41586-024-07192-8 · Cited by 47
Causal composition: Structural differences among dynamically equivalent systems
Albantakis, L., & Tononi, G. (2019) · Entropy · doi:10.3390/e21100989 · Cited by 30
Metadata
- Authors
- Timme, N. M., & Lapish, C.
- Venue
- eneuro
- Publisher
- Society for Neuroscience
- Published
- 2018-05
- Volume
- 5
- Issue
- 3
- Pages
- ENEURO.0052-18.2018
- ISSN
- 2373-2822 (ISSN-L)
- OpenAlex
- W2810939025
- Cited by
- 242
- References
- 100
- FWCI
- 11.62
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.