Elements of Information Theory

Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observations

Golatkar, A., Achille, A., & Soatto, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58526-6_23 · Cited by 122

Fluctuating fitness shapes the clone-size distribution of immune repertoires

Desponds, J., Mora, T., & Walczak, A. M. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1512977112 · Cited by 96

Entanglement-Assisted Communication of Classical and Quantum Information

Hsieh, M.-H., & Wilde, M. M. (2010) · IEEE Transactions on Information Theory · doi:10.1109/tit.2010.2053903 · Cited by 112

On nonlocality as a resource theory and nonlocality measures

de Vicente, J. I. (2014) · arXiv · doi:10.48550/arxiv.1401.6941 · Cited by 112

Deep learning generalizes because the parameter-function map is biased towards simple functions

Valle-Pérez, G., Camargo, C. Q., & Louis, A. A. (2018) · arXiv · doi:10.48550/arxiv.1805.08522 · Cited by 105

Objects of consciousness

Hoffman, D. D., & Prakash, C. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00577 · Cited by 103

Evolution of integrated causal structures in animats exposed to environments of increasing complexity

Albantakis, L., Hintze, A., Koch, C., Adami, C., & Tononi, G. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003966 · Cited by 101

Nonlocal Resources in the Presence of Superselection Rules

Schuch, N., Verstraete, F., & Cirac, J. I. (2004) · Physical Review Letters · doi:10.1103/physrevlett.92.087904 · Cited by 102

Hippocampo-cortical circuits for selective memory encoding, routing, and replay

Harvey, R. E., Robinson, H. L., Liu, C., Oliva, A., & Fernandez-Ruiz, A. (2023) · Neuron · doi:10.1016/j.neuron.2023.04.015 · Cited by 101

Psychophysical Characterization and Testbed Validation of a Wearable Vibrotactile Glove for Telemanipulation

Murray, A. M., Klatzky, R. L., & Khosla, P. K. (2003) · Presence: Teleoperators and Virtual Environments · doi:10.1162/105474603321640923 · Cited by 99

Information transfer by leaky, heterogeneous, protein kinase signaling systems

Voliotis, M., Perrett, R. M., McWilliams, C., McArdle, C. A., & Bowsher, C. G. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1314446111 · Cited by 98

The vocal joystick: evaluation of voice-based cursor control techniques

Harada, S., Landay, J. A., Malkin, J., Li, X., & Bilmes, J. A. (2006) · Proceedings of the 8th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/1168987.1169021 · Cited by 95

Data-driven identification of structural alerts for mitigating the risk of drug-induced human liver injuries

Liu, R., Yu, X., & Wallqvist, A. (2015) · Journal of Cheminformatics · doi:10.1186/s13321-015-0053-y · Cited by 92

Finding Haplotype Block Boundaries by Using the Minimum-Description-Length Principle

Anderson, E. C., & Novembre, J. (2003) · The American Journal of Human Genetics · doi:10.1086/377106 · Cited by 90

ACTIVE LEZI: AN INCREMENTAL PARSING ALGORITHM FOR SEQUENTIAL PREDICTION

GOPALRATNAM, K., & COOK, D. J. (2004) · International Journal on Artificial Intelligence Tools · doi:10.1142/s0218213004001892 · Cited by 89

The representation of prediction error in auditory cortex

Rubin, J., Ulanovsky, N., Nelken, I., & Tishby, N. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005058 · Cited by 87

Privacy-Preserving Adversarial Networks

Tripathy, A., Wang, Y., & Ishwar, P. (2019) · 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton) · doi:10.1109/allerton.2019.8919758 · Cited by 87

Measuring statistical heterogeneity: The Pietra index

Eliazar, I. I., & Sokolov, I. M. (2010) · Physica A: Statistical Mechanics and its Applications · doi:10.1016/j.physa.2009.08.006 · Cited by 85

Beyond heat baths: Generalized resource theories for small-scale thermodynamics

Yunger Halpern, N., & Renes, J. M. (2016) · Physical Review E · doi:10.1103/physreve.93.022126 · Cited by 85

Distributed Variational Representation Learning

Aguerri, I. E., & Zaidi, A. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2928806 · Cited by 84

Sharing neuroimaging studies of human cognition

Horn, J. D. V., Grafton, S. T., Rockmore, D., & Gazzaniga, M. S. (2004) · Nature Neuroscience · doi:10.1038/nn1231 · Cited by 83

Neural coding and decoding: communication channels and quantization

Dimitrov, A. G., & Miller, J. P. (2001) · Network: Computation in Neural Systems · doi:10.1080/net.12.4.441.472 · Cited by 76

Dashing hopes? The predictive accuracy of domestic abuse risk assessment by police

Turner, E., Medina, J., & Brown, G. (2019) · The British Journal of Criminology · doi:10.1093/bjc/azy074 · Cited by 78

Improved read/write cost tradeoff in DNA-based data storage using LDPC codes

Chandak, S., Tatwawadi, K., Lau, B., Mardia, J., Kubit, M., Neu, J., Griffin, P., Wootters, M., Weissman, T., & Ji, H. (2019) · 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton) · doi:10.1109/allerton.2019.8919890 · Cited by 80

From classification to quantification in tweet sentiment analysis

Gao, W., & Sebastiani, F. (2016) · Social Network Analysis and Mining · doi:10.1007/s13278-016-0327-z · Cited by 75

Wireless Network Intelligence at the Edge

Park, J., Samarakoon, S., Bennis, M., & Debbah, M. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2941458 · Cited by 92

Signal detection theory, uncertainty, and Poisson-like population codes

Ma, W. J. (2010) · Vision Research · doi:10.1016/j.visres.2010.08.035 · Cited by 60

The thermodynamic efficiency of computations made in cells across the range of life

Kempes, C. P., Wolpert, D., Cohen, Z., & Pérez-Mercader, J. (2017) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2016.0343 · Cited by 68

Trading classical communication, quantum communication, and entanglement in quantum Shannon theory

Hsieh, M.-H., & Wilde, M. M. (2010) · IEEE Transactions on Information Theory · doi:10.1109/tit.2010.2054532 · Cited by 68

Managing Your Private and Public Data: Bringing Down Inference Attacks Against Your Privacy

Salamatian, S., Zhang, A., du Pin Calmon, F., Bhamidipati, S., Fawaz, N., Kveton, B., Oliveira, P., & Taft, N. (2015) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2015.2442227 · Cited by 66

Contextual predictability shapes signal autonomy

Winters, J., Kirby, S., & Smith, K. (2018) · Cognition · doi:10.1016/j.cognition.2018.03.002 · Cited by 54

A 3 $\mu$W CMOS True Random Number Generator With Adaptive Floating-Gate Offset Cancellation

Holleman, J., Bridges, S., Otis, B. P., & Diorio, C. (2008) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2008.920327 · Cited by 63

Considerate approaches to constructing summary statistics for ABC model selection

Barnes, C. P., Filippi, S., Stumpf, M. P. H., & Thorne, T. (2012) · Statistics and Computing · doi:10.1007/s11222-012-9335-7 · Cited by 61

Computational Evolutionary Perception

Hoffman, D. D., & Singh, M. (2012) · Perception · doi:10.1068/p7275 · Cited by 61

Battery state of health monitoring by estimation of the number of cyclable Li-ions

Zhou, X., Stein, J. L., & Ersal, T. (2017) · Control Engineering Practice · doi:10.1016/j.conengprac.2017.05.009 · Cited by 60

Playing 20 Questions with the Mind: Collaborative Problem Solving by Humans Using a Brain-to-Brain Interface

Stocco, A., Prat, C. S., Losey, D. M., Cronin, J. A., Wu, J., Abernethy, J. A., & Rao, R. P. N. (2015) · PLOS ONE · doi:10.1371/journal.pone.0137303 · Cited by 57

Reduced predictable information in brain signals in autism spectrum disorder

Gómez, C., Lizier, J. T., Schaum, M., Wollstadt, P., Grützner, C., Uhlhaas, P., Freitag, C. M., Schlitt, S., Bölte, S., Hornero, R., et al. (2014) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2014.00009 · Cited by 59

Conservation of Information in Search: Measuring the Cost of Success

Dembski, W. A., & Marks II, R. J. (2009) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/tsmca.2009.2025027 · Cited by 59

Machine learning in NextG networks via generative adversarial networks

Ayanoglu, E., Davaslioglu, K., & Sagduyu, Y. E. (2022) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2022.3153004 · Cited by 55

Averaging Is Probably Not the Optimum Way of Aggregating Parameters in Federated Learning

Xiao, P., Cheng, S., Stankovic, V., & Vukobratovic, D. (2020) · Entropy · doi:10.3390/e22030314 · Cited by 54

Dynamic structure of locomotor behavior in walking fruit flies

Katsov, A. Y., Freifeld, L., Horowitz, M., Kuehn, S., & Clandinin, T. R. (2017) · eLife · doi:10.7554/elife.26410 · Cited by 43

Information-theoretic latent distribution modeling: Distinguishing discrete and continuous latent variable models.

Markon, K. E., & Krueger, R. F. (2006) · Psychological Methods · doi:10.1037/1082-989x.11.3.228 · Cited by 53

The tyrosine phosphatase SHP2 increases robustness and information transfer within IL-6-induced JAK/STAT signalling

Fiebelkow, J., Guendel, A., Guendel, B., Mehwald, N., Jetka, T., Komorowski, M., Waldherr, S., Schaper, F., & Dittrich, A. (2021) · Cell Communication and Signaling · doi:10.1186/s12964-021-00770-7 · Cited by 53

Information theoretic focal length selection for real-time active 3D object tracking

Denzler, Zobel, & Niemann (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238372 · Cited by 53

Dissecting natural sensory plasticity: hormones and experience in a maternal context

Miranda, J. A., & Liu, R. C. (2009) · Hearing Research · doi:10.1016/j.heares.2009.04.014 · Cited by 52

Learning Anonymized Representations with Adversarial Neural Networks

Feutry, C., Piantanida, P., Bengio, Y., & Duhamel, P. (2018) · arXiv · doi:10.48550/arxiv.1802.09386 · Cited by 52

Exponential random simplicial complexes

Zuev, K., Eisenberg, O., & Krioukov, D. (2015) · Journal of Physics A: Mathematical and Theoretical · doi:10.1088/1751-8113/48/46/465002 · Cited by 49

Breakdown of local information processing may underlie isoflurane anesthesia effects

Wollstadt, P., Sellers, K. K., Rudelt, L., Priesemann, V., Hutt, A., Fröhlich, F., & Wibral, M. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005511 · Cited by 49

True Random Number Generation Via Sampling From Flat Band-Limited Gaussian Processes

Gov, N. C., Mihcak, M. K., & Ergun, S. (2011) · IEEE Transactions on Circuits and Systems I: Regular Papers · doi:10.1109/tcsi.2010.2089561 · Cited by 48

Uncertainty Estimation in Autoregressive Structured Prediction

Gales, M., & Malinin, A. (2021) · Apollo - University of Cambridge Repository · doi:10.17863/cam.63497 · Cited by 48

Evaluation of glass samples for forensic purposes-an application of likelihood ratios and an information–theoretical approach

Zadora, G., & Ramos, D. (2010) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2010.03.007 · Cited by 48

Bi-weighting domain adaptation for cross-language text classification

(2011) · International Joint Conference on Artificial Intelligence · doi:10.5591/978-1-57735-516-8/ijcai11-258 · Cited by 47

A detailed comparison of optimality and simplicity in perceptual decision making.

Shen, S., & Ma, W. J. (2016) · Psychological Review · doi:10.1037/rev0000028 · Cited by 46

Identifying the Information Gain of a Quantum Measurement

Berta, M., Renes, J. M., & Wilde, M. M. (2014) · IEEE Transactions on Information Theory · doi:10.1109/tit.2014.2365207 · Cited by 60

Shared information—new insights and problems in decomposing information in complex systems

Bertschinger, N., Rauh, J., Olbrich, E., & Jost, J. (2013) · Springer Proceedings in Complexity · doi:10.1007/978-3-319-00395-5_35 · Cited by 45

Geometric robustness theory and biological networks

AY, N., & KRAKAUER, D. (2006) · Theory in Biosciences · doi:10.1016/j.thbio.2006.06.002 · Cited by 45

How many days are enough?: capturing routine human mobility

Stanley, K., Yoo, E.-H., Paul, T., & Bell, S. (2018) · International Journal of Geographical Information Science · doi:10.1080/13658816.2018.1434888 · Cited by 42

Using probabilistic reasoning

Doust, J. (2009) · BMJ · doi:10.1136/bmj.b3823 · Cited by 41

Information-Theoretic Probing for Linguistic Structure

Pimentel, T., Valvoda, J., Hall Maudslay, R., Zmigrod, R., Williams, A., & Cotterell, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.420 · Cited by 41

Fisher-information-based data compression for estimation using two sensors

Fowler, M. L., & Chen, M. (2005) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2005.1541459 · Cited by 41

Quantum proofs for classical theorems

(2011) · Theory of Computing · doi:10.4086/toc.gs.2011.002 · Cited by 41

Steps Towards a Theory of Visual Information: Active Perception, Signal-to-Symbol Conversion and the Interplay Between Sensing and Control

Soatto, S. (2011) · arXiv · doi:10.48550/arxiv.1110.2053 · Cited by 40

Online portfolio selection strategy based on combining experts' advice

Zhang, Y., & Yang, X. (2017) · Computational Economics · doi:10.1007/s10614-016-9585-0 · Cited by 38

A Framework for Investigating the Performance of Chaotic-Map Truly Random Number Generators

Beirami, A., & Nejati, H. (2013) · IEEE Transactions on Circuits and Systems II: Express Briefs · doi:10.1109/tcsii.2013.2258274 · Cited by 36

Parameterizing and Assembling IR-Based Solutions for SE Tasks Using Genetic Algorithms

Panichella, A., Dit, B., Oliveto, R., Di Penta, M., Poshyvanyk, D., & De Lucia, A. (2016) · 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER) · doi:10.1109/saner.2016.97 · Cited by 36

Reverse engineering and analysis of large genome-scale gene networks

Aluru, M., Zola, J., Nettleton, D., & Aluru, S. (2013) · Nucleic Acids Research · doi:10.1093/nar/gks904 · Cited by 34

Nonlinear dynamic analysis of oscillatory repetitive movements in Parkinson's disease and essential tremor

Costa, J., González, H. A., Valldeoriola, F., Gaig, C., Tolosa, E., & Valls‐Solé, J. (2010) · Movement Disorders · doi:10.1002/mds.23334 · Cited by 33

Modularity and directionality in genetic interaction maps

Jaimovich, A., Rinott, R., Schuldiner, M., Margalit, H., & Friedman, N. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq197 · Cited by 32

Scoring Rules and Decision Analysis Education

Bickel, J. E. (2010) · Decision Analysis · doi:10.1287/deca.1100.0184 · Cited by 31

Witnessing incompatibility of quantum channels

Carmeli, C., Heinosaari, T., Miyadera, T., & Toigo, A. (2019) · Journal of Mathematical Physics · doi:10.1063/1.5126496 · Cited by 30

Causal composition: Structural differences among dynamically equivalent systems

Albantakis, L., & Tononi, G. (2019) · Entropy · doi:10.3390/e21100989 · Cited by 30

Toward Optimal Target Placement for Neural Prosthetic Devices

Cunningham, J. P., Yu, B. M., Gilja, V., Ryu, S. I., & Shenoy, K. V. (2008) · Journal of Neurophysiology · doi:10.1152/jn.90833.2008 · Cited by 25

On the Power of Multiple Anonymous Messages

Ghazi, B., Golowich, N., Kumar, R., Pagh, R., & Velingker, A. (2019) · arXiv · doi:10.48550/arxiv.1908.11358 · Cited by 26

High-throughput phenotyping with temporal sequences

Estiri, H., Strasser, Z. H., & Murphy, S. N. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa288 · Cited by 26

Statistical Patterns in Written Language

Zanette, D. H. (2014) · arXiv · doi:10.48550/arxiv.1412.3336 · Cited by 25

Bayesian active probabilistic classification for psychometric field estimation

Song, X. D., Sukesan, K. A., & Barbour, D. L. (2018) · Attention, Perception, & Psychophysics · doi:10.3758/s13414-017-1460-0 · Cited by 22

Moment information and entropy evaluation for probability densities

Milev, M., Inverardi, P. N., & Tagliani, A. (2012) · Applied Mathematics and Computation · doi:10.1016/j.amc.2011.11.093 · Cited by 24

Natural Selection and Shape Perception

Singh, M., & Hoffman, D. D. (2013) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-1-4471-5195-1_12 · Cited by 23

Dynamical decomposition of Markov processes without detailed balance

Ao, P., Chen, T.-Q., & Shi, J.-H. (2013) · Chinese Physics Letters · doi:10.1088/0256-307x/30/7/070201 · Cited by 21

The characterization of a measure of information discrepancy

Fang, W. (2000) · Information Sciences · doi:10.1016/s0020-0255(00)00008-6 · Cited by 23

Detecting sexual predators in chats using behavioral features and imbalanced learning

CARDEI, C., & REBEDEA, T. (2017) · Natural Language Engineering · doi:10.1017/s1351324916000395 · Cited by 21

Natural selection. V. How to read the fundamental equations of evolutionary change in terms of information theory

Frank, S. A. (2012) · Journal of Evolutionary Biology · doi:10.1111/jeb.12010 · Cited by 21

A measure of survey mode differences

Homola, J., Jackson, N., & Gill, J. (2016) · Electoral Studies · doi:10.1016/j.electstud.2016.06.010 · Cited by 21

“Security of quantum key distribution protocols,”

Mafu, M., & Senekane, M. (2018) · Advanced Technologies of Quantum Key Distribution · doi:10.5772/intechopen.74234 · Cited by 20

Online perceptual learning and natural language acquisition for autonomous robots

Alomari, M., Li, F., Hogg, D. C., & Cohn, A. G. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2021.103637 · Cited by 17

Is life impossible? Information, sex, and the origin of complex organisms.

Peck, J. R., & Waxman, D. (2010) · Evolution · doi:10.1111/j.1558-5646.2010.01074.x · Cited by 13

How could we make a social robot? A virtual bargaining approach

Chater, N. (2023) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2022.0040 · Cited by 14

Scoring two-species local alignments to try to statistically separate neutrally evolving from selected DNA segments

Roskin, K. M., Diekhans, M., & Haussler, D. (2003) · Proceedings of the seventh annual international conference on Research in computational molecular biology · doi:10.1145/640075.640109 · Cited by 14

Is cosmological tuning fine or coarse?

Díaz-Pachón, D. A., Hössjer, O., & Marks II, R. J. (2021) · Journal of Cosmology and Astroparticle Physics · doi:10.1088/1475-7516/2021/07/020 · Cited by 10

Generalized synchrony of coupled stochastic processes with multiplicative noise

Jafri, H. H., Singh, R. K. B., & Ramaswamy, R. (2016) · Physical Review E · doi:10.1103/physreve.94.052216 · Cited by 12

On the improbability of algorithmic specified complexity

Ewert, W., Marks, R. J., & Dembski, W. A. (2013) · 45th Southeastern Symposium on System Theory · doi:10.1109/ssst.2013.6524962 · Cited by 11

Estimating Exceptionally Rare Germline and Somatic Mutation Frequencies via Next Generation Sequencing

Eboreime, J., Choi, S.-K., Yoon, S.-R., Arnheim, N., & Calabrese, P. (2016) · PLOS ONE · doi:10.1371/journal.pone.0158340 · Cited by 11

Sampling bias corrections for discrete and Gaussian partial information decompositions

Lorenz, G. M., Engel, N. M., Koçillari, L., Celotto, M., Orsenigo, D., Curreli, S., Malerba, S. B., Engel, A. K., Kayser, C., Fellin, T., et al. (2026) · Patterns · doi:10.1016/j.patter.2026.101619 · Cited by 9

Deception as a Semantic Attack

Thompson, P. (2006) · Chapman & Hall/CRC Computer & Information Science Series · doi:10.1201/9781420011012.ch2.2 · Cited by 8

Optimizing Epitope Conformational Ensembles Using α-Synuclein Cyclic Peptide “Glycindel” Scaffolds: A Customized Immunogen Method for Generating Oligomer-Selective Antibodies for Parkinson’s Disease

Hsueh, S. C. C., Aina, A., Roman, A. Y., Cashman, N. R., Peng, X., & Plotkin, S. S. (2022) · ACS Chemical Neuroscience · doi:10.1021/acschemneuro.1c00567 · Cited by 8

Decomposing information into copying versus transformation

Kolchinsky, A., & Corominas-Murtra, B. (2020) · Journal of The Royal Society Interface · doi:10.1098/rsif.2019.0623 · Cited by 5

Integrated information as a measure of cognitive processes in coupled genetic repressilators

Abrego, L., & Zaikin, A. (2019) · Entropy · doi:10.3390/e21040382 · Cited by 5

A Joint Data Compression and Time-Delay Estimation Method for Distributed Systems via Extremum Encoding

Weiss, A., Kochman, Y., & Wornell, G. W. (2024) · ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp48485.2024.10446180 · Cited by 5

Extremum Encoding for Joint Baseband Signal Compression and Time-Delay Estimation for Distributed Systems

Weiss, A., Kochman, Y., & Wornell, G. W. (2025) · ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp49660.2025.10888999 · Cited by 3

Quantification of the bone healing process using information of B-Mode ultrasound image

Al-Nashash, H., Mir, H., Al-Marzouqi, S., Al-Kendi, S., & Khalaf, K. (2012) · 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society · doi:10.1109/embc.2012.6346952 · Cited by 2