Decomposing information into copying versus transformation
Open accessPublished version of a preprint
Abstract
In many real-world systems, information can be transmitted in two qualitatively different ways: by copying or by transformation . Copying occurs when messages are transmitted without modification, e.g. when an offspring receives an unaltered copy of a gene from its parent. Transformation occurs when messages are modified systematically during transmission, e.g. when mutational biases occur during genetic replication. Standard information-theoretic measures do not distinguish these two modes of information transfer, although they may reflect different mechanisms and have different functional consequences. Starting from a few simple axioms, we derive a decomposition of mutual information into the information transmitted by copying versus the information transmitted by transformation. We begin with a decomposition that applies when the source and destination of the channel have the same set of messages and a notion of message identity exists. We then generalize our decomposition to other kinds of channels, which can involve different source and destination sets and broader notions of similarity. In addition, we show that copy information can be interpreted as the minimal work needed by a physical copying process, which is relevant for understanding the physics of replication. We use the proposed decomposition to explore a model of amino acid substitution rates. Our results apply to any system in which the fidelity of copying, rather than simple predictability, is of critical relevance.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Landauer's bound gives the minimal energy required to perform an abstract computation and many string writing operations, including copies and transformations.
“Landauer’s bound [132] gives us the minimal energy required to perform an abstract computation and many string writing operations, including copies and transformations of it [133].”
From Fundamental constraints to the logic of living systems (2024) · cites this paper as [133]
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486 · see citing works
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 37,139 · see citing works
Measuring Information Transfer
Schreiber, T. (2000) · Physical Review Letters · doi:10.1103/physrevlett.85.461 · Cited by 4,571 · see citing works
An Improved General Amino Acid Replacement Matrix
Le, S. Q., & Gascuel, O. (2008) · Molecular Biology and Evolution · doi:10.1093/molbev/msn067 · Cited by 3,127 · see citing works
Modeling and Simulation of Genetic Regulatory Systems: A Literature Review
de Jong, H. (2002) · Journal of Computational Biology · doi:10.1089/10665270252833208 · Cited by 2,660 · see citing works
Amino Acid Difference Formula to Help Explain Protein Evolution
Grantham, R. L. (1974) · Science · doi:10.1126/science.185.4154.862 · Cited by 2,363 · see citing works
Bayesian Experimental Design: A Review
Chaloner, K. M., & Verdinelli, I. (1995) · Statistical Science · doi:10.1214/ss/1177009939 · Cited by 1,843 · see citing works
A New Interpretation of Information Rate
Kelly, J. L. (1956) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1956.tb03809.x · Cited by 1,643 · see citing works
On a measure of the information provided by an experiment
Lindley, D. V. (1956) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177728069 · Cited by 1,509 · see citing works
Parrondo, J. M. R., Horowitz, J. M., & Sagawa, T. (2015) · Nature Physics · doi:10.1038/nphys3230 · Cited by 1,271 · see citing works
Reference Posterior Distributions for Bayesian Inference
Bernardo, J. M. (1979) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1979.tb01066.x · Cited by 1,083 · see citing works
Estimating the pattern of nucleotide substitution
Yang, Z. (1994) · Journal of Molecular Evolution · doi:10.1007/bf00178256 · Cited by 1,076 · see citing works
Modern Image Quality Assessment
Wang, Z., & Bovik, A. C. (2006) · Synthesis Lectures on Image, Video, and Multimedia Processing · doi:10.1007/978-3-031-02238-8 · Cited by 967 · see citing works
Vervet monkey alarm calls: Semantic communication in a free-ranging primate
Seyfarth, R., Cheney, D., & Marler, P. (1980) · Animal Behaviour · doi:10.1016/s0003-3472(80)80097-2 · Cited by 957 · see citing works
Butte, A. J., & Kohane, I. S. (1999) · Pacific Symposium on Biocomputing (PSB) · doi:10.1142/9789814447331_0040 · Cited by 956 · see citing works
Nonnegative Decomposition of Multivariate InformationPreprint
Williams, P. L., & Beer, R. (2010) · arXiv.org · doi:10.48550/arxiv.1004.2515 · Cited by 814 · see citing works
An Introduction to Information Theory.
Mallows, C. L., & Reza, F. M. (1964) · American Mathematical Monthly · doi:10.2307/2311344 · Cited by 778 · see citing works
Expected Information as Expected Utility
Bernardo, J. (1979) · The Annals of Statistics · doi:10.1214/aos/1176344689 · Cited by 705 · see citing works
Entropy Optimization Principles and Their Applications
Kapur, J. N., & Kesavan, H. K. (1992) · Water science and technology library · doi:10.1007/978-94-011-2430-0_1 · Cited by 520 · see citing works
Modern Image Quality Assessment
Wang, Z., & Bovik, A. C. (2006) · Synthesis lectures on image, video, and multimedia processing · doi:10.2200/s00010ed1v01y200508ivm003 · Cited by 500 · see citing works
Simulating the Evolution of Language
Cangelosi, A., & Parisi, D. (2002) · Springer eBooks · doi:10.1007/978-1-4471-0663-0 · Cited by 489 · see citing works
Nowak, M. A., & Krakauer, D. C. (1999) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.96.14.8028 · Cited by 484 · see citing works
Second Law of Thermodynamics with Discrete Quantum Feedback Control
Sagawa, T., & Ueda, M. (2008) · Physical Review Letters · doi:10.1103/physrevlett.100.080403 · Cited by 442 · see citing works
Models of amino acid substitution and applications to mitochondrial protein evolution
Yang, Z., Nielsen, R., & Hasegawa, M. (1998) · Molecular Biology and Evolution · doi:10.1093/oxfordjournals.molbev.a025888 · Cited by 414 · see citing works
Properties of cross-entropy minimization
Shore, J. E., & Johnson, R. (1981) · IEEE Transactions on Information Theory · doi:10.1109/tit.1981.1056373 · Cited by 397 · see citing works
The formal definition of reference priors
Berger, J. O., Bernardo, J. M., & Sun, D. (2009) · The Annals of Statistics · doi:10.1214/07-aos587 · Cited by 394 · see citing works
Li, W., Wu, C., & Luo, C. (1984) · Journal of Molecular Evolution · doi:10.1007/bf02100628 · Cited by 378 · see citing works
Evolving grounded communication for robots
Steels, L. (2003) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(03)00129-3 · Cited by 371 · see citing works
Bayesian Surprise Attracts Human Attention
Itti, L., & Baldi, P. (2005) · neural information processing systems · Cited by 363 · see citing works
The Computational Nature of Language Learning and Evolution
Niyogi, P. (2006) · The MIT Press eBooks · doi:10.7551/mitpress/2024.001.0001 · Cited by 361 · see citing works
Universal portfolios with side information
Cover, T. M., & Ordentlich, E. (1996) · IEEE Transactions on Information Theory · doi:10.1109/18.485708 · Cited by 353 · see citing works
Biological evolution of the Saussurean sign as a component of the language acquisition device
Hurford, J. R. (1989) · Lingua · doi:10.1016/0024-3841(89)90015-6 · Cited by 349 · see citing works
Three faces of the second law. I. Master equation formulation
Esposito, M., & den Broeck, C. V. (2010) · Physical Review E · doi:10.1103/physreve.82.011143 · Cited by 339 · see citing works
Two types of amino acid substitutions in protein evolution
Miyata, T., Miyazawa, S., & Yasunaga, T. (1979) · Journal of Molecular Evolution · doi:10.1007/bf01732340 · Cited by 321 · see citing works
Relations between chemical structure and biological activity in peptides
Sneath, P. H. A. (1966) · Journal of Theoretical Biology · doi:10.1016/0022-5193(66)90112-3 · Cited by 291 · see citing works
Exploiting the Co-evolution of Interacting Proteins to Discover Interaction Specificity
Ramani, A. K., & Marcotte, E. M. (2003) · Journal of Molecular Biology · doi:10.1016/s0022-2836(03)00114-1 · Cited by 220 · see citing works
The fitness value of information
Donaldson-Matasci, M. C., Bergstrom, C. T., & Lachmann, M. (2010) · Oikos · doi:10.1111/j.1600-0706.2009.17781.x · Cited by 212 · see citing works
Hanel, R., & Stefan Thurner, A. (2011) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/93/20006 · Cited by 184 · see citing works
Optimal decoding of cellular identities in a genetic network
Petkova, M. D., Tkačik, G., Bialek, W., Wieschaus, E. F., & Gregor, T. (2019) · Cell · doi:10.1016/j.cell.2019.01.007 · Cited by 178 · see citing works
How to measure the information gained from one symbol
DeWeese, M. R., & Meister, M. (1999) · Network Computation in Neural Systems · doi:10.1088/0954-898x_10_4_303 · Cited by 168 · see citing works
Semantic information, autonomous agency and non-equilibrium statistical physics
Kolchinsky, A., & Wolpert, D. H. (2018) · Interface Focus · doi:10.1098/rsfs.2018.0041 · Cited by 163 · see citing works
Self-citationMISTIC: mutual information server to infer coevolution
Simonetti, F. L., Teppa, E., Chernomoretz, A., Nielsen, M., & Marino‐Buslje, C. (2013) · Nucleic Acids Research · doi:10.1093/nar/gkt427 · Cited by 159 · see citing works
Unified framework for information integration based on information geometry
Oizumi, M., Tsuchiya, N., & Amari, S. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1603583113 · Cited by 156 · see citing works
Bits from Brains for Biologically Inspired Computing
Wibral, M., Lizier, J. T., & Priesemann, V. (2015) · Frontiers in Robotics and AI · doi:10.3389/frobt.2015.00005 · Cited by 131 · see citing works
Unifying Divergence Minimization and Statistical Inference Via Convex Duality
Altün, Y., & Smola, A. J. (2006) · Lecture notes in computer science · doi:10.1007/11776420_13 · Cited by 108 · see citing works
Amino acid composition and the evolutionary rates of protein-coding genes
Graur, D. (1985) · Journal of Molecular Evolution · doi:10.1007/bf02105805 · Cited by 99 · see citing works
Pyvolve: A Flexible Python Module for Simulating Sequences along Phylogenies
Spielman, S. J., & Wilke, C. O. (2015) · PLoS ONE · doi:10.1371/journal.pone.0139047 · Cited by 88 · see citing works
Physics, Computation, and Why Biology Looks so Different
Hopfield, J. J. (1994) · Journal of Theoretical Biology · doi:10.1006/jtbi.1994.1211 · Cited by 87 · see citing works
Mathematical Models in Sociology
Sørensen, A. B. (1978) · Annual Review of Sociology · doi:10.1146/annurev.so.04.080178.002021 · Cited by 87 · see citing works
The transmission sense of information
Bergstrom, C. T., & Rosvall, M. (2009) · Biology & Philosophy · doi:10.1007/s10539-009-9180-z · Cited by 86 · see citing works
Application of a Measure of Information to the Design and Comparison of Regression Experiments
Stone, M. (1959) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177706359 · Cited by 82 · see citing works
How much information is associated with a particular stimulus?
Butts, D. A. (2003) · Network Computation in Neural Systems · doi:10.1088/0954-898x_14_2_301 · Cited by 77 · see citing works
A bound on the financial value of information
Barron, A. R., & Cover, T. M. (1988) · IEEE Transactions on Information Theory · doi:10.1109/18.21241 · Cited by 58 · see citing works
Optimizing the mutual intelligibility of linguistic agents in a shared world
Komarova, N. L., & Niyogi, P. (2003) · Artificial Intelligence · doi:10.1016/j.artint.2003.08.005 · Cited by 56 · see citing works
Maximum Entropy Distribution Estimation with Generalized Regularization
Dudı́k, M., & Schapire, R. E. (2006) · Lecture notes in computer science · doi:10.1007/11776420_12 · Cited by 47 · see citing works
Fundamental Costs in the Production and Destruction of Persistent Polymer Copies
Ouldridge, T. E., & ten Wolde, P. R. (2017) · Physical Review Letters · doi:10.1103/physrevlett.118.158103 · Cited by 44 · see citing works
Complexity measures from interaction structures
Kahle, T., Olbrich, E., Jost, J., & Ay, N. (2009) · Physical Review E · doi:10.1103/physreve.79.026201 · Cited by 40 · see citing works
Stefan Thurner, A., Corominas‐Murtra, B., & Hanel, R. (2017) · Physical review. E · doi:10.1103/physreve.96.032124 · Cited by 38 · see citing works
Self-citationBeyond integrated information: A taxonomy of information dynamics phenomenaPreprint
Mediano, P. A. M., Rosas, F. E., Carhart-Harris, R. L., Seth, A. K., & Barrett, A. B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.02297 · Cited by 34 · see citing works
The minimum information principle and its application to neural code analysis
Globerson, A., Stark, E., Vaadia, E., & Tishby, N. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0806782106 · Cited by 32 · see citing works
Sequence Alignment, Mutual Information, and Dissimilarity Measures for Constructing Phylogenies
Penner, O., Grassberger, P., & Paczuski, M. (2011) · PLoS ONE · doi:10.1371/journal.pone.0014373 · Cited by 26 · see citing works
Nonequilibrium correlations in minimal dynamical models of polymer copying
Poulton, J. M., ten Wolde, P. R., & Ouldridge, T. E. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1808775116 · Cited by 23 · see citing works
Towards a mathematical theory of meaningful communication
Corominas‐Murtra, B., Fortuny, J., & Solé, R. V. (2014) · Scientific Reports · doi:10.1038/srep04587 · Cited by 18 · see citing works
Self-citationThermodynamics of Duplication Thresholds in Synthetic Protocell Systems
Corominas-Murtra, B. (2019) · Life · doi:10.3390/life9010009 · Cited by 13 · see citing works
Self-citationA Representation Approach for Relative Entropy Minimization with Expectation Constraints
Koyejo, S., & Ghosh, J. (2013) · Cited by 11 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Fundamental constraints to the logic of living systems
Solé, R., Kempes, C. P., Corominas-Murtra, B., De Domenico, M., Kolchinsky, A., Lachmann, M., Libby, E., Saavedra, S., Smith, E., & Wolpert, D. (2024) · Interface Focus · doi:10.1098/rsfs.2024.0010 · Cited by 34 · see citing works
Self-citationBehavior of information flow near criticality
Meijers, M., Ito, S., & ten Wolde, P. R. (2021) · Physical review. E · doi:10.1103/physreve.103.l010102 · Cited by 18 · see citing works
Direct and Indirect Effects—An Information Theoretic Perspective
Schamberg, G., Chapman, W. E., Xie, S., & Coleman, T. P. (2020) · Entropy · doi:10.3390/e22080854 · Cited by 9 · see citing works
A Survey of Text Deduplication: From Syntactic Matching to Semantic Understanding
Kim, J., & Park, Y. (2026) · IEEE Access · doi:10.1109/access.2026.3658439 · Cited by 2 · see citing works
- Authors
- Kolchinsky, A., & Corominas-Murtra, B.
- Affiliations
- Santa Fe Institute (US)
- Institute of Science and Technology Austria, Am Campus 1, 3400 Klosterneuburg, Austria
- Venue
- Journal of The Royal Society Interface
- Publisher
- The Royal Society
- Published
- 2020-01
- Volume
- 17
- Issue
- 162
- Pages
- 20190623
- ISSN
- 1742-5662 (ISSN-L)
- 1742-5689
- PMC
- PMC7014805
- OpenAlex
- W2950348941
- Cited by
- 5
- References
- 65
- Citations by year
- 2026: 1 · 2024: 1 · 2021: 1 · 2020: 2
- FWCI
- 0.32
- Type
- journal article
- Language
- en
- Open access
- green
- europepmc.org
- Locations
- Journal of The Royal Society Interface (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Advanced Thermodynamics and Statistical Mechanics
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Complex Systems and Dynamics
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy and History of Science
- Keywords
- information transmission, DNA replication, mutational bias, amino acid substitution rates, information theory
- MeSH terms
- Copying Processes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
