CoChat

A measure for intrinsic information

Open access
Barbosa, Leonardo S., Marshall, William, Streipert, Sabrina, Albantakis, Larissa, Tononi, Giulio · 2020 · Scientific Reports

The DOI matches its Crossref record; 21 of the 23 references it cites were found in a registry or an index.

Abstract

We introduce an information measure that reflects the intrinsic perspective of a receiver or sender of a single symbol, who has no access to the communication channel and its source or target. The measure satisfies three desired properties—causality, specificity, intrinsicality—and is shown to be unique. Causality means that symbols must be transmitted with probability greater than chance. Specificity means that information must be transmitted by an individual symbol. Intrinsicality means that a symbol must be taken as such and cannot be decomposed into signal and noise. It follows that the intrinsic information carried by a specific symbol increases if the repertoire of symbols increases without noise (expansion) and decreases if it does so without signal (dilution). An optimal balance between expansion and dilution is relevant for systems whose elements must assess their inputs and outputs from the intrinsic perspective, such as neurons in a network.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41598-020-75943-4

Cites

21 of 21 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Information from the intrinsic perspective. Different channels constituted of wires carrying a physical signal with two states (for example, high or low voltage); the brackets pointing up represents the encoding of the symbols into the physical signal(s) and the brackets pointing down represent the decoding. (A) Extrinsic perspective: encoders and decoders can be designed to find the alphabet that maximizes information transmission (channel capacity), and this alphabet is shared by both sender and received. (B) Intrinsic perspective: has fixed encoders and decoders for sending and receiving information. The remaining panels depict three enclosures where your only interface with the environment is an input channel (top) connected to a screen and an output channel (bottom) connected to a keyboard. (C) a channel with one noiseless wire, conveying one bit that is encoded (decoded) into a symbol (a dot or a dash); (D, E) channels with eight wires each and an encoder (decoder) that converts the eight bits (a byte) into one out of 256 characters (or vice-versa). In (D), all eight wires are noiseless. In (E), the first wire is noiseless while the other seven wires are fully noisy (indicated by the lightning signs) so they rarely convey the original signal. In all cases the keyboard is connected to noiseless output wires (bottom) and has as many keys as possible input symbols. How should you rank the three enclosures with respect to the information you could relay?
Figure 1 Information from the intrinsic perspective. Different channels constituted of wires carrying a physical signal with two states (for example, high or low voltage); the brackets pointing up represents the encoding of the symbols into the physical signal(s) and the brackets pointing down represent the decoding. (A) Extrinsic perspective: encoders and decoders can be designed to find the alphabet that maximizes information transmission (channel capacity), and this alphabet is shared by both sender and received. (B) Intrinsic perspective: has fixed encoders and decoders for sending and receiving information. The remaining panels depict three enclosures where your only interface with the environment is an input channel (top) connected to a screen and an output channel (bottom) connected to a keyboard. (C) a channel with one noiseless wire, conveying one bit that is encoded (decoded) into a symbol (a dot or a dash); (D, E) channels with eight wires each and an encoder (decoder) that converts the eight bits (a byte) into one out of 256 characters (or vice-versa). In (D), all eight wires are noiseless. In (E), the first wire is noiseless while the other seven wires are fully noisy (indicated by the lightning signs) so they rarely convey the original signal. In all cases the keyboard is connected to noiseless output wires (bottom) and has as many keys as possible input symbols. How should you rank the three enclosures with respect to the information you could relay?CC BY · PubMed Central
Figure 2 Intrinsic information for different levels of noise. To facilitate comparison between input channels and to provide a read-out of the enclosure, all output channels are eight-bit noiseless wires that reproduce exactly what was sent. For bit-sized input channels, eight key presses are accumulated in the encoder (bracket pointing up) to reproduce one output character every eight bits, representing a common read-out at the target (bracket pointing down). The colors represent whether the output character is correct (green) or incorrect (red). (A) Similar to Fig. 1C, a one-wire input channel that is noiseless, meaning it transmits the original signal with probability one (noiseless bit-channel). (B) Similar to Fig. 1D, an eight-wire input channel with seven additional wires that are also noiseless and transmit the original signal with probability one (noiseless byte-channel). (C) Similar to Fig. 1E, an input channel with one noiseless wire and seven fully noisy wires that transmit the original signal with probability half (noisy byte-channel). Both the intrinsic difference measure (ID) and the KL measures attribute the same amount of information to the noiseless bit-channel and the noiseless byte-channel (1 and 8, respectively). However, the KL measure attributes 1 bit to the noisy byte-channel, whereas the ID measure attributes to it close to 0 ibits. This is because the ID reflects the amount of information available from the perspective of the receiver in the sealed enclosure (intrinsic information), whereas KL reflects the information available from the extrinsic perspective of a channel designer, which could be extracted using error correcting codes. (D) A byte-channel with additional noisy (iid) wires that transmit the original signal with decreasing probability ri=s,i∈{2,…,8} (or increasing levels of noise 1/s), from s=1 (noiseless) to s=1/2 (fully noisy). Due to the additivity property, the KL measure (dashed line) will always attribute at least 1 bit of information to the noisy-byte channel. By contrast, ID starts from nearly 8 ibits and decreases to nearly 0 ibits. (E) Due to the smoothness of ID, there is a level of noise (s≈0.78) for which a byte-channel conveys 1 ibit. On a typical run with eight transmissions such a byte-channel outputs one correct byte-size character, similar to the noiseless bit-channel.
Figure 2 Intrinsic information for different levels of noise. To facilitate comparison between input channels and to provide a read-out of the enclosure, all output channels are eight-bit noiseless wires that reproduce exactly what was sent. For bit-sized input channels, eight key presses are accumulated in the encoder (bracket pointing up) to reproduce one output character every eight bits, representing a common read-out at the target (bracket pointing down). The colors represent whether the output character is correct (green) or incorrect (red). (A) Similar to Fig. 1C, a one-wire input channel that is noiseless, meaning it transmits the original signal with probability one (noiseless bit-channel). (B) Similar to Fig. 1D, an eight-wire input channel with seven additional wires that are also noiseless and transmit the original signal with probability one (noiseless byte-channel). (C) Similar to Fig. 1E, an input channel with one noiseless wire and seven fully noisy wires that transmit the original signal with probability half (noisy byte-channel). Both the intrinsic difference measure (ID) and the KL measures attribute the same amount of information to the noiseless bit-channel and the noiseless byte-channel (1 and 8, respectively). However, the KL measure attributes 1 bit to the noisy byte-channel, whereas the ID measure attributes to it close to 0 ibits. This is because the ID reflects the amount of information available from the perspective of the receiver in the sealed enclosure (intrinsic information), whereas KL reflects the information available from the extrinsic perspective of a channel designer, which could be extracted using error correcting codes. (D) A byte-channel with additional noisy (iid) wires that transmit the original signal with decreasing probability ri=s,i∈{2,…,8} (or increasing levels of noise 1/s), from s=1 (noiseless) to s=1/2 (fully noisy). Due to the additivity property, the KL measure (dashed line) will always attribute at least 1 bit of information to the noisy-byte channel. By contrast, ID starts from nearly 8 ibits and decreases to nearly 0 ibits. (E) Due to the smoothness of ID, there is a level of noise (s≈0.78) for which a byte-channel conveys 1 ibit. On a typical run with eight transmissions such a byte-channel outputs one correct byte-size character, similar to the noiseless bit-channel.CC BY · PubMed Central
Figure 3 Intrinsic information for different channel sizes. As before, all output channels are eight-bit noiseless wires that reproduce exactly what was sent (common read-out), and the colors represent whether the output character is correct or not. (A) A balance between expansion and dilution can be seen for an increasing number N of wires, given a fixed level of noise per wire (the probability of receiving the correct signal in each wire is r=0.88). As the informativeness log(pα/qα)=Nlog(r/2) grows linearly and the selectivity pα=r^N decreases exponentially, their product ID (black dots) peaks at N=8 wires. (B) On a typical run with 8 transmissions, a channel with too few input wires (one) does not output any correct byte-size character; (C) a channel with an optimal number of wires (eight) maximizes ID and outputs three correct ASCII characters; (D) a channel with too many input wires (16), could transmit thousands of different Kanji characters (for instance using unicode), but ID is lower and the enclosure outputs only two correct ASCII characters after eight transmissions.
Figure 3 Intrinsic information for different channel sizes. As before, all output channels are eight-bit noiseless wires that reproduce exactly what was sent (common read-out), and the colors represent whether the output character is correct or not. (A) A balance between expansion and dilution can be seen for an increasing number N of wires, given a fixed level of noise per wire (the probability of receiving the correct signal in each wire is r=0.88). As the informativeness log(pα/qα)=Nlog(r/2) grows linearly and the selectivity pα=r^N decreases exponentially, their product ID (black dots) peaks at N=8 wires. (B) On a typical run with 8 transmissions, a channel with too few input wires (one) does not output any correct byte-size character; (C) a channel with an optimal number of wires (eight) maximizes ID and outputs three correct ASCII characters; (D) a channel with too many input wires (16), could transmit thousands of different Kanji characters (for instance using unicode), but ID is lower and the enclosure outputs only two correct ASCII characters after eight transmissions.CC BY · PubMed Central
Figure 4 Information from the intrinsic perspective of elements in a system. (A) Probability of firing of one simplified neuron, given the input states for three different input sizes N and three different levels of noise (or value of parameter t). (B) Information measured by the intrinsic difference (ID) and KL divergence (y-axis) for groups of simplified neurons with varying numbers of outputs N (x-axis). The number of neurons in the group is always the same as the number of neurons in the output. Each panel shows the behavior of ID and KL for different levels of noise. Notice that each panel has two y-axis, one for each measure, since they are not commensurable (bits and ibits) and the goal is not to compare their absolute values, but their qualitative behavior: While KL always reaches its maximum values when the neurons are connected to all available outputs, ID shows a peak that changes with the level of noise. (C) For the noise level t=1, neurons with one output convey 0.72 ibits; (D) the maximum of ID is reached by neurons with 8 outputs (2.90 ibits), again showing a balance between expansion and dilution; (E) neurons with 16 outputs have lower values of ID (2.10 ibits) than those with eight outputs.
Figure 4 Information from the intrinsic perspective of elements in a system. (A) Probability of firing of one simplified neuron, given the input states for three different input sizes N and three different levels of noise (or value of parameter t). (B) Information measured by the intrinsic difference (ID) and KL divergence (y-axis) for groups of simplified neurons with varying numbers of outputs N (x-axis). The number of neurons in the group is always the same as the number of neurons in the output. Each panel shows the behavior of ID and KL for different levels of noise. Notice that each panel has two y-axis, one for each measure, since they are not commensurable (bits and ibits) and the goal is not to compare their absolute values, but their qualitative behavior: While KL always reaches its maximum values when the neurons are connected to all available outputs, ID shows a peak that changes with the level of noise. (C) For the noise level t=1, neurons with one output convey 0.72 ibits; (D) the maximum of ID is reached by neurons with 8 outputs (2.90 ibits), again showing a balance between expansion and dilution; (E) neurons with 16 outputs have lower values of ID (2.10 ibits) than those with eight outputs.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 21 references

Cited by

See all 56

Metadata

Authors
Barbosa, L. S., Marshall, W., Streipert, S., Albantakis, L., & Tononi, G.
Affiliations
University of Wisconsin–Madison (US)
Corresponding author
Tononi, Giulio
Venue
Scientific Reports
Published
2020-11-02
Volume
10
Issue
1
Pages
18803-18803
ISSN
2045-2322 (ISSN-L)
OpenAlex
W3095411193
Cited by
42
References
23
Citations by year
2026: 5 · 2025: 3 · 2024: 10 · 2023: 10 · 2022: 8 · 2021: 4 · 2020: 2
FWCI
2.54
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
Templeton World Charity Foundation (#TWCF0216)
Topics
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Complex Systems and Dynamics
  • Physical Sciences › Computer Science › Artificial Intelligence › Diverse Interdisciplinary Research Studies
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Advanced Thermodynamics and Statistical Mechanics
Keywords
intrinsic information, information measures, causality, specificity, information theory, neural information processing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A measure for intrinsic information. https://app.cochat.ai/papers/10.1038/s41598-020-75943-4
The paper
Barbosa, L. S., Marshall, W., Streipert, S., Albantakis, L., & Tononi, G. (2020). A measure for intrinsic information. Scientific Reports. https://doi.org/10.1038/s41598-020-75943-4

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.