Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 38 of the 40 references it cites were found in a registry or an index.

Integrated Information in Discrete Dynamical Systems: Motivation and Theoretical Framework

Open access
Balduzzi, David, Tononi, Giulio · 2008 · PLoS Computational Biology

Abstract

This paper introduces a time- and state-dependent measure of integrated information, phi, which captures the repertoire of causal states available to a system as a whole. Specifically, phi quantifies how much information is generated (uncertainty is reduced) when a system enters a particular state through causal interactions among its elements, above and beyond the information generated independently by its parts. Such mathematical characterization is motivated by the observation that integrated information captures two key phenomenological properties of consciousness: (i) there is a large repertoire of conscious experiences so that, when one particular experience occurs, it generates a large amount of information by ruling out all the others; and (ii) this information is integrated, in that each experience appears as a whole that cannot be decomposed into independent parts. This paper extends previous work on stationary systems and applies integrated information to discrete networks as a function of their dynamics and causal architecture. An analysis of basic examples indicates the following: (i) phi varies depending on the state entered by a network, being higher if active and inactive elements are balanced and lower if the network is inactive or hyperactive. (ii) phi varies for systems with identical or similar surface dynamics depending on the underlying causal architecture, being low for systems that merely copy or replay activity states. (iii) phi varies as a function of network architecture. High phi values can be obtained by architectures that conjoin functional specialization with functional integration. Strictly modular and homogeneous systems cannot generate high phi because the former lack integration, whereas the latter lack information. Feedforward and lattice architectures are capable of generating high phi but are inefficient. (iv) In Hopfield networks, phi is low for attractor states and neutral states, but increases if the networks are optimized to achieve tension between local and global interactions. These basic examples appear to match well against neurobiological evidence concerning the neural substrates of consciousness. More generally, phi appears to be a useful metric to characterize the capacity of any physical system to integrate information.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pcbi.1000091

Cites

38 of 38 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Effective information generated by entering a particular state. A system of three connected AND-gates transitions from state x 0 = 110 at time zero to x 1 = 001 at time one. The a priori repertoire is the uniform distribution on the 8 possible outputs of the elements of the system. The causal architecture of the system specifies that state 110 is the unique cause of x 1, so the a posteriori repertoire (shown in cyan) assigns probability 1 to state 110 and 0 to all other states. Effective information generated by the system transitioning to x 1 is 3 bits.
Figure 1 Effective information generated by entering a particular state. A system of three connected AND-gates transitions from state x 0 = 110 at time zero to x 1 = 001 at time one. The a priori repertoire is the uniform distribution on the 8 possible outputs of the elements of the system. The causal architecture of the system specifies that state 110 is the unique cause of x 1, so the a posteriori repertoire (shown in cyan) assigns probability 1 to state 110 and 0 to all other states. Effective information generated by the system transitioning to x 1 is 3 bits.CC BY · PubMed Central
Figure 2 Effective information: a few examples. Each panel depicts a different system, which has entered a particular state. The a priori and a posteriori repertoires are shown and effective information is measured. (A) is a simple system of two elements that copy each other's previous outputs (a couple). Effective information is 2 bits, less than for the system in Figure 1 since the repertoire of outputs is smaller. (B) shows the AND-gate system of Figure 1 entering the state 000. This state is less informative than 001 since the a posteriori repertoire specified by the system includes four perturbations; effective information is reduced to 1 bit. The systems in (C) and (D) generate no effective information. In (C) the elements always fire regardless of their inputs, corresponding to an inescapable fixed point. In (D) the elements fire or are silent at random, so that the prior state is irrelevant. In both cases the a posteriori repertoire is the maximum entropy distribution since no alternatives have been ruled out, so effective information is zero.
Figure 2 Effective information: a few examples. Each panel depicts a different system, which has entered a particular state. The a priori and a posteriori repertoires are shown and effective information is measured. (A) is a simple system of two elements that copy each other's previous outputs (a couple). Effective information is 2 bits, less than for the system in Figure 1 since the repertoire of outputs is smaller. (B) shows the AND-gate system of Figure 1 entering the state 000. This state is less informative than 001 since the a posteriori repertoire specified by the system includes four perturbations; effective information is reduced to 1 bit. The systems in (C) and (D) generate no effective information. In (C) the elements always fire regardless of their inputs, corresponding to an inescapable fixed point. In (D) the elements fire or are silent at random, so that the prior state is irrelevant. In both cases the a posteriori repertoire is the maximum entropy distribution since no alternatives have been ruled out, so effective information is zero.CC BY · PubMed Central
Figure 3 Integrated information for a system of two disjoint couples. The panels analyze the same system of two disjoint couples from three different perspectives. The interactions in the system are displayed in cyan. Those interactions that occur within a part are shown in red, and those between parts are in dark blue. (A) computes effective information for the entire system X, finding it to be 4 bits. (B) computes effective information generated by each of the couples independently and then computes integrated information φ(x 1), finding it to be 0 bits since the two couples do not interact. Notice that the combined a posteriori repertoire of the parts coincides with the a posteriori repertoire of the system; the parts account for all the interactions within X. (C) considers a partition of the system other than the minimum information partition. Since is not isolated it cannot account for the effect of interactions with internally; they are treated as extrinsic noise and result in specifying a maximum entropy a posteriori repertoire. Effective information generated across the partition is 4 bits.
Figure 3 Integrated information for a system of two disjoint couples. The panels analyze the same system of two disjoint couples from three different perspectives. The interactions in the system are displayed in cyan. Those interactions that occur within a part are shown in red, and those between parts are in dark blue. (A) computes effective information for the entire system X, finding it to be 4 bits. (B) computes effective information generated by each of the couples independently and then computes integrated information φ(x 1), finding it to be 0 bits since the two couples do not interact. Notice that the combined a posteriori repertoire of the parts coincides with the a posteriori repertoire of the system; the parts account for all the interactions within X. (C) considers a partition of the system other than the minimum information partition. Since is not isolated it cannot account for the effect of interactions with internally; they are treated as extrinsic noise and result in specifying a maximum entropy a posteriori repertoire. Effective information generated across the partition is 4 bits.CC BY · PubMed Central
Figure 4 Effective information generated across the minimum information partition. (A) depicts the interactions within the system that are quantified by effective information of the entire system. (B) disentangles the interactions, showing interactions within parts in red, and interactions between parts in dark blue. (C) is a schematic of the relationship between the repertoires specified by the system and the parts. Effective information, represented by the arrows, is the entropy of a lower repertoire relative to an upper one. φ(x 1) is the entropy of the a posteriori repertoire of the system relative to the combined a posteriori repertoire of the minimal parts.
Figure 4 Effective information generated across the minimum information partition. (A) depicts the interactions within the system that are quantified by effective information of the entire system. (B) disentangles the interactions, showing interactions within parts in red, and interactions between parts in dark blue. (C) is a schematic of the relationship between the repertoires specified by the system and the parts. Effective information, represented by the arrows, is the entropy of a lower repertoire relative to an upper one. φ(x 1) is the entropy of the a posteriori repertoire of the system relative to the combined a posteriori repertoire of the minimal parts.CC BY · PubMed Central
Figure 5 Decomposing systems into overlapping complexes. In this example elements are parity gates: they fire if they receive an odd number of spikes. Links without arrows are bidirectional. The system is decomposed into three of its complexes, shown in shades of gray. Observe that: i) complexes can overlap; ii) a complex can interact causally with elements not part of it; iii) groups of elements with identical architectures generate different amounts of integrated information, depending on their ports-in and ports-out (compare subset A, the dark gray filled-in circle, with subset B, the right-hand circle).
Figure 5 Decomposing systems into overlapping complexes. In this example elements are parity gates: they fire if they receive an odd number of spikes. Links without arrows are bidirectional. The system is decomposed into three of its complexes, shown in shades of gray. Observe that: i) complexes can overlap; ii) a complex can interact causally with elements not part of it; iii) groups of elements with identical architectures generate different amounts of integrated information, depending on their ports-in and ports-out (compare subset A, the dark gray filled-in circle, with subset B, the right-hand circle).CC BY · PubMed Central
Figure 6 Elements driven by a complex do not contribute to integrated information. The system is constructed using the AND-gate system of Figure 1, with the addition of three elements copying the inner triple. The AND-triple forms a main complex, as do the couples. However, the entire system generates no integrated information and does not form a complex, since X generates no information over and above that generated by subset A.
Figure 6 Elements driven by a complex do not contribute to integrated information. The system is constructed using the AND-gate system of Figure 1, with the addition of three elements copying the inner triple. The AND-triple forms a main complex, as do the couples. However, the entire system generates no integrated information and does not form a complex, since X generates no information over and above that generated by subset A.CC BY · PubMed Central
Figure 7 Analyzing systems in terms of elementary components. (A) and (C) show systems that on the surface appear to generate a large amount of integrated information. The units in (A) have a repertoire of 2n outputs, with the bottom unit copying the top. Integrated information is n bits. Analyzing the internal structure of the system in (B) we find n disjoint couples, each integrating 1 bit of information; the entire system however is not integrated. (C) shows a system of binary units. The top unit receives inputs from 8 other units and performs an AND-gate like operation, firing if and only if all 8 inputs are spikes. Increasing the number of inputs appears to easily increase φ without limit. (D) examines a possible implementation of the internal architecture of the top unit using binary AND-gates. The architecture has a bottleneck, shown in red, so that φ = 1 bit no matter the number of input units.
Figure 7 Analyzing systems in terms of elementary components. (A) and (C) show systems that on the surface appear to generate a large amount of integrated information. The units in (A) have a repertoire of 2n outputs, with the bottom unit copying the top. Integrated information is n bits. Analyzing the internal structure of the system in (B) we find n disjoint couples, each integrating 1 bit of information; the entire system however is not integrated. (C) shows a system of binary units. The top unit receives inputs from 8 other units and performs an AND-gate like operation, firing if and only if all 8 inputs are spikes. Increasing the number of inputs appears to easily increase φ without limit. (D) examines a possible implementation of the internal architecture of the top unit using binary AND-gates. The architecture has a bottleneck, shown in red, so that φ = 1 bit no matter the number of input units.CC BY · PubMed Central
Figure 8 Integrated information and extrinsic inputs. The gray box represents a main complex. Red arrows are input from the environment. Black arrows depict strong feedforward connections; gray arrows are weaker modulatory connections. The black zig-zag represents the bulk of the main complex. The current state of row Ra is determined by extrinsic inputs, which are treated as extrinsic noise. However the current state of row Rb together with the feedforward architecture of the system together specify the prior state of Ra, so that the system is able to distinguish extrinsic inputs once they have caused an interaction between elements within the main complex. Similarly row Rc specifies higher-order invariants in the prior state of row Rb.
Figure 8 Integrated information and extrinsic inputs. The gray box represents a main complex. Red arrows are input from the environment. Black arrows depict strong feedforward connections; gray arrows are weaker modulatory connections. The black zig-zag represents the bulk of the main complex. The current state of row Ra is determined by extrinsic inputs, which are treated as extrinsic noise. However the current state of row Rb together with the feedforward architecture of the system together specify the prior state of Ra, so that the system is able to distinguish extrinsic inputs once they have caused an interaction between elements within the main complex. Similarly row Rc specifies higher-order invariants in the prior state of row Rb.CC BY · PubMed Central
Figure 9 Integrated information peaks in balanced states. (A–D) show four discrete systems; lines represent bi-directional connections. Elements fire if they receive two or more spikes. The graph shows integrated information as a function of the number of elements firing. Integrated information is computed by averaging over all states with a particular number of elements firing. Integrated information is low for hyperactive and inactive states when too many or too few elements are firing, and high for balanced states lying between the two extremes. Note that in (A) the value of φ for 7 elements firing is undefined, since no state with seven elements firing is possible given the causal architecture.
Figure 9 Integrated information peaks in balanced states. (A–D) show four discrete systems; lines represent bi-directional connections. Elements fire if they receive two or more spikes. The graph shows integrated information as a function of the number of elements firing. Integrated information is computed by averaging over all states with a particular number of elements firing. Integrated information is low for hyperactive and inactive states when too many or too few elements are firing, and high for balanced states lying between the two extremes. Note that in (A) the value of φ for 7 elements firing is undefined, since no state with seven elements firing is possible given the causal architecture.CC BY · PubMed Central
Figure 10 Bistable dynamics. The system has connectivity as in Figure 9C, with altered element behavior. If an element receives less than two spikes it fires with probability .15. If it receives 2 or more spikes it fires with certainty, unless more than half the elements fired in the two times step prior, in which case all elements are silent. The graph plots φ and the percentage of elements firing, as the system is run for 120 time steps. The system implements a bistable dynamics, and is unable to sustain high values of φ.
Figure 10 Bistable dynamics. The system has connectivity as in Figure 9C, with altered element behavior. If an element receives less than two spikes it fires with probability .15. If it receives 2 or more spikes it fires with certainty, unless more than half the elements fired in the two times step prior, in which case all elements are silent. The graph plots φ and the percentage of elements firing, as the system is run for 120 time steps. The system implements a bistable dynamics, and is unable to sustain high values of φ.CC BY · PubMed Central
Figure 11 System cycling (via binary counting) through 16 firing patterns. The system cycles through the firing patterns 0000, 0001, 0010, …, 1101, 1110, 1111; counting in binary from 0 to 15 and repeating. (A) and (B) show the system, implemented with interacting memoryless elements, in two different states. (C) shows a system with identical dynamics, implemented using four elements independently replaying a list of instructions. Since there are no causal interactions, the replay generates no integrated information, in contrast to the memoryless system.
Figure 11 System cycling (via binary counting) through 16 firing patterns. The system cycles through the firing patterns 0000, 0001, 0010, …, 1101, 1110, 1111; counting in binary from 0 to 15 and repeating. (A) and (B) show the system, implemented with interacting memoryless elements, in two different states. (C) shows a system with identical dynamics, implemented using four elements independently replaying a list of instructions. Since there are no causal interactions, the replay generates no integrated information, in contrast to the memoryless system.CC BY · PubMed Central
Figure 12 Optimized network of AND-gates. The network is optimized to generate high integrated information in a single state, that shown. Each element implements an AND-gate.
Figure 12 Optimized network of AND-gates. The network is optimized to generate high integrated information in a single state, that shown. Each element implements an AND-gate.CC BY · PubMed Central
Figure 13 Integrated information in a strongly modular network. The system is composed of three four-element modules. The elements fire if they receive two or more spikes. The entire system forms a complex (light gray) with φ(x 1) = .7 bits; however, the architecture is strongly modular so that the main complexes (dark gray) are the modules, each generating φ = 1.2 bits of integrated information across the total partition, more than the system as a whole.
Figure 13 Integrated information in a strongly modular network. The system is composed of three four-element modules. The elements fire if they receive two or more spikes. The entire system forms a complex (light gray) with φ(x 1) = .7 bits; however, the architecture is strongly modular so that the main complexes (dark gray) are the modules, each generating φ = 1.2 bits of integrated information across the total partition, more than the system as a whole.CC BY · PubMed Central
Figure 14 Integrated information in homogeneous systems. The systems have all-to-all connectivity, including self-connections. (A) shows a parity system: each element fires if it receives an odd number of spikes and is silent otherwise. The MIP is the total partition and integrated information is 1 bit. (B) shows a majority-rule system where elements fire if they receive three or more spikes. The MIP is the bipartition shown. The a posteriori repertoire specified by each part contains three perturbations, with weights .09, .09, and .8 respectively. The combined a posteriori repertoire contains 9 perturbations of varying weights, as shown. The a posteriori repertoire of the system contains 5 equally weighted perturbations. Integrated information is .79 bits.
Figure 14 Integrated information in homogeneous systems. The systems have all-to-all connectivity, including self-connections. (A) shows a parity system: each element fires if it receives an odd number of spikes and is silent otherwise. The MIP is the total partition and integrated information is 1 bit. (B) shows a majority-rule system where elements fire if they receive three or more spikes. The MIP is the bipartition shown. The a posteriori repertoire specified by each part contains three perturbations, with weights .09, .09, and .8 respectively. The combined a posteriori repertoire contains 9 perturbations of varying weights, as shown. The a posteriori repertoire of the system contains 5 equally weighted perturbations. Integrated information is .79 bits.CC BY · PubMed Central
Figure 15 Integrated information for lattice architectures. (A) is an n×n XOR-lattice. The minimum information partition is given by a vertical or horizontal midpartition. Integrated information is n bits; and so can be increased without limit by scaling up the (highly inefficient) architecture. (B) and (C) show integrated information for a Game of Life grid in two different configurations. Cells in the grid are either ON or OFF. Each cell has 8 neighbors, the grid is assumed to wrap around to form a torus. A cell that is OFF switches to ON in the next time step if exactly 3 of its neighbors are ON. An ON cell remains ON if two or three neighbors are ON; otherwise it switches to OFF. (D) shows long-range connections short-circuiting the perimeter bottleneck intrinsic to lattice architectures.
Figure 15 Integrated information for lattice architectures. (A) is an n×n XOR-lattice. The minimum information partition is given by a vertical or horizontal midpartition. Integrated information is n bits; and so can be increased without limit by scaling up the (highly inefficient) architecture. (B) and (C) show integrated information for a Game of Life grid in two different configurations. Cells in the grid are either ON or OFF. Each cell has 8 neighbors, the grid is assumed to wrap around to form a torus. A cell that is OFF switches to ON in the next time step if exactly 3 of its neighbors are ON. An ON cell remains ON if two or three neighbors are ON; otherwise it switches to OFF. (D) shows long-range connections short-circuiting the perimeter bottleneck intrinsic to lattice architectures.CC BY · PubMed Central
Figure 16 Integrated information in feedforward networks. (A) shows a tree-like hierarchical feedforward network. Effective information from the sensory sheet below the grid (not shown) is high, 1.6 bits, but the ability of the network to integrate information is limited by the bottleneck at the grandmother cell n 1. (B) and (C) show the network with an additional grandmother cell (the network is flattened out, so the second grandmother appears at the bottom). A redundant grandmother results in zero integrated information, whereas if the grandmother is not redundant φ increase to .5 bits. (D) and (E) depict a grid-like feedforward architecture that does not suffer from the bottleneck of (A). Integrated information increases with network size.
Figure 16 Integrated information in feedforward networks. (A) shows a tree-like hierarchical feedforward network. Effective information from the sensory sheet below the grid (not shown) is high, 1.6 bits, but the ability of the network to integrate information is limited by the bottleneck at the grandmother cell n 1. (B) and (C) show the network with an additional grandmother cell (the network is flattened out, so the second grandmother appears at the bottom). A redundant grandmother results in zero integrated information, whereas if the grandmother is not redundant φ increase to .5 bits. (D) and (E) depict a grid-like feedforward architecture that does not suffer from the bottleneck of (A). Integrated information increases with network size.CC BY · PubMed Central
Figure 17 Integrated information in a Hopfield network. (A-D) show a sample run of a Hopfield network with 8 elements and all-to-all connectivity. The network has embedded attractors 11110000, 11001100, 10101010 and their mirror images. A sample run is depicted at T = .45 and initial state 11111111. (E-G) show integrated information as a function of temperature (computed using bipartitions) for the corresponding states; the colored enclosures are matched with the graphs. The system forms a complex for low temperatures (blue), but breaks down for higher temperatures (red), so that subcomplexes arise.
Figure 17 Integrated information in a Hopfield network. (A-D) show a sample run of a Hopfield network with 8 elements and all-to-all connectivity. The network has embedded attractors 11110000, 11001100, 10101010 and their mirror images. A sample run is depicted at T = .45 and initial state 11111111. (E-G) show integrated information as a function of temperature (computed using bipartitions) for the corresponding states; the colored enclosures are matched with the graphs. The system forms a complex for low temperatures (blue), but breaks down for higher temperatures (red), so that subcomplexes arise.CC BY · PubMed Central
Figure 18 A functionally integrated and functionally specialized probabilistic network can sustain high φ. (A) shows a functionally integrated and functionally specialized network; black arrows represent connections of weight ≥.5 and red arrows connections with weight ≤−.5. Weaker connections are not shown to reduce clutter; see Text S1, section 13. The elements operate according to the rules of a Hopfield network with T = .35. The network is initialized with a random firing patter and allowed to run for 800 time steps. (B) shows φ for each firing pattern that occurs during the run.
Figure 18 A functionally integrated and functionally specialized probabilistic network can sustain high φ. (A) shows a functionally integrated and functionally specialized network; black arrows represent connections of weight ≥.5 and red arrows connections with weight ≤−.5. Weaker connections are not shown to reduce clutter; see Text S1, section 13. The elements operate according to the rules of a Hopfield network with T = .35. The network is initialized with a random firing patter and allowed to run for 800 time steps. (B) shows φ for each firing pattern that occurs during the run.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Cited by

See all 425

Metadata

Authors
Balduzzi, D., & Tononi, G.
Affiliations
University of Wisconsin–Madison (US)
Corresponding author
Tononi, Giulio
Venue
PLoS Computational Biology
Published
2008-06-13
Volume
4
Issue
6
Pages
e1000091
ISSN
  • 1553-734X (ISSN-L)
  • 1553-7358
OpenAlex
W2108714018
Cited by
419
References
40
Citations by year
2026: 19 · 2025: 27 · 2024: 20 · 2023: 20 · 2022: 23 · 2021: 28 · 2020: 22 · 2019: 21 · 2018: 40 · 2017: 29 · 2016: 27 · 2015: 24 · 2014: 22 · 2013: 23 · 2012: 19
FWCI
24.64
Type
journal article
Language
en
Open access
Locations
  • PLoS Computational Biology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • MPG.PuRe (Max Planck Society) (submittedVersion)
Funders
  • James S. McDonnell Foundation
  • National Institutes of Health (DP1OD000579)
Topics
  • Social Sciences › Arts and Humanities › Philosophy › Philosophy and Theoretical Science
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
Keywords
integrated information, phi, consciousness, discrete dynamical systems, causal architecture, network architecture, Hopfield neural network, attractor states, functional integration, functional specialization, information integration
MeSH terms
Animals, Brain, Cognition, Computer Simulation, Consciousness, Humans, Models, Neurological, Nerve Net, Signal Processing, Computer-Assisted, Information Storage and Retrieval, Systems Integration

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.