CoChat

Untangling Brain-Wide Dynamics in Consciousness by Cross-Embedding

Open access
Tajima, Satohiro, Yanagawa, Toru, Fujii, Naotaka, Toyoizumi, Taro · 2015 · PLOS Computational Biology

The DOI matches its Crossref record; 78 of the 78 references it cites were found in a registry or an index.

Abstract

Brain-wide interactions generating complex neural dynamics are considered crucial for emergent cognitive functions. However, the irreducible nature of nonlinear and high-dimensional dynamical interactions challenges conventional reductionist approaches. We introduce a model-free method, based on embedding theorems in nonlinear state-space reconstruction, that permits a simultaneous characterization of complexity in local dynamics, directed interactions between brain areas, and how the complexity is produced by the interactions. We demonstrate this method in large-scale electrophysiological recordings from awake and anesthetized monkeys. The cross-embedding method captures structured interaction underlying cortex-wide dynamics that may be missed by conventional correlation-based analysis, demonstrating a critical role of time-series analysis in characterizing brain state. The method reveals a consciousness-related hierarchy of cortical areas, where dynamical complexity increases along with cross-area information flow. These findings demonstrate the advantages of the cross-embedding method in deciphering large-scale and heterogeneous neuronal systems, suggesting a crucial contribution by sensory-frontoparietal interactions to the emergence of complex brain dynamics during consciousness.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pcbi.1004537

Cites

78 of 78 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig 1 Cross-embedding analysis of complex network dynamics based on state-space reconstruction. (A) Example of randomized delay-coordinate reconstruction assuming we observe dynamics determined with three-dimensional differential equation. (1) Global state is embedded using (2) temporal sequence of single observable of system. Attractor topology in global state space is fully recovered in (3) delay-coordinate of single observable. This relationship is maintained after (4) random linear projection of delay-coordinate space. Thus, there exists unique one-to-one map from (4) to (1), given sufficient number of coordinates. (B) The reconstruction needs sufficient dimensionality. Whether we have embedding or not depends on the relation between the reconstruction dimension (d) and the underlying attractor dimension (d A). For example, a two-dimensional delay-coordinate space generically embeds one-dimensional attractor (e.g., a limit cycle, top row), except for a finite number of points. Higher dimensionality is required to embed more complex dynamics (bottom row). Generally, a dimensionality larger than that of an attractor (d > d A) is necessary and sufficient for the embedding—to distinguish almost all states of the system [16] although perfect embedding without no self-intersection requires d > 2d A [15]; the current cross-embedding protocol utilizes the former property. The Figure explores 1, 2 or 3 embedding dimensions after randomly projecting original delay-coordinates. Red regions indicate self-intersection with unignorable volume, reflecting incomplete embedding. (C) Unidirectional interaction between two modeled systems (directionally coupled Rössler systems, see Materials and Methods for equations). Downstream has information for whole system (light green) whereas upstream only has information on itself (light magenta). (D) Example of observed upstream and downstream signals. (E) The cross-embedding based on the random-coordinate state-space reconstructions. M x and M y respectively represent the downstream (x) and upstream (y) attractors based on single observed variable from each system. Colored arrows depict how nearby states in one attractor (disk: 20 nearest-neighbor states) are mapped onto states (dots) in other attractor (green: from x to y and magenta: from y to x). (F) Relationship between coordinate dimensions (number of randomized coordinates used for attractor reconstruction) and embeddedness (how accurately one node’s history can predict another node’s state). The circled dots represent complexity estimates, which were determined as minimum embedding dimension that provided ≥ 95% of optimal embeddedness measure (correlation between actual and forecast signals. See Materials and Methods).
Fig 1 Cross-embedding analysis of complex network dynamics based on state-space reconstruction. (A) Example of randomized delay-coordinate reconstruction assuming we observe dynamics determined with three-dimensional differential equation. (1) Global state is embedded using (2) temporal sequence of single observable of system. Attractor topology in global state space is fully recovered in (3) delay-coordinate of single observable. This relationship is maintained after (4) random linear projection of delay-coordinate space. Thus, there exists unique one-to-one map from (4) to (1), given sufficient number of coordinates. (B) The reconstruction needs sufficient dimensionality. Whether we have embedding or not depends on the relation between the reconstruction dimension (d) and the underlying attractor dimension (d A). For example, a two-dimensional delay-coordinate space generically embeds one-dimensional attractor (e.g., a limit cycle, top row), except for a finite number of points. Higher dimensionality is required to embed more complex dynamics (bottom row). Generally, a dimensionality larger than that of an attractor (d > d A) is necessary and sufficient for the embedding—to distinguish almost all states of the system [16] although perfect embedding without no self-intersection requires d > 2d A [15]; the current cross-embedding protocol utilizes the former property. The Figure explores 1, 2 or 3 embedding dimensions after randomly projecting original delay-coordinates. Red regions indicate self-intersection with unignorable volume, reflecting incomplete embedding. (C) Unidirectional interaction between two modeled systems (directionally coupled Rössler systems, see Materials and Methods for equations). Downstream has information for whole system (light green) whereas upstream only has information on itself (light magenta). (D) Example of observed upstream and downstream signals. (E) The cross-embedding based on the random-coordinate state-space reconstructions. M x and M y respectively represent the downstream (x) and upstream (y) attractors based on single observed variable from each system. Colored arrows depict how nearby states in one attractor (disk: 20 nearest-neighbor states) are mapped onto states (dots) in other attractor (green: from x to y and magenta: from y to x). (F) Relationship between coordinate dimensions (number of randomized coordinates used for attractor reconstruction) and embeddedness (how accurately one node’s history can predict another node’s state). The circled dots represent complexity estimates, which were determined as minimum embedding dimension that provided ≥ 95% of optimal embeddedness measure (correlation between actual and forecast signals. See Materials and Methods).CC BY · PubMed Central
Fig 2 Cross-embedding with random coordinates dissociates interaction-relevant complexity from timescale or noise variations. (A) Complexity estimates based on embedding y with x in the system used in Fig 1C–1F, where timescale of signal was varied. The estimates with each method were shown as relative values to those at timescale = 0.5. The results were similar in embedding x with y. (B) The results where the observation noise level was varied. The noise level corresponds to the standard deviation of the (zero-mean) normal distribution, from which noise were generated. The results are shown as relative values to those at noise level = 0. Although both the standard- and random-coordinate embedding methods (circle symbols) are invariant to noise level in terms of the relative values, the random-coordinate method estimates the true complexity more accurately in absolute values (see panel C). (C) Coordinate randomization avoids overestimation of complexity. The markers and error bars indicate the averages ±s.e.m. across 10 trials with different initial states of dynamics. Here, the pre-normalized complexity estimates using regular delay coordinates, (x t, x t-τ …) (open symbols), are compared with results in Fig 1C (y embedded by x) which used random delay coordinates (closed symbols); other conventions follows those in Fig 1F. The theoretically predicted embedding dimension of x (shaded bar) were accurately estimated using random coordinates, while the previous method using standard delay-coordinates tends to overestimate it, even exceeding the total number of nodes (dotted line) in the simulated system, which upper bounds the theoretically possible attractor dimensions by construction.
Fig 2 Cross-embedding with random coordinates dissociates interaction-relevant complexity from timescale or noise variations. (A) Complexity estimates based on embedding y with x in the system used in Fig 1C–1F, where timescale of signal was varied. The estimates with each method were shown as relative values to those at timescale = 0.5. The results were similar in embedding x with y. (B) The results where the observation noise level was varied. The noise level corresponds to the standard deviation of the (zero-mean) normal distribution, from which noise were generated. The results are shown as relative values to those at noise level = 0. Although both the standard- and random-coordinate embedding methods (circle symbols) are invariant to noise level in terms of the relative values, the random-coordinate method estimates the true complexity more accurately in absolute values (see panel C). (C) Coordinate randomization avoids overestimation of complexity. The markers and error bars indicate the averages ±s.e.m. across 10 trials with different initial states of dynamics. Here, the pre-normalized complexity estimates using regular delay coordinates, (x t, x t-τ …) (open symbols), are compared with results in Fig 1C (y embedded by x) which used random delay coordinates (closed symbols); other conventions follows those in Fig 1F. The theoretically predicted embedding dimension of x (shaded bar) were accurately estimated using random coordinates, while the previous method using standard delay-coordinates tends to overestimate it, even exceeding the total number of nodes (dotted line) in the simulated system, which upper bounds the theoretically possible attractor dimensions by construction.CC BY · PubMed Central
Fig 3 Cross-embedding analysis reveals large-scale cortical interaction, which can be missed by correlation-based statistics. (A) ECoG electrode loci in a representative subject. (B) Illustration of cortical areas covered by the present recording system. (C) Example of ECoG signals. Gray traces represent the signal from the all electrodes superimposed; green and magenta traces show the signals in an example electrode pair (electrodes 41 and 118). (D) Cross-embedding analysis based on the state-space reconstructions for the example electrode pair, corresponding to Fig 1E and 1F. (E) Relationship between coordinate dimensions and embeddedness in the example electrode pair, corresponding to Fig 1F. (F) Optimal embeddedness values as functions of data length (number of data points) used in the analysis. The line colors corresponds to those in panel E. Note that the embeddedness values improve as data length increases, which is a hallmark of causally coupled deterministic dynamics [21]. (G-K) Cross-embedding results. (G) The complexity of area-to-area interaction are shown in the matrix formats, which show the results of interaction from areas specifying columns to areas specifying rows. (H) Scatter plot of complexity in awake vs. anesthetized conditions; each dot represents a complexity value for each “effect” electrode, averaged across “cause” electrodes. Dashed red line shows equality. (I) (Top) The directionality of area-to-area interaction are shown in the matrix formats. (Bottom) The directionality among individual electrodes in a single subject. (J) Scatter plot of directionality in awake vs. anesthetized conditions; each dot represents the values for each electrode pair. (K) The same as panels I, except for that the directionality values were computed based on the cross-embedding analysis with limiting the embedding dimension to 1. (L-N) Correlation analysis of snapshot electrode signals. (L) (Top) Average correlation among areas. (Bottom) Correlation among individual electrodes in a single subject. (M) Scatter plots of complexity, directionality, and correlation coefficient in awake vs. anesthetized conditions; each dot represents the values for each electrode pair. (N) Comparison between cross-embedding-based directionality and correlation. Dashed red line shows zero value of correlation. Panels G-L, N and O show the pooled results across the four subjects. LVC: lower visual cortex; HVC: higher visual cortices; TC: temporal cortex; PC: parietal cortex; MSC motor and somatosensory cortex; PMC: premotor cortex; lPFC: lateral prefrontal cortex; mPFC: medial prefrontal cortex
Fig 3 Cross-embedding analysis reveals large-scale cortical interaction, which can be missed by correlation-based statistics. (A) ECoG electrode loci in a representative subject. (B) Illustration of cortical areas covered by the present recording system. (C) Example of ECoG signals. Gray traces represent the signal from the all electrodes superimposed; green and magenta traces show the signals in an example electrode pair (electrodes 41 and 118). (D) Cross-embedding analysis based on the state-space reconstructions for the example electrode pair, corresponding to Fig 1E and 1F. (E) Relationship between coordinate dimensions and embeddedness in the example electrode pair, corresponding to Fig 1F. (F) Optimal embeddedness values as functions of data length (number of data points) used in the analysis. The line colors corresponds to those in panel E. Note that the embeddedness values improve as data length increases, which is a hallmark of causally coupled deterministic dynamics [21]. (G-K) Cross-embedding results. (G) The complexity of area-to-area interaction are shown in the matrix formats, which show the results of interaction from areas specifying columns to areas specifying rows. (H) Scatter plot of complexity in awake vs. anesthetized conditions; each dot represents a complexity value for each “effect” electrode, averaged across “cause” electrodes. Dashed red line shows equality. (I) (Top) The directionality of area-to-area interaction are shown in the matrix formats. (Bottom) The directionality among individual electrodes in a single subject. (J) Scatter plot of directionality in awake vs. anesthetized conditions; each dot represents the values for each electrode pair. (K) The same as panels I, except for that the directionality values were computed based on the cross-embedding analysis with limiting the embedding dimension to 1. (L-N) Correlation analysis of snapshot electrode signals. (L) (Top) Average correlation among areas. (Bottom) Correlation among individual electrodes in a single subject. (M) Scatter plots of complexity, directionality, and correlation coefficient in awake vs. anesthetized conditions; each dot represents the values for each electrode pair. (N) Comparison between cross-embedding-based directionality and correlation. Dashed red line shows zero value of correlation. Panels G-L, N and O show the pooled results across the four subjects. LVC: lower visual cortex; HVC: higher visual cortices; TC: temporal cortex; PC: parietal cortex; MSC motor and somatosensory cortex; PMC: premotor cortex; lPFC: lateral prefrontal cortex; mPFC: medial prefrontal cortexCC BY · PubMed Central
Fig 4 Dynamical complexity and directed interaction within awake and anesthetized brain. Unlike anesthetized brain, awake brain exhibited clear differentiation between frontoparietal and occipital areas. This figure provides results in a single representative experiment, where subject state was changed from awake (A and B), eyes-closed condition to anesthesia by injection of propofol (C and D). Loci of bubbles correspond to electrode positions; the colors of bubbles indicate (A and C) the complexity or (B and D) the directionality averaged across forecast electrodes. (The absolute values of complexity/directionality are also indicated by sizes of bubbles, for visibility.) (E) Area-wise complexity and directionality. Thin lines correspond to the individual electrodes, and thick lines indicate averages and standard errors across electrodes within each area.
Fig 4 Dynamical complexity and directed interaction within awake and anesthetized brain. Unlike anesthetized brain, awake brain exhibited clear differentiation between frontoparietal and occipital areas. This figure provides results in a single representative experiment, where subject state was changed from awake (A and B), eyes-closed condition to anesthesia by injection of propofol (C and D). Loci of bubbles correspond to electrode positions; the colors of bubbles indicate (A and C) the complexity or (B and D) the directionality averaged across forecast electrodes. (The absolute values of complexity/directionality are also indicated by sizes of bubbles, for visibility.) (E) Area-wise complexity and directionality. Thin lines correspond to the individual electrodes, and thick lines indicate averages and standard errors across electrodes within each area.CC BY · PubMed Central
Fig 5 Universality of complex dynamics across different awake conditions. (A) Conscious and unconscious states are differentiated in the space of complexity averaged across all areas inside and outside the visual cortices. Markers represent individual experiments. Results from four monkeys have been superimposed. Note that the data from Reaching, Awake-Eyes-Open, and Awake-Eyes-Closed conditions are overlapped. (B) In conscious states, the complexity and directionality simultaneously increase in the outside visual cortex relative to the inside visual cortex. The convention follows that of panel A. (C) Complexity and directionality are correlated even in anesthetized condition. Each marker shows averaged complexity and directionality values (as in Fig 4) in each single electrode. (D) Anesthetization decreases the bottom-up interaction. The relative embeddedness reflects the dynamical coupling after the baseline change correlation is subtracted (Materials and Methods). The black and gray bars indicate the strength of bottom-up (from visual areas to other areas) and top-down (form other areas to visual areas), respectively. In panels A–C, the ellipses indicate the data covariance within individual conditions; the darker gray ellipse indicates the covariance for the data pooling the ketamine-medetomidine- and propofol-induced anesthesia. The dotted line indicates equality of average complexity. VC: visual cortex.
Fig 5 Universality of complex dynamics across different awake conditions. (A) Conscious and unconscious states are differentiated in the space of complexity averaged across all areas inside and outside the visual cortices. Markers represent individual experiments. Results from four monkeys have been superimposed. Note that the data from Reaching, Awake-Eyes-Open, and Awake-Eyes-Closed conditions are overlapped. (B) In conscious states, the complexity and directionality simultaneously increase in the outside visual cortex relative to the inside visual cortex. The convention follows that of panel A. (C) Complexity and directionality are correlated even in anesthetized condition. Each marker shows averaged complexity and directionality values (as in Fig 4) in each single electrode. (D) Anesthetization decreases the bottom-up interaction. The relative embeddedness reflects the dynamical coupling after the baseline change correlation is subtracted (Materials and Methods). The black and gray bars indicate the strength of bottom-up (from visual areas to other areas) and top-down (form other areas to visual areas), respectively. In panels A–C, the ellipses indicate the data covariance within individual conditions; the darker gray ellipse indicates the covariance for the data pooling the ketamine-medetomidine- and propofol-induced anesthesia. The dotted line indicates equality of average complexity. VC: visual cortex.CC BY · PubMed Central
Fig 6 Increased complexity at the cortical downstream differentiates conscious form unconscious brain states. (A) Summary of task conditions. (B) Cortical hierarchy defined by complexity. Areas aligned by complexity and relative directed interaction between them have been visualized. Arrows indicate relative information flows from one area to another, which were quantified by average directionality for interactions between two areas (only strong information flows exceeding 0.01 are shown). Complexity in visual areas was smaller than that in other areas during awake conditions (reaching: P<0.004; awake-eyes-open and awake-eyes-closed: P<0.00007; sign test, matched samples), and this inter-area difference in complexity was reduced by anesthesia (P<3×10−8; Wilcoxon rank sum test, independent samples). (C) Schematic summary of the main findings.
Fig 6 Increased complexity at the cortical downstream differentiates conscious form unconscious brain states. (A) Summary of task conditions. (B) Cortical hierarchy defined by complexity. Areas aligned by complexity and relative directed interaction between them have been visualized. Arrows indicate relative information flows from one area to another, which were quantified by average directionality for interactions between two areas (only strong information flows exceeding 0.01 are shown). Complexity in visual areas was smaller than that in other areas during awake conditions (reaching: P<0.004; awake-eyes-open and awake-eyes-closed: P<0.00007; sign test, matched samples), and this inter-area difference in complexity was reduced by anesthesia (P<3×10−8; Wilcoxon rank sum test, independent samples). (C) Schematic summary of the main findings.CC BY · PubMed Central
Fig 7 Robustness to variations in timescale. The main result is not affected by lengthening the unit delay although forecasting accuracy (embeddedness) was generally degraded by using suboptimal unit delay size. (A) The replications of the results shown in Fig 5A, respectively, where a longer unit delay (100 ms) was used for the state space reconstruction. (B) Optimal embeddedness in the all electrode pairs decreased by 0.21 times with the longer unit delay (p<10−5, sign test, paired sample across all electrode pairs). The figure shows the embeddedness averaged inside (dashed lines) or outside (solid lines) visual cortex. (C) Complexity (averaged inside or outside visual cortex) showed only small or not significant dependency to the unit delay variation (inside visual cortex: changed by 0.90 times, p <10−4; outside visual cortex: by 0.80 times, p = 0.5; sign test, paired sample, across all experimental sessions). This robustness validates that complexity reflects dimensions of dynamical attractor rather than variations in timescale of signals. In panels D-F, the color code follows that of panel B.
Fig 7 Robustness to variations in timescale. The main result is not affected by lengthening the unit delay although forecasting accuracy (embeddedness) was generally degraded by using suboptimal unit delay size. (A) The replications of the results shown in Fig 5A, respectively, where a longer unit delay (100 ms) was used for the state space reconstruction. (B) Optimal embeddedness in the all electrode pairs decreased by 0.21 times with the longer unit delay (p<10−5, sign test, paired sample across all electrode pairs). The figure shows the embeddedness averaged inside (dashed lines) or outside (solid lines) visual cortex. (C) Complexity (averaged inside or outside visual cortex) showed only small or not significant dependency to the unit delay variation (inside visual cortex: changed by 0.90 times, p <10−4; outside visual cortex: by 0.80 times, p = 0.5; sign test, paired sample, across all experimental sessions). This robustness validates that complexity reflects dimensions of dynamical attractor rather than variations in timescale of signals. In panels D-F, the color code follows that of panel B.CC BY · PubMed Central
Fig 8 Possible mechanisms of changing frontoparietal dynamics in the conscious brain state. (A) Simplified architecture of nervous system under unconscious (anesthetized) state. In the unconscious state, three clusters (frontoparietal, visual, and subcortical systems) have no strong nonlinear interaction. (B) Model 1: the frontoparietal complexity is increased by the enhanced bottom-up interaction. (C) Model 2: the frontoparietal complexity is increased by subcortical input. (D) Model 3: the frontoparietal complexity is increased by change in the connectivity inside of the frontoparietal system itself. Only Model 1 accounts for the increase in the directionality from the visual to the frontoparietal cortex, which was observed in the experimental data. (From left to right columns) Network architecture, mechanistic connectivity matrices, directionality matrices derived based on cross-embedding analysis with simulated dynamics, average complexity in the frontoparietal cortex, the averaged directionality from the visual to the frontoparietal cortex. V: visual cortex. FP: frontoparietal cortex.
Fig 8 Possible mechanisms of changing frontoparietal dynamics in the conscious brain state. (A) Simplified architecture of nervous system under unconscious (anesthetized) state. In the unconscious state, three clusters (frontoparietal, visual, and subcortical systems) have no strong nonlinear interaction. (B) Model 1: the frontoparietal complexity is increased by the enhanced bottom-up interaction. (C) Model 2: the frontoparietal complexity is increased by subcortical input. (D) Model 3: the frontoparietal complexity is increased by change in the connectivity inside of the frontoparietal system itself. Only Model 1 accounts for the increase in the directionality from the visual to the frontoparietal cortex, which was observed in the experimental data. (From left to right columns) Network architecture, mechanistic connectivity matrices, directionality matrices derived based on cross-embedding analysis with simulated dynamics, average complexity in the frontoparietal cortex, the averaged directionality from the visual to the frontoparietal cortex. V: visual cortex. FP: frontoparietal cortex.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 2

  1. M Riedl, 2013, Practical considerations of permutation entropy, Eur Phys J, 222, 249
  2. AN Kolmogorov, 1965, Three approaches to the definition of the concept of quantity of information, IEEE Trans Inf Theory, 14, 662

Cited by

See all 76

Metadata

Authors
Tajima, S., Yanagawa, T., Fujii, N., & Toyoizumi, T.
Affiliations
  • University of Geneva (CH)
  • RIKEN Center for Brain Science (JP)
  • Tokyo Institute of Technology (JP)
Corresponding author
  • Tajima, Satohiro
  • Yanagawa, Toru
  • Fujii, Naotaka
  • Toyoizumi, Taro
Venue
PLOS Computational Biology
Published
2015-11-19
Volume
11
Issue
11
Pages
e1004537
ISSN
  • 1553-734X (ISSN-L)
  • 1553-7358
OpenAlex
W2173362524
Cited by
76
References
78
Citations by year
2026: 2 · 2025: 5 · 2024: 8 · 2023: 9 · 2022: 8 · 2021: 5 · 2020: 12 · 2019: 7 · 2018: 6 · 2017: 11 · 2016: 2 · 2015: 1
FWCI
4.01
Type
journal article
Language
en
Open access
Locations
  • PLoS Computational Biology (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Figshare (submittedVersion, cc-by)
Funders
  • Japan Agency for Medical Research and Development
  • Japan Society for the Promotion of Science
  • RIKEN
  • RIKEN Brain Science Institute
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Chaos control and synchronization
Keywords
consciousness, dynamical complexity, cortical hierarchy, information flow, electrophysiological recording, model-independent analysis
MeSH terms
Algorithms, Animals, Brain, Consciousness, Electroencephalography, Macaca, Wakefulness, Computational Biology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.