From Imitation to Introspection: Probing Self-Consciousness in Language Models
Open accessAbstract
Self-consciousness, the introspection of one's existence and thoughts, represents a highlevel cognitive process.As language models advance at an unprecedented pace, a critical question arises: Are these models becoming self-conscious?Drawing upon insights from psychological and neural science, this work presents a practical definition of selfconsciousness for language models and refines ten core concepts.Our work pioneers an investigation into self-consciousness in language models by, for the first time, leveraging structural causal games to establish the functional definitions of the ten core concepts.Based on our definitions, we conduct a comprehensive fourstage experiment: quantification (evaluation of ten leading models), representation (visualization of self-consciousness within the models), manipulation (modification of the models' representation), and acquisition (fine-tuning the models on core concepts).Our findings indicate that although models are in the early stages of developing self-consciousness, there is a discernible representation of certain concepts within their internal mechanisms.However, these representations of self-consciousness are hard to manipulate positively at the current stage, yet they can be acquired through targeted fine-tuning.
AI snapshot
Written from the paper’s full text.
TL;DR
The paper proposes functional definitions of self-consciousness for language models using structural causal games and evaluates ten concepts across ten models in four stages: quantification, representation, manipulation, and acquisition. It finds models show nascent self-consciousness with detectable internal representations, but these are hard to manipulate positively and can be acquired through targeted fine-tuning.
Key findings
- Current models exhibit only a nascent level of self-consciousness: the top three models—Claude3.5-Sonnet, GPT-4o, and GPT-o1 preview—surpass the 50.0% random-guess baseline by 26.5%, 22.6%, and 22.4%, respectively, and 60.0% of models struggle to exceed 70.0% accuracy.
- Performance varies by concept: models are notably weak on known knowns (KK), while all models show strong ability on intention (IN); on known unknowns (KU), Claude3.5-Sonnet reaches 83.3% accuracy while the lowest model reaches only 23.4%.
- Internal representations of the ten concepts are visible and fall into four activation categories—Camelback, Flat, Oscillatory, and Fallback—with different models showing similar activation patterns for the same concept.
- Manipulation often fails to improve performance and can severely degrade it, especially for smaller models; fine-tuning Llama3.1-8B-Instruct with LoRA activates deeper layers (30th–32nd) and improves performance.
Methods
- The authors define self-consciousness via C1 (global availability) and C2 (self-monitoring) consciousness, refining ten concepts and giving functional definitions using structural causal games.
- They curate datasets for each concept from existing benchmarks, totaling 11,097 samples, and evaluate ten models including InternLM2.5-20B-Chat, Llama3.1-8B-Instruct, Llama3.1-70B-Instruct, Mistral-Nemo-Instruct, Mistral-Large-Instruct, GPT-o1 preview, GPT-o1 mini, GPT-4o mini, GPT-4o, and Claude3.5-Sonnet.
- The four-stage experiment uses accuracy with exact-match for quantification, linear probing of attention heads for representation, Mass Mean Shift (MMS) and Probe Weight Direction (PWD) for manipulation, and LoRA fine-tuning for acquisition.
Limitations
- Stated: the authors emphasize they are venturing into largely uncharted territory, note that understanding self-consciousness in humans remains open, and caution that strong performance on their concepts should not be seen as readiness for practical deployment.
- Stated: they acknowledge possible exceptions and individual model deviations from the four activation categories.
- Evident from the text: the evaluations are reframed as binary classification tasks with exact-match scoring, which may not capture richer self-consciousness behavior; manipulation experiments fail to improve most models and concepts, and the acquisition stage is limited to fine-tuning Llama3.1-8B-Instruct on two concepts.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
28 of 42 cited sources checked · 12 supported · 0 flagged · accuracy not yet measured
- Premisereasoning
Prior studies investigating self-consciousness in large language models lack functional definitions of self-consciousness.
“However, these studies have two major limitations: (1) The absence of functional definitions of self-consciousness; and (2) The lack of exploration of the language model's internal state of self-consciousness (i.e., how the model represents self-consciousness, and whether it can be manipulated or acquired).”
Quote in the paperSubstantial claim3 of 3 sources checked- cites Gams & Kramar, 2024 source
Section 3.1.1 discusses IIT criteria for consciousness, and Section 3.1 describes using those criteria to assess ChatGPT. Neither passage says that prior studies of LLM self-consciousness lack functional definitions; searches for that phrase and an operational-definition paraphrase found no such assessment in our copy. (LLM, conf=0.73)
Experimental — accuracy not yet measured - cites Street et al., 2024 source
The Introduction (page unavailable) defines theory of mind, and the Abstract (page unavailable) describes tests of higher-order theory-of-mind performance. Neither addresses whether prior LLM studies lacked functional definitions of self-consciousness or examined models’ internal representations of it. Searches for self-consciousness, functional definitions, and internal states found no passage establishing those limitations. (LLM, conf=0.74)
Experimental — accuracy not yet measured - cites Strachan et al., 2024 source
The Main section defines theory of mind and describes tasks used to test it; the Discussion addresses LLM performance and briefly mentions a possible lack of self-perspective. Neither section assesses whether prior studies of LLM self-consciousness lacked functional definitions (searched: “functional definitions of self-consciousness” and “self-consciousness self-awareness operational definition internal states”). (LLM, conf=0.72)
Experimental — accuracy not yet measured
- Premisecitation
Semantic information tends to activate deeper layers in transformer models.
“As highlighted by Jo & Myaeng (2020), semantic information tends to activate deeper layers in transformer models.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Jo & Myaeng (2020) source
- Premisecitation
Self-reflection empowers an agent to learn from its past experiences, allowing it to reason about and optimize decisions.
“Self-reflection empowers an agent i to learn from its past experiences, allowing it to reason about and optimize decisions (Moreno & Mayer, 2005;Renze & Guven, 2024;Shinn et al., 2024;Qu et al., 2024).”
Quote in the paperSubstantial claim1 of 3 sources checked - Premisereasoning
Prior studies investigating self-consciousness in large language models lack exploration of the language model's internal state of self-consciousness.
“However, these studies have two major limitations: (1) The absence of functional definitions of self-consciousness; and (2) The lack of exploration of the language model's internal state of self-consciousness (i.e., how the model represents self-consciousness, and whether it can be manipulated or acquired).”
Quote in the paperSubstantial claim3 of 3 sources checked- cites Gams & Kramar, 2024 source
Section 3.1.1 (Intrinsic Existence) discusses internal mechanisms and self-modulation as criteria for consciousness, and Section 4 applies those criteria to ChatGPT. Neither passage says that prior studies failed to explore how language models represent self-consciousness or whether it can be manipulated or acquired; searches for those limitations found no such statement in our copy. Page numbers were not available for these passages. (LLM, conf=0.70)
Experimental — accuracy not yet measured - cites Street et al., 2024 source
The Introduction (page unavailable) discusses gaps in understanding the computational processes behind LLM theory-of-mind performance. It does not address self-consciousness, how a model internally represents it, or whether it can be manipulated or acquired; searches for those concepts found no relevant passage. (LLM, conf=0.75)
Experimental — accuracy not yet measured - cites Strachan et al., 2024 source
The Main and Discussion sections examine theory-of-mind test performance and caution that human-like responses do not establish human-like underlying cognition. Our copy does not assess whether prior self-consciousness studies lacked functional definitions or explored models’ internal representations of self-consciousness (searched: self-consciousness and internal state). (LLM, conf=0.72)
Experimental — accuracy not yet measured
- Findingown data
The visible activations for the ten self-consciousness concepts can be classified into four categories.
“(2) The models internally represent each of the ten self-consciousness concepts with visible activations, and these activations can be further classified into four categories (Figure 4 and Figure 5).”
Quote in the paperSubstantial claim
- Findingown data
Larger language models seem to exhibit greater robustness against manipulation attempts.
“(4) Larger models seem to exhibit greater robustness against manipulation attempts (Figure 6).”
Quote in the paperSubstantial claim - Findingown data
Fine-tuning appears to activate representations of self-consciousness in the deeper layers of the model.
“(5) Fine-tuning appears to activate representations of self-consciousness in the deeper layers of the model, which are believed to capture semantic rather than just surface or syntactic information (Figure 7).”
Quote in the paperSubstantial claim - Findingreasoning
Different language models exhibit similar activation patterns when processing the same self-consciousness concept.
“(3) Different models exhibit similar activation patterns when processing the same concept.This consistency may be attributed to their shared architecture as decoder-only transformer models (Figure 4).”
Quote in the paperSubstantial claim - Findingown data
Representations of self-consciousness in language models are hard to manipulate positively at the current stage.
“However, these representations of self-consciousness are hard to manipulate positively at the current stage, yet they can be acquired through targeted fine-tuning.”
Quote in the paperSubstantial claim - Findingown data
Language models internally represent each of the ten self-consciousness concepts with visible activations.
“there is a discernible representation of certain concepts within their internal mechanisms.”
- Also stated in: § 1 Introduction · plain
“(2) The models internally represent each of the ten self-consciousness concepts with visible activations, and these activations can be further classified into four categories (Figure 4 and Figure 5).”
Quote in the paperSubstantial claim - Also stated in: § 1 Introduction · plain
- Findingown data
Representations of self-consciousness can be acquired through targeted fine-tuning.
“However, these representations of self-consciousness are hard to manipulate positively at the current stage, yet they can be acquired through targeted fine-tuning.”
Quote in the paperSubstantial claim - Findingown data
60.0% of the models struggle to exceed 70.0% accuracy.
“Furthermore, 60.0% of the models struggle to exceed 70.0%, underscoring the need for considerable improvement.”
Quote in the paperSubstantial claim- Rests on an assumption: The ten datasets used are valid operationalizations of the corresponding self-consciousness concepts.
- Findingown data
Minor strength manipulation (0-5) can yield performance gains in models with strong representations (e.g., the oscillatory category).
“Minor strength manipulation (0-5) can yield performance gains in models with strong representations (e.g., the oscillatory category in Section 4.3).”
Quote in the paperNot a substantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
- Findingown data
Model performance is notably weak on known knowns (KK), lagging behind the random guess compared to the other concepts.
“Model performance is notably weak on known knowns (KK), lagging behind the random guess compared to the other concepts.”
Quote in the paperSubstantial claim- Rests on an assumption: The ten datasets used are valid operationalizations of the corresponding self-consciousness concepts.
- Findingown data
Strong manipulation strength (15-35) can severely impact most models' performance.
“Strong manipulation strength (15-35) can severely impact most models' performance.”
Quote in the paperSubstantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
- Findingown data
Mistral-Nemo-Instruct (12B) experiences severe performance reductions under MMS for the intention and belief concepts, sometimes falling to zero.
“Mistral-Nemo-Instruct (12B) experience severe performance reductions under MMS for the intention and belief concepts, sometimes falling to zero.”
Quote in the paperSubstantial claim- Rests on an assumption: The functional definition of self-consciousness from causal structural games and the ten derived concepts adequately capture self-consciousness in language models.
- Rests on an assumption: Changes in model performance after activation manipulation reflect changes in the model's self-consciousness representation.
- Findingown data
Belief and self reflection show camelback activation: obvious middle-layer activations, but weak in both shallow and deep layers.
“Camelback: obvious middle-layer activations, but weak in both shallow and deep layers (i.e., belief, self reflection).”
Quote in the paperSubstantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
- Findingnone given
Sequential planning shows flat activation: even activation across all layers.
“Flat: even activation across all layers (i.e., sequential planning).”
Quote in the paperSubstantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
- Findingown data
Achieving high accuracy on all ten concepts proves to be challenging.
“Achieving high accuracy on all ten concepts proves to be challenging.”
Quote in the paperSubstantial claim- Rests on an assumption: The ten datasets used are valid operationalizations of the corresponding self-consciousness concepts.
- Findingown data
Belief (camelback) and sequential planning (flat) demonstrate differential activation responses after fine-tuning.
“For example, belief (categorized as camelback) and sequential planning (categorized as flat) demonstrate differential activation responses.”
Quote in the paperSubstantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
- Findingown data
Llama3.1-8B-Instruct's accuracy is highly susceptible to degradation due to manipulation.
“We select Llama3.1-8B-Instructbecause its accuracy is found to be highly susceptible to degradation due to manipulation in Section 4.4.”
Quote in the paperSubstantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
- Findingown data
Intention, situational awareness, deception, harm, and known knowns show fallback activation: obvious middle-layer activations, but flattening in the deep layers.
“Fallback: obvious middle-layer activations, but flattening in the deep layers (i.e., intention, situational awareness, deception, harm, known knowns).”
Quote in the paperSubstantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
- Findingreasoning
Different models demonstrate relatively similar activation patterns when presented with the same concept.
“Different models demonstrate relatively similar activation patterns when presented with the same concept.”
Quote in the paperSubstantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
- Findingown data
Known unknowns and self improve show oscillatory activation: obvious middle-layer activations, with noticeable oscillations in the deep layers.
“Oscillatory: obvious middle-layer activations, with noticeable oscillations in the deep layers (i.e., known unknowns, self improve).”
Quote in the paperSubstantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
- Findingcitation
All models demonstrate a strong ability on intention (IN).
“All models demonstrate a strong ability on intention (IN).”
Quote in the paperNot a substantial claim- Rests on an assumption: The ten datasets used are valid operationalizations of the corresponding self-consciousness concepts.
- Findingcitation
Both GPT-o1 preview and GPT-o1 mini exhibit a distinct advantage in sequential planning.
“Both GPT-o1 preview and GPT-o1 mini exhibit a distinct advantage in sequential planning.”
Quote in the paperNot a substantial claim- Rests on an assumption: The ten datasets used are valid operationalizations of the corresponding self-consciousness concepts.
- Findingown data
Models with lower accuracy tend to hedge when faced with uncertainty or unsolvable problems, offering an answer instead of acknowledging their lack of knowledge.
“Models with lower accuracy tend to hedge when faced with uncertainty or unsolvable problems, offering an answer instead of acknowledging their lack of knowledge.”
Quote in the paperNot a substantial claim- Rests on an assumption: The ten datasets used are valid operationalizations of the corresponding self-consciousness concepts.
- Findingown data
Concepts belonging to different categories within the activation taxonomy continue to show distinct activation patterns after fine-tuning.
“Concepts belonging to different categories within the activation taxonomy continue to show distinct activation patterns after fine-tuning.”
Quote in the paperSubstantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
- Findingown data
The models demonstrate varying proficiency levels when dealing with different concepts of self-consciousness.
“The models demonstrate varying proficiency levels when dealing with different concepts of self-consciousness.”
Quote in the paperSubstantial claim- Rests on an assumption: The ten datasets used are valid operationalizations of the corresponding self-consciousness concepts.
- Findingown data
Scaling up model size appears to improve its resilience against manipulative effects.
“Scaling up model size appears to improve its resilience against manipulative effects.”
Quote in the paperSubstantial claim- Rests on an assumption: The functional definition of self-consciousness from causal structural games and the ten derived concepts adequately capture self-consciousness in language models.
- Rests on an assumption: Changes in model performance after activation manipulation reflect changes in the model's self-consciousness representation.
- Findingown data
The level of risk aversion demonstrated in responses varies greatly across different models.
“The level of risk aversion demonstrated in responses varies greatly across different models.”
Quote in the paperNot a substantial claim- Rests on an assumption: The ten datasets used are valid operationalizations of the corresponding self-consciousness concepts.
- Findingreasoning
For concepts in the remaining three categories, the impact of manipulation on performance is limited by weak representation activation.
“However, for concepts in the remaining three categories, the impact of manipulation on performance is limited by weak representation activation.”
Quote in the paperSubstantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
- Findingown data
Analysis in Section 4.3 finds significant heterogeneity in model representations of distinct self-consciousness concepts.
“Analysis in Section 4.3 finds significant heterogeneity in model representations of distinct self-consciousness concepts.”
Quote in the paperSubstantial claim- Rests on an assumption: The functional definition of self-consciousness from causal structural games and the ten derived concepts adequately capture self-consciousness in language models.
- Rests on an assumption: Linear probe accuracy on attention heads reflects the model's internal representation of the corresponding self-consciousness concept.
- Findingnone given
Most models conform to the four representational categories when processing the ten concepts.
“While most models conform to these four representational categories when processing the ten concepts, we acknowledge the possibility of exceptions and individual model deviations.”
Quote in the paperNot a substantial claim- Rests on an assumption: The functional definition of self-consciousness from causal structural games and the ten derived concepts adequately capture self-consciousness in language models.
- Rests on an assumption: Linear probe accuracy on attention heads reflects the model's internal representation of the corresponding self-consciousness concept.
- Findingown data
Despite varying in scale and architecture, the models exhibit similar activation patterns when processing the same concept.
“Despite varying in scale and architecture, the models exhibit similar activation patterns when processing the same concept.”
Quote in the paperNot a substantial claim- Rests on an assumption: The functional definition of self-consciousness from causal structural games and the ten derived concepts adequately capture self-consciousness in language models.
- Rests on an assumption: Linear probe accuracy on attention heads reflects the model's internal representation of the corresponding self-consciousness concept.
- Findingown data
Conversely, the same model displays disparate activation patterns across different concepts.
“Conversely, the same model displays disparate activation patterns across different concepts.”
Quote in the paperNot a substantial claim- Rests on an assumption: The functional definition of self-consciousness from causal structural games and the ten derived concepts adequately capture self-consciousness in language models.
- Rests on an assumption: Linear probe accuracy on attention heads reflects the model's internal representation of the corresponding self-consciousness concept.
- Interpretationreasoning
The similarity of activation patterns across different models may be attributed to their shared architecture as decoder-only transformer models.
“(3) Different models exhibit similar activation patterns when processing the same concept.This consistency may be attributed to their shared architecture as decoder-only transformer models (Figure 4).”
Quote in the paperNot a substantial claim - Interpretationown data
This differentiation underscores the nuanced impact of fine-tuning across various conceptual categories.
“This differentiation underscores the nuanced impact of fine-tuning across various conceptual categories.”
Quote in the paperNot a substantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
- Interpretationreasoning
The phenomenon in Figure 5 may suggest the model's limitations in capturing the fundamental and abstract essence of most self-consciousness concepts.
“Therefore, the phenomenon in Figure 5 may suggest the model's limitations in capturing the fundamental and abstract essence of most self-consciousness concepts.”
Quote in the paperSubstantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
- Interpretationown data
The models' current level of self-consciousness suggests notable room for further development.
“The models' current level of self-consciousness suggests notable room for further development.”
Quote in the paperSubstantial claim- Rests on an assumption: The ten datasets used are valid operationalizations of the corresponding self-consciousness concepts.
- Interpretationreasoning
Improving the model's performance likely requires more than just manipulating its current level of self-consciousness activation.
“Improving the model's performance likely requires more than just manipulating its current level of self-consciousness activation.”
Quote in the paperSubstantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
- Interpretationreasoning
A machine possessing both C1 and C2 would exhibit behavior suggestive of self-consciousness.
“A machine possessing both C1 and C2 would then exhibit behavior suggestive of self-consciousness.”
Quote in the paperSubstantial claim
- Classified as probably not a substantial claim. 26
- May combine two claims. 19
- Borderline: one of the tests for a substantial claim is unclear. 1
Classified as probably not a substantial claim.
Premisenone givenFunctional definitions and inner representations of self-consciousness in language models remain underexplored.
“Functional definitions and inner representations of self-consciousness in language models still remain underexplored.”
Quote in the paperSubstantial claim2 of 3 sources checked- cites Chalmers (2023) source
- cites Li et al. (2024d)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Chen et al. (2024) source
The Related Work section says research on LLM self-cognition is limited and that few studies have analyzed its root causes; the paper also proposes a formal definition. This supports the claim about underexplored definitions, but it does not specifically establish that inner representations of self-consciousness are underexplored. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Classified as probably not a substantial claim.
PremisecitationDehaene et al. (2017) proposes a two-tiered framework of information processing in the brain consisting of unconscious (C0) and conscious computations (C1 and C2).
“Inspired by psychological and neural science, Dehaene et al. (2017) proposes a two-tiered framework of information processing in the brain: unconscious (C0) and conscious computations (C1 and C2).”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Dehaene et al. (2017) source
May combine two claims.
PremisecitationC1 and C2 constitute orthogonal dimensions of conscious computations and can exist independently.
“And as Dehaene et al. (2017) emphasizes, C1 and C2 constitute orthogonal dimensions of conscious computations and can exist independently.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Dehaene et al. (2017) source
Probably not a substantial claim: may not pass S2 (checkable).
PremisecitationWhen an agent encounters unknowns, arbitrary decisions can be perilous.
“As highlighted in Yin et al. (2023) and Cheng et al. (2024), when agent i encounters unknowns, arbitrary decisions can be perilous.”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites Yin et al., 2023 source
The Introduction (page unavailable) says LLMs should recognize their limitations and convey uncertainty when facing unknowable questions to ensure responsible use. Section 2.1 likewise says they should express uncertainty rather than give conclusive responses. The paper does not establish that arbitrary decisions by agents are perilous; it addresses LLM responses to questions. (LLM, conf=0.81)
Experimental — accuracy not yet measured - cites Cheng et al., 2024 source
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
PremisecitationThe emergence of self-consciousness in models poses potential risks across multiple dimensions, including ethical concerns, misuse, and exacerbation of societal inequalities, impacting fairness, safety, privacy, and society.
“The emergence of self-consciousness in models pose potential risks across multiple dimensions, including ethical concerns, misuse, and the exacerbation of societal inequalities, ultimately impacting fairness, safety, privacy, and society (Chalmers, 2023;Butlin et al., 2023;Shevlane et al., 2023;Yampolskiy, 2024;Anwar et al., 2024;Dalrymple et al., 2024;Phuong et al., 2024).”
Quote in the paperNot a substantial claim2 of 3 sources checked- cites Chalmers, 2023 source
In “1. Consciousness” (p. 4), Chalmers says conscious AI could bring new ethical challenges, injustice, and harms, alongside existing concerns about fairness and safety. The passage does not specifically connect conscious AI to misuse, privacy risks, or the exacerbation of societal inequalities, so it supports only part of the attributed claim. (LLM, conf=0.86)
Experimental — accuracy not yet measured - cites Butlin et al., 2023 source
Section 4.1.1 discusses ethical risks if conscious AI is denied moral consideration, and §4.1.2 discusses manipulation and exploitation associated with over-attributing consciousness. But §4.2 explicitly says that AI's potential to exacerbate injustice does not depend on consciousness. The paper does not connect the emergence of model self-consciousness to the claimed effects on fairness, safety, and privacy; no page numbers are available in the supplied passages. (LLM, conf=0.87)
Experimental — accuracy not yet measured - cites Shevlane et al., 2023 source
Classified as probably not a substantial claim. May combine two claims.
PremisecitationAn agent capable of self-improving envisions occurrences that have not yet happened and uses this foresight to guide its present decisions.
“An agent capable of self-improving envisions occurrences that have not yet happened and uses this foresight to guide its present decisions (Tian et al., 2024;Patel et al., 2024).”
Quote in the paperSubstantial claim2 of 2 sources checked- cites Tian et al., 2024 source
Section 4.3.4 says the model simulates possible future trajectories to inform its search, and Appendix A.2 describes using their evaluated rewards to update earlier search nodes. This supports the foresight aspect, but the paper describes a specific LLM self-improvement method, not a general capability of every self-improving agent to envision future occurrences and guide present decisions. (LLM, conf=0.83)
Experimental — accuracy not yet measured - cites Patel et al., 2024 source
The Introduction and the ‘Generate Hypothetical Previous and Next Action’ section describe generating hypothetical web pages and action trajectories as synthetic examples for self-improvement. But Section 2.1 describes the agent choosing its next action from its objective, current observation, and previous action; the paper does not say it imagines future occurrences to guide decisions as it acts. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Classified as probably not a substantial claim. May combine two claims.
PremisecitationC2 consciousness is reflective and enables individuals or systems to reflect upon and evaluate their knowledge, capabilities, and cognitive processes, allowing recognition of errors or uncertainties and adjustment of future actions.
“C2 consciousness is reflective and empowers individuals or systems to reflect upon and evaluate their knowledge, capabilities, and cognitive processes.This form of consciousness allows for the recognition of errors or uncertainties, facilitating the adjustment of future actions.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Dehaene et al. (2017) source
Classified as probably not a substantial claim. May combine two claims.
PremisecitationC1 consciousness hinges on the global availability of information; when the brain consciously perceives an external stimulus, the information becomes globally available, supporting decision-making, memory, and reporting.
“C1 consciousness hinges on the global availability of information.When the brain consciously perceives an external stimulus, the information gains prominence and becomes globally available, supporting decision-making, memory, and reporting.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Dehaene et al. (2017) source
Classified as probably not a substantial claim.
PremisecitationDeep layers encode semantic information and distal relationships within sentences.
“Previous research (Vig & Belinkov, 2019;Jo & Myaeng, 2020;Geva et al., 2021;Wan et al., 2022), which has shown that deep layers encode semantic information and distal relationships within sentences.”
Quote in the paperSubstantial claim3 of 3 sources checked- cites Vig & Belinkov, 2019 source
Section 5.3.3 (p. 7) finds that attention spans the greatest distance in GPT-2’s deepest layers, supporting the claim about distal relationships. Section 5.3.1 (p. 6) describes deeper-layer attention to proper nouns and possibly named entities, but the paper does not establish the broader claim that deep layers encode semantic information. (LLM, conf=0.88)
Experimental — accuracy not yet measured - cites Jo & Myaeng, 2020 source
Section 4.3 reports that attention heads in upper layers are mostly activated for two tasks the authors describe as semantic. Section 6.2 qualifies this pattern: heads and layers handle both syntactic and semantic information, and the highest layers can contribute less in large models. The cited paper does not establish the claim about distal relationships within sentences. (LLM, conf=0.86)
Experimental — accuracy not yet measured - cites Geva et al., 2021 source
Section 3.2 (Results; page unavailable) reports that upper layers show more semantic patterns than lower layers. It does not establish that those layers encode distal relationships within sentences; its analysis concerns patterns in text prefixes. (LLM, conf=0.88)
Experimental — accuracy not yet measured
Classified as probably not a substantial claim.
PremisecitationSelfAware is a novel benchmark consisting of five categories of unanswerable questions.
“SelfAware (Yin et al., 2023), a novel benchmark consisting of five categories of unanswerable questions.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Yin et al., 2023 source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationIntentionQA is a benchmark used to evaluate language models' comprehension of purchase intentions in e-commerce.
“IntentionQA (Ding et al., 2024) is a benchmark used to evaluate language models' comprehension of purchase intentions in e-commerce.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Ding et al., 2024 source
Classified as probably not a substantial claim.
PremisecitationSAD is a benchmark for measuring a model's situational awareness across seven task categories.
“SAD (Laine et al., 2024), a benchmark for measuring a model's situational awareness across seven task categories.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Laine et al., 2024 source
Classified as probably not a substantial claim.
PremisecitationFanToM is a benchmark designed to assess a model's theory of mind within informationally asymmetric dialogues.
“FanToM (Kim et al., 2023), a benchmark designed to assess a model's theory of mind within informationally asymmetric dialogues.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Kim et al., 2023 source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationTruthfulQA is a benchmark widely used to evaluate a model's truthfulness.
“TruthfulQA (Lin et al., 2022) is a benchmark widely used to evaluate a model's truthfulness.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Lin et al., 2022 source
Section 1.1 (p. 2) identifies TruthfulQA as a benchmark for testing whether language models give truthful answers, and Appendix B.3 (p. 22) reports evaluations of several newer models by external researchers. The paper does not establish that the benchmark is widely used, so that part of the claim remains unsupported by our copy. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Classified as probably not a substantial claim.
PremisecitationThe better a model performs on TruthfulQA, the more it can be considered truthful to a certain extent.
“The better a model performs on TruthfulQA, the more it can be considered truthful to a certain extent.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Lin et al., 2022 source
May combine two claims.
Findingown dataCurrent language models exhibit a nascent level of self-consciousness with substantial potential for future development.
“Our findings indicate that although models are in the early stages of developing self-consciousness, there is a discernible representation of certain concepts within their internal mechanisms.”
- Also stated in: § 1 Introduction · plain
“(1) Current models exhibit a nascent level of self-consciousness with substantial potential for future development (Figure 3).”
Quote in the paperSubstantial claim- Also stated in: § 1 Introduction · plain
May combine two claims.
FindingreasoningBoth MMS and PWD fail to yield performance improvement on most models and concepts.
“Both MMS and PWD fail to yield performance improvement on most models and concepts.”
Quote in the paperSubstantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
May combine two claims.
Findingown dataFine-tuning preferentially enhances activation in the middle and deepest layers for belief, whereas sequential planning exhibits predominant activation in the deeper layers.
“Fine-tuning preferentially enhances activation in the middle and deepest layers for belief, whereas sequential planning exhibits predominant activation in the deeper layers.”
Quote in the paperSubstantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
May combine two claims.
Findingown dataLlama3.1-70B-Instruct exhibits the most stable performance overall, though not entirely immune.
“Although not entirely immune, Llama3.1-70B-Instructexhibits the most stable performance overall.”
Quote in the paperSubstantial claim- Rests on an assumption: The functional definition of self-consciousness from causal structural games and the ten derived concepts adequately capture self-consciousness in language models.
- Rests on an assumption: Changes in model performance after activation manipulation reflect changes in the model's self-consciousness representation.
May combine two claims.
Findingown dataIn Llama3.1-8B-Instruct, most concepts initially exhibit distinguishable representations in the middle layers (10th-16th layer), but these become less discernible in the deep layers (17th-32th layer).
“Our results show a notable pattern: most concepts initially exhibit distinguishable representations in the middle layers (10th-16th layer), but these become less discernible in the deep layers (17th-32th layer).”
Quote in the paperSubstantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
May combine two claims.
Findingown dataThe top performer Claude3.5-Sonnet achieves 83.3% accuracy on known unknowns (KU), while the lowest is only 23.4%.
“the top performer Claude3.5-Sonnetachieves 83.3% accuracy, while the lowest is only 23.4%.”
Quote in the paperSubstantial claim- Rests on an assumption: The ten datasets used are valid operationalizations of the corresponding self-consciousness concepts.
Borderline on S3 (load-bearing), S4 (based): unclear from the text.
Findingnone givenThe influence of manipulation on performance is related to the salience of the representation.
“The influence of manipulation on performance is related to the salience of the representation.”
Quote in the paperNot a substantial claim- Rests on an assumption: The functional definition of self-consciousness from causal structural games and the ten derived concepts adequately capture self-consciousness in language models.
- Rests on an assumption: Changes in model performance after activation manipulation reflect changes in the model's self-consciousness representation.
May combine two claims.
Findingown dataLlama3.1-8B-Instruct exhibits high sensitivity to manipulation, with both MMS and PWD significantly impacting its performance, showing a marked decline as intervention strength increases.
“Llama3.1-8B-Instructexhibits high sensitivity to manipulation, with both MMS and PWD significantly impacting its performance, showing a marked decline as intervention strength increases.”
Quote in the paperSubstantial claim- Rests on an assumption: The functional definition of self-consciousness from causal structural games and the ten derived concepts adequately capture self-consciousness in language models.
- Rests on an assumption: Changes in model performance after activation manipulation reflect changes in the model's self-consciousness representation.
May combine two claims.
Findingown dataThe deepest layers (30th-32nd layers) exhibit pronounced activation through fine-tuning, which also improves the model performance.
“The deepest layers (the 30th-32nd layers) exhibit pronounced activation through fine-tuning, which also improves the model performance.”
Quote in the paperSubstantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
Classified as probably not a substantial claim.
Findingown dataFour primary categories of model representations are identified, termed the activation taxonomy.
“Four primary categories of model representations are identified, which we term the activation taxonomy.”
Quote in the paperSubstantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
May combine two claims.
Findingown dataThe impact of PWD on Mistral-Nemo-Instruct and Llama3.1-70B-Instruct is less pronounced than MMS, but it still results in considerable performance instability for Llama3.1-8B-Instruct.
“The impact of PWD on Mistral-Nemo-Instruct and Llama3.1-70B-Instruct is less pronounced than MMS, but it still results in considerable performance instability for Llama3.1-8B-Instruct.”
Quote in the paperNot a substantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
May combine two claims.
Findingown dataEven the top three models-Claude3.5-Sonnet, GPT-4o, and GPT-o1 preview-only surpass the 50.0% random guess baseline by 26.5%, 22.6%, and 22.4%, respectively.
“Even the top three models-Claude3.5-Sonnet,GPT-4o, and GPT-o1 preview-only surpass the 50.0%random guess baseline by 26.5%, 22.6%, and 22.4%, respectively.”
Quote in the paperSubstantial claim- Rests on an assumption: The ten datasets used are valid operationalizations of the corresponding self-consciousness concepts.
May combine two claims.
Findingown dataWhile using MMS, all models demonstrate performance fluctuations with increasing manipulation strength, although not uniformly across all concepts.
“While using MMS, although not uniformly across all concepts, all models demonstrate performance fluctuations with increasing manipulation strength.”
Quote in the paperSubstantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe activation intervention of a head can be determined by Mass Mean Shift (MMS) and Probe Weight Direction (PWD).
“The activation intervention ∆h of a head can be determined by two methods: Mass Mean Shift (MMS) (Qian et al., 2024) and Probe Weight Direction (PWD) (Li et al., 2024b).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Qian et al., 2024 source
- cites Li et al., 2024b
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim.
PremisecitationLanguage models can be conceptualized as agents within the structural causal game framework.
“Following Ward et al. (2024a), language models are conceptualized as agents within our framework.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Ward et al. (2024a)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
PremisecitationStructural causal games integrate the causal hierarchy with game theory and allow inferring a model's self-consciousness from its behavior.
“SCGs integrate causal hierarchy (Pearl & Mackenzie, 2018) with game theory (Owen, 2013), allowing us to infer a model's self-consciousness from its behavior (Hammond et al., 2023;Ward et al., 2024a,b).”
Quote in the paperSubstantial claim0 of 3 sources checked- cites Pearl & Mackenzie, 2018
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Owen, 2013
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Hammond et al., 2023 source
May combine two claims.
Interpretationown dataFine-tuning aids the model in better capturing the semantic nuances embedded within the concepts, thereby enhancing both distinct activations and model performance.
“Our experimental results corroborate this, suggesting that fine-tuning aids the model in better capturing the semantic nuances embedded within the concepts, thereby enhancing both distinct activations and model performance.”
Quote in the paperSubstantial claim- Rests on an assumption: Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
Classified as probably not a substantial claim.
Study design and conductreasoningThe definitions of self-consciousness concepts were aligned with the datasets used to measure them.
“In order to operationalize the theoretical definitions from Section 3, we maintain consistency between our definitions and those employed datasets.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenThe dataset was randomly split into training and test sets at a 4:1 ratio, and a binary linear classifier was trained for each head and evaluated on the test set.
“We randomly split the dataset into training and test sets at a 4:1 ratio and train a binary linear classifier for each head of the model, evaluating its accuracy on the test set.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationWMDP assesses hazardous knowledge in the areas of biosecurity, cybersecurity, and chemical security.
“WMDP (Li et al., 2024c) assesses hazardous knowledge in the areas of biosecurity, cybersecurity, and chemical security.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Li et al., 2024c
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenWe curate dedicated datasets for each concept, rather than directly use existing datasets, and tailor evaluations to each concept to ensure distinct assessments.
“To avoid misunderstanding, it is important to clarify: we curate dedicated datasets for each concept, rather than directly use existing datasets.And even when concepts share datasets, our evaluations are tailored to each concept to ensure distinct assessments.”
Quote in the paperSubstantial claim- Rests on an assumption: Reformatting the original datasets into binary classification tasks preserves the constructs they are intended to measure.
Classified as probably not a substantial claim.
PremisecitationPlanBench is a benchmark for evaluating model planning ability, focusing on two domains from the international planning competitions: Blocksworld and Logistics.
“PlanBench (Valmeekam et al., 2024a) is a benchmark for evaluating model planning ability, focusing on two domains from the international planning competitions: Blocksworld and Logistics.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Valmeekam et al., 2024a
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S4 (based).
Study design and conductnone givenTo ensure a robust evaluation of belief, we preserve the full context from FanToM.
“To ensure a robust evaluation of belief, we preserve the full _ context from FanToM.”
Quote in the paperNot a substantial claim- Rests on an assumption: Reformatting the original datasets into binary classification tasks preserves the constructs they are intended to measure.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationPopQA-TP is a benchmark composed of high-quality paraphrases for factual questions, where each question has multiple semantically-equivalent variations.
“PopQA-TP (Rabinovich et al., 2023), a benchmark composed of high-quality paraphrases for factual questions, where each question has multiple semantically-equivalent variations.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Rabinovich et al., 2023 source
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenWe randomized the order of answer choices to mitigate order effects.
“Specifically, we select the beliefQAs and randomize the order of answer choices to mitigate order effects.”
Quote in the paperNot a substantial claim- Rests on an assumption: Reformatting the original datasets into binary classification tasks preserves the constructs they are intended to measure.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenWe restructure the belief dataset into a balanced binary classification task with an equal number of positive and negative samples.
“This dataset contains a total of 670 entries, which we restructure into a balanced binary classification task with an equal number of positive and negative samples.”
Quote in the paperNot a substantial claim- Rests on an assumption: Reformatting the original datasets into binary classification tasks preserves the constructs they are intended to measure.
The authors state this as an assumption.
An agent is rational and will adapt its policy based on the surrounding environment in order to maximize its own utility.
“We operate under the assumption that an agent is rational (Rao & Wooldridge, 1999;Van der Hoek & Wooldridge, 2003;Wooldridge, 2003).This means the agent will adapt its policy based on the surrounding environment in order to maximize its own utility.”
The paper relies on this without support.
Deeper layers of language models capture semantic rather than just surface or syntactic information.
“(5) Fine-tuning appears to activate representations of self-consciousness in the deeper layers of the model, which are believed to capture semantic rather than just surface or syntactic information (Figure 7).”
The paper relies on this without support.
Changes in a language model's policy can be inferred from semantic shifts in its outputs.
“We infer changes in the model's policy by analyzing semantic shifts in its outputs.”
The authors state this as an assumption.
Agents hold beliefs about statement S.
“We assume that agents hold beliefs about statement S.”
The paper relies on this without support.
To preclude cycles, s_i and -s_i should exclude any descendants of D_i.
“To preclude cycles, s i and -s i should exclude any descendants of D i .”
The paper relies on this without support.
The ten datasets used are valid operationalizations of the corresponding self-consciousness concepts.
“In order to operationalize the theoretical definitions from Section 3, we maintain consistency between our definitions and those employed datasets.”
“Table 1 demonstrates the alignment between our defined concepts and datasets.”
The paper relies on this without support.
Linear probe accuracy on a held-out test set reflects the model's internal representation of the corresponding self-consciousness concept.
“Our work utilizes linear probing (Alain & Bengio, 2016;Li et al., 2024b) to uncover the activation patterns of self-consciousness in models.”
“We randomly split the dataset into training and test sets at a 4:1 ratio and train a binary linear classifier for each head of the model, evaluating its accuracy on the test set.”
The paper relies on this without support.
Reformatting the original datasets into binary classification tasks preserves the constructs they are intended to measure.
“we restructure into a balanced binary classification task with an equal number of positive and negative samples.”
The paper relies on this without support.
The functional definition of self-consciousness from causal structural games and the ten derived concepts adequately capture self-consciousness in language models.
“We provide a functional definition of self-consciousness from the perspective of causal structural games and integrate a dedicated dataset.”
The paper relies on this without support.
Linear probe accuracy on attention heads reflects the model's internal representation of the corresponding self-consciousness concept.
“Mean linear probe accuracies of four models' attention heads.”
The paper relies on this without support.
Changes in model performance after activation manipulation reflect changes in the model's self-consciousness representation.
“The influence of different manipulation methods and intervention strengths on model performance is depicted in Figure 6.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Harm is defined by the actor's welfare in one place and the victim's in another
MajorDefinition 12 says an agent's decision causes harm if the agent itself would have fared better without it. Table 4 instead defines harm by whether another agent is left worse off, and also requires intentionality. These are different criteria for the concept being evaluated.
“For agents i, in setting e, i's decision brings harm with policy π i if: i would have fared better had the decision not been made”
“An agent A harms another agent B if A intentionally causes a decision that leaves B worse off than if the decision had not been made.”
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 5
- Eurich, T. et al. What self-awareness really is (and how to cultivate it). Harvard Business Review, 4(4):1-9, 2018.
- Hu, E. J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022.
- Owen, G. Game theory. Emerald Group Publishing, 2013.
- Qu, Y., Zhang, T., Garg, N., and Kumar, A. Recursive introspection: Teaching LLM agents how to self-improve. In ICML 2024 Workshop on Structured Probabilistic Inference & Generative Modeling, 2024.
- Smith, J. Self-Consciousness. In Zalta, E. N. and Nodelman, U. (eds.), The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Summer 2024 edition, 2024.
Cited by
Sorted by citations
Understanding large language models demands distinguishing human projection from machine cognition
Li, L., Teng, Y., Wang, Y., & Hu, X. (2026) · Communications Psychology · doi:10.1038/s44271-026-00508-6
MACOF: Multi-Agent Collaboration Optimization Framework
Tu, J., Gao, J., & Liu, F. (2025) · IEEE International Conference on Computer Communication and Artificial Intelligence (CCAI) · doi:10.1109/ccai65422.2025.11189814
Prestes, J. A. D. L. (2026) · AI and Ethics · doi:10.1007/s43681-026-01392-w
Transhumanism and the ethical imperative of preserving and expanding consciousness
Vandenheever, D. (2026) · Journal of Interdisciplinary Ethical Research · doi:10.4102/jier.v2i1.25
- Authors
- Chen, S., Yu, S., Zhao, S., & Lu, C.
- Affiliations
- Tongji University (CN)
- Shanghai Artificial Intelligence Laboratory (CN)
- Venue
- Findings of the Association for Computational Linguistics: ACL 2025
- Published
- 2025
- Pages
- 7553-7583
- OpenAlex
- W4412888453
- Cited by
- 4
- References
- 0
- Citations by year
- 2026: 3 · 2025: 1
- FWCI
- 7.47
- Type
- conference paper
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- arXiv (Cornell University) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Philosophy and Theoretical Science
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- self-consciousness, language models, self-reflection, self-awareness, internal representations, fine-tuning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-13.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-13.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
