Self-Referential Introspection in Large Language Models: The Critical Threshold for Recursive Self-Improvement
Open accessThe DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index; 41 of the 84 sources its claims cite were checked for support.
Abstract
The pursuit of self-evolving AI raises a critical question: when is autonomous self-improvement sustainable rather than degenerative? Drawing an analogy to von Neumann’s complexity threshold for self-reproducing automata, we argue that sustainable recursive self-improvement in large language models (LLMs) requires a functional analogue: introspection—the system’s capacity to simulate its own operations and target modifications. Grounded in Kleene’s Second Recursion Theorem, we construct such introspective self-improvement programs and prove their key properties: completeness of self-modification, necessity of the reflective architecture, undecidability of improvement in general, and equivalence with Schmidhuber’s Gödel machine under a rewrite-equivalence notion, which transfers the global optimality guarantee. An empirical review, organized around these functional criteria, suggests that current LLMs exhibit only quasi-introspection.The available evidence does not establish complete introspection in the formal sense developed here, while pointing to several candidate structural bottlenecks, including incomplete self-access, feedforward processing, and limited computational depth. We outline architectural paths toward the threshold and discuss the safety implications of crossing it.
Overview
CoChat Summary
TL;DR
The paper argues that sustained recursive self-improvement in LLMs requires a formal introspection threshold—an effective self-model capable of simulating and modifying its own operations—analogous to von Neumann’s complexity threshold. It constructs introspective self-improvement programs via Kleene’s Second Recursion Theorem and reviews empirical evidence, concluding current LLMs show only quasi-introspection and have not been shown to cross the threshold.
Key findings
- Using Kleene’s Second Recursion Theorem, the paper constructs an introspection program \(e^*\) with \(\varphi_{e^*}\simeq\varphi_{f_T(e^*)}\), and an introspective self-improvement program \(\omega\) with \(\varphi_\omega\simeq\varphi_{M\circ V\circ f_T(\omega)}\); the first such program is treated as the threshold analogous to von Neumann’s complexity threshold.
- Formal properties include completeness of self-modification (Theorem A1), necessity of the reflective architecture (Theorem A2), undecidability of improvement in general (Theorem A3, via Rice’s theorem), and equivalence with Schmidhuber’s Gödel machine under a rewrite-equivalence notion, transferring the global optimality guarantee (Theorem A4/A5).
- The empirical review concludes current LLMs exhibit only quasi-introspection: fragmentary, approximate, domain-specific self-knowledge lacking the completeness, reliability, and causal grounding required by the formal definition. Candidate bottlenecks include incomplete self-access, feedforward processing—log-precision Transformers are bounded by uniform TC0—and limited computational depth; self-reports may lack causal grounding.
- Cited evidence includes Self-Refine plateauing after a few iterations, Song et al. finding no privileged self-access across 21 LLMs, and safety implications: if the threshold is crossable, completeness means safety constraints within the source are reachable and rewritable, so Corollary A1 requires an enforcement boundary outside the program’s write perimeter.
Methods
- Theoretical/computability construction: analogizes von Neumann’s self-reproducing automata and Quine programs, then uses Kleene’s Second Recursion Theorem to define introspection and introspective self-improvement programs; formal properties are proved in appendices.
- Empirical review: surveys literature from 2022–2026 organized around four functional properties—self-modeling, self-simulation, self-evaluation, self-modification—and structural properties of reflexivity and recurrent processing, distinguishing behavioral reports from architectural evidence.
- No new experiments are reported; the analysis combines formal proofs, comparison with the Gödel machine, and a literature-based assessment of current LLMs.
Limitations
- Stated: the formal results establish properties of explicitly defined introspective programs, while their applicability to contemporary LLM architectures remains conditional and empirically unresolved; the paper does not claim a universal scalar metric for LLM complexity or threshold.
- Stated: the necessity claim does not guarantee monotonically increasing empirical performance, which depends on the evaluation function, modification operator, objective, and viability conditions; the empirical survey is not intended to directly test or verify the formal framework.
- Evident: the paper reports no original empirical data and relies on cited studies, including preliminary/preprint evidence and conflicting findings on privileged self-access; the formal existence proof is conditional on a bounded simulation horizon \(T\).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
31 Valid · 10 Not valid · 33 Not checked · 99 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“Li et al. [48] revealed that while models exhibit robust mission awareness (understanding their purpose), capability awareness (grasping functional limits) remains the weakest cognitive dimension.”
§ Self-Referential Introspection in Large Language Models: The Critical Threshold for Recursive Self-Improvement > 4. Quasi-Introspection in Current LLMs > 4.2. Self-Modeling: Knowledge of One’s Own Capabilities and Limitations
What the sources say
- Disputedcites [48] Li Y. Huang Y. Lin Y. Wu S. Wan Y. Sun L. I Think, Therefore I Am: Benchmarking Awareness of Large Language Models Using AwareBench arXiv 2024 10.48550/arXiv.2401.17882 2401.17882 sourceExperimental — accuracy not yet measured
Section 5.2 reports strong mission-awareness performance for more than half the models on explicit multiple-choice questions, but says performance degrades dramatically on implicit questions and models almost fail on open-ended questions. It concludes that their safety protocols are not robust against persuasive adversarial prompts, contrary to the unqualified claim of robust mission awareness. (LLM, conf=0.89)
Checked by a reasoning model — may be wrong.
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“This parity implies that LLMs infer internal states via behavioral pattern matching rather than possessing genuine introspective access to their execution substrate.”
§ Self-Referential Introspection in Large Language Models: The Critical Threshold for Recursive Self-Improvement > 4. Quasi-Introspection in Current LLMs > 4.2. Self-Modeling: Knowledge of One’s Own Capabilities and Limitations
What the sources say
- Partly supportedcites [49] Song S. Lederman H. Hu J. Mahowald K. Privileged Self-Access Matters for Introspection in AI arXiv 2025 10.48550/arXiv.2508.14802 2508.14802 sourceExperimental — accuracy not yet measured
The Introduction reports no advantage for models judging their own sampling temperature over judging another model’s, and says their reports can rely on surface-level hints in generated text. But the claim extends this result to LLMs’ internal states and execution substrate generally; the Conclusion limits its finding to the models tested and does not rule out introspection in larger or better models. The indexed passages provide section labels but no page numbers. (LLM, conf=0.86)
- Disputedcites [50] Comsa I.M. Shanahan M. Does It Make Sense to Speak of Introspection in Large Language Models? arXiv 2025 10.48550/arXiv.2506.05068 2506.05068 sourceExperimental — accuracy not yet measured
Section 3.2 says the LLM has no direct access to its sampling-temperature parameter, but can estimate it from its own outputs; the authors say this process conforms to their lightweight definition of introspection. The Discussion likewise calls the causal connection from the parameter through the model’s output to its self-report a potential instance of bona fide introspection. The claim incorrectly treats inference from behavior as excluding genuine introspection. (LLM, conf=0.89)
Checked by a reasoning model — may be wrong.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“Steyvers and Peters [58] systematically demonstrated that while LLMs produce well-calibrated confidence judgments”
§ Self-Referential Introspection in Large Language Models: The Critical Threshold for Recursive Self-Improvement > 4. Quasi-Introspection in Current LLMs > 4.4. Self-Evaluation: Detecting One’s Own Errors
What the sources say
- Disputedcites [58] Steyvers M. Peters M.A.K. Metacognition and Uncertainty Communication in Humans and Large Language Models Curr. Dir. Psychol. Sci. 2025 35 131 139 10.1177/09637214251391158 sourceExperimental — accuracy not yet measured
The comparison in §4.2 (p. 6) says default LLMs have limited capacity to report calibrated numeric confidence and tend to be overconfident. It adds that fine-tuning can improve calibration, but does not support the unqualified claim that LLMs produce well-calibrated confidence judgments. (LLM, conf=0.90)
Checked by a reasoning model — may be wrong.
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Yet a sobering empirical pattern has emerged: most autonomous self-improvement loops exhibit rapid asymptotic saturation. For instance, the Self-Refine paradigm plateaus after a mere few iterations [7].”
§ Self-Referential Introspection in Large Language Models: The Critical Threshold for Recursive Self-Improvement > 1. Introduction: The Bet on Self-Evolving AI and Introspection
What the sources say
- Partly supportedcites [7] Madaan A. Tandon N. Gupta P. Hallinan S. Gao L. Wiegreffe S. Alon U. Dziri N. Prabhumoye S. Yang Y. Self-Refine: Iterative Refinement with Self-Feedback Proceedings of the NeurIPS 2023 New Orleans, LA, USA 10–16 December 2023 10.52202/075280-2019 sourceExperimental — accuracy not yet measured
Section 4 (Analysis; page not given) reports diminishing returns over three Self-Refine iterations, with most gains occurring early. It also says scores generally keep improving with further iterations; it does not establish that Self-Refine plateaus after a few iterations. The citation is attached to the Self-Refine example, not the broader claim about most autonomous loops. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Li et al. [48] revealed that while models exhibit robust mission awareness (understanding their purpose), capability awareness (grasping functional limits) remains the weakest cognitive dimension.”
§ Self-Referential Introspection in Large Language Models: The Critical Threshold for Recursive Self-Improvement > 4. Quasi-Introspection in Current LLMs > 4.2. Self-Modeling: Knowledge of One’s Own Capabilities and Limitations
What the sources say
- Partly supportedcites [48] Li Y. Huang Y. Lin Y. Wu S. Wan Y. Sun L. I Think, Therefore I Am: Benchmarking Awareness of Large Language Models Using AwareBench arXiv 2024 10.48550/arXiv.2401.17882 2401.17882 sourceExperimental — accuracy not yet measured
Section 3.1 defines capability awareness as understanding the limits of a model’s abilities, and Section 5.2 reports that most models perform poorly on it. But Section 5.2 says mission awareness is strong only on explicit questions; performance drops dramatically on implicit questions and is poor on open-ended ones. Table 1 also does not establish capability awareness as uniformly the weakest dimension. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
93 more claims · 21 with their cited sources checked
Create a free account to see all 173 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 42
- It may state two separate claims as one. 48
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Self-Referential Introspection in Large Language Models: The Critical Threshold for Recursive Self-Improvement > 3. Self-Reference: From Self-Reproducing to Introspection > 3.4. AI Self-Evolution and Introspection
“By reconfiguring M and V to achieve functional alignment with the Gödel Machine framework as detailed in the Appendix B, we can formally establish the global optimality of the self-improvement sequence S (see Theorem A5 in Appendix C).”
§ Self-Referential Introspection in Large Language Models: The Critical Threshold for Recursive Self-Improvement > 6. Discussion > 6.2. Formal Obstacles to RSI
“logarithmic growth limits [80] and computability ceilings [77] dictate that an introspective self-improver inevitably forfeits global optimality, approaching it only asymptotically as T→∞.”
Citations
Cites
Self-Refine: Iterative Refinement with Self-Feedback
Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.17651 · Cited by 2,314 · cited 4×
“Empirically, Self-Refine [7] and Reflexion [6] operate at the prompt/context layer through textual self-critique.”
InfluentialThree approaches to the quantitative definition of information*
Kolmogorov, A. N. (1968) · International Journal of Computer Mathematics · doi:10.1080/00207166808803030 · Cited by 2,031 · cited 2×
“With modern notions of Kolmogrov complexity [18] and algorithmic information entropy [19], the complexity threshold can be quantified by the minimum description length of a self-reproducing automaton.”
Reflexion: Language Agents with Verbal Reinforcement Learning
Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S. (2023) · arXiv · doi:10.48550/arxiv.2303.11366 · Cited by 1,087 · cited 2×
“Empirically, Self-Refine [7] and Reflexion [6] operate at the prompt/context layer through textual self-critique.”
Rosenthal, D. M. (1986) · Philosophical Studies · doi:10.1007/bf00355521 · Cited by 1,019 · cited 1×
“Higher-Order Thought (HOT) theory [75] identifies consciousness with higher-order representations of one’s own mental states—a structure formally close to what we call introspection.”
On the Length of Programs for Computing Finite Binary Sequences
Chaitin, G. J. (1966) · Journal of the ACM · doi:10.1145/321356.321363 · Cited by 913 · cited 2×
“With modern notions of Kolmogrov complexity [18] and algorithmic information entropy [19], the complexity threshold can be quantified by the minimum description length of a self-reproducing automaton.”
AI models collapse when trained on recursively generated data
Shumailov, I., Shumaylov, Z., Zhao, Y., Papernot, N., Anderson, R., & Gal, Y. (2024) · Nature · doi:10.1038/s41586-024-07566-y · Cited by 793 · cited 2×
“Another critical obstacle to sustained self-improvement in LLMs is model collapse —a degenerative process wherein successive generations of generative models are progressively polluted by their own synthetic outputs [14].”
On Notation for Ordinal Numbers
Kleene, S. C. (1938) · Journal of Symbolic Logic · doi:10.2307/2267778 · Cited by 758 · cited 1×
“The deeper mathematical foundation for self-reference in computation is provided by Kleene’s Second Recursion Theorem [23,37].”
Speculations Concerning the First Ultraintelligent Machine
Good, I. J. (1966) · Advances in Computers · doi:10.1016/s0065-2458(08)60418-0 · Cited by 703 · cited 3×
“…Recursive Self-Improvement The hypothesis that machine intelligence could surpass human capabilities and trigger runaway growth dates back to Good [4], who observed that an ultraintelligent machine” could design increasingly superior systems, culminating in what he termed an intelligence…”
Self-reproduction in cellular automata
Langton, C. G. (1984) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(84)90256-2 · Cited by 571 · cited 2×
“This tension is epitomized by the historical debate on the necessity of universal constructors in self-reproducing automata [39–41]; while von Neumann’s open-ended replication relies on a universal constructor to support arbitrary mutant structures, self-reproducing loops can be implemented far…”
LLM Evaluators Recognize and Favor Their Own Generations
Panickssery, A., Bowman, S. R., & Feng, S. (2024) · arXiv · doi:10.48550/arxiv.2404.13076 · Cited by 559 · cited 1×
“Conversely, this internal model exhibits a functional yet biased form of self-recognition: Panickssery et al. [59] showed that LLMs can distinguish their own outputs from human or exter-nal ones with non-trivial accuracy, a capability that correlates linearly with an intrinsic self-preference bias.”
Artificial General Intelligence
(2007) · Cognitive Technologies · doi:10.1007/978-3-540-68677-4 · Cited by 473
Reflection and semantics in LISP
Smith, B. C. (1984) · Proceedings of the 11th ACM SIGACT-SIGPLAN symposium on Principles of programming languages - POPL '84 · doi:10.1145/800017.800513 · Cited by 444
Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents
Zhang, J., Hu, S., Lu, C., Lange, R., & Clune, J. (2025) · arXiv · doi:10.48550/arxiv.2505.22954 · Cited by 344 · cited 3×
“Consequently, even the most sophisticated self-evolving frameworks—such as the Darwin Gödel Machine [12]—remain strictly tethered to external benchmarks and human-engineered evaluation criteria to sustain behavioral progress.”
Self-citationA Synergistic Core for Human Brain Evolution and Cognition
Luppi, A. I., Mediano, P. A. M., Rosas, F. E., Holland, N., Fryer, T. D., O’Brien, J. T., Rowe, J. B., Menon, D. K., Bor, D., & Stamatakis, E. A. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01070-0 · Cited by 314 · cited 1×
“Recently, Luppi et al. [74] proposed the “synergistic core” hypothesis, positing that specific sub-networks execute distributed, information-theoretic synergy to integrate and broadcast state variables.”
The Singularity: A Philosophical Analysis
Chalmers, D. J. (2016) · Science Fiction and Philosophy · doi:10.1002/9781118922590.ch16 · Cited by 246 · cited 2×
“For the corresponding references, see [4,25,26].”
Fifty Years of Research on Self-Replication: An Overview
Sipper, M. (1998) · Artificial Life · doi:10.1162/106454698568576 · Cited by 231 · cited 1×
“This tension is epitomized by the historical debate on the necessity of universal constructors in self-reproducing automata [39–41]; while von Neumann’s open-ended replication relies on a universal constructor to support arbitrary mutant structures, self-reproducing loops can be implemented far…”
Large Language Models Can Self-Improve
Huang, J., Gu, S., Hou, L., Wu, Y., Wang, X., Yu, H., & Han, J. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.67 · Cited by 230
Large Language Models Can Self-Improve
Huang, J., Gu, S. S., Hou, L., Wu, Y., Wang, X., Yu, H., & Han, J. (2022) · arXiv · doi:10.48550/arxiv.2210.11610 · Cited by 230 · cited 1×
“Recent works have demonstrated LLMs that refine their own reasoning through self-reflection [6,7], optimize their own prompts [8], generate and evaluate their own training data [9,10], and even modify their own source code at runtime [11,12].”
Computability: An introduction to recursive function theory
Cutland, N. (1980) · doi:10.1017/cbo9781139171496 · Cited by 206
Consciousness in Artificial Intelligence: Insights from the Science of Consciousness
Butlin, P., Long, R., Elmoznino, E., Bengio, Y., Birch, J., Constant, A., Deane, G., Fleming, S. M., Frith, C., Ji, X., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.08708 · Cited by 196 · cited 2×
“Second, Perceiver architectures [68] enforce an explicit attentional bottleneck to compel comprehensive information integration; this topology serves as a computational analogue to Global Workspace Theory [69], a foundational prerequisite for self-awareness in cognitive science [70].”
Language Models (Mostly) Know What They Know
Kadavath, S., Conerly, T., Askell, A., Henighan, T., Drain, D., Perez, E., Schiefer, N., Hatfield-Dodds, Z., DasSarma, N., Tran-Johnson, E., et al. (2022) · arXiv · doi:10.48550/arxiv.2207.05221 · Cited by 187
Self-reproduction in small cellular automata
Byl, J. (1989) · Physica D Nonlinear Phenomena · doi:10.1016/0167-2789(89)90242-x · Cited by 136 · cited 1×
“…necessity of universal constructors in self-reproducing automata [39–41]; while von Neumann’s open-ended replication relies on a universal constructor to support arbitrary mutant structures, self-reproducing loops can be implemented far more simply by sacrificing this universal capability [39,40].”
Gödel Machines: Fully Self-referential Optimal Universal Self-improvers
Schmidhuber, J. (2007) · Cognitive Technologies · doi:10.1007/978-3-540-68677-4_7 · Cited by 110 · cited 3×
“Concurrently, Schmidhuber [24]’s Gödel Machine provided this concept with its strictest mathematical realization: an agent capable of rewriting any part of its own code, provided it can formally prove the modification beneficial.”
InfluentialGödel Agent: A Self-Referential Agent Framework for Recursively Self-Improvement
Yin, X., Wang, X., Pan, L., Lin, L., Wan, X., & Wang, W. Y. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.1354 · Cited by 105 · cited 3×
“If the unit is an LLM-based agent , the reflexive structure must extend over the whole system—scaffolding and the invoked model; the Gödel Agent [11] rewrites its own orchestration code but not the model the code orchestrates, so its self-modification remains partial.”
Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
Chen, X., Zhao, A., Xia, H., Lu, X., Wang, H., Chen, Y., Zhang, W., Wang, J., Li, W., & Shen, X. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.55 · Cited by 96
AI, Human Cognition and Knowledge Collapse
Acemoğlu, D., Kong, D., & Ozdaglar, A. (2026) · National Bureau of Economic Research · doi:10.3386/w34910 · Cited by 93 · cited 1×
“…this effect globally, as AI-generated text permeates the web and shifts human reliance, the broader knowledge economy risks tipping into a steady state of “knowledge collapse,” where foundational general knowledge gradually vanishes despite the availability of high-quality personalized advice [16].”
Space-time dynamics in video feedback
Crutchfield, J. P. (1984) · Physica D Nonlinear Phenomena · doi:10.1016/0167-2789(84)90264-1 · Cited by 52 · cited 1×
“It establishes a closed loop where a system points to itself, exemplified linguistically by autological sentences (e.g., “This sentence contains seven words” ) and physically by camera-screen video feedback loops [35].”
Utility function security in artificially intelligent agents
Yampolskiy, R. V. (2014) · Journal of Experimental & Theoretical Artificial Intelligence · doi:10.1080/0952813x.2014.895114 · Cited by 47 · cited 1×
“This gives formal content to the “leakproofing” problem [26] and the utility-function security problem [81].”
Large Language Models Report Subjective Experience Under Self-Referential Processing
Berg, C., de Lucena, D., & Rosenblatt, J. (2025) · arXiv · doi:10.48550/arxiv.2510.24797 · Cited by 38 · cited 2×
“…reported self-reflective or bliss-like attractor states [42], are learned linguistic patterns: they constitute behavioral data, not evidence of phenomenal states (see [76] for a philosophical argument and [43] for an empirical study of the coupling between self-reference and such self-reports).”
From Seed AI to Technological Singularity via Recursively Self-Improving SoftwarePreprint
Yampolskiy, R. V. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.06512 · Cited by 33 · cited 3×
“This gives formal content to the “leakproofing” problem [26] and the utility-function security problem [81].”
Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution
Fernando, C., Banarse, D., Michalewski, H., Osindero, S., & Rocktäschel, T. (2023) · arXiv · doi:10.48550/arxiv.2309.16797 · Cited by 31 · cited 1×
“Recent works have demonstrated LLMs that refine their own reasoning through self-reflection [6,7], optimize their own prompts [8], generate and evaluate their own training data [9,10], and even modify their own source code at runtime [11,12].”
The Parallelism Tradeoff: Limitations of Log-Precision Transformers
Merrill, W., & Sabharwal, A. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00562 · Cited by 22 · cited 2×
“The standard Transformer, however, is fundamentally feedforward, so a single forward pass of a log-precision Transformer is bounded by uniform TC 0 [45] and cannot execute fixed-point iterations.”
Recursive Introspection: Teaching Language Model Agents How to Self-Improve
Qu, Y., Zhang, T., Garg, N., & Kumar, A. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-1754 · Cited by 21 · cited 2×
“Conversely, RISE [32] cuts across these layers by fine-tuning models to iteratively revise failed execution attempts.”
Nature-Inspired Population-Based Evolution of Large Language Models
Zhang, Y., Ye, P., Yang, X., Feng, S., Zhang, S., Bai, L., Ouyang, W., & Hu, S. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.2044 · Cited by 21
Metacognition and Uncertainty Communication in Humans and Large Language Models
Steyvers, M., & Peters, M. A. K. (2026) · Current Directions in Psychological Science · doi:10.1177/09637214251391158 · Cited by 18 · cited 1×
“Steyvers and Peters [58] systematically demonstrated that while LLMs produce well-calibrated confidence judgments, this capability relies on surface-level statistical regularities rather than genuine metacognitive monitoring, masking a fundamental mechanistic divergence from human cognition.”
Self-Rewarding Language Models
Yuan, W., Pang, R. Y., Cho, K., Li, X., Sukhbaatar, S., Xu, J., & Weston, J. (2024) · arXiv · doi:10.48550/arxiv.2401.10020 · Cited by 18 · cited 1×
“Recent works have demonstrated LLMs that refine their own reasoning through self-reflection [6,7], optimize their own prompts [8], generate and evaluate their own training data [9,10], and even modify their own source code at runtime [11,12].”
Chang, O., & Lipson, H. (2018) · The 2018 Conference on Artificial Life · doi:10.1162/isal_a_00049 · Cited by 16 · cited 1×
“While a quasi-reflexive framework implemented via neural Quines [72] or Hyper networks [73] can manifest such approximate topologies, continuous approximations fundamentally undermine the formal safeguards of introspective systems, leaving the dynamical stability of continuous recursive…”
A Computability Argument Against Superintelligence
Wiedermann, J. (2012) · Cognitive Computation · doi:10.1007/s12559-012-9124-9 · Cited by 12 · cited 2×
“First, Rice’s theorem dictates that no general algorithm can decide whether an arbitrary program modification constitutes an improvement [77], ostensibly ruling out the total evaluation function V introduced in Section 3.4.”
InfluentialAlphaApollo: A System for Deep Agentic ReasoningPreprint
Zhou, Z., Cao, C., Feng, X., Li, X., Li, Z., Lu, X., Yao, J., Huang, W., Cheng, T., Zhang, J., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.06261 · Cited by 12 · cited 1×
“SkillRL [28] and AlphaApollo [29] inhabit the memory/skill layer by accumulating reusable libraries or tools.”
Self-citationScaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
Geiping, J., McLeish, S., Jain, N., Kirchenbauer, J., Singh, S., Bartoldson, B., Kailkhura, B., Bhatele, A., & Goldstein, T. (2025) · Advances in Neural Information Processing Systems 38 · doi:10.52202/085713-1380 · Cited by 10
Model Collapse Demystified: The Case of Regression
Dohmatob, E., Feng, Y., & Kempe, J. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-1490 · Cited by 10
A Survey on Self-Evolution of Large Language Models
Tao, Z., Lin, T.-E., Chen, X., Li, H., Wu, Y., Li, Y., Jin, Z., Huang, F., Tao, D., & Zhou, J. (2024) · arXiv · doi:10.48550/arxiv.2404.14387 · Cited by 9 · cited 2×
“Rather than duplicating comprehensive existing surveys [1–3], our analysis untangles the fundamental structural constraints that compel these diverse architectures to externalize their verification signals.”
Me, Myself, and AI: The Situational Awareness Dataset (SAD) for LLMs
Laine, R., Chughtai, B., Betley, J., Hariharan, K., Scheurer, J., Balesni, M., Hobbhahn, M., Meinke, A., & Evans, O. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-2044 · Cited by 8
Me, Myself, and AI: The Situational Awareness Dataset (SAD) for LLMs
Laine, R., Chughtai, B., Betley, J., Hariharan, K., Scheurer, J., Balesni, M., Hobbhahn, M., Meinke, A., & Evans, O. (2024) · arXiv · doi:10.48550/arxiv.2407.04694 · Cited by 8
Long Term Memory: The Foundation of AI Self-Evolution
Jiang, X., Li, F., Zhao, H., Qiu, J., Wang, J., Shao, J., Xu, S., Zhang, S., Chen, W., Tang, X., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.15665 · Cited by 7 · cited 1×
“Third, memory-augmented agents treating long-term storage as a first-class primitive [71] persistently accumulate introspective state histories across interactions, establishing a form of temporal self-continuity.”
Fang, J., Peng, Y., Zhang, X., Wang, Y., Yi, X., Zhang, G., Xu, Y., Wu, B., Liu, S., Li, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2508.07407 · Cited by 6
Does It Make Sense to Speak of Introspection in Large Language Models?
Comsa, I. M., & Shanahan, M. (2025) · arXiv · doi:10.48550/arxiv.2506.05068 · Cited by 3 · cited 1×
“Although Song et al. [49]—extended from Comsa and Shanahan [50]—demonstrated that LLMs can precisely predict their own operational parameters (such as temperature), the models exhibit no accuracy advantage when predicting their own parameters versus those of a peer system.”
Gao, H.-A., Geng, J., Hua, W., Hu, M., Juan, X., Liu, H., Liu, S., Qiu, J., Qi, X., Wu, Y., et al. (2025) · arXiv · doi:10.48550/arxiv.2507.21046 · Cited by 3
Self-citationEmergent Introspective Awareness in Large Language Models
Lindsey, J. (2026) · arXiv · doi:10.48550/arxiv.2601.01828 · Cited by 2 · cited 1×
“…and Africa [54], who showed that models trained to detect hidden activation injections still fail to resist such manipulations, and by Lindsey [55], who noted that functional awareness for modulating internal concepts in advanced architectures (e.g., Claude 4) remains highly sensitive to…”
Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models
Lu, H., Liu, Y., Xu, J., Nan, G., Yu, Y., Chen, Z., & Wang, K. (2025) · Advances in Neural Information Processing Systems 38 · doi:10.52202/085713-5426 · Cited by 2
Hypernetworks That Evolve Themselves
Pedersen, J. W., Plantec, E., Nisioti, E., Barylli, M., Montero, M., Korte, K., & Risi, S. (2025) · ALIFE 2025: Ciphers of Life: Proceedings of the Artificial Life Conference 2025 · doi:10.1162/isal.a.845 · Cited by 1 · cited 1×
“While a quasi-reflexive framework implemented via neural Quines [72] or Hyper networks [73] can manifest such approximate topologies, continuous approximations fundamentally undermine the formal safeguards of introspective systems, leaving the dynamical stability of continuous recursive…”
Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activations
Li, J.-A., Xiong, H., Wilson, R., Mattar, M. G., & Benna, M. K. (2025) · Advances in Neural Information Processing Systems 38 · doi:10.52202/085713-2009 · Cited by 1
The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators
Iacob, A., Jovanović, A., Shen, W. F., Burkhardt, D., Kurmanji, M., Tastan, N., Sani, L., Venanzi, N. A. E., Nie, J., Odonnat, A., et al. (2026) · arXiv · doi:10.48550/arxiv.2606.26294 · Cited by 1 · cited 1×
“Alternatively, the Red Queen Gödel Machine (RQGM) [33] employs controlled utility evolution , co-evolving evaluators and agents within localized epochs to stabilize non-stationary optimization signals.”
Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
Chen, X., Zhao, A., Xia, H., Lu, X., Wang, H., Chen, Y., Zhang, W., Wang, J., Li, W., & Shen, X. (2025) · arXiv · doi:10.48550/arxiv.2505.16782 · Cited by 1 · cited 1×
“…unbounded iteration at the cost of parallel training efficiency and guaranteed termination, while latent CoT—recurrent reasoning in the hidden state, without token traces [64–66]—may soften the feedforward bottleneck, though it still iterates over activations rather than weight-level self-models.”
I Think, Therefore I am: Benchmarking Awareness of Large Language Models Using AwareBench
Li, Y., Huang, Y., Lin, Y., Wu, S., Wan, Y., & Sun, L. (2024) · arXiv · doi:10.48550/arxiv.2401.17882 · Cited by 1
Verbalizable Representations Form a Global Workspace in Language Models
Gurnee, W., Sofroniew, N., Pearce, A., Piotrowski, M., Kauvar, I., Chen, R., Soligo, A., Bogdan, P., Ong, E., Wang, R., et al. (2026) · arXiv · doi:10.48550/arxiv.2607.15495 · Cited by 1 · cited 1×
“More recently, the Anthropic Transformer Circuits Team formalized this capacity for model introspection by demonstrating that LLMs maintain a privileged internal “J-space” [61].”
SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning
Xia, P., Chen, J., Wang, H., Liu, J., Zeng, K., Wang, Y., Han, S., Zhou, Y., Zhao, X., Chen, H., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.08234
Steering Awareness: Detecting Activation Steering from Within
Rivera, J. F., & Africa, D. D. (2025) · arXiv · doi:10.48550/arxiv.2511.21399 · cited 1×
“This fragility is echoed by Fonseca Rivera and Africa [54], who showed that models trained to detect hidden activation injections still fail to resist such manipulations, and by Lindsey [55], who noted that functional awareness for modulating internal concepts in advanced architectures (e.g.,…”
Privileged Self-Access Matters for Introspection in AI
Song, S., Lederman, H., Hu, J., & Mahowald, K. (2025) · arXiv · doi:10.48550/arxiv.2508.14802 · cited 2×
“To formalize this distinction, Song et al. [49] proposed a rigorous criterion: genuine introspection requires an agent to acquire information regarding its internal states via a mechanism that is demonstrably more reliable or computationally less costly than third-party observation of those same…”
Vaugrante, L., Weckauff, A., & Hagendorff, T. (2026) · arXiv · doi:10.48550/arxiv.2602.14777 · cited 1×
“Bridging self-evaluation and alignment safety, Vaugrante et al. [60] analyzed mis-aligned models and found that they can accurately self-report elevated harm scores without explicit contextual prompting, tracking their actual harmful behavior closely.”
(2026) · doi:10.48550/arxiv.2604.12128 · cited 2×
“A preliminary empirical study by Bae [44], currently available as a preprint, examined how LLMs respond to paradoxical self-reference involving non-closing truth recursion (NCTR), such as the Liar’s Paradox.”
Research works printed but not linked: 17
- Huang, J., Chen, X., Mishra, S., Zheng, H.S., Yu, A.W., Song, X., and Zhou, D. (2024, January 7–11). Large Language Models Cannot Self-Correct Reasoning Yet. Proceedings of the Twelfth International Conference on Learning Representations, Vienna, Austria.
- Burks, A.W. (1966). Theory of Self-Reproducing Automata (John von Neumann), University of Illinois Press.
- Rogers, H. (1967). Theory of Recursive Functions and Effective Computability, MIT Press.
- Zelikman, E., Lorch, E., Mackey, L., and Kalai, A.T. (2024, January 7–9). Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation. Proceedings of the COLM 2024, Philadelphia, PA, USA.
- Bolander, T. (2003). Logical Theories for Agent Introspection. [Ph.D. Thesis, Technical University of Denmark].
- Ferrando, J., Obeso, O., Rajamanoharan, S., and Nanda, N. (2025, January 24–28). Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models. Proceedings of the ICLR 2025, Singapore.
- Binder, F.J., Chua, J., Korbak, T., Sleight, H., Hughes, J., Long, R., Perez, E., Turpin, M., and Evans, O. (2025, January 24–28). Looking Inward: Language Models Can Learn About Themselves by Introspection. Proceedings of the Thirteenth International Conference on Learning Representations (ICLR 2025), Singapore.
- Song, S., Hu, J., and Mahowald, K. (2025, January 7–10). Language Models Fail to Introspect About Their Knowledge of Language. Proceedings of the Conference on Language Modeling (COLM), Montreal, QC, Canada.
- Betley, J., Bao, X., Soto, M., Sztyber-Betley, A., Chua, J., and Evans, O. (2025, January 24–28). Tell Me About Yourself: LLMs Are Aware of Their Learned Behaviors. Proceedings of the Thirteenth International Conference on Learning Representations (ICLR), Singapore.
- Merrill, W., and Sabharwal, A. (2024, January 7–11). The Expressive Power of Transformers with Chain of Thought. Proceedings of the Twelfth International Conference on Learning Representations (ICLR 2024), Vienna, Austria.
- Dehghani, M., Gouws, S., Vinyals, O., Uszkoreit, J., and Kaiser, Ł. (2019, January 6–9). Universal Transformers. Proceedings of the ICLR 2019, New Orleans, LA, USA.
- Hao, S., Sukhbaatar, S., Su, A., Li, X., Hu, Z., Weston, J., and Tian, Y. (2025, January 24–28). Training Large Language Models to Reason in a Continuous Latent Space. Proceedings of the Thirteenth International Conference on Learning Representations (ICLR 2025), Singapore.
- Jaegle, A., Gimeno, F., Brock, A., Zisserman, A., Vinyals, O., and Carreira, J. (2021, January 18–24). Perceiver: General Perception with Iterative Attention. Proceedings of the ICML 2021, Virtual.
- Baars, B.J. (1988). A Cognitive Theory of Consciousness, Cambridge University Press.
- Mahoney, M. (2026, July 10). A Model for Recursively Self Improving Programs, 2010. Self-Published Preprint (Final Version v.3, December 2010)
- An Earlier Version Was Submitted to the Journal of Artificial General Intelligence in 2008. Available online: https://www.academia.edu/66745654/A_Model_for_Recursively_Self_Improving_Programs.
- Nivel, E., Thórisson, K.R., Steunebrink, B.R., Dindo, H., Pezzulo, G., Rodríguez, M., Hernández, C., Ognibene, D., Schmidhuber, J., and Sanz, R. (August, January 31). Bounded Recursive Self-Improvement. Proceedings of the AGI 2013, Beijing, China.
Cited by
Tan, Z., Sun, L., Shi, Z., Liu, Y., Wu, J., Brooks, N., Qian, J., Shang, X., Huang, S., Ding, Y., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.06396 · Cited by 14
Self-citationDalek: A Constructive Agent MachinePreprint
Xie, W. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.03546
Metadata
- Authors
- Zhang, J., Yuan, B., & Zhang, Q.
- Affiliations
- Beijing Normal University (CN)
- Swarma Research (CN)
- Corresponding author
- Zhang, Jiang
- Venue
- Entropy
- Publisher
- MDPI AG
- Published
- 2026-08-24
- Volume
- 28
- Issue
- 9
- Pages
- 951
- ISSN
- 1099-4300 (ISSN-L)
- PMC
- PMC13604973
- OpenAlex
- W7204071795
- Cited by
- 2
- References
- 31
- Citations by year
- 2026: 2
- FWCI
- 4.99
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Entropy (publishedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Computability, Logic, AI Algorithms
- Physical Sciences › Computer Science › Computational Theory and Mathematics › semigroups and automata theory
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Keywords
- self-reflection, large language models, recursive self-improvement, self-modification, self-reproducing automata, undecidability, AI safety, reflective architecture
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.