Self-Referential Introspection in Large Language Models: The Critical Threshold for Recursive Self-Improvement
Open accessThe DOI matches its Crossref record; 57 of the 61 references it cites were found in a registry or an index, and 3 are not checked yet; 41 of the 84 sources its claims cite were checked for support.
Abstract
The pursuit of self-evolving AI raises a critical question: when is autonomous self-improvement sustainable rather than degenerative? Drawing an analogy to von Neumann’s complexity threshold for self-reproducing automata, we argue that sustainable recursive self-improvement in large language models (LLMs) requires a functional analogue: introspection—the system’s capacity to simulate its own operations and target modifications. Grounded in Kleene’s Second Recursion Theorem, we construct such introspective self-improvement programs and prove their key properties: completeness of self-modification, necessity of the reflective architecture, undecidability of improvement in general, and equivalence with Schmidhuber’s Gödel machine under a rewrite-equivalence notion, which transfers the global optimality guarantee. An empirical review, organized around these functional criteria, suggests that current LLMs exhibit only quasi-introspection.The available evidence does not establish complete introspection in the formal sense developed here, while pointing to several candidate structural bottlenecks, including incomplete self-access, feedforward processing, and limited computational depth. We outline architectural paths toward the threshold and discuss the safety implications of crossing it.
Verified 2026-11-19 · 3 references not checked yet
Overview
CoChat Summary
TL;DR
The paper argues that sustained recursive self-improvement in LLMs requires a formal introspection threshold—an effective self-model capable of simulating and modifying its own operations—analogous to von Neumann’s complexity threshold. It constructs introspective self-improvement programs via Kleene’s Second Recursion Theorem and reviews empirical evidence, concluding current LLMs show only quasi-introspection and have not been shown to cross the threshold.
Key findings
- Using Kleene’s Second Recursion Theorem, the paper constructs an introspection program \(e^*\) with \(\varphi_{e^*}\simeq\varphi_{f_T(e^*)}\), and an introspective self-improvement program \(\omega\) with \(\varphi_\omega\simeq\varphi_{M\circ V\circ f_T(\omega)}\); the first such program is treated as the threshold analogous to von Neumann’s complexity threshold.
- Formal properties include completeness of self-modification (Theorem A1), necessity of the reflective architecture (Theorem A2), undecidability of improvement in general (Theorem A3, via Rice’s theorem), and equivalence with Schmidhuber’s Gödel machine under a rewrite-equivalence notion, transferring the global optimality guarantee (Theorem A4/A5).
- The empirical review concludes current LLMs exhibit only quasi-introspection: fragmentary, approximate, domain-specific self-knowledge lacking the completeness, reliability, and causal grounding required by the formal definition. Candidate bottlenecks include incomplete self-access, feedforward processing—log-precision Transformers are bounded by uniform TC0—and limited computational depth; self-reports may lack causal grounding.
- Cited evidence includes Self-Refine plateauing after a few iterations, Song et al. finding no privileged self-access across 21 LLMs, and safety implications: if the threshold is crossable, completeness means safety constraints within the source are reachable and rewritable, so Corollary A1 requires an enforcement boundary outside the program’s write perimeter.
Methods
- Theoretical/computability construction: analogizes von Neumann’s self-reproducing automata and Quine programs, then uses Kleene’s Second Recursion Theorem to define introspection and introspective self-improvement programs; formal properties are proved in appendices.
- Empirical review: surveys literature from 2022–2026 organized around four functional properties—self-modeling, self-simulation, self-evaluation, self-modification—and structural properties of reflexivity and recurrent processing, distinguishing behavioral reports from architectural evidence.
- No new experiments are reported; the analysis combines formal proofs, comparison with the Gödel machine, and a literature-based assessment of current LLMs.
Limitations
- Stated: the formal results establish properties of explicitly defined introspective programs, while their applicability to contemporary LLM architectures remains conditional and empirically unresolved; the paper does not claim a universal scalar metric for LLM complexity or threshold.
- Stated: the necessity claim does not guarantee monotonically increasing empirical performance, which depends on the evaluation function, modification operator, objective, and viability conditions; the empirical survey is not intended to directly test or verify the formal framework.
- Evident: the paper reports no original empirical data and relies on cited studies, including preliminary/preprint evidence and conflicting findings on privileged self-access; the formal existence proof is conditional on a bounded simulation horizon \(T\).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
![Figure 1 Recursive self-improvement and introspection in current LLMs. The diagram relates recursive self-improvement to complexity and introspection thresholds, summarizes evidence for quasi-introspective capabilities in current LLMs, and distinguishes proposed structural limitations from corresponding research directions. Dashed lines denote the proposed introspection threshold. For the corresponding references, see [4,25,26].](https://figures.cochat.ai/paper-figures/fce0e838f12e44d89f5ecf8c66d8ad35/self-referential-introspection-in-large-language-models-the-critical-threshold-figure-1.png)



Verification
Claims the paper makes
31 Valid · 10 Not valid · 33 Not checked · 99 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“Li et al. [48] revealed that while models exhibit robust mission awareness (understanding their purpose), capability awareness (grasping functional limits) remains the weakest cognitive dimension.”
§ Self-Referential Introspection in Large Language Models: The Critical Threshold for Recursive Self-Improvement > 4. Quasi-Introspection in Current LLMs > 4.2. Self-Modeling: Knowledge of One’s Own Capabilities and Limitations
What the sources say
- Disputedcites [48] Li Y. Huang Y. Lin Y. Wu S. Wan Y. Sun L. I Think, Therefore I Am: Benchmarking Awareness of Large Language Models Using AwareBench arXiv 2024 10.48550/arXiv.2401.17882 2401.17882 sourceExperimental — accuracy not yet measured
Section 5.2 reports strong mission-awareness performance for more than half the models on explicit multiple-choice questions, but says performance degrades dramatically on implicit questions and models almost fail on open-ended questions. It concludes that their safety protocols are not robust against persuasive adversarial prompts, contrary to the unqualified claim of robust mission awareness. (LLM, conf=0.89)
Checked by a reasoning model — may be wrong.
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“This parity implies that LLMs infer internal states via behavioral pattern matching rather than possessing genuine introspective access to their execution substrate.”
§ Self-Referential Introspection in Large Language Models: The Critical Threshold for Recursive Self-Improvement > 4. Quasi-Introspection in Current LLMs > 4.2. Self-Modeling: Knowledge of One’s Own Capabilities and Limitations
What the sources say
- Partly supportedcites [49] Song S. Lederman H. Hu J. Mahowald K. Privileged Self-Access Matters for Introspection in AI arXiv 2025 10.48550/arXiv.2508.14802 2508.14802 sourceExperimental — accuracy not yet measured
The Introduction reports no advantage for models judging their own sampling temperature over judging another model’s, and says their reports can rely on surface-level hints in generated text. But the claim extends this result to LLMs’ internal states and execution substrate generally; the Conclusion limits its finding to the models tested and does not rule out introspection in larger or better models. The indexed passages provide section labels but no page numbers. (LLM, conf=0.86)
- Disputedcites [50] Comsa I.M. Shanahan M. Does It Make Sense to Speak of Introspection in Large Language Models? arXiv 2025 10.48550/arXiv.2506.05068 2506.05068 sourceExperimental — accuracy not yet measured
Section 3.2 says the LLM has no direct access to its sampling-temperature parameter, but can estimate it from its own outputs; the authors say this process conforms to their lightweight definition of introspection. The Discussion likewise calls the causal connection from the parameter through the model’s output to its self-report a potential instance of bona fide introspection. The claim incorrectly treats inference from behavior as excluding genuine introspection. (LLM, conf=0.89)
Checked by a reasoning model — may be wrong.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“Steyvers and Peters [58] systematically demonstrated that while LLMs produce well-calibrated confidence judgments”
§ Self-Referential Introspection in Large Language Models: The Critical Threshold for Recursive Self-Improvement > 4. Quasi-Introspection in Current LLMs > 4.4. Self-Evaluation: Detecting One’s Own Errors
What the sources say
- Disputedcites [58] Steyvers M. Peters M.A.K. Metacognition and Uncertainty Communication in Humans and Large Language Models Curr. Dir. Psychol. Sci. 2025 35 131 139 10.1177/09637214251391158 sourceExperimental — accuracy not yet measured
The comparison in §4.2 (p. 6) says default LLMs have limited capacity to report calibrated numeric confidence and tend to be overconfident. It adds that fine-tuning can improve calibration, but does not support the unqualified claim that LLMs produce well-calibrated confidence judgments. (LLM, conf=0.90)
Checked by a reasoning model — may be wrong.
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Yet a sobering empirical pattern has emerged: most autonomous self-improvement loops exhibit rapid asymptotic saturation. For instance, the Self-Refine paradigm plateaus after a mere few iterations [7].”
§ Self-Referential Introspection in Large Language Models: The Critical Threshold for Recursive Self-Improvement > 1. Introduction: The Bet on Self-Evolving AI and Introspection
What the sources say
- Partly supportedcites [7] Madaan A. Tandon N. Gupta P. Hallinan S. Gao L. Wiegreffe S. Alon U. Dziri N. Prabhumoye S. Yang Y. Self-Refine: Iterative Refinement with Self-Feedback Proceedings of the NeurIPS 2023 New Orleans, LA, USA 10–16 December 2023 10.52202/075280-2019 sourceExperimental — accuracy not yet measured
Section 4 (Analysis; page not given) reports diminishing returns over three Self-Refine iterations, with most gains occurring early. It also says scores generally keep improving with further iterations; it does not establish that Self-Refine plateaus after a few iterations. The citation is attached to the Self-Refine example, not the broader claim about most autonomous loops. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Li et al. [48] revealed that while models exhibit robust mission awareness (understanding their purpose), capability awareness (grasping functional limits) remains the weakest cognitive dimension.”
§ Self-Referential Introspection in Large Language Models: The Critical Threshold for Recursive Self-Improvement > 4. Quasi-Introspection in Current LLMs > 4.2. Self-Modeling: Knowledge of One’s Own Capabilities and Limitations
What the sources say
- Partly supportedcites [48] Li Y. Huang Y. Lin Y. Wu S. Wan Y. Sun L. I Think, Therefore I Am: Benchmarking Awareness of Large Language Models Using AwareBench arXiv 2024 10.48550/arXiv.2401.17882 2401.17882 sourceExperimental — accuracy not yet measured
Section 3.1 defines capability awareness as understanding the limits of a model’s abilities, and Section 5.2 reports that most models perform poorly on it. But Section 5.2 says mission awareness is strong only on explicit questions; performance drops dramatically on implicit questions and is poor on open-ended ones. Table 1 also does not establish capability awareness as uniformly the weakest dimension. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
93 more claims · 21 with their cited sources checked
Create a free account to see all 173 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 42
- It may state two separate claims as one. 48
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Self-Referential Introspection in Large Language Models: The Critical Threshold for Recursive Self-Improvement > 3. Self-Reference: From Self-Reproducing to Introspection > 3.4. AI Self-Evolution and Introspection
“By reconfiguring M and V to achieve functional alignment with the Gödel Machine framework as detailed in the Appendix B, we can formally establish the global optimality of the self-improvement sequence S (see Theorem A5 in Appendix C).”
§ Self-Referential Introspection in Large Language Models: The Critical Threshold for Recursive Self-Improvement > 6. Discussion > 6.2. Formal Obstacles to RSI
“logarithmic growth limits [80] and computability ceilings [77] dictate that an introspective self-improver inevitably forfeits global optimality, approaching it only asymptotically as T→∞.”
Citations
Cites
Self-Refine: Iterative Refinement with Self-Feedback
Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.17651 · Cited by 2,314 · cited 4×
“Empirically, Self-Refine [7] and Reflexion [6] operate at the prompt/context layer through textual self-critique.”
InfluentialThree approaches to the quantitative definition of information*
Kolmogorov, A. N. (1968) · International Journal of Computer Mathematics · doi:10.1080/00207166808803030 · Cited by 2,031 · cited 2×
“With modern notions of Kolmogrov complexity [18] and algorithmic information entropy [19], the complexity threshold can be quantified by the minimum description length of a self-reproducing automaton.”
Reflexion: Language Agents with Verbal Reinforcement Learning
Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S. (2023) · arXiv · doi:10.48550/arxiv.2303.11366 · Cited by 1,087 · cited 2×
“Empirically, Self-Refine [7] and Reflexion [6] operate at the prompt/context layer through textual self-critique.”
Rosenthal, D. M. (1986) · Philosophical Studies · doi:10.1007/bf00355521 · Cited by 1,019 · cited 1×
“Higher-Order Thought (HOT) theory [75] identifies consciousness with higher-order representations of one’s own mental states—a structure formally close to what we call introspection.”
On the Length of Programs for Computing Finite Binary Sequences
Chaitin, G. J. (1966) · Journal of the ACM · doi:10.1145/321356.321363 · Cited by 913 · cited 2×
“With modern notions of Kolmogrov complexity [18] and algorithmic information entropy [19], the complexity threshold can be quantified by the minimum description length of a self-reproducing automaton.”
AI models collapse when trained on recursively generated data
Shumailov, I., Shumaylov, Z., Zhao, Y., Papernot, N., Anderson, R., & Gal, Y. (2024) · Nature · doi:10.1038/s41586-024-07566-y · Cited by 793 · cited 2×
“Another critical obstacle to sustained self-improvement in LLMs is model collapse —a degenerative process wherein successive generations of generative models are progressively polluted by their own synthetic outputs [14].”
On Notation for Ordinal Numbers
Kleene, S. C. (1938) · Journal of Symbolic Logic · doi:10.2307/2267778 · Cited by 758 · cited 1×
“The deeper mathematical foundation for self-reference in computation is provided by Kleene’s Second Recursion Theorem [23,37].”
Speculations Concerning the First Ultraintelligent Machine
Good, I. J. (1966) · Advances in Computers · doi:10.1016/s0065-2458(08)60418-0 · Cited by 703 · cited 3×
“…Recursive Self-Improvement The hypothesis that machine intelligence could surpass human capabilities and trigger runaway growth dates back to Good [4], who observed that an ultraintelligent machine” could design increasingly superior systems, culminating in what he termed an intelligence…”
Self-reproduction in cellular automata
Langton, C. G. (1984) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(84)90256-2 · Cited by 571 · cited 2×
“This tension is epitomized by the historical debate on the necessity of universal constructors in self-reproducing automata [39–41]; while von Neumann’s open-ended replication relies on a universal constructor to support arbitrary mutant structures, self-reproducing loops can be implemented far…”
LLM Evaluators Recognize and Favor Their Own Generations
Panickssery, A., Bowman, S. R., & Feng, S. (2024) · arXiv · doi:10.48550/arxiv.2404.13076 · Cited by 559 · cited 1×
“Conversely, this internal model exhibits a functional yet biased form of self-recognition: Panickssery et al. [59] showed that LLMs can distinguish their own outputs from human or exter-nal ones with non-trivial accuracy, a capability that correlates linearly with an intrinsic self-preference bias.”
Research works printed but not linked: 17
- Huang J. Chen X. Mishra S. Zheng H.S. Yu A.W. Song X. Zhou D. Large Language Models Cannot Self-Correct Reasoning Yet Proceedings of the Twelfth International Conference on Learning Representations Vienna, Austria 7–11 May 2024
- Burks A.W. Theory of Self-Reproducing Automata (John von Neumann) University of Illinois Press Champaign, IL, USA 1966
- Rogers H. Theory of Recursive Functions and Effective Computability MIT Press Cambridge, MA, USA 1967
- Zelikman E. Lorch E. Mackey L. Kalai A.T. Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation Proceedings of the COLM 2024, Philadelphia, PA, USA 7–9 October 2024
- Bolander T. Logical Theories for Agent Introspection Ph.D. Thesis Technical University of Denmark Kongens Lyngby, Denmark 2003
- Ferrando J. Obeso O. Rajamanoharan S. Nanda N. Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models Proceedings of the ICLR 2025 Singapore 24–28 April 2025
- Binder F.J. Chua J. Korbak T. Sleight H. Hughes J. Long R. Perez E. Turpin M. Evans O. Looking Inward: Language Models Can Learn About Themselves by Introspection Proceedings of the Thirteenth International Conference on Learning Representations (ICLR 2025) Singapore 24–28 April 2025
- Song S. Hu J. Mahowald K. Language Models Fail to Introspect About Their Knowledge of Language Proceedings of the Conference on Language Modeling (COLM) Montreal, QC, Canada 7–10 October 2025
- Betley J. Bao X. Soto M. Sztyber-Betley A. Chua J. Evans O. Tell Me About Yourself: LLMs Are Aware of Their Learned Behaviors Proceedings of the Thirteenth International Conference on Learning Representations (ICLR) Singapore 24–28 April 2025
- Merrill W. Sabharwal A. The Expressive Power of Transformers with Chain of Thought Proceedings of the Twelfth International Conference on Learning Representations (ICLR 2024) Vienna, Austria 7–11 May 2024
- Dehghani M. Gouws S. Vinyals O. Uszkoreit J. Kaiser Ł. Universal Transformers Proceedings of the ICLR 2019, New Orleans, LA, USA 6–9 May 2019
- Hao S. Sukhbaatar S. Su A. Li X. Hu Z. Weston J. Tian Y. Training Large Language Models to Reason in a Continuous Latent Space Proceedings of the Thirteenth International Conference on Learning Representations (ICLR 2025) Singapore 24–28 April 2025
- Jaegle A. Gimeno F. Brock A. Zisserman A. Vinyals O. Carreira J. Perceiver: General Perception with Iterative Attention Proceedings of the ICML 2021 Virtual 18–24 July 2021
- Baars B.J. A Cognitive Theory of Consciousness Cambridge University Press New York, USA 1988
- Mahoney, M. (2026, July 10). A Model for Recursively Self Improving Programs, 2010. Self-Published Preprint (Final Version v.3, December 2010)
- Mahoney M. A Model for Recursively Self Improving Programs, 2010. Self-Published Preprint (Final Version v.3, December 2010); An Earlier Version Was Submitted to the Journal of Artificial General Intelligence in 2008 Available online: https://www.academia.edu/66745654/A_Model_for_Recursively_Self_Improving_Programs (accessed on 10 July 2026)
- Nivel E. Thórisson K.R. Steunebrink B.R. Dindo H. Pezzulo G. Rodríguez M. Hernández C. Ognibene D. Schmidhuber J. Sanz R. Bounded Recursive Self-Improvement Proceedings of the AGI 2013 Beijing, China 31 July–3 August 2013
Cited by
Tan, Z., Sun, L., Shi, Z., Liu, Y., Wu, J., Brooks, N., Qian, J., Shang, X., Huang, S., Ding, Y., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.06396 · Cited by 14
Self-citationDalek: A Constructive Agent MachinePreprint
Xie, W. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.03546
Metadata
- Authors
- Zhang, J., Yuan, B., & Zhang, Q.
- Affiliations
- Beijing Normal University (CN)
- Swarma Research (CN)
- Corresponding author
- Zhang, Jiang
- Venue
- Entropy
- Publisher
- MDPI AG
- Published
- 2026-08-24
- Volume
- 28
- Issue
- 9
- Pages
- 951
- ISSN
- 1099-4300 (ISSN-L)
- PMC
- PMC13604973
- OpenAlex
- W7204071795
- Cited by
- 2
- References
- 31
- Citations by year
- 2026: 2
- FWCI
- 4.99
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Entropy (publishedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Computability, Logic, AI Algorithms
- Physical Sciences › Computer Science › Computational Theory and Mathematics › semigroups and automata theory
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Keywords
- self-reflection, large language models, recursive self-improvement, self-modification, self-reproducing automata, undecidability, AI safety, reflective architecture
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.