Towards an Actual Gödel Machine Implementation: a Lesson in Self-Reflective Systems
Abstract
Recently, interest has been revived in self-reflective systems in the context of Artificial General Intelligence (AGI). An AGI system should be intelligent enough to be able to reason about its own program code, and make modifications where it sees fit, improving on the initial code written by human programmers. A pertinent example is the Gödel Machine, which employs a proof searcher— in parallel to its regular problem solves duties—to find a self-rewrite of which it can prove that it will be beneficial. Obviously there are technical challenges involved in attaining such a level of self-reflection in an AGI system, but many of them are not widely known or properly appreciated. In this chapter we go back to the theoretical foundations of self-reflection and examine the (often subtle) issues encountered when embarking on actually implementing a self-reflective AGI system in general and a Gödel Machine in particular.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Our work is first to leverage LMs for recursively self-improving code generation.
“our work is first to leverage LMs for recursively self-improving code generation.”
From Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation (2023) · cites this paper as Steunebrink & Schmidhuber, 2012
Cites
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972 · see citing works
Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme I
Gödel, K. (2006) · Monatshefte für Mathematik · doi:10.1007/s00605-006-0423-7 · Cited by 2,096 · see citing works
Structure and Interpretation of Computer Programs
Abelson, H., Sussman, G. J., & Sussman, J. (1985) · Cited by 1,439 · see citing works
Revised 3 report on the algorithmic language scheme
Rees, J., & Clinger, W. (1986) · ACM SIGPLAN Notices · doi:10.1145/15042.15043 · Cited by 509 · see citing works
Reflection and semantics in LISP
Smith, B. C. (1984) · ACM SIGACT-SIGPLAN Symposium on Principles of Programming Languages (POPL) · doi:10.1145/800017.800513 · Cited by 445 · see citing works
Gödel Machines: Fully Self-referential Optimal Universal Self-improvers
Schmidhuber, J. (2007) · Cognitive Technologies · doi:10.1007/978-3-540-68677-4_7 · Cited by 110 · see citing works
Self-citationFriedman, D. P., & Wand, M. (1984) · ACM Conference on LISP and Functional Programming (LFP) · doi:10.1145/800055.802051 · Cited by 108 · see citing works
Artificial Neural Networks: Biological Inspirations – ICANN 2005
(2005) · Lecture notes in computer science · doi:10.1007/11550822 · Cited by 96 · see citing works
The implementation of procedurally reflective languages
Rivières, J. D., & Smith, B. C. (1984) · ACM Conference on LISP and Functional Programming (LFP) · doi:10.1145/800055.802050 · Cited by 82 · see citing works
The mystery of the tower revealed: A nonreflective description of the reflective tower
Wand, M., & Friedman, D. P. (1988) · LISP and Symbolic Computation · doi:10.1007/bf01806174 · Cited by 79 · see citing works
The mystery of the tower revealed
Wand, M., & Friedman, D. P. (1986) · ACM Conference on LISP and Functional Programming (LFP) · doi:10.1145/319838.319871 · Cited by 64 · see citing works
THE FASTEST AND SHORTEST ALGORITHM FOR ALL WELL-DEFINED PROBLEMS
Hütter, M. (2002) · International Journal of Foundations of Computer Science · doi:10.1142/s0129054102001199 · Cited by 61 · see citing works
Schmidhuber, J. (2009) · Cognitive Computation · doi:10.1007/s12559-009-9014-y · Cited by 59 · see citing works
Self-citationIntensions and extensions in a reflective tower
Olivier Danvy, A., & Malmkjær, K. (1988) · ACM Conference on LISP and Functional Programming (LFP) · doi:10.1145/62678.62725 · Cited by 59 · see citing works
Queinnec, C. (1996) · Cambridge University Press eBooks · doi:10.1017/cbo9781139172974 · Cited by 42 · see citing works
Completely Self-referential Optimal Reinforcement Learners
Schmidhuber, J. (2005) · Lecture notes in computer science · doi:10.1007/11550907_36 · Cited by 28 · see citing works
Self-citationA simple reflective interpreter
Jefferson, S., & Friedman, D. P. (1996) · LISP and Symbolic Computation · doi:10.1007/bf01806112 · Cited by 28 · see citing works
Reification without evaluation
Bawden, A. (1988) · ACM Conference on LISP and Functional Programming (LFP) · doi:10.1145/62678.62726 · Cited by 25 · see citing works
Schmidhuber, J. (2003) · arXiv · doi:10.48550/arxiv.cs/0309048 · Cited by 24 · see citing works
Self-citationA Family of Gödel Machine Implementations
Steunebrink, B. R., & Schmidhuber, J. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-22887-2_29 · Cited by 14 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Zenil, H., Badillo, L., Hernández-Orozco, S., & Hernández-Quiroz, F. (2018) · International Journal of Parallel Emergent and Distributed Systems · doi:10.1080/17445760.2018.1448932 · Cited by 33 · see citing works
Learning in brain and machine—complexity, Gödel, Aristotle
PERLOVSKY, L. I. (2013) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2013.00023 · Cited by 17 · see citing works
A Review and Proposed Framework for Artificial General Intelligence
Long, L. N., & Cotner, C. F. (2019) · IEEE Aerospace Conference (AERO) · doi:10.1109/aero.2019.8742125 · Cited by 15 · see citing works
A Brain-Inspired Cognitive Control Framework for Artificial Intelligence Dynamic System
Yalawar, M. S., Babu, K. V., Mahender, B., & Singh, H. (2023) · Cognitive science and technology · doi:10.1007/978-981-19-8086-2_70 · Cited by 9 · see citing works
Gödel Agent: A Self-Referential Agent Framework for Recursively Self-Improvement
Yin, X., Wang, X., Pan, L., Lin, L., Wan, X., & Wang, W. Y. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.1354 · Cited by 5 · see citing works
Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation
Zelikman, E., Lorch, E., Mackey, L., & Kalai, A. T. (2023) · arXiv · doi:10.48550/arxiv.2310.02304 · Cited by 4 · see citing works
“Regarding implementations, while efforts have been made for Gödel machines (Hall, 2007; Steunebrink & Schmidhuber, 2012), our work is first to leverage LMs for recursively self-improving code generation.”
MethodGödel Agent: A Self-Referential Agent Framework for Recursive Self-Improvement
Yin, X., Wang, X., Pan, L., Lin, L., Wan, X., & Wang, W. Y. (2024) · arXiv · doi:10.48550/arxiv.2410.04444 · Cited by 2 · see citing works
“In the early days, there were also some discussions of self-improving agents that were not based on LLM (Hall, 2007; Steunebrink and Schmidhuber, 2012).”
Continually Learning Planning Agent for Large Environments guided by LLMs
Paul, S. K. (2024) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai59869.2024.00076 · Cited by 2 · see citing works
Paul, S. K., & Bhaumik, P. (2021) · IETE Journal of Research · doi:10.1080/03772063.2021.2004461 · Cited by 2 · see citing works
Do the Self-Knowing Machines Dream of Knowing Their Factivity?
Aldini, A., Fano, V., & Graziani, P. (2015) · CINECA IRIS Institutional Research information system (University of Urbino) · Cited by 1 · see citing works
Towards Formalization of Constructivist Seed AI
Paul, S. K., & Bhaumik, P. (2022) · Lecture notes in networks and systems · doi:10.1007/978-981-19-2130-8_6
Deconstructing Superintelligence: Identity, Self-Modification and Différance
Perrier, E. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-33195-3_14
- Authors
- Steunebrink, B. R., & Schmidhuber, J.
- Affiliations
- Dalle Molle Institute for Artificial Intelligence Research (CH)
- Venue
- Atlantis Thinking Machines
- Publisher
- Atlantis Press
- Published
- 2012
- Pages
- 173-195
- ISSN
- 1877-3273 (ISSN-L)
- OpenAlex
- W2103519081
- Cited by
- 11
- References
- 20
- Citations by year
- 2026: 1 · 2024: 1 · 2023: 1 · 2022: 1 · 2021: 1 · 2019: 1 · 2018: 1 · 2017: 1 · 2016: 1 · 2015: 1 · 2013: 1
- FWCI
- 1.14
- Type
- book chapter
- Language
- en
- Open access
- closed
- Licence
- Locations
- Atlantis thinking machines (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Computability, Logic, AI Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Keywords
- artificial general intelligence, proof search, self-modification, implementation barriers, self-reflection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
