The DOI matches its DataCite record; 59 of the 59 references it cites were found in a registry or an index; 10 of the 39 sources its claims cite were checked for support.
Levels of AGI for Operationalizing Progress on the Path to AGI
Open accessAbstract
We propose a framework for classifying the capabilities and behavior of Artificial General Intelligence (AGI) models and their precursors. This framework introduces levels of AGI performance, generality, and autonomy, providing a common language to compare models, assess risks, and measure progress along the path to AGI. To develop our framework, we analyze existing definitions of AGI, and distill six principles that a useful ontology for AGI should satisfy. With these principles in mind, we propose "Levels of AGI" based on depth (performance) and breadth (generality) of capabilities, and reflect on how current systems fit into this ontology. We discuss the challenging requirements for future benchmarks that quantify the behavior and capabilities of AGI models against these levels. Finally, we discuss how these levels of AGI interact with deployment considerations such as autonomy and risk, and emphasize the importance of carefully selecting Human-AI Interaction paradigms for responsible and safe deployment of highly capable AI systems.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This position paper proposes a leveled ontology for AGI that separates performance (depth) from generality (breadth), with six principles for operationalizing AGI. It argues current frontier language models are best classified as Level 1 “Emerging AGI” and that no public system has reached “Competent AGI” as of September 2023.
Key findings
- Proposes a Levels of AGI matrix: performance levels Emerging, Competent (≥50th percentile of skilled adults), Expert (≥90th), Exceptional (≥99th), and Superhuman (outperforms 100% of humans), crossed with Narrow vs. General generality; Competent AGI, Expert AGI, Exceptional AGI, and ASI are listed as not yet achieved.
- Distills six principles for an operationalizable AGI definition: focus on capabilities not processes; generality and performance; cognitive/metacognitive rather than physical tasks; potential not deployment; ecological validity; and the path to AGI rather than a single endpoint.
- Classifies current frontier LLMs (ChatGPT, Bard, Llama 2, Gemini) as Level 1 “Emerging AGI” overall, with “Competent” performance on some tasks (e.g., short essay writing, simple coding) but “Emerging” performance on most tasks (e.g., mathematical abilities, factuality).
- Introduces six Levels of Autonomy (No AI, AI as Tool, AI as Consultant, AI as Collaborator, AI as Expert, AI as Agent) and argues autonomy is unlocked but not determined by AGI level, with risk depending on the combination.
Methods
- Position/analysis paper: examines nine prominent AGI definitions (Turing Test, strong AI, brain analogies, human-level cognitive tasks, ability to learn tasks, economically valuable work, flexible/general “Coffee Test,” Artificial Capable Intelligence, and SOTA LLMs as generalists) and distills six principles.
- Proposes a two-dimensional leveled taxonomy (performance × generality) plus a companion autonomy taxonomy, and discusses requirements for future AGI benchmarks and risk assessment.
- No empirical study, dataset, or benchmark is introduced; classifications of example systems are described as approximate.
Limitations
- Stated: the authors do not propose a benchmark; they leave open what task set constitutes generality, what proportion of tasks must be mastered, and whether certain tasks are mandatory, calling instead for a living benchmark.
- Stated: assignment of example systems to cells is approximate, unambiguous classification requires a standardized benchmark, and the risk analysis is incomplete; predictions about which autonomy levels are unlocked are speculative.
- Evident: as a position paper, it provides no empirical validation of the levels, no inter-rater reliability for system classifications, and no quantitative evaluation of the proposed framework.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
46 Own interpretation · 19 Sources not found · 7 No basis given · 6 Partly supported · 2 Supported · 1 No full text of the sources · accuracy not yet measured
Every source the paper cites for this claim was checked and supports it.
In the paper
“has the additional downside of potentially introducing alignment risks (Kenton et al., 2021) by only targeting fiscal profit.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 2. Defining AGI: Case Studies
What the sources say
- Supportedcites Kenton et al., 2021 source
Section 3.1, AI Alignment (p. 3), says that optimizing the wrong objective can lead to bad outcomes, supporting the cited point about potential alignment risks. The fiscal-profit framing is the citing author's application of that general risk, not a claim that Kenton et al. discuss profit specifically. (LLM, conf=0.87)
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“there is no scientific consensus on methods for determining whether machines possess strong AI attributes such as consciousness (Butlin et al., 2023)”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 2. Defining AGI: Case Studies
What the sources say
- Partly supportedcites Butlin et al., 2023 sourceExperimental — accuracy not yet measured
The Introduction says expert opinions about AI consciousness are highly divergent, and the Executive Summary describes the authors’ assessment rubric as provisional. This supports the lack of a settled approach to assessing consciousness, but the paper does not address methods for determining other “strong AI attributes.” (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“similarly, it is not a necessary precursor for AGI that systems possess qualities such as consciousness (subjective awareness) (Butlin et al., 2023) or sentience (the ability to have feelings), since these qualities have a process focus.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 3. Defining AGI: Six Principles
What the sources say
- Partly supportedcites Butlin et al., 2023 sourceExperimental — accuracy not yet measured
Section 4.2 (page unavailable) says we may find ways to build high-performing AI systems that are not conscious, supporting the idea that consciousness need not accompany greater AI capability. It does not specifically establish that consciousness is unnecessary for AGI, and it frames the possibility cautiously. The citation is attached to consciousness, not the separate claim about sentience. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Despite recent advances in robotics (Brohan et al., 2023), physical capabilities for AI systems seem to be lagging behind non-physical capabilities.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 3. Defining AGI: Six Principles
What the sources say
- Supportedcites Brohan et al., 2023 source
The citation is attached to “recent advances in robotics,” not to the citing authors’ assessment that physical capabilities lag behind non-physical ones. The Introduction (p. 3) describes RT-2’s new robotic capabilities, including improved generalization and deploying learned physical skills in new ways. (LLM, conf=0.91)
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“we posit that AlphaFold (Jumper et al., 2021;Varadi et al., 2021) is a Level 5 Narrow AI (“Superhuman Narrow AI”) since it performs a single task (predicting a protein's 3D structure from an amino acid sequence) above the level of the world's top scientists.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > Competent AGI not yet achieved
What the sources say
- Partly supportedcites Jumper et al., 2021 sourceExperimental — accuracy not yet measured
The AlphaFold network section describes predicting a protein’s 3D coordinates from its amino acid sequence and related inputs. The Main section reports that AlphaFold was substantially more accurate than competing methods in the CASP14 assessment, but it does not compare its performance directly with the world’s top scientists. The Level 5 label is the citing authors’ own proposed classification. (LLM, conf=0.84)
- Partly supportedcites Varadi et al., 2021 sourceExperimental — accuracy not yet measured
The Introduction (page unavailable) identifies AlphaFold as an AI system that predicts protein structures from amino-acid sequences and describes its accuracy as unprecedented. It does not compare AlphaFold’s performance with that of the world’s top scientists. The Level 5 classification is explicitly the citing authors’ own proposal, not a classification attributed to this source. (LLM, conf=0.85)
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We estimate the “Expert” level of performance since DALLE-2 produces images of higher quality than most people are able to draw”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > Competent AGI not yet achieved
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“however, the system has failure modes (e.g., drawing hands with incorrect numbers of digits, rendering nonsensical or illegible text) that prevent it from achieving an “Exceptional” performance designation.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > Competent AGI not yet achieved
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Consider DALLE-2 (Ramesh et al., 2022), which we estimate as a Level 3 Narrow AI (“Expert Narrow AI”) in our taxonomy.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > Competent AGI not yet achieved
What the sources say
- Not matchedcites Ramesh et al., 2022
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“the success of transformer-based architectures (Vaswani et al., 2023) whose performance is not reliant on humanlike learning suggests that strict brain-based processes and benchmarks are not inherently necessary for AGI.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 2. Defining AGI: Case Studies
What the sources say
- Not matchedcites Vaswani et al., 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Given that modern LLMs pass some framings of the Turing Test, it seems clear that this criteria is insufficient for operationalizing or benchmarking AGI.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 2. Defining AGI: Case Studies
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Modern LLMs pass some framings of the Turing Test.
- Probably not a substantial claim. 53
- It may state two separate claims as one. 9
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Conversely, the role of human-AI interaction research can be viewed as ensuring new AI systems are usable by and useful to people such that AI systems successfully extend people's capabilities (i.e.,”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6.3. Human-AI Interaction and Risk Assessment
What the sources say
- Partly supportedcites Brynjolfsson, 2022 sourceExperimental — accuracy not yet measured
The paper argues that AI can augment human capabilities, enabling people to do things they could not do before (p. 3), and that augmentation opens new opportunities (p. 10). It does not describe the role of human-AI interaction research or say that ensuring systems are usable and useful is how that augmentation is achieved. (LLM, conf=0.84)
- Not matchedcites Englebart, 1962
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“we posit that as of this writing in September 2023, frontier language models (e.g., ChatGPT (OpenAI, 2023), Bard (Anil et al., 2023), Llama2 (Touvron et al., 2023), etc.) exhibit “Competent” performance levels for some tasks (e.g., short essay writing, simple coding), but are still at “Emerging” performance levels for most tasks (e.g., mathematical abilities, tasks involving factuality).”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 4. Levels of AGI
What the sources say
- Supportedcites OpenAI, 2023 source
The citation is attached to ChatGPT as an example of a frontier language model, rather than to the citing authors’ proposed “Competent” and “Emerging” ratings. The report identifies ChatGPT as a deployment of GPT-4 (Economic Impacts, p. 58) and discusses earlier ChatGPT versions based on GPT-3.5 (Limitations, p. 10), supporting that identification. (LLM, conf=0.87)
- Supportedcites Anil et al., 2023 source
The citation is attached to “Bard” in the list of example models, rather than to the citing authors’ performance assessment. The PaLM 2 model card (Additional information; page unavailable) identifies Bard as an application built on PaLM 2, supporting that identification. (LLM, conf=0.90)
- Not matchedcites Touvron et al., 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Unlike prior taxonomies of computer automation (Sheridan et al., 1978;Sheridan & Parasuraman, 2005;Parasuraman et al., 2000) that take a computer-centric perspective (fram-ing automation in terms of how much control the designer relinquishes to computers), we characterize the concept of autonomy through the lens of the nature of human-AI interaction style”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6.2. Capabilities vs. Autonomy
What the sources say
- Not matchedcites Sheridan et al., 1978
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites Sheridan & Parasuraman, 2005
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Partly supportedcites Parasuraman et al., 2000 sourceExperimental — accuracy not yet measured
Section III (p. 2) describes automation levels as increasing the computer’s autonomy over human action, supporting the claim’s description of how levels are framed. But Section II (p. 1) says the model is guided by human-centered automation, and the Introduction (p. 1) makes human-performance consequences its primary design criterion. Calling this paper’s perspective simply computer-centric therefore overstates the contrast; the citing authors’ human-AI interaction approach is their own characterization, not a claim this source must establish. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The majority of definitions focus on what an AGI can accomplish, not on the mechanism by which it accomplishes tasks.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 3. Defining AGI: Six Principles
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The assignment of example systems to cells is approximate.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 4. Levels of AGI
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“All of the above definitions emphasize generality to varying degrees, but some exclude performance criteria.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 3. Defining AGI: Six Principles
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Most definitions focus on cognitive tasks, by which we mean non-physical tasks.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 3. Defining AGI: Six Principles
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The “Competent AGI” level, which has not been achieved by any public systems at the time of writing, best corresponds to many prior conceptions of AGI, and may precipitate rapid societal change once achieved.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 4. Levels of AGI
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“A shortcoming of this definition is that it does not capture all of the criteria that may be part of “general intelligence.””
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 2. Defining AGI: Case Studies
What the sources say
- Not matchedcites OpenAI, 2018
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
- Not matched
In the paper
“This definition captures both generality and performance (via the inclusion of reliability)”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 2. Defining AGI: Case Studies
What the sources say
- Not matchedcites Marcus, 2022b
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
- Not matched
In the paper
“such tasks may have more ecological validity than many current AI benchmarks”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 2. Defining AGI: Case Studies
What the sources say
- Not matchedcites Mustafa Suleyman and Michael Bhaskar, 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
- Not matched
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“This framing is more narrow than OpenAI's definition of economically valuable work”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 2. Defining AGI: Case Studies
What the sources say
- Not matchedcites Mustafa Suleyman and Michael Bhaskar, 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites OpenAI, 2018
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
In the paper
“Another challenge with framing AGI in terms of economic value is the implied need for deployment in order to realize that value, whereas a focus on capabilities might only require the potential for an AGI to execute a task.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 2. Defining AGI: Case Studies
What the sources say
- Not matchedcites OpenAI, 2018
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Not matched
In the paper
“While we agree that generality is a crucial characteristic of AGI, we posit that it must also be paired with a measure of performance (i.e., if an LLM can write code or perform math, but is not reliably correct, then its generality is not yet sufficiently performant).”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 2. Defining AGI: Case Studies
What the sources say
- Not matchedcites Agüera y Arcas & Norvig, 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This focus on capabilities implies that AGI systems need not necessarily think or understand in a human-like way (since this focuses on processes)”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 3. Defining AGI: Six Principles
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There are tasks associated with intelligence that may not have a well-defined economic value (e.g., artistic creativity or emotional intelligence).”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 2. Defining AGI: Case Studies
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Note that general systems that broadly perform at a level N may be able to perform a narrow subset of tasks at higher levels.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 4. Levels of AGI
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“This level-based approach to defining AGI supports the coexistence of many prominent formulations -for example, Aguera y Arcas & Norvig's definition (Agüera y Arcas & Norvig, 2023) would fall into the “Emerging AGI” category of our ontology, while OpenAI's threshold of labor replacement (OpenAI, 2018) better matches “Exceptional AGI.””
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6. Focus on the Path to AGI, not a Single Endpoint.
What the sources say
- Not matchedcites Agüera y Arcas & Norvig, 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites OpenAI, 2018
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Not matched
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“Our “Competent AGI” level is probably the best catch-all for many existing definitions of AGI (e.g., the Legg (Legg, 2008), Shanahan (Shanahan, 2015), and Suleyman (Mustafa Suleyman and Michael Bhaskar, 2023) formulations).”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6. Focus on the Path to AGI, not a Single Endpoint.
What the sources say
- Not matchedcites Legg, 2008
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites Shanahan, 2015
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites Mustafa Suleyman and Michael Bhaskar, 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We introduced Levels of Autonomy that are unlocked, but not determined by, progression through the Levels of AGI.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 7. Conclusion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“While Level 5 Self-Driving (SAE International, 2021) vehicles would likely be a Level 4 or 5 Narrow AI under our taxonomy, the same considerations regarding human vs. computer autonomy apply to AGIs.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6.2. Capabilities vs. Autonomy
What the sources say
- Not matchedcites SAE International, 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, in the domain of self-driving vehicles, when Level 5 Self-Driving technology is widely available, there may be reasons for using a Level 0 (No Automation) vehicle.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6.2. Capabilities vs. Autonomy
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Including items matched to ASL capabilities in any AGI benchmark would connect points in our AGI taxonomy to specific risks and mitigations.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6.1. Levels of AGI as a Framework for Risk Assessment
What the sources say
- Not matchedcites Anthropic, 2023b
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On the other hand, reaching “Expert AGI” likely alleviates some risks introduced by “Emerging AGI” and “Competent AGI,” such as the risk of incorrect task execution.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6.1. Levels of AGI as a Framework for Risk Assessment
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Considering AGI systems in the context of use by people allows us to reflect on the interplay between advances in models and advances in human-AI interaction paradigms.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6.3. Human-AI Interaction and Risk Assessment
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Requiring deployment as a condition of measuring AGI introduces nontechnical hurdles such as legal and social considerations, as well as ethical and safety concerns.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 3. Defining AGI: Six Principles
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
In the paper
“Knowing when to ask for help is necessary to support alignment and appropriate human-AI interaction (Terry et al., 2023)”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 5. Testing for AGI
What the sources say
- Not checked — no full text of the sourcecites Terry et al., 2023 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Determining that something is not an AGI at a given level simply requires identifying tasks that people can typically do but the system cannot adequately perform.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 5. Testing for AGI
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, this framing also implies that Superhuman systems may be able to perform an even broader generality of tasks than lower levels of AGI, since the ability to execute tasks that qualitatively differ from existing human skills would by definition outperform all humans (who fundamentally cannot do such tasks).”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > Competent AGI not yet achieved
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With these principles in mind, we introduced our Levels of AGI ontology, which offers a more nuanced way to define progress toward AGI by considering generality (either Narrow or General) in tandem with five levels of performance (Emerging, Competent, Expert, Exceptional, and Superhuman).”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 7. Conclusion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These interaction paradigms in turn introduce new classes of risk.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6.3. Human-AI Interaction and Risk Assessment
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
In the paper
“While Level 5 Self-Driving (SAE International, 2021) vehicles would likely be a Level 4 or 5 Narrow AI under our taxonomy, the same considerations regarding human vs. computer autonomy apply to AGIs.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6.2. Capabilities vs. Autonomy
What the sources say
- Not matchedcites SAE International, 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We argue that both generality and performance are key components of AGI.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 3. Defining AGI: Six Principles
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The “Competent AGI” level, which has not been achieved by any public systems at the time of writing, best corresponds to many prior conceptions of AGI, and may precipitate rapid societal change once achieved.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 4. Levels of AGI
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“Concurrent with this work, Anthropic released Version 1.0 of its Responsible Scaling Policy (RSP) (Anthropic, 2023b).This policy uses a levels-based approach (inspired by biosafety levels (Richmond & McKinney, 2009)) to define the level of risk associated with an AI system, identifying what dangerous capabilities may be associated with each AI Safety Level (ASL), and what containment or deployment”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6.1. Levels of AGI as a Framework for Risk Assessment
What the sources say
- Not matchedcites Anthropic, 2023b
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites Richmond & McKinney, 2009
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Notably, we observed that AGI is not necessarily synonymous with autonomy.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 7. Conclusion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On the other hand, metacognitive capabilities (such as the ability to learn new tasks or the ability to know when to ask for clarification or assistance from a human) are key prerequisites for systems to achieve generality.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 3. Defining AGI: Six Principles
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This discrepancy reflects the fact that for General systems, capability development is likely to be uneven; for example, a Level 1 General AI (”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6.3. Human-AI Interaction and Risk Assessment
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The role of model building research can be seen as helping systems' capabilities progress along the path to AGI in their performance and generality, such that an AI system's abilities will overlap an increasingly large portion of human abilities.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6.3. Human-AI Interaction and Risk Assessment
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These Levels of Autonomy are correlated with the Levels of AGI.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6.2. Capabilities vs. Autonomy
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We may develop an AGI, but choose not to deploy it autonomously, or choose to deploy it with differentiated autonomy levels in distinct circumstances as dictated by contextual considerations.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6.2. Capabilities vs. Autonomy
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Increasing capabilities unlock new interaction paradigms, but do not determine them.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6.2. Capabilities vs. Autonomy
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This position paper introduces a novel ontology that supports discussing progress toward AGI in a nuanced manner, with the aim of supporting clear communication among researchers, practitioners, and policymakers about systems' capabilities and associated risks.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > Impact Statement
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Higher levels of autonomy are”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6.2. Capabilities vs. Autonomy
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We illustrated how considering AGI Level jointly with Autonomy Level can provide more nuanced insights into risks associated with AI systems, underscoring the importance of investing in human-AI interaction research in tandem with model improvements.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 7. Conclusion
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“though lower levels of autonomy may be desirable for particular tasks and contexts even as we reach higher levels of AGI.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6.2. Capabilities vs. Autonomy
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The order in which stronger skills in specific cognitive areas are acquired may have serious implications for AI safety (e.g., acquiring strong knowledge of chemical engineering before acquiring strong ethical reasoning skills may be a dangerous combination).”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 4. Levels of AGI
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Unambiguous classification of AI systems will require a standardized benchmark of tasks, as we discuss in Section 5.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 4. Levels of AGI
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Tasks that can be used to benchmark progress toward AGI are critical to operationalizing any proposed definition.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 5. Focus on Ecological Validity.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Certain aspects of generality may be required to make particular interaction paradigms desirable.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6.2. Capabilities vs. Autonomy
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“further, our ontology considers how AI capabilities may enable particular interaction paradigms and how the combination of level of autonomy and level of AGI may impact risk.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6.2. Capabilities vs. Autonomy
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The interplay of model capabilities and interaction design will enable more nuanced risk assessments and responsible deployment decisions than considering model capabilities alone.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 6.3. Human-AI Interaction and Risk Assessment
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The “Competent AGI” level, which has not been achieved by any public systems at the time of writing, best corresponds to many prior conceptions of AGI, and may precipitate rapid societal change once achieved.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > 4. Levels of AGI
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This definition means that Level 5 General AI (“ASI”) systems will be able to do a wide range of tasks at a level that no human can match.”
§ Position: Levels of AGI for Operationalizing Progress on the Path to AGI > Competent AGI not yet achieved
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
The paper relies on this without support.
Modern LLMs pass some framings of the Turing Test.
“Given that modern LLMs pass some framings of the Turing Test”
The paper relies on this without support.
The concept of AGI is related to a prediction about progress in AI, namely that it is toward greater generality, approaching and exceeding human generality.
“The concept of AGI is related to a prediction about progress in AI, namely that it is toward greater generality, approaching and exceeding human generality.”
The paper relies on this without support.
The ability to perform physical tasks increases a system's generality.
“We suggest that the ability to perform physical tasks increases a system's generality”
The paper relies on this without support.
The ability of systems to accurately model end-users is a necessary component of alignment for AGI systems.
“the ability of systems to accurately model end-users is a necessary component of alignment for AGI systems.”
The paper relies on this without support.
It is impossible to enumerate the full set of tasks achievable by a sufficiently general intelligence.
“It is impossible to enumerate the full set of tasks achievable by a sufficiently general intelligence.”
The paper relies on this without support.
Broad but imperfect generality is impactful.
“since it seems clear that broad but imperfect generality is impactful”
The paper relies on this without support.
Individual humans lack consistent performance across all possible tasks but are generally intelligent.
“individual humans also lack consistent performance across all possible tasks, but are generally intelligent”
The paper relies on this without support.
Most dangerous capabilities tend to be dual use.
“since most such skills tend to be dual use (having valid applications to socially positive scenarios as well as nefarious ones).”
The paper relies on this without support.
Including dangerous capability tests in a public benchmark may allow malicious actors to optimize for these abilities.
“However, including such tests in a public benchmark may allow malicious actors to optimize for these abilities”
The paper relies on this without support.
Capabilities provide prerequisites for AI risks.
“While capabilities provide prerequisites for AI risks, AI systems (including AGI systems) do not and will not operate in a vacuum.”
The paper relies on this without support.
AI systems, including AGI systems, do not and will not operate in a vacuum; they are deployed with particular interfaces and used to achieve particular tasks in specific scenarios.
“Rather, AI systems are deployed with particular interfaces and used to achieve particular tasks in specific scenarios.”
The paper relies on this without support.
System designers and end-users will settle on a mode of human-AI interaction that balances a variety of considerations, including safety.
“Rather, system designers and end-users will settle on a mode of human-AI interaction (Morris et al., 2023) that balances a variety of considerations, including safety.”
The paper relies on this without support.
AGI is a concept of both aspirational and practical consequences.
“Artificial General Intelligence is a concept of both aspirational and practical consequences.”
The paper relies on this without support.
The choice of appropriate autonomy level need not be the maximum achievable given the capabilities of the underlying model.
“The choice of appropriate autonomy level need not be the maximum achievable given the capabilities of the underlying model.”
The paper relies on this without support.
One consideration in the choice of autonomy level are resulting risks.
“One consideration in the choice of autonomy level are resulting risks.”
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
I.—COMPUTING MACHINERY AND INTELLIGENCE
TURING, A. M. (1950) · Mind · doi:10.1093/mind/lix.236.433 · Cited by 10,290
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Varadi, M., Anyango, S., Deshpande, M., Nair, S., Natassia, C., Yordanova, G., Yuan, D., Stroe, O., Wood, G., Laydon, A., et al. (2022) · Nucleic Acids Research · doi:10.1093/nar/gkab1061 · Cited by 7,850
Searle, J. R. (1980) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00005756 · Cited by 6,954
ELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452
A model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2018) · Science · doi:10.1126/science.aar6404 · Cited by 3,562
Research works printed but not linked: 6
- Gubrud, M. Nanotechnology and International Security. Fifth Foresight Conference on Molecular Nanotechnology, November 1997.
- Legg, S. Machine Super Intelligence. Doctoral Dissertation submitted to the Faculty of Informatics of the University of Lugano, June 2008.
- McCarthy, J., Minsky, M., Rochester, N., and Shannon, C. A Proposal for The Dartmouth Summer Research Project on Artificial Intelligence. Dartmouth Workshop, 1955.
- Pressley, M., Borkowski, J., and Schneider, W. Cognitive strategies: Good strategy users coordinate metacognition and knowledge. Annals of Child Development, 4:89-129, 1987.
- Shah, R., Freire, P., Alex, N., Freedman, R., Krasheninnikov, D., Chan, L., Dennis, M. D., Abbeel, P., Dragan, A., and Russell, S. Benefits of Assistance over Reward Learning, 2021. URL https://openreview.net/forum? id=DFIoGDZejIB.
- Shanahan, M. Embodiment and the Inner Life. Oxford University Press, 2010.
Cited by
Sorted by citations
MMMU: A Massive Multi-Discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
Yue, X., Ni, Y., Zheng, T., Zhang, K., Liu, R., Zhang, G., Stevens, S., Jiang, D., Ren, W., Sun, Y., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00913 · Cited by 429
Empowering biomedical discovery with AI agents
Gao, S., Fang, A., Huang, Y., Giunchiglia, V., Noori, A., Schwarz, J. R., Ektefaie, Y., Kondic, J., & Zitnik, M. (2024) · Cell · doi:10.1016/j.cell.2024.09.022 · Cited by 315
Foundational Challenges in Assuring Alignment and Safety of Large Language ModelsPreprint
Anwar, U., Saparov, A., Ramírez, J. R., Paleka, D., Turpin, M., Hase, P., Lubana, E. S., Jenner, E., Casper, S., Sourbut, O., et al. (2024) · arXiv (Cornell University) · doi:10.3929/ethz-b-000706962 · Cited by 320
Distributed agency in second language learning and teaching through generative AI
Godwin‐Jones, R. (2024) · Language learning & technology · doi:10.64152/10125/73570 · Cited by 282
A review of large language models and autonomous agents in chemistry
Ramos, M. C., Collison, C. J., & White, A. D. (2025) · Chemical Science · doi:10.1039/d4sc03921a · Cited by 274
Bridging the Gulf of Envisioning: Cognitive Challenges in Prompt Based Interactions with LLMs
Subramonyam, H., Pea, R. D., Pondoc, C., Agrawala, M., & Seifert, C. M. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642754 · Cited by 247
Leng, J., Zhu, X., Huang, Z., Li, X., Zheng, P., Zhou, X., Mourtzis, D., Wang, B., Qi, Q., Shao, H., et al. (2024) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2024.02.010 · Cited by 228
Defining intelligence: Bridging the gap between human and artificial perspectives
Gignac, G. E., & Szodorai, E. T. (2024) · Intelligence · doi:10.1016/j.intell.2024.101832 · Cited by 176
Ying, K., Meng, F., Wang, J., Li, Z., Lin, H., Yang, Y., Zhang, H., Zhang, W., Lin, Y., Liu, S., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.16006 · Cited by 164
Theory Is All You Need: AI, Human Cognition, and Causal Reasoning
(2024) · doi:10.1287/stsc.2024.0189 · Cited by 151
Large language models for reticular chemistry
Zheng, Z., Rampal, N., Inizan, T. J., Borgs, C., Chayes, J. T., & Yaghi, O. M. (2025) · Nature Reviews Materials · doi:10.1038/s41578-025-00772-8 · Cited by 134
MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark
Wang, Y., Ma, X., Zhang, G., Ni, Y., Chandra, A., Guo, S., Ren, W., Arulraj, A., He, X., Jiang, Z., et al. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-3018 · Cited by 131
Addressing 6 challenges in generative AI for digital health: A scoping review
Templin, T., Perez, M. W., Sylvia, S., Leek, J. T., & Sinnott-Armstrong, N. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000503 · Cited by 129
Generative Ghosts: Anticipating Benefits and Risks of AI Afterlives
Morris, M. R., & Brubaker, J. R. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713758 · Cited by 106
Self-citationMcIntosh, T. R., Susnjak, T., Liu, T., Watters, P., Xu, D., Liu, D., & Halgamuge, M. N. (2025) · Technologies · doi:10.3390/technologies13020051 · Cited by 102
EgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level Planning
Chen, Y., Ge, Y., Ge, Y., Ding, M., Li, B., Wang, R., Xu, R., Shan, Y., & Liu, X. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02676-0 · Cited by 91
Xiao, T., & Xu, P. (2024) · Energy and Buildings · doi:10.1016/j.enbuild.2024.114691 · Cited by 86
Huber, S. E., Kiili, K., Nebel, S., Ryan, R. M., Sailer, M., & Ninaus, M. (2024) · Educational Psychology Review · doi:10.1007/s10648-024-09868-z · Cited by 85
Chan, A., Ezell, C., Kaufmann, M., Wei, K., Hammond, L., Bradley, H., Bluemke, E., Rajkumar, N., Krueger, D., Kolt, N., et al. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658948 · Cited by 84
GIScience in the era of Artificial Intelligence: a research agenda towards Autonomous GIS
Li, Z., Ning, H., Gao, S., Janowicz, K., Li, W., Arundel, S. T., Yang, C. P., Bhaduri, B., Wang, S., Zhu, A.-X., et al. (2025) · Annals of GIS · doi:10.1080/19475683.2025.2552161 · Cited by 78
Herrera, F. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103133 · Cited by 68
Towards evaluations-based safety cases for AI schemingPreprint
Balesni, M., Hobbhahn, M., Lindner, D., Meinke, A., Korbak, T., Clymer, J., Shlegeris, B., Scheurer, J., Stix, C., Shah, R., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.03336 · Cited by 69
AI Safety in Generative AI Large Language Models: A SurveyPreprint
Chua, J., Li, Y., Yang, S., Wang, C., & Yao, L. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.18369 · Cited by 67
Pedersen, C. L., & Ritter, T. (2024) · Industrial Marketing Management · doi:10.1016/j.indmarman.2024.10.005 · Cited by 64
How Far Are We From AGI: Are LLMs All We Need?Preprint
Feng, T., Jin, C., Liu, J., Zhu, K., Tu, H., Cheng, Z., Lin, G., & You, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.10313 · Cited by 62
“Generate” the Future of Work through AI: Empirical Evidence from Online Labor Markets
Liu, J., Xu, X., Nan, X., Li, Y., & Tan, Y. (2026) · Information Systems Research · doi:10.1287/isre.2023.0461 · Cited by 57
OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI
Huang, Z., Wang, Z., Xia, S., Li, X., Zou, H., Xu, R., Fan, R.-Z., Ye, L., Chern, E., Ye, Y., et al. (2024) · neural information processing systems · doi:10.52202/079017-0607 · Cited by 55
Li, J., & Yeo, R. K. (2024) · Human Resource Development International · doi:10.1080/13678868.2024.2348987 · Cited by 50
Can Language Models Learn to Skip Steps?
Liu, T., Guo, Q., Hu, X., Jiayang, C., Zhang, Y., Qiu, X., & Zhang, Z. (2024) · neural information processing systems · doi:10.52202/079017-1441 · Cited by 48
LLM as OS, Agents as Apps: Envisioning AIOS, Agents and the AIOS-Agent EcosystemPreprint
Ge, Y., Ren, Y., Hua, W., Xu, S., Tan, J., & Zhang, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.03815 · Cited by 49
Frontier AI developers need an internal audit function
Schuett, J. (2024) · Risk Analysis · doi:10.1111/risa.17665 · Cited by 50
EgoPlan-Bench2: A Benchmark for Multimodal Large Language Model Planning in Real-World Scenarios
Qiu, L., Chen, Y., Ge, Y., Ge, Y., Shan, Y., & Liu, X. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02826-y · Cited by 47
Introduction to AI Safety, Ethics, and Society
Hendrycks, D. (2024) · CRC Press eBooks · doi:10.1201/9781003530336 · Cited by 46
ValueCompass: A Framework for Measuring Contextual Value Alignment Between Human and LLMs
Shen, H. P., Knearem, T., Ghosh, R., Yang, Y.-J., Clark, N., Mitra, T., & Huang, Y. (2025) · Widening NLP Workshop (WiNLP) · doi:10.18653/v1/2025.winlp-main.15 · Cited by 42
McIntosh, T. R., Sušnjak, T., Liu, T., Watters, P., Ng, A., & Halgamuge, M. N. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3394392 · Cited by 41
Vitron: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing
Fei, H., Wu, S., Zhang, H., Chua, T., & Yan, S. (2024) · neural information processing systems · doi:10.52202/079017-1823 · Cited by 41
Nuclear Deployed!: Analyzing Catastrophic Risks in Decision-making of Autonomous LLM Agents
Xu, R., Li, X., Chen, S., & Xu, W. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.67 · Cited by 40
Challenges and Paths Towards AI for Software EngineeringPreprint
Gu, A., Jain, N., Li, W., Shetty, M., Shao, Y., Li, Z., Yang, D., Ellis, K., Sen, K., & Solar-Lezama, A. (2025) · Qeios · doi:10.32388/vv1661 · Cited by 40
The Algorithmic State Architecture (ASA): An Integrated Framework for AI-Enabled GovernmentPreprint
Engin, Z., Crowcroft, J., Hand, D., & Treleaven, P. C. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.08725 · Cited by 39
MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in VideosPreprint
He, X., Feng, W., Zheng, K., Lu, Y., Zhu, W., Li, J., Fan, Y., Wang, J., Li, L., Yang, Z., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.08407 · Cited by 41
Lu, H., Fang, L., Zhang, R., Li, X., Cai, J., Cheng, H., Tang, L., Liu, Z., Sun, Z., Wang, T., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.19672 · Cited by 37
Levels of AI Agents: from Rules to Large Language ModelsPreprint
Huang, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.06643 · Cited by 39
Religion and Artificial Intelligence
Singler, B. (2024) · Routledge eBooks · doi:10.4324/9781003256113 · Cited by 35
General Scales Unlock AI Evaluation with Explanatory and Predictive PowerPreprint
Zhou, L., Pacchiardi, L., Martínez‐Plumed, F., Collins, K. M., Moros-Daval, Y., Zhang, S., Zhao, Q., Huang, Y., Sun, L., Prunty, J. E., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.06378 · Cited by 34
Satyanarayan, A., & Jones, G. M. (2024) · doi:10.21428/e4baedd9.2d7598a2 · Cited by 33
The Code That Binds Us: Navigating the Appropriateness of Human-AI Assistant Relationships
Manzini, A., Keeling, G., Alberts, L., Vallor, S., Morris, M. R., & Gabriel, I. (2024) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v7i1.31694 · Cited by 33
Self-citationCan Large Language Model Agents Simulate Human Trust Behavior?
Xie, C., Chen, C., Jia, F., Ye, Z., Lai, S., Shu, K., Gu, J., Bibi, A., Hu, Z., Jurgens, D., et al. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-0501 · Cited by 31
Anurin, A., Ng, J., Schaffer, K., Schreiber, J. M., & Kran, E. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.09114 · Cited by 29
Liu, C.-Y., & Yin, B. (2024) · Computers in Human Behavior · doi:10.1016/j.chb.2024.108406 · Cited by 28
Appraising Regulatory Framework Towards Artificial General Intelligence (AGI) Under Digital Humanism
Cheng, L., & Gong, X. (2024) · International Journal of Digital Law and Governance · doi:10.1515/ijdlg-2024-0015 · Cited by 28
Artificial Intelligence/Machine Learning Technology in Power System Applications
Chen, Y., USDOE Office of Electricity (OE), Fan, X., Huang, Q., Huang, Q., Li, A., & Guddanti, K. P. (2024) · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · doi:10.2172/2340760 · Cited by 28
Long, R., Sebo, J., Butlin, P., Finlinson, K., Fish, K., Harding, J., Pfau, J., Sims, T., Birch, J., & Chalmers, D. (2024) · arXiv · doi:10.48550/arxiv.2411.00986 · Cited by 27
Generative AI Security: Challenges and CountermeasuresPreprint
Zhu, B., Mu, N., Jiao, J., & Wagner, D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.12617 · Cited by 27
Telecom’s Artificial General Intelligence (AGI) Vision: Beyond the GenAI Frontier
Chaccour, C., Karapantelakis, A., Murphy, T., & Döhler, M. (2024) · IEEE Network · doi:10.1109/mnet.2024.3425594 · Cited by 26
MemoVis: A GenAI-Powered Tool for Creating Companion Reference Images for 3D Design Feedback
Chen, C., Nguyen, C., Groueix, T., Kim, V. G., & Weibel, N. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3694681 · Cited by 25
Shin, C., Seo, D. G., Jin, S.-Y., Lee, S. H., & Park, H. J. (2024) · IEEE Access · doi:10.1109/access.2024.3447067 · Cited by 24
Game-TARS: Pretrained Foundation Models for Scalable Generalist Multimodal Game AgentsPreprint
Wang, Z., Li, X., Ye, Y., Fang, J., Wang, H., Liu, L., Liang, S., Lu, J., Wu, Z., Feng, J., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.23691 · Cited by 23
Morris, M. R., Bernstein, M. S., Bigham, J. P., Bruckman, A. S., & Monroy‐Hernández, A. (2024) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3678884.3689134 · Cited by 22
Self-citationGrand challenges for human factors and ergonomics
Karwowski, W., Salvendy, G., Endsley, M. R., Rouse, W. B., Salmon, P. M., Stanney, K. M., Thatcher, A., Andre, T., Yang, X. J., Ayaz, H., et al. (2025) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922x.2024.2431336 · Cited by 23
Context Engineering 2.0: The Context of Context EngineeringPreprint
Hua, Q., Ye, L., Fu, D., Xiao, Y., Cai, X., Wu, Y., Lin, J., Wang, J., & Liu, P. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.26493 · Cited by 23
Interactive AI Alignment: Specification, Process, and Evaluation Alignment
Terry, M., Kulkarni, C., Wattenberg, M., Dixon, L., & Morris, M. R. (2023) · arXiv · doi:10.48550/arxiv.2311.00710 · Cited by 20
Self-citationLanguage Models Do Hard Arithmetic Tasks Easily and Hardly Do Easy Arithmetic Tasks
Gambardella, A., Iwasawa, Y., & Matsuo, Y. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-short.8 · Cited by 20
Risk and artificial general intelligence
Faroldi, F. L. G. (2024) · AI & Society · doi:10.1007/s00146-024-02004-z · Cited by 19
Unsocial Intelligence: An Investigation of the Assumptions of AGI Discourse
Blili-Hamelin, B., Hancox-Li, L., & Smart, A. (2024) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v7i1.31625 · Cited by 18
Measuring AI Agent Autonomy: Towards a Scalable Approach with Code Inspection
Cihon, P., Stein, M., Bansal, G., Manning, S., & Xu, K. S. (2025) · SuperIntelligence - Robotics - Safety & Alignment · doi:10.70777/si.v2i3.15295 · Cited by 18
AGI, Governments, and Free Societies
Bullock, J. B., Hammond, S., & Krier, S. (2025) · Public administration, governance and globalization · doi:10.1007/978-3-032-00514-4_17 · Cited by 17
What is Meant by AGI? On the Definition of Artificial General IntelligencePreprint
Xu, B. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.10731 · Cited by 17
Sanguino, T. J. M. (2024) · Applied Sciences · doi:10.3390/app14093780 · Cited by 16
Toward Embodied AGI: A Review of Embodied AI and the Road AheadPreprint
Wang, Y., & Sun, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.14235 · Cited by 16
Xu, R., Le, N., Park, R. H., Murray, L., Das, V., Kumar, D., & Goldberg, B. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.02522 · Cited by 15
Boundless Socratic Learning with Language GamesPreprint
Schaul, T. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.16905 · Cited by 15
Code-Vision: Evaluating Multimodal LLMs Logic Understanding and Code Generation CapabilitiesPreprint
Wang, H., Zhou, X., Xu, Z., Cheng, K., Zuo, Y., Tian, K., Song, J., Lu, J., Hu, W., & Liu, X. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.11829 · Cited by 15
Kapoor, S., Stroebl, B., Siegel, Z. S., Nadgir, N., & Narayanan, A. (2024) · arXiv · doi:10.48550/arxiv.2407.01502 · Cited by 13
Can Transformative AI Shape a New Age for Our Civilization?
Lobo, J. L., & Ser, J. D. (2026) · CRC Press eBooks · doi:10.1201/9781003727071 · Cited by 13
The Routledge Handbook of Artificial Intelligence and Philanthropy
Ugazio, G., & Maricic, M. (2024) · Routledge eBooks · doi:10.4324/9781003468615 · Cited by 12
VIVA: A Benchmark for Vision-Grounded Decision-Making with Human Values
Hu, Z., Ren, Y., Li, J., & Yin, Y. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.137 · Cited by 12
MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmark
Xing, J., He, Y., Zhou, M., Dong, H., Han, S. Z., Chen, L., Zhang, D., Chaudhuri, S., & Jagadish, H. V. (2025) · neural information processing systems · doi:10.52202/085713-2319 · Cited by 12
A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents
Su, H., Luo, J., Liu, C., Yang, X., Zhang, Y., Dong, Y., & Zhu, J. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3688650 · Cited by 11
Evaluating Frontier Models for Dangerous Capabilities
Phuong, M., Aitchison, M., Catt, E., Cogan, S., Kaskasoli, A., Krakovna, V., Lindner, D., Rahtz, M., Assael, Y., Hodkinson, S., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.13793 · Cited by 11
Self-citationCorradetti, D., & Rodrigues, J. D. (2024) · Journal of Cultural Heritage · doi:10.1016/j.culher.2024.11.017 · Cited by 11
Suzuki, R., Abtahi, P., Chen, Z., Dogan, M. D., Colaço, A., Gonzalez, E. J., Ahuja, K., & González-Franco, M. (2025) · Frontiers in Virtual Reality · doi:10.3389/frvir.2025.1649785 · Cited by 11
Thinking Preference OptimizationPreprint
Wang, Y., Jin, H., Yang, J., Chaudhary, V., & Han, X. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.13173 · Cited by 11
Zhou, P., Zhang, F., Peng, X., Xu, Z., Ai, J., Qiu, Y., Li, C., Li, Z., Li, M., Feng, Y., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.05782 · Cited by 11
Thais, S. J. (2024) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v7i1.31737 · Cited by 10
Disrupting Education: Artificial Intelligence in Higher Education
Alangari, H. (2024) · The Evolution of Artificial Intelligence in Higher Education · doi:10.1108/978-1-83549-486-820241004 · Cited by 9
Transforming artificial intelligence into artificial wisdom
Jeste, D. V., Paulus, M. P., Alexopoulos, G. S., Barnard, M., Otte, W. M., Satake, Y., Na, P. J., Torous, J. B., Prodan, A., & Occhipinti, J. (2026) · Nature Mental Health · doi:10.1038/s44220-026-00640-6 · Cited by 9
On Safety Risks in Experience-Driven Self-Evolving Agents
Zhao, W., Zhang, Y., Wang, Y., Deng, Y., Zhao, Y., Zhi, X., Huang, Y., He, H., Che, W., Qin, B., et al. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.2091 · Cited by 9
Large language models: assessment for singularity
Ishizaki, R., & Sugiyama, M. (2025) · AI & Society · doi:10.1007/s00146-025-02271-4 · Cited by 9
An Evaluation of GPT-4V and Gemini in Online VQAPreprint
Liu, M., Chen, C., & Gurari, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.10637 · Cited by 9
Crafting Knowledge: Exploring the Creative Mechanisms of Chat-Based Search EnginesPreprint
Ma, L., Xu, X., & Tan, Y. S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.19421 · Cited by 9
Satyanarayan, A. (2024) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3672539.3695751 · Cited by 8
The future of work and capital: analyzing AGI in a CES production model
Stiefenhofer, P. (2025) · Applied Mathematical Sciences · doi:10.12988/ams.2025.919196 · Cited by 8
Between world models and model worlds: on generality, agency, and worlding in machine learning
Mitrokhov, K. (2024) · AI & Society · doi:10.1007/s00146-024-02086-9 · Cited by 8
Wasil, A. R., Smith, E., Katze, C., & Bullock, J. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4884541 · Cited by 8
Learning to Adopt Generative AIPreprint
Ma, L., Xu, X., He, Y., & Tan, Y. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4990170 · Cited by 8
An Operational Kardashev-Style Scale for Autonomous AI - Towards AGI and SuperintelligencePreprint
Chojecki, P. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.13411 · Cited by 8
BehaveGPT: A Foundation Model for Large-scale User Behavior ModelingPreprint
Gong, J., Ding, J., Meng, F., Yang, C., Chen, H., Wang, Z., Lu, H., & Li, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.17631 · Cited by 7
“Control Is a Trajectory, Not a Point”: Conceptualizing Control in Human-AI Co-Creativity
Issak, A., Rezwana, J., & Harteveld, C. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790861 · Cited by 6
Emerging Questions in AI Welfare
Keeling, G., & Street, W. (2026) · Cambridge University Press eBooks · doi:10.1017/9781009732000 · Cited by 6
An Outlook for AI Innovation in Multimodal Communication Research
Henlein, A., Bauer, A., Bhattacharjee, R., Ćwiek, A., Gregori, A., Kügler, F., Lemanski, J., Lücking, A., Mehler, A., Prieto, P., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-61066-0_13 · Cited by 6
Metadata
- Authors
- Morris, M. R., Sohl-Dickstein, J., Fiedel, N., Warkentin, T., Dafoe, A., Faust, A., Farabet, C., & Legg, S.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2311.02462
- OpenAlex
- W4388481599
- Semantic Scholar
- c0a822ac56e47b6e4b99d5552d2998647abe8234
- Cited by
- 274
- References
- 45
- Citations by year
- 2026: 74 · 2025: 107 · 2024: 85 · 2023: 8
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University)
- Funders
- DeepMind
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- artificial general intelligence, AI autonomy, AI risk assessment, human-AI interaction, responsible AI deployment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
