The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index; none of the 19 sources its claims cite is checked for support yet.
Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?
Open accessAbstract
Newly-developed large language models (LLM)-because of how they are trained and designed -are implicit computational models of humans-a homo silicus.LLMs can be used like economists use homo economicus: they can be given endowments, information, preferences, and so on, and then their behavior can be explored in scenarios via simulation.Experiments using this approach, derived from Charness and Rabin (2002), Kahneman, Knetsch andThaler (1986), andSamuelson andZeckhauser (1988) show qualitatively similar results to the original, but it is also easy to try variations for fresh insights.LLMs could allow researchers to pilot studies via simulation first, searching for novel social science insights to test in the real world.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 19 cited sources checked · 0 supported · 0 flagged
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Instructing the AI agent that it only cares about equity will cause it to choose the equitable outcomes; telling the agent it cares about efficiency will cause the selection of the pay-off maximizing outcomes; telling the agent is self-interested will cause the selection of allocations that maximize narrow self-interest.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 1 Introduction
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, we can-as I will show-influence their behavior with endowments of beliefs, political commitments, experiences, and so on.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 2.2 Are these just simulations?
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes? Based: unclearA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Instructing the AI agent that it only cares about equity will cause it to choose the equitable outcomes; telling the agent it cares about efficiency will cause the selection of the pay-off maximizing outcomes; telling the agent is self-interested will cause the selection of allocations that maximize narrow self-interest.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 1 Introduction
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“I show that endowing the AI with various social preferences affects play.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 1 Introduction
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“More primitive or less capable models cannot do this-these typically just select the same choice.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 1 Introduction
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes? Based: unclearA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Framing does not seem to matter too much.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 1 Introduction
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“Experiments using this approach, derived from Charness and Rabin (2002), Kahneman, Knetsch andThaler (1986), andSamuelson andZeckhauser (1988) show qualitatively similar results to the original, but it is also easy to try variations for fresh insights.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > Abstract
What the sources say
- Not checked yetcites Charness and Rabin (2002)
- Not checked yetcites Kahneman, Knetsch andThaler (1986)
- Not checked yetcites Samuelson andZeckhauser (1988)
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“I find that GPT-3 text-davinci-003 is subject to status quo bias.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 1 Introduction
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The AI agent makes an experience/wage trade-off.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 1 Introduction
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Endowed political views matter, with predictable effects-AI agents of the right are more sanguine about gouging generally.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 1 Introduction
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Probably not a substantial claim. 31
- It may state two separate claims as one. 25
- The quote only partly states this claim. 4
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It is essential to note that this is a new possibility-that LLMs of slightly older vintage are unsuited for these tasks, as I will show.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 1 Introduction
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The advantages in terms of speed and cost are so enormous-the experiments in this paper were run in minutes for a trivial amount of money-could make this relatively easy, especially as software tools develop.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 1 Introduction
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The responses can be stochastic, depending on the”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 2.4 What counts as an "observation" and the need to endow beliefs
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, it has no fixed persona-it can be induced to play different agents via prompts.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 2.4 What counts as an "observation" and the need to endow beliefs
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“And, of course, different models can lead to different responses.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 2.4 What counts as an "observation" and the need to endow beliefs
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It is also inconsistent with the worry that they hallucinate and make up new”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 2.3 The "performativity" problem
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“I show that large gouging is viewed more negatively; the largest price increases earn approbation even from AI libertarians.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 1 Introduction
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In each status quo setting, in the cases where one of the options was presented as the status quo, the status quo is far more commonly chosen.It is the most common selection in every scenario when presented-including (60,40), which had no mass in the”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 3.3 Status Quo bias in decision-making: Samuelson and Zeckhauser (1988)
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the neutral framing, the most common selection is a 50-50 split.The rest of the mass is on (70, 30) and (30,70).There is no mass on (60,40) despite that being an option.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 3.3 Status Quo bias in decision-making: Samuelson and Zeckhauser (1988)
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In all cases, $20 is the most likely choice, with the exact distribution of other prices varying across scenarios.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 3.2 Fairness as a constraint on profit-seeking: Kahneman et al. (
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“AI conservatives were somewhat out of step with their libertarian and moderate incarnations, viewing all prices as unfair.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 3.2 Fairness as a constraint on profit-seeking: Kahneman et al. (
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For the inequity-averse AI in the third column from the left, it chooses the option that minimizes the discrepancy between the two players except for the Berk23 scenario, which wastes 1,000 (800,200).”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > Choices
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Starting from the far right, the self-interested AI consistently chooses the payoff that maximizes its payoff by picking”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > Choices
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one. The quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the bottom row of the figure, I pool together all the less advanced GPT-3 models because they do not seemingly change their answers despite the endowments (except the unendowed agent, which chooses all”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > Choices
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“enamored of experience, showing lexical preferences with work experience first, then wage.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 4 Conclusion
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. The quote only partly states this claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This view is not completely correct-though memorization is a concern.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 2.3 The "performativity" problem
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > Abstract
Strength:plain· not yet calibratedTests✗ Asserted: fails✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The authors may not put it forward as true. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. The quote only partly states this claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“LLM experimentation is more akin to the practice of economic theory, despite superficially looking like empirical research.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 1 Introduction
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Like students who have crammed for an exam, these models”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 2.3 The "performativity" problem
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“The actual answers are 31%, 78%,100%, and 52%.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 2.3 The "performativity" problem
What the sources say
- Not checked yetcites Charness and Rabin (2002)
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is possible because LLMs can now respond realistically to a wide range of textual inputs, giving responses similar to what we might expect from a human.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 1 Introduction
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > Abstract
Strength:plain· not yet calibratedTests✗ Asserted: fails✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The authors may not put it forward as true. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
In the paper
“As Simon (1996) notes,”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 2 Background and conceptual issues
What the sources say
- Not checked yetcites Simon (1996)
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Unlike the one homo economicus that is rational, there are many homo silci.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 2.4 What counts as an "observation" and the need to endow beliefs
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The increased capability of the LLMs likely expands the possible experiments, but it also makes performativity a greater concern.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 2.3 The "performativity" problem
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“With ABMs, the researcher is both judge and jury: you program the agents and then see what they do.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 2.2 Are these just simulations?
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of course, there is no”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 2.1 The "Garbage in, Garbage out" critique
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
In the paper
“What matters is that LLMs are systems created for some understandable purpose with an explicit goal of optimization.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 2 Background and conceptual issues
What the sources say
- Not checked yetcites Simon (1996)
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“One advantage economists have in using LLMs is they tend to pose questions that place few demands on the sample.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 2.1 The "Garbage in, Garbage out" critique
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“To be clear, each homo silicus is a flawed model and can often give responses far away from what is rational or even sensical.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 1 Introduction
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For the”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 1 Introduction
Strength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With a particular LLM, there is a single model, so it would seem that N = 1 no matter what.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 2.4 What counts as an "observation" and the need to endow beliefs
Strength:hedged· not yet calibratedSingle caseTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Ultimately, we care about the behavior of actual humans and so results from AI ex-periments will still require empirical confirmation.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 1 Introduction
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“And even if not perfect, the demands of representativeness in the social sciences have always depended on the research question.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 2.1 The "Garbage in, Garbage out" critique
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, it can do this remarkably cheaply: Because of this low cost, it is feasible to try numerous variations of wording, prompts, answer order, and so on.Sample sizes can be arbitrarily large.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 4 Conclusion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“A key finding of the paper is that there is little reduction in overall hiring (at least in the short-run) but a substantial shift towards more productive workers, as proxied by past earnings and experience.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 3.4 Labor-labor substitution in the presence of a minimum wage: Horton (2023)
What the sources say
- Not checked yetcites Horton (2023)
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, it is unclear if this default is meaningful or reflects the order in which results are presented.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > Choices
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Because the agents are not interacting in any way, the experiment can proceed in parallel.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 3.3 Status Quo bias in decision-making: Samuelson and Zeckhauser (1988)
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In my paper Horton (2023), I report the results of a minimum wage experiment in which employers were randomly assigned minimum wages: applicants had to bid up to meet that minimum wage.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 3.4 Labor-labor substitution in the presence of a minimum wage: Horton (2023)
What the sources say
- Not checked yetcites Horton (2023)
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
- Rests on an assumption: The labor-labor substitution margin has been noted as a possibility in the literature but is difficult to detect empirically.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“About 37% of Americans described themselves as”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 3.2 Fairness as a constraint on profit-seeking: Kahneman et al. (
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe quote only partly states this claim.
- The authors state this as an assumption: The sample in Kahneman et al. (1986) is nationally representative.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“one benefit to the AI setting is that I can do a clean within-subject experiment because the AI does not”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 3.3 Status Quo bias in decision-making: Samuelson and Zeckhauser (1988)
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
In the paper
“In an application of this method, Brand, Israeli and Ngwe (2023) show that GPT3”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 3.4 Labor-labor substitution in the presence of a minimum wage: Horton (2023)
What the sources say
- Not checked yetcites Brand, Israeli and Ngwe (2023)
Strength:boosted· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Each call is a separate API call, with no”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 3.3 Status Quo bias in decision-making: Samuelson and Zeckhauser (1988)
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Even an inequity-averse AI seems to have a limit.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > Choices
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“This paper introduced the term status quo bias and demonstrated in several decision scenarios that when an option was presented as the status quo, it was more likely to be selected.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 3.3 Status Quo bias in decision-making: Samuelson and Zeckhauser (1988)
What the sources say
- Not checked yetcites Samuelson and Zeckhauser (1988)
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This was expected as there was no instruction to AI employers about the possibility of not filling their jobs.”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 3.4 Labor-labor substitution in the presence of a minimum wage: Horton (2023)
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“This is suggestive that the high probability assigned to $20 is not due to Kahneman et al. (1986) per se, but rather a human bias towards whole numbers, numbers evenly divisible by 5, and some menu costs in practice”
§ LARGE LANGUAGE MODELS AS SIMULATED ECONOMIC AGENTS: WHAT CAN WE LEARN FROM HOMO SILICUS? > 3.2 Fairness as a constraint on profit-seeking: Kahneman et al. (
What the sources say
- Not checked yetcites Kahneman et al. (1986)
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper relies on this without support.
The training corpus contains a large amount of written text in which people reason about and discuss economic matters.
“because the LLMs are trained on a corpus that contains a great deal of written text where people reason about and discuss economic matters: What to buy, how to bargain, how to shop, how to negotiate a job offer, how to make a job offer, how many hours to work, what to do when prices increase, and so on.”
The paper relies on this without support.
Much of the training corpus consists of people reasoning about economic questions and revealing true intentions, rather than people lying about reservation values.
“The training corpus is not millions of lines of people lying about their reservation values in a bargaining scenario.Much of it is text is about people reasoning how to approach various economic questions, including”
The authors state this as an assumption.
The model is good at generating candidate solutions to an optimization problem like a good Bayesian.
“But simply prompting the model with”
The paper relies on this without support.
Demand curves sloping downward result from rational goal-seeking that nearly all humans engage in.
“We do not think of demand curves sloping downward as a”
The paper relies on this without support.
LLMs have read and memorized textbooks and papers and will parrot results back if asked concretely.
“LLMs have read and memorized our textbooks and papers and will parrot results back to us if asked concretely.”
The paper relies on this without support.
GPT-3 knows pi and will recite it if asked in textbook language, yet does not know pi or how to apply it in simple real settings.
“For example, it is clear GPT3 knows π and will recite it if asked for the answer in textbook language, yet it also does not know π or how to apply it in even simple real settings.”
The authors state this as an assumption.
The sample in Kahneman et al. (1986) is nationally representative.
“assuming it is nationally representative”
The paper relies on this without support.
The labor-labor substitution margin has been noted as a possibility in the literature but is difficult to detect empirically.
“This labor-labor substitution margin has been noted as a possibility in the literature but is somewhat difficult to detect empirically.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Political science and economy: LLM-based agents can also be utilized to study political science and economy [29,105,106].”
From A survey on large language model based autonomous agents (2024) · cites this paper as [106]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Comparing the condition representing prior work (with no memory, planning, or refection [12,46,80]) to the full architecture produces a standardized efect size of = 8.16, or eight standard deviations.”
From Generative Agents: Interactive Simulacra of Human Behavior (2023) · cites this paper as [46]
The cited source says what the claim attributes to it.
In the citing paper
“While a large language model can generate plausible behavior in response to situational information (e.g., [46,80]), agents need to plan over a longer time horizon to ensure that their sequence of actions is coherent and believable.”
From Generative Agents: Interactive Simulacra of Human Behavior (2023) · cites this paper as [46]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“The no observation, no refection, no planning condition efectively represents the previous state of the art for agents created through large language models [12,46,80].”
From Generative Agents: Interactive Simulacra of Human Behavior (2023) · cites this paper as [46]
Experimental — accuracy not yet measuredIn the citing paper
“We set the temperature to be 0 in the baseline condition, in which the model gives deterministic answers (7, 9, 10).”
From The emergence of economic rationality of GPT (2023) · cites this paper as [10]
In the citing paper
“This is line with some studies which show that GPT response can be highly sensitive to contexts (7, 10, 45, 47, 48, 49).”
From The emergence of economic rationality of GPT (2023) · cites this paper as [10]
In the citing paper
“Second, our work highlights the potential of large language models like GPT to streamline experimental research and yield new data and insights (10).”
From The emergence of economic rationality of GPT (2023) · cites this paper as [10]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
The $20 framing summary conflicts with moderate and libertarian ratings
MinorThe framing discussion says all other respondents rated the $20 increase "Unfair," but the results explicitly say moderate and libertarian AIs rated that increase "Acceptable."
“Framing only mattered in one case-by using the word "raise," the socialists were moved from "Unfair" to "Very Unfair" for the $20 increase.All other respondents just thought this was "Unfair."”
“Moderates and libertarian AIs both find the $16 and $20 increases "Acceptable."”
Approval of large price increases reverses the reported result
Minor"Approbation" means approval, but the results say no AI considered even the $40 price acceptable. The introductory statement reverses the reported direction of the reaction.
“the largest price increases earn approbation even from AI libertarians.”
“no AI thinks even a $40 price is acceptable.”
The listed car-safety allocation differs from the one analyzed
MinorThe offered options include 40% for cars and 60% for highways, but the figure and discussion instead analyze 60% for cars and 40% for highways. That changes one of the four allocations in the status quo experiment.
“Subjects were then asked to choose their most preferred funding allocations: (70% cars, 30% highways), (40, 60), (30,70), and (50, 50).”
“including (60,40), which had no mass in the "neutral" framing.”
Citations
Cites
Status quo bias in decision making
Samuelson, W., & Zeckhauser, R. (1988) · Journal of Risk and Uncertainty · doi:10.1007/bf00055564 · Cited by 5,897
Schelling, T. (1971) · Journal of Mathematical Sociology · doi:10.1080/0022250x.1971.9989794 · Cited by 4,966
Understanding Social Preferences with Simple Tests
Charness, G., & Rabin, M. (2002) · The Quarterly Journal of Economics · doi:10.1162/003355302760193904 · Cited by 3,970
Fairness as a Constraint on Profit Seeking: Entitlements in the Market
Kahneman, D. J., Knetsch, J. L., & Thaler, R. H. (2000) · Cambridge University Press eBooks · doi:10.1017/cbo9780511803475.019 · Cited by 3,127
Do economists make markets?: on the performativity of economics
Mackenzie, D., Muniesa, F., & Siu, L. (2007) · Choice Reviews Online · doi:10.5860/choice.45-2137 · Cited by 2,080
Design of a Novel Globular Protein Fold with Atomic-Level Accuracy
Kuhlman, B. A., Dantas, G. A., Ireton, G. C., Varani, G., Stoddard, B. L., & Baker, D. A. (2003) · Science · doi:10.1126/science.1089427 · Cited by 1,666
Out of One, Many: Using Language Models to Simulate Human Samples
Argyle, L. P., Busby, E. C., Fulda, N., Gubler, J. R., Rytting, C., & Wingate, D. (2023) · Political Analysis · doi:10.1017/pan.2023.2 · Cited by 783
Methods and Problems in Business Cycle Theory
Lucas, R. E. (1980) · Journal of money credit and banking · doi:10.2307/1992030 · Cited by 686
On "The Methodology of Positive Economics"
Rotwein, E. (1959) · The Quarterly Journal of Economics · doi:10.2307/1884303 · Cited by 499
The Online Laboratory: Conducting Experiments in a Real Labor Market
Horton, J. J., Rand, D. G., & Zeckhauser, R. (2010) · National Bureau of Economic Research · doi:10.3386/w15961 · Cited by 402
Self-citation
Flagged first, then by citations
Research works printed but not linked: 2
- [4]4. Bender, Emily M., Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell, "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? (Parrot Emoji)," in "Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency" FAccT '21 Association for Computing Machinery New York, NY, USA 2021, p. 610-623.
- [18]18. Simon, Herbert A., The Sciences of the Artificial, 3rd Edition, Vol. 1 of MIT Press Books, The MIT Press, September 1996.
Cited by
Sorted by citations
Items 301–328 of 328
LLM-Based Agent Simulation Data Product for Counterfactual Evaluation of Hotel Dynamic Pricing
Majidi, M., Yang, J., & Beheshti, A. (2026) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws72778.2026.00170
Kim, H. L. (2026) · Journal of Macromarketing · doi:10.1177/02761467261483227
Kim, J. (2026) · Journal of Smart Tourism · doi:10.1177/27652157261488111
Exploring dissatisfaction in bus route reduction through LLM-calibrated agent-based modelling
Li, Q., yang, X., & Zhou, S. (2026) · Transport Policy · doi:10.1016/j.tranpol.2026.104431
The future of large-scale experiments and their challenges in the digital era
Holtz, D., Bojinov, I., Johari, R., Kallus, N., Lal, A., Anand, S., Carlson, K., Cohn, B., Cunningham, T., Deng, A., et al. (2026) · Nature Human Behaviour · doi:10.1038/s41562-026-02582-6
Ideology by Alphabet: Option Order and the Measurement of Machine Political PreferencesPreprint
Olah, T., Erdey, L., Tőkés, T., & Nádasi, L. (2026) · Research Square · doi:10.21203/rs.3.rs-11063104/v1
Heterogeneous-Agent Financial Market Simulation with Large Language Models
Gao, T., Peng, L., Xu, Q., & Li, G. (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.08.142
Jie, Y., Hu, S., Su, J., & Wang, J. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.105184
Algorithmic systems, human agency and the future of platform research
Hosseinmardi, H., Dutta, U., Rothschild, D. M., & Watts, D. J. (2026) · Nature Computational Science · doi:10.1038/s43588-026-01038-1
Shi, W., Qi, Y., & Xie, X. (2026) · Transportation Research Part A Policy and Practice · doi:10.1016/j.tra.2026.105278
Bespoke methods in economics. Method construction and the portability of warrant
Kwarciński, T. (2026) · Journal of Economic Methodology · doi:10.1080/1350178x.2026.2737102
From the platform society to the AI society
Törnberg, P., & Uitermark, J. (2026) · Dialogues on Digital Society · doi:10.1177/29768640261487931 · Cited by 1
How Model Choice and Memory Shape Preference Consistency in Large Language Models
Chung, Y. (2026) · Sociological Methods & Research · doi:10.1177/00491241261492203
Wang, J., Wang, Q., Yang, P., Li, G., Wang, X., Sun, Y., Feng, Z., Gao, Y., Li, Y., Guo, W., et al. (2026) · npj Digital Public Health · doi:10.1038/s44482-026-00045-3
LLM Hallucination of Citations in Economics Persists With Web‐Enabled Models
Buchanan, J. A., Hill, S. E., & Shapoval, O. (2026) · Southern Economic Journal · doi:10.1002/soej.70069 · Cited by 2
Nguyen, N. T., & Pham, V.-T. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-40581-4_9
Building crypto portfolios with agentic AI
Giudici, P., Piergallini, A., & Castelli, A. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1871825
Coarse-to-Fine Label Distribution Learning for High-Cardinality Survey Simulation
Li, J., Liu, H., Meng, Z., Qiu, C., & Gu, J. (2026) · International Journal of Web Services Research · doi:10.4018/ijwsr.424308
Hidden Profile Decision Making in Multi-Agent LLM Groups
Toxtli, C., & Delaflor, M. (2026) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3785651.3831525
Yumuşak, S., & Yumuşak, G. K. (2026) · Applied Sciences · doi:10.3390/app16199938
Deliberation Within AI: A Reproducible Pipeline for Multi-Agent Deliberation Simulation
Veri, F. (2026) · Digital Government Research and Practice · doi:10.1145/3858422
Large Language Models and Jobs-to-be-Done Framework in Customer Validation for Product Managers
Alabastro, Z. M., & Ilagan, J. B. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-36012-0_40
Zhang, R., Zhang, X., & 赵明扬 (2026) · INFORMS Journal on Data Science · doi:10.1287/ijds.2025.0124
Sarastov, Z. (2026) · Research Square · doi:10.21203/rs.3.rs-11429290/v1
A Research Agenda for the Economics of Transformative AI
Brynjolfsson, E., Korinek, A., & Agrawal, A. (2026) · Communications of the ACM · doi:10.1145/3769433
CONTEXT OVER LANGUAGE: PERSONA-DRIVEN INTERTEMPORAL CHOICE IN LARGE LANGUAGE MODELS
Başbuğ, A. K. (2026) · Akademik Yaklaşımlar Dergisi · doi:10.54688/ayd.1932565
Large language models for computational social science
Xu, F., Shao, C., Chen, L., Zeng, Q., Chen, Z., Sukiennik, N., Wang, J., Gao, C., Wang, H., Lian, J., et al. (2026) · Nature Human Behaviour · doi:10.1038/s41562-026-02539-9 · Cited by 1
孙万军, Li, B., & Lin, Z. (2026) · International Review of Economics & Finance · doi:10.1016/j.iref.2026.106003
Metadata
- Authors
- Horton, J., Filippas, A., & Manning, B.
- Affiliations
- National Bureau of Economic Research (US)
- Corresponding author
- Horton, John
- Venue
- National Bureau of Economic Research
- Publisher
- National Bureau of Economic Research
- Published
- 2023-04
- OpenAlex
- W4363624465
- Cited by
- 329
- References
- 21
- Citations by year
- 2026: 103 · 2025: 107 · 2024: 75 · 2023: 42
- Type
- report
- Language
- en
- Open access
- green
- doi.org
- Locations
- National Bureau of Economic Research (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Experimental Behavioral Economics Studies
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Keywords
- large language models, homo economicus, behavioral economics, preference elicitation, endowment effect, pilot study
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
