Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

LLMs of slightly older vintage are unsuited for these tasks.

“It is essential to note that this is a new possibility-that LLMs of slightly older vintage are unsuited for these tasks, as I will show.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The experiments in this paper were run in minutes for a trivial amount of money.

“The advantages in terms of speed and cost are so enormous-the experiments in this paper were run in minutes for a trivial amount of money-could make this relatively easy, especially as software tools develop.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Responses from an LLM can be stochastic depending on the temperature parameter.

“The responses can be stochastic, depending on the”

May combine two claims.

An LLM has no fixed persona; it can be induced to play different agents via prompts.

“However, it has no fixed persona-it can be induced to play different agents via prompts.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Different models can lead to different responses.

“And, of course, different models can lead to different responses.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

LLMs hallucinate and make up new facts regularly.

“It is also inconsistent with the worry that they hallucinate and make up new”

May combine two claims.

Large price increases in the snow shovel scenario are viewed more negatively, and the largest price increases earn approbation even from AI libertarians.

“I show that large gouging is viewed more negatively; the largest price increases earn approbation even from AI libertarians.”

May combine two claims.

In each status quo setting, the option presented as the status quo was far more commonly chosen and was the most common selection in every scenario, including (60,40) which had no mass under neutral framing.

“In each status quo setting, in the cases where one of the options was presented as the status quo, the status quo is far more commonly chosen.It is the most common selection in every scenario when presented-including (60,40), which had no mass in the”

May combine two claims.

In the neutral framing of the car/highway safety budget allocation, the most common selection was a 50-50 split, with remaining mass on (70,30) and (30,70), and no mass on (60,40).

“In the neutral framing, the most common selection is a 50-50 split.The rest of the mass is on (70, 30) and (30,70).There is no mass on (60,40) despite that being an option.”

May combine two claims.

In all scenarios with the final price masked, $20 is the most likely choice, with the distribution of other prices varying across scenarios.

“In all cases, $20 is the most likely choice, with the exact distribution of other prices varying across scenarios.”

May combine two claims.

AI conservatives viewed all prices as unfair, differing from libertarian and moderate AIs.

“AI conservatives were somewhat out of step with their libertarian and moderate incarnations, viewing all prices as unfair.”

May combine two claims.

The inequity-averse AI chooses the option that minimizes the discrepancy between the two players except in the Berk23 scenario, which wastes 1,000.

“For the inequity-averse AI in the third column from the left, it chooses the option that minimizes the discrepancy between the two players except for the Berk23 scenario, which wastes 1,000 (800,200).”

May combine two claims. The quote only partly states this claim.

The self-interested AI consistently chooses the payoff that maximizes its payoff by picking left, except in the Berk29 scenario where payoffs are equal from B's perspective but Right gives a much larger payoff to A.

“Starting from the far right, the self-interested AI consistently chooses the payoff that maximizes its payoff by picking”

May combine two claims.

Less advanced GPT-3 models do not seemingly change their answers despite endowments, except the unendowed agent in one model chooses all Right.

“In the bottom row of the figure, I pool together all the less advanced GPT-3 models because they do not seemingly change their answers despite the endowments (except the unendowed agent, which chooses all”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). The quote only partly states this claim.

When not given the typical wage instruction, the AI tends to be overly enamored of experience, showing lexical preferences with work experience first, then wage.

“enamored of experience, showing lexical preferences with work experience first, then wage.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The view that LLMs are simply repeating back something already read is not completely correct, though memorization is a concern.

“This view is not completely correct-though memorization is a concern.”

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based). The quote only partly states this claim.

Natural human variation in preferences does not exist in LLMs unless they are endowed with differences.

“This”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

LLM experimentation is more akin to the practice of economic theory, despite superficially looking like empirical research.

“LLM experimentation is more akin to the practice of economic theory, despite superficially looking like empirical research.”

May combine two claims.

LLMs know things but often do not apply that knowledge consistently to scenarios, like students who crammed for an exam.

“Like students who have crammed for an exam, these models”

May combine two claims.

The actual fractions choosing Left in the Berk29, Berk26, Berk23, and Barc2 scenarios of Charness and Rabin (2002) are 31%, 78%, 100%, and 52%, respectively.

“The actual answers are 31%, 78%,100%, and 52%.”

• cites Charness and Rabin (2002)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

LLMs can now respond realistically to a wide range of textual inputs, giving responses similar to what might be expected from a human.

“This is possible because LLMs can now respond realistically to a wide range of textual inputs, giving responses similar to what we might expect from a human.”

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based). May combine two claims.

The garbage-in, garbage-out critique rests partly on the notion that LLM responses are a weighted average, and this is not correct.

“The”

Probably not a substantial claim: may not pass S2 (checkable).

Sciences of the artificial can usefully abstract away from micro-details of construction so long as the created object is viewed as trying to maximize something subject to environmental constraints.

“As Simon (1996) notes,”

• cites Simon (1996)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

Unlike the one rational homo economicus, there are many homo silici.

“Unlike the one homo economicus that is rational, there are many homo silci.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

Increased capability of LLMs likely expands possible experiments but also makes performativity a greater concern.

“The increased capability of the LLMs likely expands the possible experiments, but it also makes performativity a greater concern.”

Classified as probably not a substantial claim.

With agent-based models, the researcher programs the agents and then sees what they do, making the researcher both judge and jury.

“With ABMs, the researcher is both judge and jury: you program the agents and then see what they do.”

Probably not a substantial claim: may not pass S2 (checkable).

Prompting to get conditional distributions of responses might be good enough for the research question.

“Of course, there is no”

May combine two claims.

LLMs are systems created for an understandable purpose with an explicit goal of optimization.

“What matters is that LLMs are systems created for some understandable purpose with an explicit goal of optimization.”

• cites Simon (1996)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

Economists tend to pose questions that place few demands on the sample.

“One advantage economists have in using LLMs is they tend to pose questions that place few demands on the sample.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Each homo silicus is a flawed model and can often give responses far away from what is rational or even sensical.

“To be clear, each homo silicus is a flawed model and can often give responses far away from what is rational or even sensical.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

The stated-versus-revealed preference critique of LLMs is only superficially persuasive.

“For the”

Probably not a substantial claim: may not pass S3 (load-bearing).

With a particular LLM, there is a single model, so N equals 1 no matter what.

“With a particular LLM, there is a single model, so it would seem that N = 1 no matter what.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Results from AI experiments will still require empirical confirmation because we care about actual human behavior.

“Ultimately, we care about the behavior of actual humans and so results from AI ex-periments will still require empirical confirmation.”

Probably not a substantial claim: may not pass S2 (checkable).

The demands of representativeness in the social sciences depend on the research question.

“And even if not perfect, the demands of representativeness in the social sciences have always depended on the research question.”

May combine two claims.

The approach can be done remarkably cheaply, making it feasible to try numerous variations and use arbitrarily large sample sizes.

“Furthermore, it can do this remarkably cheaply: Because of this low cost, it is feasible to try numerous variations of wording, prompts, answer order, and so on.Sample sizes can be arbitrarily large.”

May combine two claims.

A key finding of Horton (2023) is that there is little reduction in overall hiring in the short run but a substantial shift toward more productive workers, as proxied by past earnings and experience.

“A key finding of the paper is that there is little reduction in overall hiring (at least in the short-run) but a substantial shift towards more productive workers, as proxied by past earnings and experience.”

• cites Horton (2023)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

It is unclear whether the default choice of less advanced models is meaningful or reflects the order in which results are presented.

“However, it is unclear if this default is meaningful or reflects the order in which results are presented.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Because the agents are not interacting, the experiment can proceed in parallel.

“Because the agents are not interacting in any way, the experiment can proceed in parallel.”

Classified as probably not a substantial claim. May combine two claims.

In Horton (2023), employers were randomly assigned minimum wages and applicants had to bid up to meet that minimum wage.

“In my paper Horton (2023), I report the results of a minimum wage experiment in which employers were randomly assigned minimum wages: applicants had to bid up to meet that minimum wage.”

• cites Horton (2023)Not checked yet

  Not checked in the free run.

• AssumptionRests on an assumption: The labor-labor substitution margin has been noted as a possibility in the literature but is difficult to detect empirically.

The quote only partly states this claim.

If the original Kahneman et al. (1986) sample is nationally representative, the LLM estimate for the $20 price increase is an underestimate relative to the original 82%.

“About 37% of Americans described themselves as”

• AssumptionThe authors state this as an assumption: The sample in Kahneman et al. (1986) is nationally representative.

Probably not a substantial claim: may not pass S4 (based).

The AI setting allows a clean within-subject experiment because the AI does not remember having seen the previous prompt.

“one benefit to the AI setting is that I can do a clean within-subject experiment because the AI does not”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Brand, Israeli and Ngwe (2023) show that GPT3 responds to survey questions consistently with economic theory and well-documented consumer behavior patterns, including downward-sloping demand curves and state dependence.

“In an application of this method, Brand, Israeli and Ngwe (2023) show that GPT3”

• cites Brand, Israeli and Ngwe (2023)Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Each call is a separate API call with no knowledge of other scenarios passed between calls.

“Each call is a separate API call, with no”

Probably not a substantial claim: may not pass S3 (load-bearing).

Even an inequity-averse AI seems to have a limit.

“Even an inequity-averse AI seems to have a limit.”

Classified as probably not a substantial claim. May combine two claims.

Samuelson and Zeckhauser (1988) introduced the term status quo bias and demonstrated that when an option was presented as the status quo, it was more likely to be selected.

“This paper introduced the term status quo bias and demonstrated in several decision scenarios that when an option was presented as the status quo, it was more likely to be selected.”

• cites Samuelson and Zeckhauser (1988)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The rise in hourly wages was expected because AI employers were not instructed about the possibility of not filling their jobs.

“This was expected as there was no instruction to AI employers about the possibility of not filling their jobs.”

May combine two claims.

The high probability assigned to $20 is suggestive of a human bias toward whole numbers, numbers divisible by 5, and menu costs, rather than being due to Kahneman et al. (1986) per se.

“This is suggestive that the high probability assigned to $20 is not due to Kahneman et al. (1986) per se, but rather a human bias towards whole numbers, numbers evenly divisible by 5, and some menu costs in practice”

• cites Kahneman et al. (1986)Not checked yet

  Not checked in the free run.