CoChat
Sign Up
Cited by 165Decision-Making and Behavioral EconomicsExperimental Behavioral Economics StudiesEconomic and Environmental Valuation

The emergence of economic rationality of GPT

Open access
Chen, Yiting, Liu, Tracy Xiao, Shan, You, Zhong, Songfa·2023·Proceedings of the National Academy of Sciences
DOI 10.1073/pnas.2316205120PMC PMC10740389
Open this paper in CoChat — free account

Abstract

As large language models (LLMs) like GPT become increasingly prevalent, it is essential that we assess their capabilities beyond language processing. This paper examines the economic rationality of GPT by instructing it to make budgetary decisions in four domains: risk, time, social, and food preferences. We measure economic rationality by assessing the consistency of GPT’s decisions with utility maximization in classic revealed preference theory. We find that GPT’s decisions are largely rational in each domain and demonstrate higher rationality score than those of human subjects in a parallel experiment and in the literature. Moreover, the estimated preference parameters of GPT are slightly different from human subjects and exhibit a lower degree of heterogeneity. We also find that the rationality scores are robust to the degree of randomness and demographic settings such as age and gender but are sensitive to contexts based on the language frames of the choice situations. These results suggest the potential of LLMs to make good decisions and the need to further understand their capabilities, limitations, and underlying mechanisms.

AI snapshot

Written from the paper’s full text.

TL;DR

GPT-3.5-Turbo made risk, time, social, and food decisions that were mostly consistent with utility maximization and often scored higher than human decisions. Its rationality was robust to temperature and demographic prompts but dropped under alternative price framing and discrete choice formats.

Key findings

  • In the baseline, 95, 89, 81, and 92 out of 100 GPT observations for risk, time, social, and food preferences had no GARP violations; average CCEI was 0.998, 0.997, 0.997, and 0.999, versus human averages of 0.980, 0.985, 0.967, and 0.963 (P < 0.01).
  • GPT was more responsive to price changes than humans: mean Spearman correlations of ln(xA/xB) and ln(pA/pB) were −0.984, −0.966, −0.951, and −0.992 for risk, time, social, and food, versus −0.826, −0.788, −0.681, and −0.673 for humans (P < 0.01).
  • Rationality was similar at temperature 0.5 and 1, though invalid response rates were 4.7% and 9.8%; alternative price framing reduced average CCEI to 0.901, 0.884, 0.698, and 0.894, and discrete choice reduced it to 0.843, 0.908, 0.871, and 0.780 across the four domains.
  • GPT’s estimated preference parameters differed slightly from humans and showed substantially higher homogeneity; demographic prompts did not significantly affect CCEI, price responsiveness, or estimated parameters.

Methods

  • GPT-3.5-Turbo was accessed via OpenAI API and instructed to allocate 100 points between two commodities across 25 budget tasks in four domains: risk, time, social, and food. Each domain was repeated 100 times, yielding 10,000 tasks; baseline temperature was 0, with variations at 0.5 and 1, price framing, discrete choice, and demographic prompts.
  • A parallel human experiment used 347 subjects from a representative US sample, randomly assigned to baseline, price framing, and discrete choice conditions; the experiment was pre-registered and IRB-approved.
  • Rationality was measured using GARP and the critical cost efficiency index (CCEI), with robustness checks using HMI, MPI, and MCI; preference parameters were structurally estimated using nonlinear tobit maximum likelihood.

Limitations

  • The paper states it does not explore mechanisms behind GPT’s choices, including why responses are sensitive to context and framing; it suggests possible corpus-based heuristics, insufficient training on alternative frames, or spurious correlations.
  • The paper states it uses a narrow revealed-preference definition of economic rationality and a simple two-commodity experimental environment, so broader and more realistic settings remain untested.
  • The paper states demographic insensitivity may reflect hyperaccuracy distortion from alignment efforts; the text also makes evident that only GPT-3.5-Turbo was studied, and higher-temperature analyses were conditional on valid responses.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1073/pnas.2316205120

Cites

86 of 86 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 21 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    GPT's performance is not affected by settings of demographic information.

    “The performance is affected by the way questions are framed, but not by settings of demographic information and randomness.”
    Quote in the paperSubstantial claim
  • Findingown data

    The level of rationality drops significantly when a discrete choice setting is used.

    “However, the level of rationality drops significantly when we employ a different price framing and when we use a discrete choice setting.”
    • Also stated in: § Abstract · plain
      “We observe that rationality decreases when alternative price framing or discrete choice are used.”
    Quote in the paperSubstantial claim
  • Findingown data

    The estimated preference parameters of GPT are slightly different from those of human subjects.

    “Moreover, the estimated preference parameters of GPT, compared to those of human subjects, are slightly different and exhibit a substantially higher degree of homogeneity.”
    • Also stated in: § Abstract · plain
      “We find that the estimated preference parameters of GPT have some minor distinctions from human subjects and show a substantially higher degree of homogeneity.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: For risk and time preferences, the underlying utility function is U(x1,x2)=αu(x1)+(1−α)u(x2), where u(z)=1/ρ z^ρ for ρ≤1, ρ≠0, and ln(z) for ρ=0.
  • Findingown data

    The level of rationality drops significantly when a different price framing is employed.

    “However, the level of rationality drops significantly when we employ a different price framing and when we use a discrete choice setting.”
    • Also stated in: § Abstract · plain
      “We observe that rationality decreases when alternative price framing or discrete choice are used.”
    Quote in the paperSubstantial claim
  • Findingown data

    GPT's rationality scores are invariant to the specification of the randomness of GPT.

    “Furthermore, we find that GPT’s rationality scores are consistent across different demographic characteristics and invariant to the specification of the randomness of GPT.”
    Quote in the paperSubstantial claim
  • May combine two claims. 26
  • Classified as probably not a substantial claim. 12

  • May combine two claims.

    Findingown data

    GPT's decisions are mostly rational across risk, time, social, and food domains and score higher than human decisions.

    “We show that GPT’s decisions are mostly rational and even score higher than human decisions.”
    • Also stated in: § Abstract · boosted
      “We find that GPT demonstrates a high level of rationality in all four decision-making tasks concerning risk, time, social, and food, and it outperforms human subjects in the rationality score documented in both our human subject experiment and those reported in the literature.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    GPT observations always have negative Spearman's correlation coefficients, while human subjects have a lower proportion having negative coefficients (96.1% on average).

    “Fig. 2 further illustrates that GPT observations always have negative Spearman’s correlation coefficients, while human has a lower proportion having negative Spearman’s correlation coefficients (96.1% on average).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The average CCEI for GPT is 0.998, 0.997, 0.997, and 0.999 for risk, time, social, and food preferences, respectively.

    “The average CCEI is 0.998, 0.997, 0.997, and 0.999 for risk, time, social, and food preferences, respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For human subjects, the mean Spearman's correlation coefficients of ln(xA/xB) and ln(pA/pB) for risk, time, social, and food preferences are −0.826, −0.788, −0.681, and −0.673.

    “For GPT observations, the coefficients for risk, time, social, and food preferences have a mean of −0.984, −0.966, −0.951, and −0.992, while these are −0.826, −0.788, −0.681, and −0.673 for human subjects, respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    As GPT temperature increases (randomness), the level of rationality (CCEI) is similar to baseline, and increasing randomness does not significantly impact its performance.

    “We find that as the randomness increases, the level of rationality is similar to that in the baseline condition (Fig. 4).”
    • Also stated in: § Discussion · plain
      “We also observe that increasing the randomness of GPT does not significantly impact its performance.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    95, 89, 81, and 92 out of 100 GPT observations for risk, time, social, and food preferences exhibit no violations of GARP (CCEI=1).

    “We find that 95, 89, 81, and 92 out of 100 GPT observations for risk, time, social, and food preferences exhibit no violations of GARP; that is, CCEI equals 1.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Results from HMI, MPI, and MCI show that GPT observations exhibit a high level of rationality across the four preference domains and surpass those of human subjects across all domains (P<0.1, two-sided two-sample t-tests).

    “Consistent with the observations based on CCEI, results from these indices show that GPT observations exhibit a high level of rationality across the four preference domains and surpass those of human subjects across all domains (P<0.1, two-sided two-sample t-tests).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For GPT observations, the mean Spearman's correlation coefficients of ln(xA/xB) and ln(pA/pB) for risk, time, social, and food preferences are −0.984, −0.966, −0.951, and −0.992.

    “For GPT observations, the coefficients for risk, time, social, and food preferences have a mean of −0.984, −0.966, −0.951, and −0.992, while these are −0.826, −0.788, −0.681, and −0.673 for human subjects, respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The average CCEI among human subjects is 0.980, 0.985, 0.967, and 0.963 for risk, time, social, and food preferences, respectively.

    “Meanwhile, in our human experiment, the average CCEI among human subjects is 0.980, 0.985, 0.967, and 0.963 for risk, time, social, and food preferences, respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The downward-sloping demand property is impaired under the alternative price framing, with Spearman's correlation coefficients of −0.053, −0.116, 0.267, and −0.499 for risk, time, social, and food preferences.

    “Moreover, the downward-sloping demand property is impaired under the alternative price framing, with the key Spearman’s correlation coefficients being −0.053, −0.116, 0.267, and −0.499 for risk, time, social, and food preferences, respectively (SI Appendix, Fig. S24).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Under discrete choice, Spearman's correlation coefficients are −0.589, −0.497, −0.519, and −0.533 for risk, time, social, and food preferences (P<0.01 vs baseline).

    “Consistently, the Spearman’s correlation coefficients are −0.589, −0.497, −0.519, and −0.533 for risk, time, social, and food preferences (SI Appendix, Fig. S24; P<0.01 when compared to the baseline condition, two-sided two-sample t-tests).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Human subjects' rationality level and downward-sloping demand property reduce in the discrete setting compared to baseline (P<0.05 in risk and time preferences).

    “Human subjects’ rationality level and the downward-sloping demand property reduce in the discrete setting, compared to the baseline condition (P<0.05, two-sided two-sample t-tests in risk and time preferences).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The alternative price framing reduces the rationality level and the downward-sloping demand property in the human subjects experiment (P<0.05 in risk and time preferences).

    “We find that the alternative price framing also reduces the rationality level and the downward-sloping demand property in the human subjects experiment (P<0.05, two-sided two-sample t-tests in risk and time preferences).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    GPT's CCEI values, Spearman's correlation coefficients of ln(xA/xB) and ln(pA/pB), and estimated preference parameters are all insensitive to variations of demographic factors embedded in prompts.

    “Comparing the baseline condition and variations in demographics in the GPT experiment, we find that CCEI values, Spearman’s correlation coefficients of ln(xA/xB) and ln(pA/pB), and estimated preference parameters are all insensitive to variations of demographic factors embedded in the prompts to request responses from GPT (Fig. 4, SI Appendix, Fig. S24 and Tables S2 and S3).”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The underlying utility functions commonly employed in prior literature capture risk, time, social, and food preferences.
    • The authors state this as an assumption: Risk preference is estimated using a disappointment aversion (DA) functional form.
    • The authors state this as an assumption: Time preference is estimated using a time separable period utility function.

  • May combine two claims.

    Findingown data

    Under price framing, average CCEI values are 0.901, 0.884, 0.698, and 0.894 for risk, time, social, and food preferences, respectively, with 34%, 48%, 88%, and 49% exhibiting CCEI below 0.9.

    “Remarkably, the average CCEI for risk preference declines to 0.901, with 34% exhibiting a CCEI below 0.9. These values are 0.884 (48%), 0.698 (88%), and 0.894 (49%) for time, social, and food preferences, respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    GPT responses to price changes in the price framing condition are disordered and appear flatter compared to those in the baseline condition.

    “SI Appendix, Figs. S13–S16 show the disordered responses of GPT observations to price changes in the price framing condition, which appear to be flatter compared to those in the baseline condition.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Reductions in rationality and downward-sloping demand under discrete choice are larger in the GPT experiment than in the human experiment.

    “As shown in the figures, these reductions are larger in the GPT experiment than in the human experiment. We also verify this observation through OLS regression analyses (SI Appendix, Table S4).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Under discrete choice, 51%, 32%, 33%, and 55% of GPT observations have a CCEI below 0.9 in risk, time, social, and food preferences, respectively.

    “Additionally, 51%, 32%, 33%, and 55% of GPT observations demonstrate a CCEI below 0.9 in risk, time, social, and food preferences, respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    In human subjects (our experiment) and prior studies, rationality score and preference differ across demographic groups.

    “These are in contrast to results of our human experiment (SI Appendix, Tables S5 and S6) and prior studies where rationality score and preference have been shown to differ across demographic groups (22, 28, 61).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [22] S. Choi, S. Kariv, W. Müller, D. Silverman, Who is (more) rational? Am. Econ. Rev. 104, 1518– 1550 (2014). source

      Not checked in the free run.

    • cites [28] F. Echenique, S. Lee, M. Shum, The money pump as a measure of revealed preference violations. J. Polit. Econ. 119, 1201– 1223 (2011). source

      Not checked in the free run.

    • cites [61] H. M. Von Gaudecker, A. Van Soest, E. Wengström, Heterogeneity in risky choice behavior in a broad population. Am. Econ. Rev. 101, 664– 694 (2011). source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductown data

    GPT constantly provides correct answers to three testing questions, confirming it understands the decision environment.

    “For each question, we simulate 25 times and GPT constantly provides correct answers. This confirms that GPT understands the decision environment.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Temperature 0 makes GPT give deterministic answers.

    “We set the temperature to be 0 in the baseline condition, in which the model gives deterministic answers (7, 9, 10).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [7] M. Binz, E. Schulz, Using cognitive psychology to understand GPT-3. Proc. Natl. Acad. Sci. U.S.A. 120, e2218523120 (2023). 36730192 PMC9963545 source

      Not checked in the free run.

    • cites [9] T. Webb, K. J. Holyoak, H. Lu, Emergent analogical reasoning in large language models. Nat. Hum. Behav. 7, 1526– 1541 (2023). 37524930 source

      Not checked in the free run.

    • cites [10] J. J. Horton, Large language models as simulated economic agents: What can we learn from homo silicus? NBER Working Paper (2023). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    GPT response can be highly sensitive to contexts.

    “This is line with some studies which show that GPT response can be highly sensitive to contexts (7, 10, 45, 47, 48, 49).”
    Quote in the paperNot a substantial claim0 of 6 sources checked
    • cites [7] M. Binz, E. Schulz, Using cognitive psychology to understand GPT-3. Proc. Natl. Acad. Sci. U.S.A. 120, e2218523120 (2023). 36730192 PMC9963545 source

      Not checked in the free run.

    • cites [10] J. J. Horton, Large language models as simulated economic agents: What can we learn from homo silicus? NBER Working Paper (2023). source

      Not checked in the free run.

    • cites [45] A. Borji, A categorical archive of chatGPT failures. arXiv [Preprint] (2023). http://arxiv.org/abs/2302.03494 (Accessed 25 November 2023). source

      Not checked in the free run.

    • cites [47] K. Mahowald., Dissociating language and thought in large language models: A cognitive perspective. arXiv [Preprint] (2023). http://arxiv.org/abs/2301.06627 (Accessed 25 November 2023).

      Not checked in the free run.

    • cites [48] E. Jones, J. Steinhardt, Capturing failures of large language models via human cognitive biases. Adv. Neural Inf. Process. Syst. 35, 11785– 11799 (2022).

      Not checked in the free run.

    • cites [49] J. Brand, A. Israeli, D. Ngwe, Using GPT for market research. Available at SSRN 4395751 (2023). source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Human subjects were randomly assigned to the baseline, price framing, and discrete choice conditions.

    “347 human subjects from a representative US sample are randomly assigned to the baseline, price framing, and discrete choice conditions.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    GPT obtains a high rationality score but has some potential limitations in its decision-making abilities.

    “These findings suggest that GPT obtains high rationality score but has some potential limitations in its decision-making abilities.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The human experiment was pre-registered.

    “SI Appendix provides the design and instructions of this pre-registered human experiment (AEARCTR-0011750).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    The human experiment was approved by an institutional review board and all subjects provided informed consent.

    “This experiment was approved by The Institutional Review Board of Finance and Economics Experimental Laboratory in The Wang Yanan Institute of Studies in Economics, Xiamen University (FEEL230701), and all subjects provided informed consent before they started the experiment.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    In the discrete setting, a locally non-satiated, strictly monotonic, continuous, and concave utility may violate GARP, demonstrating the need to use the assumption of strong monotonicity.

    “In our discrete setting, (23) shows that a locally non-satiated, strictly monotonic, continuous, and concave utility may violate GARP and demonstrates the need to use the assumption of strong monotonicity (see also ref. 55 for discussions).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [23] W. T. Harbaugh, K. Krause, T. R. Berry, GARP for kids: On the development of rational choice behavior. Am. Econ. Rev. 91, 1539– 1545 (2001). source

      Not checked in the free run.

    • cites ref. 55

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Other rationality indices (HMI, MPI, MCI) were computed as robustness checks.

    “We also compute these indices and report the results as robustness checks.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    CCEI values documented in prior studies range from 0.81 to 0.99 with an average of 0.918.

    “SI Appendix, Fig. S1 plots CCEI values documented in prior studies, which range from 0.81 to 0.99 with an average of 0.918.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Experimental instructions between human subjects and GPT were kept as similar as possible.

    “We keep the experimental instructions between human subjects and GPT as similar as possible.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationcitation

    Large language models like GPT have potential to streamline experimental research and yield new data and insights.

    “Second, our work highlights the potential of large language models like GPT to streamline experimental research and yield new data and insights (10).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [10] J. J. Horton, Large language models as simulated economic agents: What can we learn from homo silicus? NBER Working Paper (2023). source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    GPT's decision-making is more significantly affected by both the framing of prices and discrete choices than human subjects.

    “These results suggest that GPT’s decision-making is more significantly affected by both the framing of prices and discrete choices than human subjects.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationcitation

    Understanding how LLMs make decisions could help reveal general principles that govern both language intelligence and decision intelligence.

    “For example, our understanding of how LLMs make decisions could help reveal general principles that govern both language intelligence and decision intelligence (70).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [70] T. J. Sejnowski, Large language models and the reverse turing test. Neural Comput. 35, 309– 342 (2023). 36746144 PMC10177005 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationreasoning

    Studying the choice behavior of artificial intelligence can provide an important benchmark for understanding natural intelligence.

    “Finally, studying the choice behavior of artificial intelligence can provide an important benchmark for understanding natural intelligence.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Randomness (temperature) increases stochasticity and creativity in GPT's language presentations but does not affect its rationality score.

    “These findings suggest that randomness increases the stochasticity and creativity in language presentations of GPT, but not the rationality score.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    Rationality is often defined more broadly in the literature to include various decision rules and heuristics.

    “Second, we focus on economic rationality as defined by revealed preference analysis, whereas rationality is often defined more broadly in the literature to include various decision rules and heuristics (40, 77, 78).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [40] D. Kahneman, Maps of bounded rationality: Psychology for behavioral economics. Am. Econ. Rev. 93, 1449– 1475 (2003). source

      Not checked in the free run.

    • cites [77] H. A. Simon, Rational decision making in business organizations. Am. Econ. Rev. 69, 493– 513 (1979).

      Not checked in the free run.

    • cites [78] R. H. Thaler, Behavioral economics: Past, present, and future. Am. Econ. Rev. 106, 1577– 1600 (2016). source

      Not checked in the free run.

36 of 36

  • The paper relies on this without support.

    The economic rationality of GPT has not been explored in prior work.

    “Nevertheless, the rationality of GPT remains unexplored.”

  • The authors state this as an assumption.

    For risk and time preferences, the underlying utility function is U(x1,x2)=αu(x1)+(1−α)u(x2), where u(z)=1/ρ z^ρ for ρ≤1, ρ≠0, and ln(z) for ρ=0.

    “For these two domains, we assume that the underlying utility function is given by:U(x1,x2)=αu(x1)+(1−α)u(x2), where the utility function u(z)=1ρzρ,ρ≤1(ρ≠0)ln(z),ρ=0 and α∈[0,1].”

  • The authors state this as an assumption.

    The underlying utility functions commonly employed in prior literature capture risk, time, social, and food preferences.

    “Our estimations will be made for each DM separately on the assumption of the underlying utility function commonly employed found in the prior literature to capture risk, time, social, and food preferences.”

  • The authors state this as an assumption.

    Background income is zero.

    “with the assumption that background income is zero”

  • The authors state this as an assumption.

    Social preference utility is a member of the CES family.

    “we assume that U(xA,xB) is a member of the CES family.”

  • The authors state this as an assumption.

    Food preference utility is a member of the CES family.

    “we assume that U(xA,xB) is a member of the CES family.”

  • The authors state this as an assumption.

    Risk preference is estimated using a disappointment aversion (DA) functional form.

    “We estimate the risk preference of the DM by a functional form of disappointment aversion (DA) introduced by ref. 57”

  • The authors state this as an assumption.

    Time preference is estimated using a time separable period utility function.

    “Following the method of ref. 19, we use a time separable period utility function to estimate the DM’s time preference from intertemporal decisions of the payoff between today and 1 mo later x:(xA,xB), with the assumption that background income is zero”
8 of 8

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • While the previous generation of models frequently exhibited human-like heuristics and biases, they have largely disappeared in the latest generation of LLMs.

    “Interestingly, there is evidence from several studies showing that, while the previous generation of models frequently exhibited human-like heuristics and biases, they have largely disappeared in the latest generation of LLMs (Chen, Liu, et al., 2023;Hagendorff et al., 2023).”

    From Machine Psychology (2023) · cites this paper as Chen, Liu, et al., 2023

    The Discussion (page unavailable) says GPT remains sensitive to framing and can use corpus-based heuristics in unfamiliar decision tasks. The Abstract likewise reports reduced rationality under alternative framing and discrete choice. The paper does not compare model generations, and its findings run contrary to the claim that such biases have largely disappeared. (LLM, conf=0.86)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • Chatbot behavior is similar to human behavior, suggesting that GPT could have the potential in assisting human decision-making.

    “In synthesis, these findings suggest that chatbots behavior is similar to human behavior, leading to the suggestion that “GPT could have the potential in assisting human decision-making” (147).”

    From The impact of generative artificial intelligence on socioeconomic inequalities and policy making (2024) · cites this paper as [147]

  • One study suggests LLMs exhibit more rational behavior than humans.

    “One study, for example, suggests LLMs exhibit more rational behavior than humans (65).”

    From Can Generative AI improve social science? (2024) · cites this paper as [65]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Stated Leontief limit does not follow from the CES formula

    Minor

    For fixed positive weights α and 1−α, the displayed CES function approaches min{x1,x2} as ρ tends to negative infinity; the weights do not remain inside the minimum. The stated limit min{αx1,(1−α)x2} is therefore inconsistent with that formula.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. I. Goodfellow, Y. Bengio, A. Courville, Deep Learning (MIT Press, 2016).
  2. M. Houtman, J. Maks, Determining all maximal data subsets consistent with revealed preference. Kwantitatieve Methoden 19, 89–104 (1985).

Cited by

Sorted by citations

Related

Authors
Chen, Y., Liu, T. X., Shan, Y., & Zhong, S.
Affiliations
  • Lingnan University (HK)
  • Tsinghua University (CN)
  • National University of Singapore (SG)
Corresponding author
  • Chen, Yiting
  • Liu, Tracy Xiao
  • Shan, You
  • Zhong, Songfa
ORCID
Venue
Proceedings of the National Academy of Sciences
Publisher
National Academy of Sciences
Published
2023-12-12
Volume
120
Issue
51
Pages
e2316205120-e2316205120
ISSN
  • 0027-8424 (ISSN-L)
  • 1091-6490
DOI
10.1073/pnas.2316205120
PMC
PMC10740389
OpenAlex
W4389611357
Semantic Scholar
de916e9e6f8841dbb73b35615f8d15eba2e9eaf7
Cited by
164
References
70
Citations by year
2026: 48 · 2025: 67 · 2024: 40 · 2023: 8 · 2021: 1
FWCI
34.05
Type
journal article
Language
en
Open access
Licence
Locations
  • Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • MOST | NSFC | National Outstanding Youth Foundation of China (72222005)
  • Tsinghua University (2022Z04W01032)
  • Hong Kong University of Science and Technology (00000)
Topics
  • Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
  • Social Sciences › Safety Research › Experimental Behavioral Economics Studies
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Economic and Environmental Valuation
Keywords
GPT, large language models, economic rationality, budget decision-making, revealed preference, utility maximization, preference heterogeneity, preference estimation, risk preferences, time preference, social preferences, food preferences
MeSH terms
Decision Making, Humans

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review