CoChat
Sign Up
PreprintCited by 35Forecasting Techniques and ApplicationsFinancial Markets and Investment StrategiesStock Market Forecasting Methods

Surveying Generative AI's Economic Expectations

Open access
Bybee, Leland·2023·arXiv
DOI 10.48550/arxiv.2305.02823arXiv 2305.02823
Open this paper in CoChat — free account

Abstract

I introduce a survey of economic expectations formed by querying a large language model (LLM)'s expectations of various financial and macroeconomic variables based on a sample of news articles from the Wall Street Journal between 1984 and 2021. I find the resulting expectations closely match existing surveys including the Survey of Professional Forecasters (SPF), the American Association of Individual Investors, and the Duke CFO Survey. Importantly, I document that LLM based expectations match many of the deviations from full-information rational expectations exhibited in these existing survey series. The LLM's macroeconomic expectations exhibit under-reaction commonly found in consensus SPF forecasts. Additionally, its return expectations are extrapolative, disconnected from objective measures of expected returns, and negatively correlated with future realized returns. Finally, using a sample of articles outside of the LLM's training period I find that the correlation with existing survey measures persists -- indicating these results do not reflect memorization but generalization on the part of the LLM. My results provide evidence for the potential of LLMs to help us better understand human beliefs and navigate possible models of nonrational expectations.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2305.02823

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • There is a lack of studies on predicting the aggregated stock market using ChatGPT.

    “However, there is a lack of studies on predicting the aggregated stock market, or equiavalently the market risk premium.”

    From ChatGPT and Deepseek: Can They Predict the Stock Market and Macroeconomy? (2025) · cites this paper as Bybee (2023)

    The Introduction (p. 3) describes this paper’s analysis of GPT’s stock-market expectations, and Return Expectations (p. 11) reports their correlation with future returns. Neither passage assesses whether studies using ChatGPT to predict the aggregate stock market or market risk premium are scarce; the existence of this study alone does not settle that claim. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

References 24 · Cited by 35

Research works printed but not linked: 30

  1. Afrouzi, Hassan, Spencer Yongwook Kwon, Augustin Landier, Yueran Ma, and David Thesmar, 2020, Overreaction in expectations: Evidence and theory, Available at SSRN 3709548 .
  2. Aher, Gati, Rosa I Arriaga, and Adam Tauman Kalai, 2022, Using large language models to simulate multiple humans, arXiv preprint arXiv:2208.10264 . · arXiv
  3. Barberis, Nicholas, Robin Greenwood, Lawrence Jin, and Andrei Shleifer, 2015, X-CAPM: An extrapolative capital asset pricing model, Journal of Financial Economics 115, 1-24.
  4. Barsky, Robert B, and J Bradford De Long, 1993, Why does the stock market fluctuate?, The Quarterly Journal of Economics 108, 291-311.
  5. Bender, Emily M., Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell, 2021, On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? xn-cc6c, in FAccT '21: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency , 610-623 (Association for Computing Machinery, New York, NY, USA).
  6. Bordalo, Pedro, Nicola Gennaioli, Yueran Ma, and Andrei Shleifer, 2020, Overreaction in macroeconomic expectations, American Economic Review 110, 2748-82.
  7. Bordalo, Pedro, Nicola Gennaioli, and Andrei Shleifer, 2018, Diagnostic expectations and credit cycles, The Journal of Finance 73, 199-227.
  8. Brand, James, Ayelet Israeli, and Donald Ngwe, 2023, Using gpt for market research, Available at SSRN 4395751 .
  9. Brown, Tom B., Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei, 2020, Language Models are Few-Shot Learners, arXiv .
  10. Brunnermeier, Markus, Emmanuel Farhi, Ralph S. J. Koijen, Arvind Krishnamurthy, Sydney C. Ludvigson, Hanno Lustig, Stefan Nagel, and Monika Piazzesi, 2021, Review Article: Perspectives on the Future of Asset Pricing, Rev. Financ. Stud. 34, 2126-2160.
  11. Bubeck, Sébastien, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang, 2023, Sparks of Artificial General Intelligence: Early experiments with GPT-4, arXiv .
  12. Chomsky, Noam, 1959, A review of bf skinner's verbal behavior, Language 35, 26-58.
  13. Coibion, Olivier, and Yuriy Gorodnichenko, 2012, What Can Survey Forecasts Tell Us about Information Rigidities?, Journal of Political Economy .
  14. Cutler, David M, James M Poterba, Lawrence H Summers, et al., 1990, Speculative dynamics and the role of feedback traders, American Economic Review 80, 63-68.
  15. Fama, Eugene F., and Kenneth R. French, 1989, Business conditions and expected returns on stocks and bonds, Journal of Financial Economics 25, 23-49.
  16. Flynn, Joel P, and Karthik Sastry, 2022, The macroeconomics of narratives, Available at SSRN 4140751 .
  17. Greenwood, Robin, and Andrei Shleifer, 2014, Expectations of Returns and Expected Returns, Review of Financial Studies 27.
  18. Jin, Lawrence J, and Pengfei Sui, 2022, Asset pricing with return extrapolation, Journal of Financial Economics 145, 273-295.
  19. Lakonishok, Josef, Andrei Shleifer, and Robert W Vishny, 1994, Contrarian investment, extrapolation, and risk, The journal of finance 49, 1541-1578.
  20. Lettau, Martin, and Sydney Ludvigson, 2001, Consumption, Aggregate Wealth, and Expected Stock Returns, Journal of Finance 56, 815-849.
  21. Lochstoer, Lars A., and Tyler Muir, 2022, Volatility Expectations and Returns, Journal of Finance 77, 1055-1096.
  22. Lucas, Robert E, 1976, Econometric policy evaluation: A critique, in Carnegie-Rochester conference series on public policy , volume 1, 19-46, North-Holland.
  23. Muth, John F, 1961, Rational expectations and the theory of price movements, Econometrica: Journal of the Econometric Society 315-335.
  24. Sargent, Thomas J, 2001, The conquest of American inflation (Princeton University Press).
  25. Shiller, Robert J., 2019, Narrative Economics (Princeton University Press, Princeton, NJ, USA).
  26. Sims, Christopher A, 1980, Macroeconomics and reality, Econometrica: journal of the Econometric Society 1-48.
  27. Touvron, Hugo, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample, 2023, LLaMA: Open and Efficient Foundation Language Models, arXiv .
  28. Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin, 2017, Attention Is All You Need, arXiv .
  29. Wei, Jason, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus, 2022, Emergent Abilities of Large Language Models, arXiv .
  30. Wu, Shijie, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann, 2023, BloombergGPT: A Large Language Model for Finance, arXiv .

Authors
Bybee, L.
Affiliations
Yale University (US)
Corresponding author
Bybee, Leland
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
1556-5068 (ISSN-L)
DOI
10.48550/arxiv.2305.02823
arXiv
2305.02823
OpenAlex
W4376455614
Cited by
35
References
24
Citations by year
2026: 7 · 2025: 8 · 2024: 12 · 2023: 8
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • SSRN Electronic Journal (acceptedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Forecasting Techniques and Applications
  • Social Sciences › Economics, Econometrics and Finance › Finance › Financial Markets and Investment Strategies
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Stock Market Forecasting Methods
Keywords
large language models, economic expectations, survey data, macroeconomic forecasting, Survey of Professional Forecasters, expected returns, underreaction, investor sentiment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review