The DOI matches its Crossref record.
Superforecasting: the art and science of prediction
Abstract
A New York Times Bestseller An Economist Best Book of 2015 most important book on decision making since Daniel Kahneman's Thinking, Fast and Slow. Jason Zweig, The Wall Street Journal Everyone would benefit from seeing further into the future, whether buying stocks, crafting policy, launching a new product, or simply planning the weeks meals. Unfortunately, people tend to be terrible forecasters. As Wharton professor Philip Tetlock showed in a landmark 2005 study, even experts predictions are only slightly better than chance. However, an important and underreported conclusion of that study was that some experts do have real foresight, and Tetlock has spent the past decade trying to figure out why. What makes some people so good? And can this talent be taught? In Superforecasting, Tetlock and coauthor Dan Gardner offer a masterwork on prediction, drawing on decades of research and the results of a massive, government-funded forecasting tournament. The Good Judgment Project involves tens of thousands of ordinary peopleincluding a Brooklyn filmmaker, a retired pipe installer, and a former ballroom dancerwho set out to forecast global events. Some of the volunteers have turned out to be astonishingly good. Theyve beaten other benchmarks, competitors, and prediction markets. Theyve even beaten the collective judgment of intelligence analysts with access to classified information. They are superforecasters. In this groundbreaking and accessible book, Tetlock and Gardner show us how we can learn from this elite group. Weaving together stories of forecasting successes (the raid on Osama bin Ladens compound) and failures (the Bay of Pigs) and interviews with a range of high-level decision makers, from David Petraeus to Robert Rubin, they show that good forecasting doesnt require powerful computers or arcane methods. It involves gathering evidence from a variety of sources, thinking probabilistically, working in teams, keeping score, and being willing to admit error and change course. Superforecasting offers the first demonstrably effective way to improve our ability to predict the futurewhether in business, finance, politics, international affairs, or daily lifeand is destined to become a modern classic.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Iterative near-term ecological forecasting: Needs, opportunities, and challenges
Dietze, M. C., Fox, A., Beck-Johnson, L. M., Betancourt, J. L., Hooten, M. B., Jarnevich, C. S., Keitt, T. H., Kenney, M. A., Laney, C. M., Larsen, L. G., et al. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1710231115 · Cited by 604
The Bias of Crowds: How Implicit Bias Bridges Personal and Systemic Prejudice
Payne, B. K., Vuletich, H. A., & Lundberg, K. B. (2017) · Psychological Inquiry · doi:10.1080/1047840x.2017.1335568 · Cited by 535
A practical guide to structured expert elicitation using the IDEA protocol
Hemming, V., Burgman, M. A., Hanea, A. M., McBride, M. F., & Wintle, B. C. (2018) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12857 · Cited by 528
Chicco, D., Warrens, M. J., & Jurman, G. (2021) · IEEE Access · doi:10.1109/access.2021.3084050 · Cited by 444
What Motivates Effort? Evidence and Expert Forecasts
DellaVigna, S., & Pope, D. (2018) · The Review of Economic Studies · doi:10.1093/restud/rdx033 · Cited by 392
Understanding algorithm aversion: When is advice from automation discounted?
Prahl, A., & Van Swol, L. (2017) · Journal of Forecasting · doi:10.1002/for.2464 · Cited by 308
The Behavioral Revolution and International Relations
Hafner-Burton, E. M., Haggard, S., Lake, D. A., & Victor, D. G. (2017) · International Organization · doi:10.1017/s0020818316000400 · Cited by 291
Towards a capability theory of (innovating) firms: implications for management and policy
Teece, D. J. (2017) · Cambridge Journal of Economics · doi:10.1093/cje/bew063 · Cited by 272
Modelling complex systems of heterogeneous agents to better design sustainability transitions policy
Mercure, J.-F., Pollitt, H., Bassi, A. M., Viñuales, J. E., & Edwards, N. R. (2016) · Global Environmental Change · doi:10.1016/j.gloenvcha.2016.02.003 · Cited by 204
The feeling of grip: novelty, error dynamics, and the predictive brain
Kiverstein, J., Miller, M., & Rietveld, E. (2019) · Synthese · doi:10.1007/s11229-017-1583-9 · Cited by 143
“Improving Intelligence Analysis with Decision Science.”
Dhami, M. K., Mandel, D. R., Mellers, B. A., & Tetlock, P. E. (2015) · Perspectives on Psychological Science · doi:10.1177/1745691615598511 · Cited by 104
“Finally, assignment of the best forecasters into so-called “superteams”, allowing elite performers to interact online with each other, yields additional improvements to forecasting accuracy (Mellers et al., 2015b; Tetlock & Gardner, 2015).”
BackgroundGoldman, A. I. (2018) · Topoi · doi:10.1007/s11245-016-9410-3 · Cited by 107
Unsolved Problems in ML Safety
Hendrycks, D., Carlini, N., Schulman, J., & Steinhardt, J. (2021) · arXiv · doi:10.48550/arxiv.2109.13916 · Cited by 95
Confidence Calibration in a Multiyear Geopolitical Forecasting Competition
Moore, D. A., Swift, S. A., Minster, A., Mellers, B., Ungar, L., Tetlock, P., Yang, H. H. J., & Tenney, E. R. (2017) · Management Science · doi:10.1287/mnsc.2016.2525 · Cited by 74
“Intelligence analysts aware of their accountability to a political establishment prone to seeking blame when things go wrong may be skittish about making their forecasts clear enough to be tested and scored (Tetlock and Gardner 2015, Tetlock and Mellers 2011).”
BackgroundFriedman, J. A., Baker, J. D., Mellers, B. A., Tetlock, P. E., & Zeckhauser, R. (2018) · International Studies Quarterly · doi:10.1093/isq/sqx078 · Cited by 58
Do People Inherently Dislike Uncertain Advice?
Gaertig, C., & Simmons, J. P. (2018) · Psychological Science · doi:10.1177/0956797617739369 · Cited by 52
“Similarly, in their book Superforecasting, Tetlock and Gardner (2015) write, “A confident yes or no is satisfying in a way that maybe never is” (p.”
BackgroundInfluentialForecasting tournaments, epistemic humility and attitude depolarization
Mellers, B., Tetlock, P., & Arkes, H. R. (2019) · Cognition · doi:10.1016/j.cognition.2018.10.021 · Cited by 46
How to deal with risks of AI suffering
Dung, L. (2025) · Inquiry · doi:10.1080/0020174x.2023.2238287 · Cited by 21
Quantifying machine influence over human forecasters
Abeliuk, A., Benjamin, D. M., Morstatter, F., & Galstyan, A. (2020) · Scientific Reports · doi:10.1038/s41598-020-72690-4 · Cited by 18
Understanding Bayesianism: Fundamentals for Process Tracers
Bennett, A., Charman, A. E., & Fairfield, T. (2022) · Political Analysis · doi:10.1017/pan.2021.23 · Cited by 12
Should I Be Worried? Worry in Dunhuang Divination
Zhao, L. (2024) · International Journal of Divination and Prognostication · doi:10.1163/25899201-bja10016
Metadata
- Venue
- Choice Reviews Online
- Publisher
- American Library Association
- Published
- 2016-04-19
- Volume
- 53
- Issue
- 09
- Pages
- 53-4021-53-4021
- ISSN
- 0009-4978 (ISSN-L)
- 1523-8253
- OpenAlex
- W2160554939
- Cited by
- 919
- References
- 0
- Type
- book review
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.