Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 236

The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.

Identifying and Cultivating Superforecasters as a Method of Improving Probabilistic Predictions

Mellers, Barbara, Stone, Eric, Murray, Terry, Minster, Angela, Rohrbaugh, Nick, Bishop, Michael, Chen, Eva, Baker, Joshua, Hou, Yuan, Horowitz, Michael, Ungar, Lyle, Tetlock, Philip · 2015 · Perspectives on Psychological Science

Abstract

Across a wide range of tasks, research has shown that people make poor probabilistic predictions of future events. Recently, the U.S. Intelligence Community sponsored a series of forecasting tournaments designed to explore the best strategies for generating accurate subjective probability estimates of geopolitical events. In this article, we describe the winning strategy: culling off top performers each year and assigning them into elite teams of superforecasters. Defying expectations of regression toward the mean 2 years in a row, superforecasters maintained high accuracy across hundreds of questions and a wide array of topics. We find support for four mutually reinforcing explanations of superforecaster performance: (a) cognitive abilities and styles, (b) task-specific skills, (c) motivation and commitment, and (d) enriched environments. These findings suggest that superforecasters are partly discovered and partly created—and that the high-performance incentives of tournaments highlight aspects of human judgment that would not come to light in laboratory paradigms focused on typical performance.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1177/1745691615577794

Cites

17 of 17 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 17 references

Research works printed but not linked: 40

  1. Atanasov P., Rescober P., Stone E., Servan-Schreiber E., Tetlock P., Ungar L., Mellers B. (2014). Distilling the wisdom of crowds: Prediction markets versus prediction polls. Manuscript under review. · DOI
  2. 10.1177/0734282909343763 · DOI
  3. Baron J., 2000, Thinking and deciding
  4. 10.1006/obhd.1997.2690 · DOI · Paper page
  5. 10.1016/S0272-7757(98)00044-2 · DOI · Paper page
  6. 10.1037/0022-3514.45.4.792 · doi:10.1037/0022-3514.45.4.792 · Paper page
  7. 10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2 · DOI · Paper page
  8. 10.1207/s15327752jpa4803_13 · DOI · Paper page
  9. 10.1111/j.1745-3984.1978.tb00065.x · DOI · Paper page
  10. 10.1016/0749-5978(91)90010-Q · DOI · Paper page
  11. 10.1016/0883-9026(88)90020-1 · DOI · Paper page
  12. Desvouges W. H., 1993, Contingent valuation, A critical assessment, 91
  13. 10.1016/B978-0-444-53707-2.00003-7 · DOI · Paper page
  14. 10.1111/0162-895X.00058 · DOI · Paper page
  15. 10.1017/CBO9780511808098 · DOI · Paper page
  16. Haran U., 2013, Journal of Decision Making, 8, 188
  17. 10.1086/296525 · doi:10.1086/296525
  18. Heisenberg W., 1999, Physics and philosophy: The revolution in modern science
  19. 10.1037/1076-8971.2.3-4.447 · doi:10.1037/1076-8971.2.3-4.447 · Paper page
  20. 10.1080/00222895.1977.10735113 · DOI
  21. Jervis R., 2010, Why intelligence fails
  22. Kahneman D., 2011, Thinking, fast and slow
  23. 10.1016/0095-0696(92)90019-S · DOI · Paper page
  24. Kahneman D., 1985, Judgment under uncertainty: Heuristics and biases, 153
  25. 10.1037/a0017905 · DOI · Paper page
  26. 10.1146/annurev.so.14.080188.001535 · DOI · Paper page
  27. 10.1177/0272989X0102100105 · DOI · Paper page
  28. Mellers B. A., Stone E., Atanasov P., Rohrbaugh N., Metz E. S., Ungar L. Tetlock P. (2014). The psychology of intelligence analysis: Drivers of prediction accuracy in world politics. Manuscript under review. · DOI · Paper page
  29. 10.1175/1520-0493(1987)115<1330:AGFFFV>2.0.CO;2 · DOI · Paper page
  30. 10.1016/0749-5978(88)90005-2 · DOI
  31. Rosenthal R., 1966, Experimenter effects in behavioral research
  32. 10.1016/j.ijforecast.2013.09.009 · DOI
  33. 10.1214/14-AOAS739 · DOI
  34. 10.1006/jeem.1994.1006 · DOI · Paper page
  35. Shipley W. C., 2009, Shipley–2 manual
  36. 10.1016/0001-6918(81)90005-6 · DOI · Paper page
  37. 10.1111/1529-1006.001 · DOI · Paper page
  38. 10.1111/1467-9280.00418 · DOI
  39. 10.1037/0096-3445.124.4.424 · DOI · Paper page
  40. 10.1016/0010-0277(72)90014-5 · DOI · Paper page

Cited by

See all 236

Metadata

Authors
Mellers, B., Stone, E., Murray, T., Minster, A., Rohrbaugh, N., Bishop, M., Chen, E., Baker, J., Hou, Y., Horowitz, M., Ungar, L., & Tetlock, P.
Venue
Perspectives on Psychological Science
Published
2015-05
Volume
10
Issue
3
Pages
267-281
ISSN
  • 1745-6916 (ISSN-L)
  • 1745-6924
OpenAlex
W2165949980
Cited by
236
References
17
FWCI
26.19
Type
journal article
Language
en
Open access
closed
Licence
journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Identifying and Cultivating Superforecasters as a Method of Improving Probabilistic Predictions. https://app.cochat.ai/papers/10.1177/1745691615577794
The paper
Mellers, B., Stone, E., Murray, T., Minster, A., Rohrbaugh, N., Bishop, M., Chen, E., Baker, J., Hou, Y., Horowitz, M., Ungar, L., & Tetlock, P. (2015). Identifying and Cultivating Superforecasters as a Method of Improving Probabilistic Predictions. Perspectives on Psychological Science. https://doi.org/10.1177/1745691615577794

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.