Psychological Strategies for Winning a Geopolitical Forecasting Tournament
Open accessThe DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index; none of the 9 sources its claims cite is checked for support yet.
Abstract
Five university-based research groups competed to recruit forecasters, elicit their predictions, and aggregate those predictions to assign the most accurate probabilities to events in a 2-year geopolitical forecasting tournament. Our group tested and found support for three psychological drivers of accuracy: training, teaming, and tracking. Probability training corrected cognitive biases, encouraged forecasters to use reference classes, and provided forecasters with heuristics, such as averaging when multiple estimates were available. Teaming allowed forecasters to share information and discuss the rationales behind their beliefs. Tracking placed the highest performers (top 2% from Year 1) in elite teams that worked together. Results showed that probability training, team collaboration, and tracking improved both calibration and resolution. Forecasting is often viewed as a statistical problem, but forecasts can be improved with behavioral interventions. Training, teaming, and tracking are psychological interventions that dramatically increased the accuracy of forecasts. Statistical algorithms (reported elsewhere) improved the accuracy of the aggregation. Putting both statistics and psychology to work produced the best forecasts 2 years in a row.
Overview
CoChat Summary
## TL;DR
This paper reports a 2-year geopolitical forecasting tournament and tests three psychological interventions—training, teaming, and tracking—to improve probabilistic forecasts. All three improved accuracy, and the top 2% of Year 1 forecasters placed into elite teams (“superforecasters”) performed best in Year 2.
## Key findings
- Training improved accuracy: In Year 1, training improved Brier scores, with probability training outperforming scenario training, and scenario training outperforming no training: F(2, 1586) = 14.29, p < .001; t(1053) = 2.23, p = .026; t(1056) = 3.25, p < .001. In Year 2, probability training again beat no training: F(1, 882) = 19.12, p < .001.
- Teaming improved accuracy: Group influence was significant in both years, F(2, 1586) = 60.68, p < .001 and F(1, 882) = 62.76, p < .001. In Year 1, teams were more accurate than crowd-belief forecasters, who were more accurate than independent forecasters: t(1036) = 5.52, p < .001 and t(1120) = 5.92, p < .001.
- Tracking the top performers worked best: The best performers from Year 1, the top 2%, were placed together in elite Year 2 teams and “outperformed all other groups by a wide margin.” The paper reports no evidence of Year 2 regression to the mean.
- Both calibration and resolution improved: Calibration improved with training in Years 1 and 2, F(2, 1586) = 3.16, p = .04 and F(1, 938) = 3.78, p < .05; resolution varied with group influence and training in both years, and tracking enhanced resolution, t(293) = 4.12, p < .001.
## Methods
- The study embedded experiments in a 2-year geopolitical forecasting tournament (Year 1: September 2011–April 2012; Year 2: June 2012–April 2013) on goodjudgmentproject.com, where participants made probability estimates for 199 geopolitical outcomes.
- Year 1 used a 3 (training: no/scenario/probability) × 4 (group influence: independent/crowd-belief/team/prediction-market) factorial design; Year 2 used a 2 (training: no/probability) × 3 (group influence: independent/team/prediction-market) design plus an elite tracking condition. Results for prediction markets are noted as not discussed in the paper.
- The main outcome was Brier score; analyses also examined calibration and resolution, using statistical tests reported as F-tests, t-tests, and regressions. The paper states that findings were robust across Brier, spherical, and logarithmic scoring rules.
## Limitations
- The paper notes that results for prediction-market forecasting are not discussed here. This limits what can be concluded from the full design as presented in this article.
- The paper itself says “No one knows how much further improvement is possible” and that there are likely “pockets of inherent uncertainty” making perfect or near-perfect Brier scores impossible. This is a stated substantive limitation of forecasting accuracy.
- Evident from the text: the Year 2 no-training condition included some participants who had previously received scenario training, which the authors say “tipped the scale against finding a statistically significant effect of training.”
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 0 Not valid · 5 Not checked · 39 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and in the middle period of Year 2, F (1, 882) = 6.10, p = .014.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For both years, training and teaming remained significant predictors even when timing and updating were included as predictors.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In addition to showing a main effect of group influence, Year 2 data showed an interaction between training and group influence, F (1, 882) = 5.78, p = .018.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In Year 2, calibration also benefited from group influence, F (1, 938) = 7.07, p = .008, and tracking.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There was no evidence of Year 2 regression to the mean;”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
19 more claims
Create a free account to see all 44 claims and every source verdict.
- It may state two separate claims as one. 16
- Probably not a substantial claim. 6
- The quote only partly states this claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Additionally, as reported elsewhere, training improved the calibration and resolution of forecasters by reducing overconfidence (Mellers et al., 2014; Moore et al., 2016).”
From Developing expert political judgment: The impact of training and practice on judgmental accuracy in geopolitical forecasting tournaments (2016) · cites this paper as Mellers et al., 2014
Experimental — accuracy not yet measured
Statistics that don’t add up
Each reported p-value recomputed from its test statistic and degrees of freedom.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“F (1, 882) = 5.78, p = .018”
Reported p = .018 · recomputed p = .016
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“F (2, 1463) = 6.20, p < .001”
Reported p < .001 · recomputed p = .002
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“t (1056) = 3.25, p < .001”
Reported p < .001 · recomputed p = .001
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“t (1588) = 2.70, p < .001”
Reported p < .001 · recomputed p = .007
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“F (1, 938) = 3.78, p < .05”
Reported p < .05 · recomputed p = .052
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Doesn’t add upt(1056) = 3.25reported p < .001 · recomputed p = .001
“t (1056) = 3.25, p < .001”
- Doesn’t add upF(1, 882) = 5.78reported p = .018 · recomputed p = .016
“F (1, 882) = 5.78, p = .018”
- Doesn’t add upF(2, 1463) = 6.2reported p < .001 · recomputed p = .002
“F (2, 1463) = 6.20, p < .001”
- Doesn’t add upt(1588) = 2.7reported p < .001 · recomputed p = .007
“t (1588) = 2.70, p < .001”
- Doesn’t add upF(1, 938) = 3.78reported p < .05 · recomputed p = .052
“F (1, 938) = 3.78, p < .05”
37 more recomputed tests
Create a free account to see every test the Stats check recomputed.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Self-check results appear here once an independent check has confirmed them.
Citations
Cites
Journal of The American Statistical Association
Fienberg, S. E. (2005) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1317 · Cited by 6,125
Groups: Interaction and Performance.
Polley, R. B., & McGrath, J. E. (1984) · Administrative Science Quarterly · doi:10.2307/2393041 · Cited by 3,290
Norm theory: Comparing reality to its alternatives.
Kahneman, D., & Miller, D. T. (1986) · Psychological Review · doi:10.1037/0033-295x.93.2.136 · Cited by 3,024
The difference: how the power of diversity creates better groups, firms, schools, and societies
(2007) · Choice Reviews Online · doi:10.5860/choice.45-1534 · Cited by 2,280
Many Hands Make Light the Work: The Causes and Consequences of Social Loafing
Latané, B., Williams, K., & Harkins, S. (1979) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.37.6.822 · Cited by 2,216
Stasser, G., & Titus, W. (1985) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.48.6.1467 · Cited by 1,850 · cited 1×
“Teams have a demonstrated failure to search for unique information, unshared information, or hypothesis-disconfirming information (Kerr, MacCoun, & Kramer, 1996; Stasser & Titus, 1985; Sunstein, 2006).”
BackgroundA Theory of Fads, Fashion, Custom, and Cultural Change as Informational Cascades
Bikhchandani, S., Hirshleifer, D., & Welch, I. (1992) · Journal of Political Economy · doi:10.1086/261849 · Cited by 1,805
Uncertain Judgements: Eliciting Experts' Probabilities
O'Hagan, A., Buck, C. E., Daneshkhah, A., Eiser, J. R., Garthwaite, P. H., Jenkinson, D. J., Oakley, J. E., & Rakow, T. (2006) · doi:10.1002/0470033312 · Cited by 1,550
Koriat, A., Lichtenstein, S., & Fischhoff, B. (1980) · Journal of Experimental Psychology: Human Learning and Memory · doi:10.1037/0278-7393.6.2.107 · Cited by 1,477
Group Performance and Decision Making
Kerr, N. L., & Tindale, R. S. (2004) · Annual Review of Psychology · doi:10.1146/annurev.psych.55.090902.142009 · Cited by 1,331
Blackwell Handbook of Judgment and Decision Making
(2004) · Wiley eBooks · doi:10.1002/9780470752937 · Cited by 961
Progress in Small Group Research
Levine, J. M., & Moreland, R. L. (1990) · Annual Review of Psychology · doi:10.1146/annurev.ps.41.020190.003101 · Cited by 925
Infotopia How Many Minds Produce Knowledge
Sunstein, C. R. (2006) · doi:10.1093/oso/9780195189285.001.0001 · Cited by 913
An Extension of the Tahiti–Darwin Southern Oscillation Index
Ropelewski, C. F., & Jones, P. D. (1987) · Monthly Weather Review · doi:10.1175/1520-0493(1987)115<2161:aeotts>2.0.co;2 · Cited by 891
Page, S. (2008) · Princeton University Press eBooks · doi:10.1515/9781400830282 · Cited by 881
Benhabib, J., Bisin, A., & Jackson, M. O. (2011) · Cited by 824
Costs and benefits of judgment errors: Implications for debiasing.
Arkes, H. R. (1991) · Psychological Bulletin · doi:10.1037/0033-2909.110.3.486 · Cited by 764
Fault trees: sensitivity of estimated failure probabilities to problem representation
Fischhoff, B., Slovic, P., & Lichtenstein, S. (1978) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037/0096-1523.4.2.330 · Cited by 708 · cited 1×
“For each bin, the relative frequency of “correct” predictions is plotted; a prediction was considered to be correct if the outcome judged most likely actually occurred (Fischhoff et al., 1978).”
BackgroundThe effects of statistical training on thinking about everyday problems
Fong, G. T., Krantz, D. H., & Nisbett, R. E. (1986) · Cognitive Psychology · doi:10.1016/0010-0285(86)90001-0 · Cited by 597
Bias in judgment: Comparing individuals and groups.
Kerr, N. L., MacCoun, R. J., & Kramer, G. P. (1996) · Psychological Review · doi:10.1037//0033-295x.103.4.687 · Cited by 491
Alive and Well after 25 Years: A Review of Groupthink Research
Esser, J. K. (1998) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1998.2758 · Cited by 417
Strategies for revising judgment: How (and how well) people use others’ opinions.
Soll, J. B., & Larrick, R. P. (2009) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/a0015145 · Cited by 416
Lichtenstein, S., & Fischhoff, B. (1980) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(80)90052-5 · Cited by 375 · cited 1×
“Remarkably, a brief probabilistic training module paid off over an extended time (see also Lichtenstein & Fischhoff, 1980).”
BackgroundLarrick, R. P. (2004) · Blackwell Handbook of Judgment and Decision Making · doi:10.1002/9780470752937.ch16 · Cited by 328
The Wisdom of Many in One Mind: Improving Individual Judgments With Dialectical Bootstrapping
Herzog, S. M., & Hertwig, R. (2009) · Psychological Science · doi:10.1111/j.1467-9280.2009.02271.x · Cited by 304
Facing uncertainty in the game of bridge: A calibration study
Keren, G. (1987) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(87)90047-1 · Cited by 302
Probability Forecasting in Meteorology
Murphy, A. H., & Winkler, R. L. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10478075 · Cited by 300
Epple, D., & Romano, R. E. (2011) · Handbook of Social Economics · doi:10.1016/b978-0-444-53707-2.00003-7 · Cited by 284
Combining multiple probability predictions using a simple logit model
Satopää, V., Baron, J. A., Foster, D. P., Mellers, B. A., Tetlock, P. E., & Ungar, L. (2014) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2013.09.009 · Cited by 192
Self-citationThe psychology of intelligence analysis: Drivers of prediction accuracy in world politics.
Mellers, B., Stone, E., Atanasov, P., Rohrbaugh, N., Metz, S. E., Ungar, L., Bishop, M. M., Horowitz, M., Merkle, E., & Tetlock, P. (2015) · Journal of Experimental Psychology: Applied · doi:10.1037/xap0000040 · Cited by 187
Self-citationSoll, J. B., Milkman, K. L., & Payne, J. W. (2015) · The Wiley Blackwell Handbook of Judgment and Decision Making · doi:10.1002/9781118468333.ch33 · Cited by 162
Hertel, G., Kerr, N. L., & Messé, L. A. (2000) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.79.4.580 · Cited by 161
The effects of averaging subjective probability estimates between and within judges.
Ariely, D., Tung Au, W., Bender, R. H., Budescu, D. V., Dietz, C. B., Gu, H., Wallsten, T. S., & Zauberman, G. (2000) · Journal of Experimental Psychology: Applied · doi:10.1037//1076-898x.6.2.130 · Cited by 144
Two Reasons to Make Aggregated Probability Forecasts More Extreme
Baron, J., Mellers, B. A., Tetlock, P. E., Stone, E., & Ungar, L. H. (2014) · Decision Analysis · doi:10.1287/deca.2014.0293 · Cited by 130
Self-citationRobust analogizing and the outside view: two empirical tests of case‐based decision making
Lovallo, D. P., Clarke, C., & Camerer, C. F. (2011) · Strategic Management Journal · doi:10.1002/smj.962 · Cited by 127
The effects of feedback and training on the performance of probability forecasters
Benson, P. G., & Önkal, D. (1992) · International Journal of Forecasting · doi:10.1016/0169-2070(92)90066-i · Cited by 92
Fault Trees: Sensitivity of Estimated Failure Probabilities to Problem Representation\
Fischhoff, B., Slovic, P., & Lichtenstein, S. (1977) · Defense Technical Information Center (DTIC) · doi:10.21236/ada057163 · Cited by 77
Key difficulties in identifying the effects of ability grouping on student achievement
Betts, J. R., & Shkolnik, J. L. (2000) · Economics of Education Review · doi:10.1016/s0272-7757(99)00022-9 · Cited by 75
Intelligent management of intelligence agencies: Beyond accountability ping-pong.
Tetlock, P. E., & Mellers, B. A. (2011) · American Psychologist · doi:10.1037/a0024285 · Cited by 71
Self-citationCalibration of Subjective Probability Judgments in a Naturalistic Setting
JOHNSON, J. E. V., & Bruce, A. (2001) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.2000.2949 · Cited by 69
Probability aggregation in time-series: Dynamic hierarchical modeling of sparse expert beliefs
Satopää, V., Jensen, S. T., Mellers, B. A., Tetlock, P. E., & Ungar, L. (2014) · The Annals of Applied Statistics · doi:10.1214/14-aoas739 · Cited by 30
Self-citationBigelow, K. (2012) · Cited by 16
VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY
BRIER, G. W. (1950) · Monthly Weather Review · doi:10.1175/1520-0493(1950)078<0001:vofeit>2.0.co;2
Verification of forecasts expressed in terms of probability
BRIER, G. W. (1950) · Monthly Weather Review · doi:10.1175/1520-0493(1950)078%3c0001:vofeit%3e2.0.co;2 · cited 1×
“All forecasters were given the goal of minimizing their average Brier score (Brier, 1950).”
Method
Research works printed but not linked: 4
- Baron J., Decision Analysis
- Fischhoff B., 2011, Intelligence analysis: Behavioral and social scientific foundations
- Janis I., 1982, Group-think
- Satopää V. A., Annals of Applied Statistics
Cited by
Algorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
“Following prior research (Mellers et al., 2014), we used Brier scores (Brier, 1950) to assess forecasting accuracy.”
Self-citationThe Bias of Crowds: How Implicit Bias Bridges Personal and Systemic Prejudice
Payne, B. K., Vuletich, H. A., & Lundberg, K. B. (2017) · Psychological Inquiry · doi:10.1080/1047840x.2017.1335568 · Cited by 535
“Research on “the wisdom of crowds” (Surowiecki, 2004) finds that, for many kinds of questions, the collective judgment of a group tends to be closer to the true answer than any one individual’s answer (Clemen, 1989; Galton, 1907; Lorge, Fox, Davitz, & Brenner, 1958; Mellers et al., 2014; Page, 2007).”
BackgroundA practical guide to structured expert elicitation using the IDEA protocol
Hemming, V., Burgman, M. A., Hanea, A. M., McBride, M. F., & Wintle, B. C. (2018) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12857 · Cited by 528
“Importantly, it should account for each step of the elicitation including the recruitment of experts, the framing of questions, the elicitation and aggregation of their judgements, using procedures that have been tested and clearly demonstrated to improve judgements (e.g. Cooke, 1991; Mellers et al., 2014).”
BackgroundMethodInfluentialNetwork dynamics of social influence in the wisdom of crowds
Becker, J., Brackbill, D., & Centola, D. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1615978114 · Cited by 397
Debiasing Decisions: Improved Decision Making With a Single Training Intervention
Morewedge, C. K., Yoon, H., Scopelliti, I., Symborski, C. W., Korris, J. H., & Kassam, K. S. (2015) · Policy Insights from the Behavioral and Brain Sciences · doi:10.1177/2372732215600886 · Cited by 384
“Researchers have produced numerous successful collaborations with government and industry partners that have reduced waste and improved the health and finances of the public (e.g., Chapman et al., 2010; Mellers et al., 2014; Schultz et al., 2007; Schwartz et al., 2014; Thaler & Benartzi, 2004).”
Background
Metadata
- Authors
- Mellers, B., Ungar, L., Baron, J., Ramos, J., Gurcay, B., Fincher, K., Scott, S. E., Moore, D., Atanasov, P., Swift, S. A., Murray, T., Stone, E., & Tetlock, P. E.
- Affiliations
- University of Pennsylvania (US)
- Rice University (US)
- Department of Psychology, University of Pennsylvania
- University of California, Berkeley (US)
- ORCID
- Venue
- Psychological Science
- Publisher
- SAGE Publications
- Published
- 2014-03-21
- Volume
- 25
- Issue
- 5
- Pages
- 1106-1115
- ISSN
- 0956-7976 (ISSN-L)
- 1467-9280
- OpenAlex
- W2131472525
- Cited by
- 312
- References
- 42
- Citations by year
- 2026: 20 · 2025: 22 · 2024: 28 · 2023: 29 · 2022: 47 · 2021: 31 · 2020: 14 · 2019: 19 · 2018: 25 · 2017: 28 · 2016: 19 · 2015: 19 · 2014: 11
- FWCI
- 32.31
- Type
- journal article
- Language
- en
- Open access
- green
- www.escholarship.org
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Psychological Science (publishedVersion)
- PubMed (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- Funders
- U.S. Department of the Interior
- IBM Center for the Business of Government
- Intelligence Advanced Research Projects Activity
- Interior Business Center
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Forecasting Techniques and Applications
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Sports Analytics and Performance
- Keywords
- geopolitical forecasting, forecasting tournaments, forecast accuracy, teaming, tracking, calibration, resolution, cognitive biases, reference classes, behavioral interventions, forecast aggregation, heuristics
- MeSH terms
- Adult, Algorithms, Female, Forecasting, Humans, Interpersonal Relations, Judgment, Male, Probability, Psychological Techniques, Social Behavior, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.