The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
“Accuracy of Forecasts in Strategic Intelligence.”
Open accessAbstract
Significance: Forecasting is a vital part of strategic intelligence, offering policy makers indications about probable future conditions and aiding sound decision making. Nevertheless, there has not been a concerted effort to study the accuracy of intelligence forecasts over a large set of assessments. This study applied quantitative measures of forecasting accuracy commonly used in other areas of expert forecasting to over 1,500 strategic intelligence forecasts spanning approximately 6 y of intelligence production by a strategic assessment unit. The findings revealed a high level of discrimination and calibration skill. We also show how calibration may be improved through postforecast transformations. The study illustrates the utility of proactively applying scoring rules to intelligence forecasts as a means of outcome-based quality control.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Thus, if we take a large body of cases where analysts estimated that their conclusions were 70 percent likely to be true, we can see whether those conclusions were actually true roughly 70 percent of the time (Rieber 2004; Tetlock 2005, 13; Mandel and Barnes 2014, 10985).”
From The Value of Precision in Probability Assessment: Evidence from a Large-Scale Geopolitical Forecasting Tournament (2018) · cites this paper as Mandel and Barnes 2014, 10985
The cited source says what the claim attributes to it.
In the citing paper
“Mandel and Barnes (2014) provide evidence that geopolitical forecasts in Canadian intelligence reports are surprisingly well calibrated.”
From The Value of Precision in Probability Assessment: Evidence from a Large-Scale Geopolitical Forecasting Tournament (2018) · cites this paper as Mandel and Barnes (2014)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A new look at the statistical model identification
Akaike, H. (1974) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1974.1100705 · Cited by 51,879
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
Chaiken, S. (1980) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.39.5.752 · Cited by 5,516
ANew Look at the Statistical Model Identification
AKAlKE, H. (2011) · Cited by 4,177
Heuristics and Biases: The Psychology of Intuitive Judgment
Bottom, W. P., Gilovich, T. D., Griffin, D., & Kahneman, D. J. (2004) · Academy of Management Review · doi:10.2307/20159081 · Cited by 3,583
Clinical versus statistical prediction: A theoretical analysis and a review of the evidence.
Meehl, P. E. (1954) · University of Minnesota Press eBooks · doi:10.1037/11281-000 · Cited by 2,227
The robust beauty of improper linear models in decision making.
Dawes, R. M. (1979) · American Psychologist · doi:10.1037/0003-066x.34.7.571 · Cited by 2,046
Evaluating the predictive performance of habitat models developed using logistic regression
Pearce, J., & Ferrier, S. (2000) · Ecological Modelling · doi:10.1016/s0304-3800(00)00322-7 · Cited by 2,004
Calibration of probabilities: the state of the art to 1980
Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1982) · Judgment under Uncertainty · doi:10.1017/cbo9780511809477.023 · Cited by 1,967
Knowing with certainty: The appropriateness of extreme confidence.
Fischhoff, B., Slovic, P., & Lichtenstein, S. (1977) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/0096-1523.3.4.552 · Cited by 1,428
Flagged first, then by citations
Research works printed but not linked: 6
- T Fingar Intelligence Analysis: Behavioral and Social Scientific Foundations, eds B Fischhoff, C Chauvin (Natl Acad Press, Washington, DC), pp. 3–27 (2011).
- , eds B Fischhoff, C Chauvin (Natl Acad Press, Washington, DC Intelligence Analysis: Behavioral and Social Scientific Foundations, , 338 pp. (2011).
- ; National Research Council Field Evaluation in the Intelligence and Counterintelligence Context: Workshop Summary (Natl Acad Press, Washington, DC, , 114 pp. (2010).
- ; National Research Council Intelligence Analysis for Tomorrow: Advances from the Behavioral and Social Sciences (Natl Acad Press, Washington, DC, , 102 pp. (2011).
- 10.1037/0022-3514.39.5.752 · DOI · Paper page
- HR Arkes, J Kajdasz Intelligence Analysis: Behavioral and Social Scientific Foundations, eds B Fischhoff, C Chauvin (Natl Acad Press, Washington, DC), pp. 143–168 (2011).
Cited by
Sorted by citations
Items 101–101 of 101
Current intelligence in the President’s Daily Brief: accuracy and uniqueness
Dolan, T. M., & Coulthart, S. (2026) · Intelligence & National Security · doi:10.1080/02684527.2026.2734820
Metadata
- Authors
- Mandel, D. R., & Barnes, A.
- Affiliations
- Defence Research and Development Canada (CA)
- Privy Council Office (CA)
- ORCID
- Mandel, David R.
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2014-07-14
- Volume
- 111
- Issue
- 30
- Pages
- 10984-10989
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC4121776
- OpenAlex
- W2085900183
- Cited by
- 101
- References
- 44
- Citations by year
- 2026: 6 · 2025: 3 · 2024: 12 · 2023: 12 · 2022: 5 · 2021: 9 · 2020: 7 · 2019: 11 · 2018: 7 · 2017: 13 · 2016: 8 · 2015: 5 · 2014: 3
- FWCI
- 62.95
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Natural Sciences and Engineering Research Council of Canada (249537-07)
- Topics
- Social Sciences › Political Science and International Relations › Intelligence, Security, War Strategy
- Social Sciences › Decision Sciences › Management Science and Operations Research › Forecasting Techniques and Applications
- Social Sciences › Business, Management and Accounting › Strategy and Management › Competitive and Knowledge Intelligence
- Keywords
- strategic intelligence, forecast accuracy, forecast calibration, underconfidence, analyst expertise, forecast informativeness, intelligence reports, policy decision-making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
