CoChat
Sign Up
PreprintCited by 73Advanced Statistical Methods and ModelsStatistical Methods and ApplicationsDecision-Making and Behavioral Economics

Illusions in Regression Analysis

Open access
Armstrong, J. Scott·2011·SSRN Electronic Journal
DOI 10.2139/ssrn.1969740
Open this paper in CoChat — free account

Abstract

Soyer and Hogarth’s article, 'The Illusion of Predictability,' shows that diagnostic statistics that are commonly provided with regression analysis lead to confusion, reduced accuracy, and overconfidence. Even highly competent researchers are subject to these problems. This overview examines the Soyer-Hogarth findings in light of prior research on illusions associated with regression analysis. It also summarizes solutions that have been proposed over the past century. These solutions would enhance the value of regression analysis.

AI snapshot

Written from the paper’s full text.

TL;DR

Armstrong reviews Soyer and Hogarth’s evidence that standard regression diagnostics mislead even expert econometricians, and traces several long-recognized illusions in regression analysis. He argues for a priori analysis, simplicity, ex ante forecast testing, combining forecasts, and avoiding statistical significance and fit statistics.

Key findings

  • Soyer and Hogarth show that commonly provided regression statistics—t, p, F, R-squared and the like—can confuse highly competent researchers, reduce accuracy, and increase overconfidence.
  • Armstrong identifies five illusions: complexity, regression sufficiency, best linear unbiased estimators, control, and “fit implies accuracy,” plus statistical significance and correlation illusions in decision-making. Proposed solutions include a priori analysis, Occam’s Razor, not estimating more than three variables, combining forecasts including the naïve model, damping toward naïve forecasts as horizon increases, and avoiding t, p, F, and R-squared.
  • Cited evidence includes combining forecasts reducing errors from about 10% to 58% versus average uncombined forecasts; Cumming’s experiment where 40% of 55 participants using significance levels reached correct conclusions versus 95% of 57 using confidence intervals; Armstrong’s random-data regression with 31 observations, 30 potential variables, 8 final variables, and adjusted R-square 0.85; and Armstrong and Patnaik’s 24 causal statements where directional effects differed for 8.
  • Armstrong recommends software default to holdout forecast accuracy, use MdRAE rather than RMSE, and support ex ante testing via jackknifing for cross-sectional forecasts and withheld data with successive updating for time series.

Methods

  • Conceptual and historical overview/commentary on Soyer and Hogarth’s “The Illusion of Predictability,” synthesizing prior research on regression illusions and proposed solutions over the past century.
  • Draws on cited experiments and examples, including S&H’s expert experiment, Cumming’s confidence-interval versus significance-level experiment, Armstrong’s random-data regression, Armstrong and Patnaik’s comparison of experimental and non-experimental causal statements, and Graefe et al. on combining forecasts.
  • No new primary data collection or formal meta-analysis is reported; the paper is a narrative review with recommendations.

Limitations

  • The paper states that further experimentation is needed to assess alternative techniques on large samples of forecasts on holdout samples, and that little such experimentation has been done over the past century.
  • It notes that a priori analyses can be expensive: the author spent many hours over a 16-year period summarizing knowledge, yielding 195 principles for advertising.
  • Evident from the text: as an overview/commentary, it relies on selected prior studies and examples rather than a systematic review or new empirical test, and it reports no formal search or selection criteria.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2139/ssrn.1969740

Cites

38 of 38 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

8 of 50 cited sources checked · 3 supported · 1 flagged · accuracy not yet measured

  • Premisecitation
    § Using regression for decision-making

    In one study comparing findings from experiments with analyses of non-experimental data for 24 causal statements, the directional effects differed for 8 of the 24 comparisons.

    “In one study, we had an opportunity to compare findings from experiments with those from analyses of nonexperimental data for 24 causal statements. The directional effects differed for 8 of the 24 comparisons (Armstrong and Patnaik 2009).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Armstrong and Patnaik 2009 source
  • Premisecitation
    § Forecast accuracy and confidence

    Fit bears little relationship to ex ante forecast accuracy, especially for time series.

    “This ignores the research showing that fit bears little relationship to ex ante forecast accuracy, especially for time series.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Zellner (2001) source

      The Abstract (page unavailable) reports that complicated macroeconometric models often forecast no better than simple time-series models. Section IV (page unavailable) also evaluates out-of-sample forecasts. Neither passage establishes that model fit bears little relationship to ex ante forecast accuracy, so the claim overstates what this source shows. (LLM, conf=0.77)

      Experimental — accuracy not yet measured
  • Premisecitation
    § Forecast accuracy and confidence

    Zellner reached the same conclusion that increased complexity of regression models typically reduced forecast accuracy.

    “Zellner (2001) reached the same conclusion in his review of the research.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Zellner (2001) source

      The Abstract reports that complicated macroeconometric models often forecast unsatisfactorily and, in some cases, no better or worse than simple time-series models. That supports Zellner’s preference for simplicity, but not the broader claim that increasing the complexity of regression models typically reduces forecast accuracy. The Summary and Conclusions also says well-chosen added complexity may improve prediction. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
  • Findingcitation
    § Discussion

    None of the evidence-based principles were found in the advertising textbooks and handbooks that the author analyzed.

    “None of these evidence-based principles were found in the advertising textbooks and handbooks that I analyzed.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Armstrong 2011 source

      In “Principles v. current practices” (page unavailable), the author says no evidence-based principles were found in the nine advertising texts examined. Our copy describes these as textbooks; it does not establish that advertising handbooks were also analyzed. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
  • Findingnone given
    § Discussion

    The vast majority of the 195 principles were sufficiently complex such that neither prior experience nor regression analyses were able to discover them.

    “The vast majority of them were sufficiently complex such that neither prior experience nor regression analyses were able to discover them.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. 21
  • May combine two claims. 20

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § Forecast accuracy and confidence

    The Federal Reserve Bank of Minneapolis scrapped its complicated vector autoregressive models after their poor performance in forecasting turning points.

    “he reported 'the Federal Reserve Bank of Minneapolis decided to scrap its complicated vector autoregressive (V AR, i.e., Very Awful Regression) models after their poor performance in forecasting turning points, etc.'”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Zellner (2001) source

      The Abstract says the Federal Reserve Bank of Minneapolis was revising its VAR models after poor turning-point forecasts, not scrapping them. It separately says Federal Reserve officials decided to scrap the Federal Reserve–MIT–PENN model; the claim conflates those two accounts. (LLM, conf=0.90)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation
    § Forecast accuracy and confidence

    Typically, fit improves as complexity increases, while ex ante forecast accuracy decreases.

    “Typically, fit improves as complexity increases, while ex ante forecast accuracy decreases - a conclusion that Zellner (2001) traced back to Sir Harold Jeffreys in the 1930s.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Zellner (2001) source

      Section IV reports that reducing model complexity improved out-of-sample forecasts, and the Introduction describes Jeffreys’s preference for simpler models. But our copy does not say that fit typically improves as complexity increases or that Jeffreys identified the claimed fit-versus-forecast relationship. Section V also notes that well-chosen added complexity can improve prediction. (LLM, conf=0.78)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § Discussion

    The index model allows analysts to take account of 'the knowledge present in a field' as recommended by Zellner.

    “The index model allows analysts to take account of 'the knowledge present in a field' as recommended by Zellner (2001).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Zellner (2001) source

  • May combine two claims.

    Premisecitation
    § Discussion

    Many of the 195 principles are counter-intuitive and are often violated by advertisers.

    “Many are counter-intuitive and are often violated by advertisers (Armstrong 2011).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Armstrong 2011 source

  • May combine two claims.

    Premisecitation
    § Forecast accuracy and confidence

    Christ found that simultaneous equations provided forecasts that were more accurate than those from simpler regression models when tested on artificial data, but not when tested out of sample using real data.

    “For example, Christ (1960) found that simultaneous equations provided forecasts that were more accurate than those from simpler regression models when tested on artificial data, but not when tested out of sample using real data.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Christ (1960) source

  • May combine two claims.

    Premisecitation
    § Abstract

    Soyer and Hogarth's article shows that diagnostic statistics commonly provided with regression analysis lead to confusion, reduced accuracy, and overconfidence.

    “Soyer and Hogarth's article, 'The Illusion of Predictability,' shows that diagnostic statistics that are commonly provided with regression analysis lead to confusion, reduced accuracy, and overconfidence.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Soyer and Hogarth (2011)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisenone given
    § Forecast accuracy and confidence

    Regression-based prediction is most effective when dealing with a small number of variables, large amounts of reliable and valid data, where changes are expected to be large and predictable, and when using well-established causal relationships.

    “Regression-based prediction is most effective when dealing with a small number of variables, large amounts of reliable and valid data, where changes are expected to be large and predictable, and when using well-established causal relationships - such as the elasticities for income, price, and advertising when forecasting demand.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation
    § Forecast accuracy and confidence

    Illusions reduce forecast accuracy and lead to overconfidence in regression analysis.

    “However, there are illusions that reduce the forecast accuracy and lead to overconfidence in regression analysis.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § Discussion

    Software developers typically fail to incorporate evidence-based findings about which methods work best, and statisticians who work on new forecasting procedures seldom cite the literature that tests the relative effectiveness of various methods for forecasting.

    “Unfortunately, software developers typically fail to incorporate evidence-based findings about which methods work best, and statisticians who work on new forecasting procedures seldom cite the literature that tests the relative effectiveness of various methods for forecasting (Fildes and Makridakis 1995).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Fildes and Makridakis 1995 source

  • May combine two claims.

    Premisecitation
    § Using regression for decision-making

    In an experiment, 40% of 55 participants who used significance levels to guide interpretation reached correct conclusions, while 95% of 57 participants who thought in terms of confidence intervals reached correct conclusions.

    “Cumming (2012, pp. 13-14) describes an experiment showing that when researchers in psychology, behavioral neuroscience, and medicine were presented with a set of results, 40% of 55 participants who used significance levels to guide their interpretation reached correct conclusions. In stark contrast, 95% of the 57 participants who thought in terms of confidence intervals reached correct conclusions”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Cumming (2012)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § Historical view of illusions in regression analysis

    Einhorn concluded that the modern researcher cannot rely on the computer to turn data into meaningful and valuable scientific information.

    “Einhorn (1972) concluded, 'Just as the alchemists were not successful in turning base metal into gold, the modern researcher cannot rely on the 'computer' to turn his data into meaningful and valuable scientific information.'”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Einhorn (1972) source

  • May combine two claims.

    Premisecitation
    § Historical view of illusions in regression analysis

    Meta-analyses produce more accurate and less biased summaries than traditional reviews.

    “Meta-analyses produce more accurate and less biased summaries than those provided by traditional reviews as shown by Cumming (2012).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Cumming (2012)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § Historical view of illusions in regression analysis

    Ord provides a simple demonstration of how standard regression procedures, applied without a priori analyses, can lead one astray.

    “Ord (2012) provides a simple demonstration of how standard regression procedures, applied without a priori analyses, can lead one astray.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Ord (2012) source

  • May combine two claims.

    Premisecitation
    § Historical view of illusions in regression analysis

    A procedure combining a priori estimates with regression analyses provided forecasts substantially more accurate than those from extrapolation methods and from stepwise regression on the complete set of causal variables considered.

    “This procedure provided forecasts that were substantially more accurate than those from extrapolation methods and from stepwise regression on the complete set of causal variables that were considered (Armstrong 1968a,b).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Armstrong 1968a,b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning
    § Forecast accuracy and confidence

    Users of regression assume that by putting variables into the equation they are somehow controlling for these variables, but this only occurs for experimental data.

    “Users of regression assume that by putting variables into the equation they are somehow controlling for these variables. This only occurs for experimental data.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § Conclusions

    Various illusions can reduce the accuracy of regression analysis, lead to a false sense of confidence, and harm decision-making.

    “Various illusions can reduce the accuracy of regression analysis, lead to a false sense of confidence, and harm decision-making.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given
    § Conclusions

    Over the past century or so, effective solutions have been developed to deal with the illusions.

    “Over the past century or so, effective solutions have been developed to deal with the illusions.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § Forecast accuracy and confidence

    Large sample sizes cannot resolve the problem of controlling for variables in non-experimental data, so statistics on the number of degrees of freedom are misleading.

    “Large sample sizes cannot resolve this problem, so statistics on the number of degrees of freedom are misleading.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given
    § Conclusions

    The basic problem is that the solutions are often ignored in practice.

    “The basic problem is that the solutions are often ignored in practice.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § Using regression for decision-making

    Two illusions, statistical significance and correlations, can reduce the value of regression analysis.

    “However, two illusions, statistical significance and correlations, can reduce the value of regression analysis.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § Discussion

    RMSE is unreliable and uninformative.

    “Do not provide RMSE (Root Mean Square Errors) as it is unreliable and uninformative.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation
    § Using regression for decision-making

    Meehl concluded that reliance on merely refuting the null hypothesis is basically unsound, poor scientific strategy, and one of the worst things that ever happened in the history of psychology.

    “Meehl (1978) concluded that 'reliance on merely refuting the null hypothesis . . . is basically unsound, poor scientific strategy, and one of the worst things that ever happened in the history of psychology”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Meehl (1978) source

  • May combine two claims.

    Premisereasoning
    § Using regression for decision-making

    Decisions based on regression are less likely to be subject to bias, they are consistent, the basis for the decisions can be fully explained and they are generally useful.

    “As a result, decisions based on regression are less likely to be subject to bias, they are consistent, the basis for the decisions can be fully explained and they are generally useful.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Grove and Meehl 1996 source
    • cites Armstrong 2001 source

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationcitation
    § Discussion

    The three characteristics of good science described by Freedman-clarity of prior reasoning, bringing together many different lines of evidence, and amount of shoe leather-are missing from most regression analyses in journals.

    “These three characteristics of good science, described by Freedman, are missing from most regression analyses that I see in journals.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Freedman (1991) source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § Discussion

    The MdRAE was designed for comparisons among methods and many software packages now provide this statistic.

    “The MdRAE (Median Relative Absolute Error) was designed for such comparisons and many software packages now provide this statistic”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Armstrong and Collopy 1992

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning
    § Discussion

    Regression analyses have met repeated failures in advertising because there may be over 50 principles that were used - or misused - in an ad.

    “In contrast, regression analyses have met repeated failures in this area because there may be over 50 principles that were used - or misused - in an ad.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § Discussion

    S&H recommend visual presentation and Ziliak adds support.

    “S&H recommend visual presentation - and Ziliak (2012) adds support.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Soyer and Hogarth (2011)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Ziliak (2012) source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § Discussion

    John Snow's discovery of the cause of cholera in London in the 1850s came about from the clarity of the prior reasoning, the bringing together of many different lines of evidence, and the amount of shoe leather Snow was willing to use to get the data.

    “For example, John Snow's discovery of the cause of cholera in London in the 1850s came about from 'the clarity of the prior reasoning, the bringing together of many different lines of evidence, and the amount of shoe leather Snow was willing to use to get the data.'”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Freedman (1991) source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § Discussion

    Allen and Fildes note that since 1985 there has been a substantial increase in the attention paid to ex ante forecast comparisons in published papers.

    “Allen and Fildes (2001) note that since 1985 there has been a substantial increase in the attention paid to ex ante forecast comparisons in published papers.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Allen and Fildes (2001) source

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given
    § Conclusions

    It is remarkable that so little experimentation has been done over the past century to determine which regression methods work best under what conditions.

    “It is remarkable that so little experimentation has been done over the past century to determine which regression methods work best under what conditions.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given
    § Conclusions

    Regression analysis can play an important role when analyzing non-experimental data.

    “Regression analysis can play an important role when analyzing non-experimental data.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § Using regression for decision-making

    Cumming describes additional examples of the harm caused by the use of statistical significance.

    “Cumming (2012) describes additional examples of the harm caused by the use of statistical significance.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Cumming (2012)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationown data
    § Discussion

    It can be expensive to do a priori analyses for complex situations.

    “It can be expensive to do a priori analyses for complex situations .”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § Discussion

    The mountains of articles on autocorrelation testing contrast with the near absence of studies on the impact of autocorrelation correction on ex ante forecast performance.

    “They also describe the evidence on the principles. For example, on the matter of evidence on one of the diagnostic statistics they state (p. 311), 'The mountains of articles on autocorrelation testing contrast with the near absence of studies on the impact of autocorrelation correction on [ ex ante ] forecast performance.'”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Allen and Fildes (2001) source
34 of 34

  • The paper relies on this without support.

    Regression analysis is widely used.

    “Given the widespread use of regression analysis, the implications of the article are important for the life and social sciences.”
1 of 1

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Soyer and Hogarth citation year differs from the reference

    Minor

    The text cites Soyer and Hogarth as a 2011 work, but the reference list dates the cited article to 2012.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. H Wold, 1953, Demand Analysis

Cited by

Sorted by citations

Related

Authors
Armstrong, J. S.
Affiliations
University of Pennsylvania (US)
Corresponding author
Armstrong, J. Scott
ORCID
Armstrong, J. Scott
Venue
SSRN Electronic Journal
Publisher
Elsevier BV
Published
2011
ISSN
1556-5068 (ISSN-L)
DOI
10.2139/ssrn.1969740
OpenAlex
W2029278301
Semantic Scholar
33d7843fbc2e537f5bf86429e2174f79ab5628ba
Cited by
73
References
38
Citations by year
2024: 35 · 2023: 1 · 2022: 6 · 2021: 1 · 2020: 2 · 2019: 2 · 2018: 7 · 2017: 6 · 2016: 4 · 2015: 4 · 2014: 1 · 2013: 3 · 2012: 1
Type
posted content
Language
en
Open access
Locations
  • SSRN Electronic Journal (acceptedVersion)
  • Scholarly Commons (University of Pennsylvania) (submittedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • ScholarlyCommons (University of Pennsylvania) (submittedVersion)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
  • Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
Keywords
regression analysis, diagnostic statistics, overconfidence, statistical interpretation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-26.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-26.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review