The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
The Well-Calibrated Bayesian
Abstract
Suppose that a forecaster sequentially assigns probabilities to events. He is well calibrated if, for example, of those events to which he assigns a probability 30 percent, the long-run proportion that actually occurs turns out to be 30 percent. We prove a theorem to the effect that a coherent Bayesian expects to be well calibrated, and consider its destructive implications for the theory of coherence.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
An introduction to probability theory and its applications
(1958) · Journal of the Franklin Institute · doi:10.1016/0016-0032(58)90688-4 · Cited by 29,621
Jeffreys, H., Moran, P. A. P., & Chatfield, C. (1922) · Nature · doi:10.1038/109132a0 · Cited by 7,877
An Introduction to Probability Theory and its Applications.
Kingmán, J. F. C., & Feller, W. (1972) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2344620 · Cited by 5,297
Journal of the Royal Statistical Society.
(1921) · The Economic Journal · doi:10.1093/ej/31.121.133 · Cited by 1,503
Sampling and Bayes' Inference in Scientific Modelling and Robustness
Box, G. E. P. (1980) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2982063 · Cited by 905
Calibration of Probabilities: The State of the Art
Lichtenstein, S., Fischhoff, B., & Phillips, L. (1977) · Springer eBooks · doi:10.1007/978-94-010-1276-8_19 · Cited by 736
Two further applications of a model for binary regression
COX, D. R. (1958) · Biometrika · doi:10.1093/biomet/45.3-4.562 · Cited by 556
Studies in Subjective Probability.
Hill, B. M., Kyburg, H. E., & Smokler, H. E. (1965) · Journal of the American Statistical Association · doi:10.2307/2283162 · Cited by 549
Reliability of Subjective Probability Forecasts of Precipitation and Temperature
Murphy, A. H., & Winkler, R. L. (1977) · Applied Statistics · doi:10.2307/2346866 · Cited by 386
Combining Expert Judgments: A Bayesian Approach
Morris, P. A. (1977) · Management Science · doi:10.1287/mnsc.23.7.679 · Cited by 356
Research works printed but not linked: 6
- Carnap R., 1971, Studies in Subjective Logic and Probability, 35
- De Finetti B., 1975, Theory of Probability
- De Groot M. H., 1979, Journal of the Royal Statistical Society, 142, 172
- De Groot M. H., 1980, Bayesian Statistics, 385
- Pratt J. W., 1962, “Must Subjective Probabilities be Realized as Relative Frequencies?”
- Roberts, H. V. “On the Meaning of the Probability of Rain,”. Paper presented to First National Conference on Statistical Meteorology.
Cited by
Sorted by citations
Items 501–600 of 618
On impossibility of sequential algorithmic forecasting
V'yugin, V. V. (2006) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/dagsemproc.06051.11
Safe learning — how to adjust Bayes and MDL when the model is wrong
Grünwald, P. (2010) · IEEE Information Theory Workshop (ITW) · doi:10.1109/itwksps.2010.5503127
“Indeed, various Bayesian statisticians have proposed that Bayesian inference should be augmented by some hypothesis-testing like “reality checks,” reminiscent of Popperian falsification [ 4 ], [5], [6], [7].”
BackgroundEssays on beliefs and behavior
Lu, J. (2014)
On Calibration Error of Randomized Forecasting Algorithms
V'yugin, V. V. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-75225-7_31
Frequentism as a positivism: a three-tiered interpretation of probability
Lingamneni, S. (2013)
Philosophy of Science Association
Seidenfeld, T. (2010)
Probabilistic Calibration of Predictive Distributions
Casarin, R., Ravazzolo, F., & Gneiting, T. (2014) · ARCA (Università Ca' Foscari Venezia)
A framework for open-minded Bayesianism
Wenmackers, S., & Romeijn, J. (2016)
Realism and the No‐Miracles Argument
Howson, C. (2000) · doi:10.1093/0198250371.003.0004
Learning the ergodic decompositionPreprint
Al‐Najjar, N. I., & Shmaya, E. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.6670
“The idea of calibration originated with Dawid [5].”
MethodHowson, C. (2000) · doi:10.1093/0198250371.003.0012
Howson, C. (2000) · doi:10.1093/0198250371.003.0010
Howson, C. (2000) · doi:10.1093/0198250371.003.0007
Howson, C. (2000) · doi:10.1093/0198250371.003.0008
Howson, C. (2000) · doi:10.1093/0198250371.003.0002
Doing Ethics in Food and Agriculture
Stannard, C. (2008) · The international library of environmental, agricultural and food ethics/The International library of environmental, agricultural and food ethics · doi:10.1007/978-1-4020-8722-6_2
Kleiven, A. (2017) · BIBSYS Brage (BIBSYS (Norway))
Inferring health conditions from fMRI-graph dataPreprint
Mana, P. P., Bachmann, C., & Morrison, A. (2018) · doi:10.31219/osf.io/r2huz
High-Confidence Predictions under Adversarial UncertaintyPreprint
Drucker, A. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1101.4446
“The second strand of related work is the so-called forecasting problem in decision theory (see [4] for an early, influential discussion).”
BackgroundFailure of Calibration is TypicalPreprint
Belot, G. (2013) · PhilSci-Archive (University of Pittsburgh) · doi:10.48550/arxiv.1306.4943
“…the data sequences encode, for each of some sequence of days, whether or not it snows in a given locale on that day (1=snow; 0=no snow); each day, the forecasting system is asked to output a probability for snow on that day, given the pattern of snowy and snowless days in the extant data set.”
BackgroundSelf-Confidence Measures of a Decision Support System Based on Bayesian Networks
Koźniewski, M. (2019) · D-Scholarship@Pitt (University of Pittsburgh)
Regularising Deep Networks with Deep Generative ModelsPreprint
Willetts, M., Camuto, A., Roberts, S., & Holmes, C. C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.11507
“A statistical metric for the quality of the uncertainty of a model is its calibration (Dawid, 1982): is a model as likely to be correct in a particular prediction as it is confident in that prediction?”
BackgroundConflict and Cooperation: AI Research and Development in terms of the Economy of ConventionsPreprint
Solans, D., Tauchmann, C., Farrell, A., Kappler, K. E., Huber, H.-H., Castillo, C., & Kristian Kersting, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12591
“This procedure is known as calibration [17] and is applied to equalize the precision classification ratio between all the models so further analysis are not biased by different precision scores.”
MethodEssays on Learning and Induction
Nielsen, M. (2019) · doi:10.7916/d8-50j7-yb60
Probability Forecasts and Prediction Markets
Mortera, J., & Dawid, A. P. (2022) · Springer series in the data sciences · doi:10.1007/978-3-030-75460-0_6
Self-citationBetter Multi-class Probability Estimates for Small Data SetsPreprint
Alasalmi, T., Suutala, J., Koskimäki, H., & Röning, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.11242
“A classifier is said to be well calibrated if the predicted probability of an event is close to the proportion of these events among a group of similar predictions [1].”
BackgroundEntropified Berk-Nash EquilibriumPreprint
Massari, F., & Newton, J. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3473630
Deep Learning Requires Explicit Regularization for Reliable Predictive ProbabilityPreprint
Joo, T., & Chung, U. (2020) · arXiv (Cornell University)
Safe ProbabilityPreprint
Grünwald, P. (2016) · arXiv (Cornell University)
ON THE TRUTH-CONVERGENCE OF OPEN-MINDED BAYESIANISM
Sterkenburg, T. F., & de Heide, R. (2021) · The Review of Symbolic Logic · doi:10.1017/s1755020321000022
“On the standard conception, there is, again, no room for the agent to later adapt this model (Dawid, 1982); there is, in particular, no room to expand the model, to incorporate new hypotheses that might be more in accord with the data (Gillies, 2001; Gelman and Shalizi, 2013).”
ResultTop-label calibration.Preprint
Gupta, C., & Ramdas, A. (2021) · arXiv (Cornell University)
Hild, M. (2006) · Philosophical Studies · doi:10.1007/s11098-005-4059-6
Online Multiobjective Minimax Optimization and ApplicationsPreprint
Noarov, G., Pai, M. M., & Roth, A. (2021) · arXiv (Cornell University)
El desarrollo industrial y su entorno
de Villalonga, A. E. (1972) · Economía industrial
Rubin, D. B. (1987) · Wiley series in probability and statistics · doi:10.1002/9780470316696.refs
Methodological Consequences: Objective Bayesianism
(2008) · Springer eBooks · doi:10.1007/978-1-4020-8817-9_5
Methodological Consequences: Mechanisms and Levels of Causation
(2008) · Springer eBooks · doi:10.1007/978-1-4020-8817-9_6
Inferring health conditions from fMRI-graph dataPreprint
Mana, P. P., Bachmann, C., & Morrison, A. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/295113
“Predictive power is a model’s capacity to give high probability to propositions that turn out to be true, during and especially after its learning phase (cf. Dawid, 1982b).”
BackgroundSupporting the Rationale of Variation
(2008) · Springer eBooks · doi:10.1007/978-1-4020-8817-9_7
Jolliffe, I. T., & Stephenson, D. B. (2011) · Wiley eBooks · doi:10.1002/9781119960003.refs
Epistemology of Causal Modelling
(2008) · Springer eBooks · doi:10.1007/978-1-4020-8817-9_4
Schervish, M. J. (1988) · Springer eBooks · doi:10.1007/978-1-4613-8768-8_14
Gani, J. (2004) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1613
Dawes, R. M. (2001) · Elsevier eBooks · doi:10.1016/b0-08-043076-7/00431-9
Arnold, B. C., & Sarabia, J. M. (2018) · Statistics for social and behavioral sciences · doi:10.1007/978-3-319-93773-1_11
DeGroot, M. H. (2004) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess2919
Wolfson, L. J. (2014) · Wiley StatsRef: Statistics Reference Online · doi:10.1002/9781118445112.stat00231
Parmigiani, G., & Inoue, L. Y. T. (2009) · Wiley series in probability and statistics · doi:10.1002/9780470746684.refs
Howson, C. (2000) · doi:10.1093/0198250371.003.0005
Wolfson, L. J. (2005) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1032.pub2
Gani, J. (2005) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1613.pub2
Howson, C. (2000) · doi:10.1093/0198250371.003.0006
(1994) · Wiley series in probability and statistics · doi:10.1002/9780470316870.refs
DeGroot, M. H. (2014) · Wiley StatsRef: Statistics Reference Online · doi:10.1002/9781118445112.stat00252
Howson, C. (2000) · doi:10.1093/0198250371.003.0003
Dawid, A. P. (2005) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess0335.pub2
Self-citation“World of Uncertainty” Game for Decision-Makers
Kyzy, J. T. (2012) · IGI Global eBooks · doi:10.4018/978-1-4666-2467-2.ch004
Wolfson, L. J. (2004) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1032
(2022) · De Gruyter eBooks · doi:10.1515/9783110785968-011
Indistinguishable Predictions and Multi-group Fair Learning
Rothblum, G. N. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-30545-0_1
A Bayesian workflow for the analysis and reporting of international large-scale surveys
Kaplan, D. E., & Harra, K. (2023) · OECD education working papers · doi:10.1787/588c4a12-en
Incorporating Uncertainty Quantification for the Performance Improvement of Academic Recommenders
Zhu, J., Novelo, L. L., & Yaseen, A. S. (2023) · Knowledge · doi:10.3390/knowledge3030020
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603213
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603205
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603196
How Do Classifiers Induce Agents to Behave Strategically?
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603201
How Do Classifiers Induce Agents to Behave Strategically?
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603210
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603214
Calibration of Deep Learning Classification Models in fNIRSPreprint
Cao, Z., & Luo, Z. (2024) · doi:10.36227/techrxiv.170861971.16301364/v1
Models: Interpretability, Accuracy, and Calibration
Arthur Charpentier, A. (2024) · Springer Actuarial · doi:10.1007/978-3-031-49783-4_4
Rehder, E., & Cordts, M. (2024) · ATZ/MTZ-Fachbuch · doi:10.1007/978-3-658-38486-9_44
Krzysztofowicz, R. (2024) · Wiley eBooks · doi:10.1002/9781394319060.ref
Variance and skewness in density forecasts: assessing world GDP growth
Ramos, F. M. (2025) · Empirical Economics · doi:10.1007/s00181-025-02720-5
Breaking the T^(2/3) Barrier for Sequential Calibration
Dagan, Y., Daskalakis, C., Fishelson, M., Golowich, N., Kleinberg, R., & Okoroafor, P. (2025) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/3717823.3718178
Collina, N., Goel, S., Gupta, V., & Roth, A. (2025) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/3717823.3718222 · Cited by 9
An Empirical Review of Uncertainty Estimation for Quality Control in CAD Model Segmentation
Vidanes, G., Toal, D. J. J., Keane, A. J., Zhang, D. X., Nuñez, M., & Gregory, J. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-96196-0_4
Randomness and imprecision: From supermartingales to randomness tests
de Cooman, G., Persiau, F., & Bock, J. D. (2025) · Information and Computation · doi:10.1016/j.ic.2025.105328
“Definition 3 (Martin-Löf randomness [5]) .”
InfluentialInferring Information Quality From What is ReportedPreprint
Penman, S. H., & Johnstone, D. J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5417516
A frequentist local false discovery rate
Xiang, D., Soloff, J. A., & Fithian, W. (2025) · Biometrika · doi:10.1093/biomet/asaf083
Inherent Tradeoffs in the Fair Determination of Risk Scores
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603206
The Externalities of Exploration and How Data Diversity Helps Exploitation
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603199
Algorithmic monoculture and social welfare
(2023) · The Societal Impacts of Algorithmic Decision-Making · doi:10.1145/3603195.3603202
Selection Problems in the Presence of Implicit Bias
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603200
Additional file 1 of Bayesian historical borrowing with longitudinal large-scale assessments
Kaplan, D. E., Chen, J., Lyu, W., & Yavuz, S. (2024) · Figshare · doi:10.6084/m9.figshare.26562283
Prompting without Panic: Attribute-Aware, Zero-Shot, Test-Time Calibration
Hebbalaguppe, R., Kandar, T., Nagpal, A., & Arora, C. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-06106-5_17 · Cited by 1
Protected Probabilistic Classification Library
Petej, I. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-15120-9_19
Online prediction for streaming observational data
Clarke, B. S., & Chanda, A. (2026) · Statistics Surveys · doi:10.1214/26-ss155
Rehder, E., & Cordts, M. (2026) · Springer eBooks · doi:10.1007/978-3-658-45276-6_44
Is This Predictor More Informative than Another? A Decision-Theoretical ComparisonPreprint
Feng, Y., Qian, L., & Tang, W. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6167146
Assessment of PHRED Score Characteristics in Illumina MiSeq Amplicon Sequencing
Sims, S., Khudyakov, Y. E., & Zelikovsky, A. (2026) · Journal of Computational Biology · doi:10.1177/15578666261436820
Biquality learning : from weakly supervised learning to distribution shifts
Nodet, P. (2023) · theses.fr (ABES) · doi:10.70675/41d22fb9z3d38z4a64z9a3cz4a385058629c
Decision Making on Spatio-temporal Stochastic Processes with Environmental Applications
Tayewo, R. (2023) · theses.fr (ABES) · doi:10.70675/142fa5edz9af6z43b4zb3abz7241a3799806
Trustworthy machine learning : explainability and distribution-free uncertainty quantification
Amoukou, S. I. (2023) · theses.fr (ABES) · doi:10.70675/a022ddcbz8047z488bzb33bz68f6b99639b2 · Cited by 1
Metadata
- Authors
- Dawid, A. P.
- Affiliations
- University College London (GB)
- Corresponding author
- Dawid, A. P.
- ORCID
- Dawid, A. P.
- Venue
- Journal of the American Statistical Association
- Publisher
- Informa UK Limited
- Published
- 1982-09
- Volume
- 77
- Issue
- 379
- Pages
- 605-610
- ISSN
- 0162-1459 (ISSN-L)
- 1537-274X
- OpenAlex
- W2023943903
- Cited by
- 625
- References
- 24
- Citations by year
- 2026: 22 · 2025: 26 · 2024: 45 · 2023: 51 · 2022: 27 · 2021: 31 · 2020: 39 · 2019: 18 · 2018: 19 · 2017: 15 · 2016: 13 · 2015: 18 · 2014: 23 · 2013: 18 · 2012: 17
- FWCI
- 6.76
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of the American Statistical Association (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Social Sciences › Decision Sciences › Management Science and Operations Research › Forecasting Techniques and Applications
- Keywords
- well calibration, Bayesian forecasting, coherence, subjective probability, Dutch book argument
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
