Verified Uncertainty Calibration
Open accessThe DOI matches its OpenAlex record; 14 of the 14 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Bootstrap methods: another look at the jackknife
Efron, B. (1979) · The Annals of Statistics · doi:10.1214/aos/1176344552 · Cited by 17,602
Probabilistic Forecasts, Calibration and Sharpness
Gneiting, T., Balabdaoui, F., & Raftery, A. E. (2007) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2007.00587.x · Cited by 1,783
Estimation of entropy and mutual information
Paninski, L. (2003) · Neural Computation · doi:10.1162/089976603321780272 · Cited by 1,442 · cited 1×
“This is analogous to the difficulty of measuring the mutual information between two continuous signals [27].”
BackgroundTransforming classifier scores into accurate multiclass probability estimates
Zadrozny, B., & Elkan, C. (2002) · Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/775047.775151 · Cited by 1,280 · cited 3×
“We recalibrated each class separately as in [13], using B bins per class, and evaluated calibration using the marginal calibration error (Definition 2.”
MethodDistribution-Free Predictive Inference for Regression
Lei, J., G’Sell, M., Rinaldo, A., Tibshirani, R. J., & Wasserman, L. (2018) · Journal of the American Statistical Association · doi:10.1080/01621459.2017.1307116 · Cited by 1,049 · cited 1×
“Besides calibration, other ways of producing and quantifying uncertainties include Bayesian methods [40] and conformal prediction [41, 42].”
MethodObtaining well calibrated probabilities using Bayesian binning
Pakdaman Naeini, M., Cooper, G., & Hauskrecht, M. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9602 · Cited by 991 · cited 1×
“1 and `1 calibration errors are also used in the literature [9, 23, 24].”
MethodDeep Anomaly Detection with Outlier Exposure
Hendrycks, D., Mazeika, M., & Dietterich, T. (2018) · arXiv · doi:10.48550/arxiv.1812.04606 · Cited by 932
Weather Forecasting with Ensemble Methods
Gneiting, T., & Raftery, A. E. (2005) · Science · doi:10.1126/science.1115255 · Cited by 636 · cited 4×
“We often want to minimize the MSE subject to a calibration budget [5, 25].”
InfluentialIncreasing the Reliability of Reliability Diagrams
Bröcker, J., & Smith, L. A. (2007) · Weather and Forecasting · doi:10.1175/waf993.1 · Cited by 316 · cited 1×
“Besides the calibration error, prior work also uses the Hosmer-Lemeshov test [37] and reliability diagrams [4, 38] to evaluate calibration.”
Method
Cited by
A review of uncertainty quantification in deep learning: Techniques, applications and challenges
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.008 · Cited by 3,111
“[329] (Scaling-binning calibrator), [330] (AugMix as a data processing approach), Możejko et al.”
MethodHendrycks, D., Zhao, K., Basart, S., Steinhardt, J., & Song, D. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01501 · Cited by 1,090
Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models
Dahl, M., Magesh, V., Suzgun, M., & Ho, D. E. (2024) · Journal of Legal Analysis · doi:10.1093/jla/laae003 · Cited by 559
(2021) · International Conference on Learning Representations · Cited by 435
A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving
Feng, D., Harakeh, A., Waslander, S. L., & Dietmayer, K. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3096854 · Cited by 316
Metadata
- Authors
- Kumar, A., Liang, P., & Ma, T.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2019-09-23
- Volume
- 32
- Pages
- 3787-3798
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1909.10155
- OpenAlex
- W2990037398
- Cited by
- 131
- References
- 14
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Verified Uncertainty Calibration. https://app.cochat.ai/papers/10.48550/arxiv.1909.10155
- The paper
- Kumar, A., Liang, P., & Ma, T. (2019). Verified Uncertainty Calibration. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1909.10155
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.