CoChat
Sign Up
PreprintCited by 3,446Adversarial Robustness in Machine LearningAdvanced Neural Network ApplicationsDomain Adaptation and Few-Shot Learning

On calibration of modern neural networks

Open access
Guo, Chuan, Pleiss, Geoff, Sun, Yu, Weinberger, Kilian Q.·2017·arXiv
DOI 10.48550/arxiv.1706.04599arXiv 1706.04599
Open this paper in CoChat — free accountOpen PDF

Abstract

Confidence calibration -- the problem of predicting probability estimates representative of the true correctness likelihood -- is important for classification models in many applications. We discover that modern neural networks, unlike those from a decade ago, are poorly calibrated. Through extensive experiments, we observe that depth, width, weight decay, and Batch Normalization are important factors influencing calibration. We evaluate the performance of various post-processing calibration methods on state-of-the-art architectures with image and document classification datasets. Our analysis and experiments not only offer insights into neural network learning, but also provide a simple and straightforward recipe for practical settings: on most datasets, temperature scaling -- a single-parameter variant of Platt Scaling -- is surprisingly effective at calibrating predictions.

AI snapshot

Written from the paper’s full text.

TL;DR

Modern neural networks are often poorly calibrated: their predicted confidence exceeds their accuracy, unlike older networks. Temperature scaling, a single-parameter post-processing method, is surprisingly effective at fixing this across vision and NLP classification tasks.

Key findings

  • Miscalibration is common in modern architectures: on CIFAR-100, a 110-layer ResNet has ECE 16.53% uncalibrated, while a 5-layer LeNet has ECE 4.85%; most datasets/models show ECE typically between 4 to 10%, with exceptions SVHN at 0.44% and Reuters at 0.85%.
  • Depth, width, Batch Normalization, and weight decay affect calibration: increased model capacity and less weight decay are associated with worse ECE, and a 6-layer ConvNet with Batch Normalization has worse calibration even when accuracy improves slightly.
  • Temperature scaling is often the most effective method: on CIFAR-100 ResNet 110 it reduces ECE from 16.53% to 1.26%, and on CIFAR-10 ResNet 110 from 4.6% to 0.83%; it outperforms vector and matrix scaling on vision tasks and performs comparably on NLP datasets.
  • NLL can overfit after classification accuracy improves: on CIFAR-100, test error drops from 29% to 27% in the region where NLL overfits, suggesting networks gain accuracy at the expense of well-modeled probabilities.

Methods

  • The paper evaluates post-processing calibration methods: histogram binning, isotonic regression, Bayesian Binning into Quantiles (BBQ), and Platt scaling variants including temperature scaling, vector scaling, and matrix scaling.
  • Experiments cover 6 image datasets (Caltech-UCSD Birds, Stanford Cars, ImageNet 2012, CIFAR-10/100, SVHN) and 4 document/sentiment datasets (20 News, Reuters, SST binary, SST fine-grained), using ResNets, stochastic-depth ResNets, Wide ResNets, DenseNets, Deep Averaging Networks, and TreeLSTMs.
  • Calibration is measured primarily by Expected Calibration Error (ECE) with M = 15 bins, with additional MCE, NLL, and test error tables; temperature scaling uses a single scalar T > 0 applied to logits and optimized by NLL on a validation set, leaving class predictions unchanged.

Limitations

  • The paper states it cannot claim causality for the observed relationships between architecture/training trends and miscalibration, and says it remains future work to understand why these trends affect calibration while improving accuracy.
  • Temperature scaling does not calibrate the Reuters dataset; the authors note this dataset is already well-calibrated (ECE ≤ 1%) and that measurements may be affected by dataset split or binning scheme.
  • Matrix scaling performs poorly on datasets with hundreds of classes and fails to converge on 1000-class ImageNet, which the authors attribute to its number of parameters scaling quadratically with classes and overfitting a small validation set.
  • The supplement states MCE is very sensitive to the binning scheme and less suited for small test sets.
  • Evident from the text: all calibration methods require a hold-out validation set and assume training, validation, and test sets are drawn from the same distribution; evaluation is limited to the classification datasets and architectures listed.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.1706.04599

Cites

49 of 49 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

2 of 7 cited sources checked · 2 supported · 0 flagged

  • Findingown data
    § 5. Results · p. 7

    The only dataset that temperature scaling does not calibrate is the Reuters dataset.

    “The only dataset that temperature scaling does not calibrate is the Reuters dataset.”
    Quote in the paperNot a substantial claim
  • Findingown data
    § 5. Results · p. 7

    The two notable exceptions to miscalibration are SVHN and Reuters, both of which experience ECE values below 1%.

    “The two notable exceptions are SVHN and Reuters, both of which experience ECE values below 1% .”
    Quote in the paperNot a substantial claim
  • Findingown data
    § Abstract · p. 1

    On most datasets, temperature scaling is surprisingly effective at calibrating predictions.

    “on most datasets, temperature scaling - a singleparameter variant of Platt Scaling - is surprisingly effective at calibrating predictions.”
    • Also stated in: § 1. Introduction · p. 2 · plain
      “we find that a single-parameter variant of Platt scaling (Platt et al., 1999) - which we refer to as temperature scaling - is often the most effective method at obtaining calibrated probabilities.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Platt et al., 1999

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • The authors state this as an assumption: The training, validation, and test sets are drawn from the same distribution.
  • Findingown data
    § 3. Observing Miscalibration · p. 4

    The worse calibration with Batch Normalization holds regardless of the hyperparameters used on the Batch Normalization model.

    “We find that this result holds regardless of the hyperparameters used on the Batch Normalization model (i.e. low or high learning rate, etc.).”
    Quote in the paperNot a substantial claim
  • Findingreasoning
    § 3. Observing Miscalibration · p. 3

    The ECE metric grows substantially with model capacity.

    “the ECE metric grows substantially with model capacity.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 14
  • May combine two claims. 22
  • The quote only partly states this claim. 1

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.2. Extension to Multiclass Models · p. 6

    Temperature scaling is commonly used in settings such as knowledge distillation and statistical mechanics.

    “Temperature scaling is commonly used in settings such as knowledge distillation (Hinton et al., 2015) and statistical mechanics (Jaynes, 1957).”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites Hinton et al., 2015 source
    • cites Jaynes, 1957 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1. Introduction · p. 1

    Recent advances in deep learning have dramatically improved neural network accuracy.

    “Recent advances in deep learning have dramatically improved neural network accuracy (Simonyan & Zisserman, 2015; Srivastava et al., 2015; He et al., 2016; Huang et al., 2016; 2017).”
    Quote in the paperNot a substantial claim1 of 3 sources checked
    • cites Simonyan & Zisserman, 2015

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Srivastava et al., 2015 source
    • cites He et al., 2016

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingown data
    § 5. Results · p. 7

    Temperature scaling outperforms all other calibration methods on the vision tasks, and performs comparably to other methods on the NLP datasets.

    “Temperature scaling outperforms all other methods on the vision tasks, and performs comparably to other methods on the NLP datasets.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning
    § 5. Results · p. 7

    SVHN and Reuters have very low error (1.98% and 2.97%, respectively), so their ratio of ECE to error is comparable to other datasets.

    “Both of these datasets have very low error ( 1 . 98% and 2 . 97% , respectively); and therefore the ratio of ECE to error is comparable to other datasets.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 5. Results · p. 7

    Vector scaling recovers essentially the same solution as temperature scaling; the learned vector has nearly constant entries and is no different than a scalar transformation.

    “In fact, vector scaling recovers essentially the same solution as temperature scaling - the learned vector has nearly constant entries, and therefore is no different than a scalar transformation.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5. Results · p. 7

    Temperature scaling outperforms the vector and matrix Platt scaling variants, which are strictly more general methods.

    “What is perhaps even more surprising is that temperature scaling outperforms the vector and matrix Platt scaling variants, which are strictly more general methods.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5. Results · p. 7

    Most datasets and models experience some degree of miscalibration, with ECE typically between 4 to 10%.

    “It is worth noting that most datasets and models experience some degree of miscalibration, with ECE typically between 4 to 10% .”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 3. Observing Miscalibration · p. 3

    Increasing depth and width negatively affect model calibration, even though they may reduce classification error.

    “Although increasing depth and width may reduce classification error, we observe that these increases negatively affect model calibration.”
    • Also stated in: § 3. Observing Miscalibration · p. 3 · plain
      “the ECE metric grows substantially with model capacity.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given
    § S4. Additional Reliability Diagrams · p. 13

    Error with temperature scaling is exactly the same as uncalibrated.

    “Error with temperature scaling is exactly the same as uncalibrated.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingreasoning
    § S1. Further Information on Calibration Metrics · p. 11

    ECE using M bins converges to the M-term Riemann-Stieltjes sum of E_P[|P(Y=Y|P=p)-p|].

    “Hence ECE using M bins converges to the M -term Riemann-Stieltjes sum of E ˆ P [∣ ∣ ∣ P ( ˆ Y = Y | ˆ P = p ) -p ∣ ∣ ∣ ] .”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § S2. Further Information on Temperature Scaling · p. 11

    Temperature scaling not only lowers the NLL but also raises the entropy of the distribution.

    “Temperature scaling not only lowers the NLL but also raises the entropy of the distribution (green line).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingreasoning
    § 5. Results · p. 8

    Using a conjugate gradient solver, the optimal temperature can be found in 10 iterations, or a fraction of a second on most modern hardware.

    “Using a conjugate gradient solver, the optimal temperature can be found in 10 iterations, or a fraction of a second on most modern hardware.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning
    § 5. Results · p. 8

    Histogram binning and isotonic regression take an order of magnitude longer than temperature scaling, and BBQ takes roughly 3 orders of magnitude more time.

    “Histogram binning and isotonic regression take an order of magnitude longer than temperature scaling, and BBQ takes roughly 3 orders of magnitude more time.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given
    § 5. Results · p. 8

    All calibration methods scale linearly with the number of validation set samples.

    “All methods scale linearly with the number of validation set samples.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning
    § 5. Results · p. 8

    Matrix scaling performs poorly on datasets with hundreds of classes (Birds, Cars, and CIFAR-100), and fails to converge on the 1000-class ImageNet dataset.

    “Matrix scaling performs poorly on datasets with hundreds of classes (i.e. Birds, Cars, and CIFAR-100), and fails to converge on the 1000-class ImageNet dataset.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given
    § Entropy vs. NLL on CIFAR-100 · p. 12

    MCE seems very sensitive to the binning scheme and is less suited for small test sets.

    “MCE seems very sensitive to the binning scheme and is less suited for small test sets.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 5. Results · p. 8

    Each of the bins in the reliability diagram after temperature scaling is well calibrated, even though all probabilities were modified by only a single parameter.

    “Each of the bins are well calibrated, which is remarkable given that all the probabilities were modified by only a single parameter.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given
    § S4. Additional Reliability Diagrams · p. 13

    NLL roughly follows the trends of ECE.

    “To summarize, NLL roughly follows the trends of ECE.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 5. Results · p. 8

    Binning methods tend to change class predictions, which hurts accuracy.

    “Additionally, binning methods tend to change class predictions which hurts accuracy (see Section S3).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning
    § 5. Results · p. 8

    Temperature scaling is by far the fastest calibration method, as it amounts to a one-dimensional convex optimization problem.

    “Temperature scaling is by far the fastest method, as it amounts to a onedimensional convex optimization problem.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 5. Results · p. 8

    Binning methods improve calibration on most datasets, but do not outperform temperature scaling.

    “Binning methods improve calibration on most datasets, but do not outperform temperature scaling.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingreasoning
    § 5. Results · p. 8

    The computational complexity of vector scaling is linear in the number of classes, and that of matrix scaling is quadratic in the number of classes.

    “The computational complexity of vector and matrix scaling are linear and quadratic respectively in the number of classes, reflecting the number of parameters in each method.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 3. Observing Miscalibration · p. 4

    High capacity models are not necessarily immune from overfitting; overfitting manifests in probabilistic error rather than classification error.

    “The observed disconnect between NLL and 0/1 loss suggests that these high capacity models are not necessarily immune from overfitting, but rather, overfitting manifests in probabilistic error rather than classification error.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § 3. Observing Miscalibration · p. 3

    Increased model capacity lowers training NLL, and thus the model will be more overconfident on average.

    “Increased model capacity will lower training NLL, and thus the model will be more (over)confident on average.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisenone given
    § 2. Definitions · p. 2

    Reliability diagrams do not display the proportion of samples in a given bin, and thus cannot be used to estimate how many samples are calibrated.

    “Note that reliability diagrams do not display the proportion of samples in a given bin, and thus cannot be used to estimate how many samples are calibrated.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisenone given
    § 2. Definitions · p. 2

    In all practical settings, achieving perfect calibration is impossible.

    “In all practical settings, achieving perfect calibration is impossible.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 2. Definitions · p. 3

    In expectation, NLL is minimized if and only if ˆπ(Y|X) recovers the ground truth conditional distribution π(Y|X).

    “It is a standard result (Friedman et al., 2001) that, in expectation, NLL is minimized if and only if ˆ π ( Y | X ) recovers the ground truth conditional distribution π ( Y | X ) .”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Friedman et al., 2001

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisereasoning
    § 2. Definitions · p. 2

    acc(B_m) is an unbiased and consistent estimator of P(ˆY=Y | ˆP∈I_m).

    “Basic probability tells us that acc( B m ) is an unbiased and consistent estimator of P ( ˆ Y = Y | ˆ P ∈ I m ) .”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given
    § 4.3. Other Related Works · p. 7

    The paper's framework returns a confidence score rather than a distribution of possible outputs.

    “In contrast, our framework, which does not augment the neural network model, returns a confidence score rather than returning a distribution of possible outputs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given
    § 4.2. Extension to Multiclass Models · p. 6

    Temperature scaling is equivalent to maximizing the entropy of the output probability distribution subject to certain constraints on the logits.

    “The model is equivalent to maximizing the entropy of the output probability distribution subject to certain constraints on the logits (see Section S2).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § 2. Definitions · p. 3

    For perfectly calibrated classifiers, MCE and ECE both equal 0.

    “For perfectly calibrated classifiers, MCE and ECE both equal 0.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data
    § S2. Further Information on Temperature Scaling · p. 11

    Overfitting with respect to NLL results in a low-entropy softmax distribution over classes, which explains the model's overconfidence.

    “This results in a low-entropy softmax distribution over classes (blue line), which explains the model's overconfidence.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 5. Results · p. 7

    Because the Reuters dataset is well-calibrated to begin with (ECE ≤ 1%), there is not much room for improvement with any method.

    “Because this dataset is well-calibrated to begin with (ECE ≤ 1% ), there is not much room for improvement with any method”
    Quote in the paperSubstantial claim
33 of 33

  • The authors state this as an assumption.

    The training, validation, and test sets are drawn from the same distribution.

    “We assume that the training, validation, and test sets are drawn from the same distribution.”

  • The authors state this as an assumption.

    There is no prior use of temperature scaling in the context of calibrating probabilistic models.

    “To the best of our knowledge, we are not aware of any prior use in the context of calibrating probabilistic models.”
2 of 2

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Confidence calibration was proposed by Guo et al. (2017), although not under that name.

    “Confidence calibration was proposed by Guo et al. (2017), although not under that name.”

    From Classifier calibration: a survey on how to assess and improve predicted class probabilities (2023) · cites this paper as Guo et al. (2017)

    The Abstract (p. 1) explicitly calls the topic “confidence calibration,” contrary to the claim that Guo et al. did not use that name. Section 4 (p. 4) says the authors review existing calibration methods and introduce variants of their own, rather than proposing confidence calibration itself. (LLM, conf=0.93)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • LLMs exhibited Expected Calibration Errors (ECE) ranging from 0.108 to 0.427, frequently expressing high confidence in incorrect answers.

    “Empirically, LLMs exhibited Expected Calibration Errors (ECE) ranging from 0.108 to 0.427, frequently expressing high confidence in incorrect answers [61].”

    From Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework (2026) · cites this paper as [61]

    The Results (p. 7) report ECE measurements for convolutional, recurrent, and deep averaging networks, and Observing Miscalibration (p. 3) describes neural networks becoming overconfident. The paper does not report ECE values of 0.108–0.427 or findings about LLMs giving confident incorrect answers; those specifics are unsupported. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • Temperature scaling is a simple yet effective approach to eliminate the over-confident problem of the output posteriors of neural networks.

    “Temperature scaling divides the logits vector by a learned scaling parameter, which is a simple yet effective approach to eliminate the over-confident problem of the output posteriors of neural networks [25].”

    From When Machine Unlearning Jeopardizes Privacy (2021) · cites this paper as [25]

    Section 4.2 (p. 6) describes temperature scaling as using a learned scalar temperature to soften the softmax output, and the Results (p. 7) find it effective despite its simplicity. But the claim that it eliminates overconfidence is too absolute: the Results (p. 7) identify a dataset that temperature scaling does not calibrate. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Traditional point predictions, even when accompanied by confidence scores from probabilistic models, often fail to quantify uncertainty reliably due to miscalibration.

    “Traditional point predictions, even when accompanied by confidence scores from probabilistic models, often fail to quantify uncertainty reliably due to miscalibration [3], [4].”

    From Optimized Certainty Equivalent Risk-Controlling Prediction Sets (2026) · cites this paper as [3]

    The Introduction (p. 2) reports that, on several computer-vision and NLP tasks, neural networks produce confidence scores that do not represent true probabilities. This supports the concern about miscalibrated confidence, but the paper does not establish the broader claim about traditional point predictions or probabilistic models generally. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • Temperature scaling leads to well-calibrated predictions on the i.i.d. test set.

    “which was shown by Guo et al. (2017) to lead to well-calibrated predictions on the i.i.d. test set.”

    From Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift (2019) · cites this paper as Guo et al. (2017)

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Temperature scaling is not best on every vision task

    Minor

    Table 1 reports lower ECE for histogram binning on Cars (1.74% versus 2.35%) and for BBQ on CIFAR-10 with ResNet 110 (0.54% versus 0.83%). Thus temperature scaling does not outperform all other methods on those vision tasks.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. Collobert, Ronan, Kavukcuoglu, Koray, and Farabet, Cl´ ement. Torch7: A matlab-like environment for machine learning. In BigLearn Workshop, NIPS , 2011.
  2. Friedman, Jerome, Hastie, Trevor, and Tibshirani, Robert. The elements of statistical learning , volume 1. Springer series in statistics Springer, Berlin, 2001.
  3. Netzer, Yuval, Wang, Tao, Coates, Adam, Bissacco, Alessandro, Wu, Bo, and Ng, Andrew Y. Reading digits in natural images with unsupervised feature learning. In Deep Learning and Unsupervised Feature Learning Workshop, NIPS , 2011.

Cited by

Sorted by citations

Related

Authors
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q.
Affiliations
Cornell University (US)
Corresponding author
Pleiss, Geoff
ORCID
Venue
arXiv
Publisher
arXiv
Published
2017
Pages
1321-1330
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1706.04599
arXiv
1706.04599
OpenAlex
W2626967530
Cited by
3,435
References
37
Citations by year
2026: 42 · 2025: 315 · 2024: 521 · 2023: 590 · 2022: 450 · 2021: 737 · 2020: 475 · 2019: 217 · 2018: 80 · 2017: 5 · 2016: 1 · 2012: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
  • International Conference on Machine Learning
Funders
  • National Science Foundation (1618134, 1149882, 1526012)
  • Bill and Melinda Gates Foundation
  • Office of Naval Research
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
confidence calibration, artificial neural networks, temperature scaling, Platt scaling, probability estimates, network depth, weight decay, batch normalization, image classification, document classification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-21.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-21.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review