CoChat
Sign Up
Cited by 112,155Advanced Neural Network ApplicationsDomain Adaptation and Few-Shot LearningAdvanced Image and Video Retrieval Techniques

ImageNet classification with deep convolutional neural networks

Open access
Krizhevsky, Alex, Sutskever, Ilya, Hinton, Geoffrey E.·2017·Communications of the ACM
DOI 10.1145/3065386
Open this paper in CoChat — free accountOpen PDF

Abstract

We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0%, respectively, which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully connected layers we employed a recently developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.

Integrity signals

Registry

Crossref · 10.1145/3065386

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

2 of 14 cited sources checked · 2 supported · 0 flagged

  • Interpretationcitation

    Dropout reduces complex co-adaptations of neurons because a neuron cannot rely on the presence of particular other neurons.

    “This technique reduces complex co adaptations of neurons, since a neuron cannot rely on the presence of particular other neurons.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [12] 12. Hinton, G., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580 (2012). source
  • Premisecitation

    There is a very efficient version of model combination that only costs about a factor of two during training.

    “There is, however, a very efficient version of model combination that only costs about a factor of two during training.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [12] 12. Hinton, G., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580 (2012). source
  • Findingown data

    The two-GPU scheme reduces top-1 and top-5 error rates by 1.7% and 1.2%, respectively, compared with a net with half as many kernels in each convolutional layer trained on one GPU.

    “This scheme reduces our top-1 and top-5 error rates by 1.7% and 1.2%, respectively, as compared with a net with half as many kernels in each convolutional layer trained on one GPU.”
    Quote in the paperSubstantial claim
  • Findingnone given

    The size of the network made overfitting a significant problem, even with 1.2 million labeled training examples.

    “The size of our network made overfitting a significant problem, even with 1.2 million labeled training examples”
    Quote in the paperSubstantial claim
  • Findingreasoning

    The data augmentation schemes are, in effect, computationally free.

    “So these data augmentation schemes are, in effect, computationally free.”
    Quote in the paperNot a substantial claim
  • May combine two claims. 18
  • Classified as probably not a substantial claim. 26
  • The quote only partly states this claim. 2

  • May combine two claims.

    Findingreasoning

    The 1000 classes of ILSVRC make each training example impose 10 bits of constraint on the mapping from image to label, but this is insufficient to learn 60 million parameters without considerable overfitting.

    “Although the 1000 classes of ILSVRC make each training example impose 10 bits of constraint on the mapping from image to label, this turns out to be insufficient to learn so many parameters without considerable overfitting.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    The resulting training examples from the first augmentation are highly interdependent.

    “though the resulting training examples are, of course, highly inter dependent.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    On the ImageNet LSVRC-2010 test data, the network achieved top-1 error 37.5% and top-5 error 17.0%.

    “On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0%, respectively”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Models with overlapping pooling find it slightly more difficult to overfit.

    “We generally observe during training that models with overlapping pooling find it slightly more difficult to overfit.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    On CIFAR-10, a four-layer CNN achieved a 13% test error rate without normalization and 11% with normalization.

    “a four-layer CNN achieved a 13% test e r r o r r a t e w i t h o u t n o r m a l i z a t i o n a n d 1 1 % w i t h normalization.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingnone given

    The techniques used for preventing overfitting were effective.

    “so we used several effective techniques for preventing overfitting”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Response normalization reduces top-1 and top-5 error rates by 1.4% and 1.2%, respectively.

    “Response normalization reduces our top-1 and top-5 error rates by 1.4% and 1.2%, respectively.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The two-GPU net takes slightly less time to train than the one-GPU net.

    “The two-GPU net takes slightly less time to train than the one-GPU net.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    The network contains a number of new and unusual features which improve its performance and reduce its training time.

    “Our network contains a number of new and unusual features which improve its performance and reduce its training time”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    Without the first data augmentation scheme, the network suffers from substantial overfitting, which would have forced the authors to use much smaller networks.

    “Without this scheme, our network suffers from substantial overfitting, which would have forced us to use much smaller networks.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    For the leopard image, only other types of cat are considered plausible labels.

    “For example, only other types of cat are considered plausible labels for the leopard.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    Initializing neuron biases in certain layers with the constant 1 accelerates the early stages of learning by providing the ReLUs with positive inputs.

    “This initialization accelerates the early stages of learning by providing the ReLUs with positive inputs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    At the pixel level, the retrieved training images are generally not close in L2 to the query images in the first column.

    “Notice that at the pixel level, the retrieved training images are generally not close in L2 to the query images in the first column.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Training one CNN with an extra sixth convolutional layer on the entire ImageNet Fall 2011 release and then finetuning it on ILSVRC-2012 gives a top-5 error rate of 16.6%.

    “Training one CNN, with an extra sixth convolutional layer over the last pooling layer, to classify the entire ImageNet Fall 2011 release (15M images, 22K categories), and then”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    On the Fall 2009 version of ImageNet, the top-1 and top-5 error rates are 67.4% and 40.9%.

    “Our top-1 and top-5 error rates on this dataset are 67.4% and 40.9%, attained by the net described above but with an additional, sixth convolutional layer over the last pooling layer.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Fall 2009 ImageNet split used by the authors, which differs from previous authors' splits, does not affect the results appreciably.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given

    The specialization of kernels between the two GPUs occurs during every run and is independent of any particular random weight initialization, modulo a renumbering of the GPUs.

    “This kind of specialization occurs during every run and is independent of any particular random weight initialization (modulo a renumbering of the GPUs).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Networks with ReLUs consistently learn several times faster than equivalents with saturating neurons, although the magnitude of the effect varies with network architecture.

    “The magnitude of the effect demonstrated here varies with network architecture, but networks with ReLUs consistently learn several times faster than equivalents with saturating neurons.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The network achieves top-1 and top-5 test set error rates of 37.5% and 17.0%, respectively, on ILSVRC-2010.

    “Our network achieves top-1 and top-5 test set error rates of 37.5% and 17.0%, respectively.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    Averaging the predictions of five similar CNNs gives a top-5 error rate of 16.4%.

    “Averaging the predictions of five similar CNNs gives an error rate of 16.4%.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    In the authors' experience, validation and test error rates do not differ by more than 0.1%.

    “In the remainder of this paragraph, we use validation and test error rates interchangeably because in our experience they do not differ by more than 0.1% (see Table 2).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Even off-center objects, such as the mite in the top-left, can be recognized by the network.

    “Notice that even off-center objects, such as the mite in the top-left, can be recognized by the net.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    All experiments suggest that the results can be improved simply by waiting for faster GPUs and bigger datasets to become available.

    “All of our experiments suggest that our results can be improved simply by waiting for faster GPUs and bigger datasets to become available.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    ReLUs do not require input normalization to prevent them from saturating.

    “ReLUs have the desirable property that they do not require input normalization to prevent them from saturating.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Convolutional neural networks (CNNs) constitute a class of models with large learning capacity and prior knowledge.

    “Convolutional neural networks (CNNs) constitute one such class of models.9,15,17,19,21,26,32”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [9] 9. Fukushima, K. Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol. Cybern. 36, 4 (1980), 193-202. source
    • cites [15] 15. Krizhevsky, A. Convolutional deep belief networks on cifar-10. Unpublished manuscript, 2010.

      Not checked in the free run.

    • cites [17] 17. LeCun, Y., Boser, B., Denker, J., Henderson, D., Howard, R., Hubbard, W., Jackel, L., et al. Handwritten digit recognition with a back-propagation network. In Advances in Neural Information Processing Systems (1990).

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    The immense complexity of the object recognition task means that the problem cannot be specified even by a dataset as large as ImageNet, so a model should also have lots of prior knowledge to compensate for missing data.

    “However, the immense complexity of the object recognition task means that this problem cannot be specified even by a dataset as large as ImageNet, so our model should also have lots of prior knowledge to compensate for all the data we do not have.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisereasoning

    Compared to standard feedforward neural networks with similarly sized layers, CNNs have much fewer connections and parameters and are easier to train, while their theoretically best performance is likely to be only slightly worse.

    “Thus, compared to standard feedforward neural networks with similarly sized layers, CNNs have much fewer connections and parameters and so they are easier to train, while their theoretically best performance is likely to be only slightly worse.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    Methods that require careful hand-engineering by a programmer who understands the domain do not scale as well as methods that replace the programmer with a powerful general-purpose learning procedure.

    “methods that require careful hand-engineering by a programmer who understands the domain do not scale as well as methods that replace the programmer with a powerful general-purpose learning procedure.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Current GPUs paired with a highly optimized implementation of 2D convolution are powerful enough to facilitate training of interestingly large CNNs.

    “Luckily, current GPUs, paired with a highly optimized implementation of 2D convolution, are powerful enough to facilitate the training of interestinglylarge CNNs”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    CNNs make strong and mostly correct assumptions about the nature of images, namely stationarity of statistics and locality of pixel dependencies.

    “they also make strong and mostly correct assumptions about the nature of images (namely, stationarity of statistics and locality of pixel dependencies).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Despite their attractive qualities, CNNs have been prohibitively expensive to apply at large scale to high-resolution images.

    “Despite the attractive qualities of CNNs, and despite the relative efficiency of their local architecture, they have still been prohibitively expensive to apply in large scale to highresolution images.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    ImageNet consists of over 15 million labeled high-resolution images in over 22,000 categories.

    “ImageNet, 7 which consists of over 15 million labeled high-resolution images in over 22,000 categories.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [7] 7. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L. ImageNet: A large-scale hierarchical image database. In CVPR09 (2009). source

  • May combine two claims.

    Interpretationown data

    The top-1 and top-5 error rates of 37.5% and 17.0% on LSVRC-2010 are considerably better than the previous state-of-the-art.

    “which is considerably better than the previous state-of-the-art”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    With enough computation and enough data, learning beats programming for complicated tasks that require the integration of many different, noisy cues.

    “With enough computation and enough data, learning beats programming for complicated tasks that require the integration of many different, noisy cues.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Recent datasets such as ImageNet contain enough labeled examples to train such models without severe overfitting.

    “and recent datasets such as ImageNet contain enough labeled examples to train such models without severe overfitting.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationnone given

    For deep neural networks to shine, they needed far more labeled data and hugely more computation.

    “for deep neural networks to shine, they needed far more labeled data and hugely more computation.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    If at least some training examples produce a positive input to a ReLU, learning will happen in that neuron.

    “If at least some training examples produce a positive input to a ReLU, learning will happen in that neuron.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationnone given

    To learn about thousands of objects from millions of images, a model with a large learning capacity is needed.

    “To learn about thousands of objects from millions of images, we need a model with a large learning capacity.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The hyperparameters k, n, α, and β were determined using a validation set.

    “The constants k, n, α, and β are hyperparameters whose values are determined using a validation set”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    On the Caltech-101 dataset, the primary concern is preventing overfitting, so the effect observed by Jarrett et al. is different from the accelerated ability to fit the training set reported when using ReLUs.

    “However, on this dataset the primary concern is preventing overfitting, so the effect they are observing is different from the accelerated ability to fit the training set which we report when using ReLUs.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    The best published results since ILSVRC-2010 are 45.7% top-1 and 25.7% top-5.

    “since then the best published results are 45.7% and 25.7% with an approach that averages the predictions of two classifiers trained on Fisher Vectors (FVs) computed from two types of densely sampled features.29”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [29] 29. Sánchez, J., Perronnin, F. High- dimensional signature compression for large-scale image classification. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011 (2011). IEEE, 1665-1672. source

  • Classified as probably not a substantial claim.

    Premisecitation

    Combining the predictions of many different models is a very successful way to reduce test errors.

    “Combining the predictions of many different models is a very successful way to reduce test errors, 1, 3”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [1] 1. Bell, R., Koren, Y. Lessons from the netflix prize challenge. ACM SIGKDD Explor. Newsl. 9, 2 (2007), 75-79. source
    • cites [3] 3. Breiman, L. Random forests. Mach. Learn. 45, 1 (2001), 5-32. source

  • Classified as probably not a substantial claim.

    Premisenone given

    Combining predictions of many models appears too expensive for big neural networks that already take several days to train.

    “but it appears to be too expensive for big neural networks that already take several days to train.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    The best published results on the Fall 2009 ImageNet dataset are 78.1% top-1 and 60.9% top-5.

    “The best published results on this dataset are 78.1% and 60.9%.23”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [23] 23. Mensink, T., Verbeek, J., Perronnin, F., Csurka, G. Metric learning for large scale image classification: Generalizing to new classes at near-zero cost. In ECCV -European Conference on Computer Vision (Florence, Italy, Oct. 2012). source
43 of 43

  • The paper relies on this without support.

    Objects in realistic settings exhibit considerable variability.

    “But objects in realistic settings exhibit considerable variability”

  • The paper relies on this without support.

    Object identity is invariant to changes in the intensity and color of illumination.

    “object identity is invariant to changes in the intensity and color of the illumination.”

  • The paper relies on this without support.

    The Fall 2009 ImageNet split used by the authors, which differs from previous authors' splits, does not affect the results appreciably.

    “Since there is no established test set, our split necessarily differs from the splits used by previous authors, but this does not affect the results appreciably.”
3 of 3

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In the field of DL, the CNN is the most famous and commonly employed algorithm.

    “In the field of DL, the CNN is the most famous and commonly employed algorithm [30, 71–75].”

    From Review of deep learning: concepts, CNN architectures, challenges, applications, future directions (2021) · cites this paper as [30]

    The Epilogue (page unavailable) says the computer-vision community switched to deep neural networks and that these are widely deployed, supporting widespread use in that field. It does not establish that CNNs are the most famous or most commonly employed algorithm across deep learning. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Fully convolutional neural networks have provided state-of-the-art performance for object classification and image segmentation.

    “Fully convolutional neural networks (fCNNs) have provided state-of-the-art performance for object classification and image segmentation (1).”

    From Fully automated, real-time 3D ultrasound segmentation to estimate first trimester placental volume using deep learning (2018) · cites this paper as [1]

    The Abstract reports better-than-previous-state-of-the-art ImageNet classification results. But §4.5 (page unavailable) says the network has three fully connected layers, so it is not fully convolutional; our copy does not report state-of-the-art image segmentation performance. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Krizhevesky et al. first proposed AlexNet and improved the CNN learning ability by increasing its depth and implementing several parameter optimization strategies.

    “Krizhevesky et al. [30] first proposed AlexNet and consequently improved the CNN learning ability by increasing its depth and implementing several parameter optimization strategies.”

    From Review of deep learning: concepts, CNN architectures, challenges, applications, future directions (2021) · cites this paper as [30]

  • AlexNet achieved innovative results in the fields of image recognition and classification.

    “In deep CNN architecture, AlexNet is highly respected [30], as it achieved innovative results in the fields of image recognition and classification.”

    From Review of deep learning: concepts, CNN architectures, challenges, applications, future directions (2021) · cites this paper as [30]

  • Many CNN models, e.g. AlexNet, GoogleNet, and ResNet, have been trained on large datasets such as ImageNet for image recognition purposes.

    “Many CNN models, e.g. AlexNet [30], GoogleNet [103], and ResNet [37], have been trained on large datasets such as ImageNet for image recognition purposes.”

    From Review of deep learning: concepts, CNN architectures, challenges, applications, future directions (2021) · cites this paper as [30]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Third convolutional layer exceeds the stated 1% parameter limit

    Minor

    The third layer alone has 384 × 3 × 3 × 256 = 884,736 kernel weights. That is about 1.47% of the network's stated 60 million parameters, contradicting the claim that every convolutional layer contains no more than 1%.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 13

  1. Berg A., 2010, Large scale visual recognition challenge
  2. Deng J., 2009, CVPR09
  3. Krizhevsky A., 2009, Department of Computer Science
  4. Krizhevsky A., 2010, Convolutional deep belief networks on cifar-10. Unpublished manuscript
  5. Krizhevsky A., 2011, ESANN
  6. LeCun Y., 1990, Advances in Neural Information Processing Systems
  7. LeCun Y., 1985, Une procedure d'apprentissage pour reseau a seuil asymmetrique (a learning scheme for asymmetric threshold networks)
  8. 10.5555/1896300.1896315 · DOI
  9. Mensink T., 2012, Italy
  10. 10.5555/3104322.3104425 · DOI
  11. Rumelhart D.E., 1985, DTIC Document
  12. 10.5555/938980.939477 · DOI
  13. Werbos P., 1974, Beyond regression: New tools for prediction and analysis in the behavioral sciences

Cited by

Sorted by citations

Related

Authors
Krizhevsky, A., Sutskever, I., & Hinton, G. E.
Affiliations
  • Google (United States) (US)
  • University of Toronto (CA)
Venue
Communications of the ACM
Publisher
Association for Computing Machinery (ACM)
Published
2017-05-24
Volume
60
Issue
6
Pages
84-90
ISSN
  • 0001-0782 (ISSN-L)
  • 1557-7317
DOI
10.1145/3065386
OpenAlex
W2163605009
Cited by
112,155
References
27
Citations by year
2026: 2710 · 2025: 7710 · 2024: 10211 · 2023: 12267 · 2022: 12280 · 2021: 15497 · 2020: 14538 · 2019: 13976 · 2018: 9999 · 2017: 6661 · 2016: 3637 · 2015: 1866 · 2014: 587 · 2013: 177 · 2012: 16
FWCI
5127.18
Type
journal article
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
  • Communications of the ACM (publishedVersion)
  • neural information processing systems
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
Keywords
ImageNet, image classification, deep convolutional neural networks, dropout regularization, overfitting

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-11.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-11.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review