A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
Open accessAbstract
We consider the two related problems of detecting if an example is misclassified or out-of-distribution. We present a simple baseline that utilizes probabilities from softmax distributions. Correctly classified examples tend to have greater maximum softmax probabilities than erroneously classified and out-of-distribution examples, allowing for their detection. We assess performance by defining several tasks in computer vision, natural language processing, and automatic speech recognition, showing the effectiveness of this baseline across all. We then show the baseline can sometimes be surpassed, demonstrating the room for future research on these underexplored detection tasks.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Most ML models can generate confidence scores that, in practice, correlate with the model's true likelihood to err.
“most ML models can generate confidence scores that, in practice, correlate with the model's true likelihood to err [32]”
From Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance (2021) · cites this paper as [32]
The Introduction (pp. 1–2) and Table 1 (p. 4) show that softmax-derived scores can help distinguish correct from incorrect predictions in the neural classifiers tested. The paper does not establish this for most ML models, and it explicitly cautions that softmax prediction probabilities have poor direct correspondence to confidence when viewed in isolation. (LLM, conf=0.88)
Experimental — accuracy not yet measuredThere are commonly seen correlations between mistakes and low-confidence.
“Relying on the commonly seen correlations between mistakes and low-confidence [32]”
From Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance (2021) · cites this paper as [32]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
No citation lists for this paper
No citation lists for this paper yet — checked again daily.
Cited by 1,968
Research works printed but not linked: 32
- Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Man´ e. Concrete problems in ai safety. arXiv , 2016.
- Jesse Davis and Mark Goadrich. The relationship between precision-recall and ROC curves. In International Conference on Machine Learning (ICML) , 2006.
- Tom Fawcett. An introduction to ROC analysis . Pattern Recognition Letters, 2005.
- Kevin Gimpel, Nathan Schneider, Brendan O ′ Connor, Dipanjan Das, Daniel Mills, Jacob Eisenstein, Michael Heilman, Dani Yogatama, Jeffrey Flanigan, and Noah A. Smith. Part-of-Speech Tagging for Twitter: Annotation, Features, and Experiments . Association for Computational Linguistics (ACL), 2011.
- Raja Giryes, Guillermo Sapiro, and Alex M. Bronstein. Deep neural networks with random gaussian weights: A universal classification strategy? arXiv , 2015.
- Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR) , 2015.
- Alex Graves, Santiago Fern´ andez, Faustino Gomez, and J¨ urgen Schmidhuber. Connectionist temporal classification: Labeling unsegmented sequence data with recurrent neural networks. In International Conference on Machine Learning (ICML) , 2006.
- Dan Hendrycks and Kevin Gimpel. Methods for detecting adversarial images and a colorful saliency map. arXiv , 2016a.
- Dan Hendrycks and Kevin Gimpel. Bridging nonlinearities and stochastic regularizers with Gaussian error linear units. arXiv , 2016b.
- Dan Hendrycks and Kevin Gimpel. Adjusting for dropout variance in batch normalization and weight initialization. arXiv , 2016c.
- Hans-G¨ unter Hirsch and David Pearce. The Aurora experimental framework for the performance evaluation of speech recognition systems under noisy conditions. ISCA ITRW ASR2000 , 2000.
- Minqing Hu and Bing Liu. Mining and Summarizing Customer Reviews . Knowledge Discovery and Data Mining (KDD), 2004.
- Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, and Hal Daum´ e Iii. Deep Unordered Composition Rivals Syntactic Methods for Text Classification . Association for Computational Linguistics (ACL), 2015.
- Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. Bag of tricks for efficient text classification. arXiv , 2016.
- Diederik Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization . International Conference for Learning Representations (ICLR), 2015.
- Ken Lang. Newsweeder: Learning to filter netnews. In International Conference on Machine Learning (ICML) , 1995.
- David D. Lewis, Yiming Yang, Tony G. Rose, and Fan Li. Rcv1: A new benchmark collection for text categorization research. Journal of Machine Learning Research (JMLR) , 2004.
- Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with restarts. arXiv , 2016.
- Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In Association for Computational Linguistics (ACL) , 2011.
- Chris Manning and Hinrich Sch¨ utze. Foundations of Statistical Natural Language Processing . MIT Press, 1999.
- Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Computer Vision and Pattern Recognition (CVPR) , 2015.
- Khanh Nguyen and Brendan O'Connor. Posterior calibration and exploratory analysis for natural language processing models. In Empirical Methods in Natural Language Processing (EMNLP) , 2015.
- Olutobi Owoputi, Brendan O'Connor, Chris Dyer, Kevin Gimpel, Nathan Schneider, and Noah A. Smith. Improved part-of-speech tagging for online conversational text with word clusters. In North American Chapter of the Association for Computational Linguistics (NAACL) , 2013.
- Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. Thumbs up? sentiment classification using machine learning techniques. In Empirical Methods in Natural Language Processing (EMNLP) , 2002.
- Foster Provost, Tom Fawcett, and Ron Kohavi. The case against accuracy estimation for comparing induction algorithms. In International Conference on Machine Learning (ICML) , 1998.
- Michael L. Seltzer, Dong Yu, and Yongqiang Wang. Investigation of deep neural networks for noise robust speech recognition. In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) , 2013.
- Jacob Steinhardt and Percy Liang. Unsupervised risk estimation using only conditional independence structure. In Neural Information Processing Systems (NIPS) , 2016.
- Gethin Williams and Steve Renals. Confidence measures for hybrid hmm/ann speech recognition. In Proceedings of EuroSpeech , 1997.
- Jianxiong Xiao, James Hays, Krista A. Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2010.
- Dong Yu, Jinyu Li, and Li Deng. Calibration of confidence measures in speech recognition. In IEEE Transactions on Audio, Speech, and Language , 2010.
- Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. British Machine Vision Conference , 2016.
- Yuting Zhang, Kibok Lee, and Honglak Lee. Augmenting supervised neural networks with unsupervised objectives for large-scale image classification. In International Conference on Machine Learning (ICML) , 2016.
- Authors
- Hendrycks, D., & Gimpel, K.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2016
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1610.02136
- OpenAlex
- W2531327146
- Cited by
- 1,968
- References
- 0
- Citations by year
- 2026: 27 · 2025: 179 · 2024: 325 · 2023: 396 · 2022: 279 · 2021: 384 · 2020: 241 · 2019: 96 · 2018: 36 · 2017: 3 · 2012: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (publishedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- misclassification detection, out-of-distribution detection, anomaly detection, computer vision, natural language processing, speech recognition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
