Generalized Out-of-Distribution Detection: A Survey
Open accessThe DOI matches its Crossref record; 335 of the 335 references it cites were found in a registry or an index.
Abstract
Out-of-distribution (OOD) detection is critical to ensuring the reliability and safety of machine learning systems. For instance, in autonomous driving, we would like the driving system to issue an alert and hand over the control to humans when it detects unusual scenes or objects that it has never seen during training time and cannot make a safe decision. The term, OOD detection, first emerged in 2017 and since then has received increasing attention from the research community, leading to a plethora of methods developed, ranging from classification-based to density-based to distance-based ones. Meanwhile, several other problems, including anomaly detection (AD), novelty detection (ND), open set recognition (OSR), and outlier detection (OD), are closely related to OOD detection in terms of motivation and methodology. Despite common goals, these topics develop in isolation, and their subtle differences in definition and problem setting often confuse readers and practitioners. In this survey, we first present a unified framework called generalized OOD detection, which encompasses the five aforementioned problems, i.e.,AD, ND, OSR, OOD detection, and OD. Under our framework, these five problems can be seen as special cases or sub-tasks, and are easier to distinguish. Despite comprehensive surveys of related fields, the summarization of OOD detection methods remains incomplete and requires further advancement. This paper specifically addresses the gap in recent technical developm…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Fischler, M. A., & Bolles, R. C. (1981) · Communications of the ACM · doi:10.1145/358669.358692 · Cited by 25,893
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879
(1996) · Cited by 20,179
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · cited 2×
“Application and Benchmark The application of OOD detection usually falls into safety-critical situations, such as autonomous driving [65], [66].”
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Chandola, V., Banerjee, A., & Kumar, V. (2009) · ACM Computing Surveys · doi:10.1145/1541880.1541882 · Cited by 11,687 · cited 1×
“Definition Anomaly detection (AD) (Chandola et al., 2009) aims to detect any anomalous samples that deviate from the predefined normality during testing.”
On Estimation of a Probability Density Function and Mode
Parzen, E. (1962) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177704472 · Cited by 10,585
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381
Ensemble methods in machine learning
Dietterich, T. G. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45014-9_1 · Cited by 8,408
Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2008) · 2008 Eighth IEEE International Conference on Data Mining · doi:10.1109/icdm.2008.17 · Cited by 6,601 · cited 6×
“Classification-based Methods AD methods like Isolation Forest [344] and OC-SVM [318, 319] can be applied to OD.”
BackgroundMethodInfluentialDropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471
Journal of The American Statistical Association
Fienberg, S. E. (2005) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1317 · Cited by 6,125
Image Style Transfer Using Convolutional Neural Networks
Gatys, L. A., Ecker, A. S., & Bethge, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.265 · Cited by 6,025
Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
Powers, D. M. W. (2020) · arXiv · doi:10.48550/arxiv.2010.16061 · Cited by 5,440
CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features
Yun, S., Han, D., Oh, S. J., Chun, S., Choe, J., & Yoo, Y. (2019) · arXiv · doi:10.48550/arxiv.1905.04899 · Cited by 5,439
CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features
Yun, S., Han, D., Chun, S., Oh, S. J., Yoo, Y., & Choe, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00612 · Cited by 5,400
LOF: identifying density-based local outliers
Breunig, M. M., Kriegel, H.-P., Ng, R. T., & Sander, J. (2000) · ACM SIGMOD Record · doi:10.1145/335191.335388 · Cited by 5,207
Bayesian Learning for Neural Networks
Neal, R. M. (1996) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-0745-0 · Cited by 4,276
Reading digits in natural images with unsupervised feature learning
Netzer, Y. (2024) · doi:10.57702/e7wwo0zk · Cited by 4,149
LOF: identifying density-based local outliers
Breunig, M. M., Kriegel, H.-P., Ng, R. T., & Sander, J. (2000) · Proceedings of the 2000 ACM SIGMOD international conference on Management of data · doi:10.1145/342009.335388 · Cited by 4,149 · cited 3×
“Density-Based Methods OD methods include Gaussian distribution (Altman & Bland, 2005; Leys et al., 2013), Mahalanobis distance (De Maesschalck et al., 2000), Gaussian mixtures (Yang et al., 2009), and Local outlier factor (LOF) (Breunig et al., 2000).”
Leys, C., Ley, C., Klein, O., Bernard, P., & Licata, L. (2013) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2013.03.013 · Cited by 4,147 · cited 1×
“Density-Based Methods OD methods include Gaussian distribution (Altman & Bland, 2005; Leys et al., 2013), Mahalanobis distance (De Maesschalck et al., 2000), Gaussian mixtures (Yang et al., 2009), and Local outlier factor (LOF) (Breunig et al., 2000).”
Places: A 10 Million Image Database for Scene Recognition
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., & Torralba, A. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2723009 · Cited by 4,042
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
Lakshminarayanan, B., Pritzel, A., & Blundell, C. (2016) · arXiv · doi:10.48550/arxiv.1612.01474 · Cited by 3,666 · cited 4×
“Recent works attempt several less principled approximations including MC-dropout (Gal & Ghahramani, 2016) and deep ensembles (Dietterich, 2000; Lakshminarayanan et al., 2017; Maddox et al., 2019) for faster and better estimates of uncertainty.”
InfluentialA Survey of Outlier Detection Methodologies
Hodge, V., & Austin, J. (2004) · Artificial Intelligence Review · doi:10.1007/s10462-004-4304-y · Cited by 3,596
A theory of learning from different domains
Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., & Vaughan, J. W. (2010) · Machine Learning · doi:10.1007/s10994-009-5152-4 · Cited by 3,515 · cited 1×
“Even worse, these sub-topics, which are supposed to be compared and learned from each other, are developing in isolation.”
Learning to Prompt for Vision-Language Models
Zhou, K., Yang, J., Loy, C. C., & Liu, Z. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01653-1 · Cited by 3,304 · cited 20×
“As for the OOD dataset, we construct near-OOD with CIFAR-100 (Krizhevsky et al., 2009) and TinyImageNet (Krizhevsky et al., 2012).”
InfluentialSelf-citationDeep neural networks are easily fooled: High confidence predictions for unrecognizable images
Nguyen, A., Yosinski, J., & Clune, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298640 · Cited by 3,258
Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images
Nguyen, A., Yosinski, J., & Clune, J. (2014) · arXiv · doi:10.48550/arxiv.1412.1897 · Cited by 3,258
Nonlinear principal component analysis using autoassociative neural networks
Kramer, M. A. (1991) · AIChE Journal · doi:10.1002/aic.690370209 · Cited by 3,037
Deep Learning for Anomaly Detection: A Review
Pang, G., Shen, C., Cao, L., & Hengel, A. V. D. (2022) · ACM Computing Surveys · doi:10.1145/3439950 · Cited by 2,837
Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels
Han, B., Yao, Q., Yu, X., Niu, G., Xu, M., Hu, W., Tsang, I., & Sugiyama, M. (2018) · arXiv · doi:10.48550/arxiv.1804.06872 · Cited by 2,806
Mixture Densities, Maximum Likelihood and the EM Algorithm
Redner, R. A., & Walker, H. F. (1984) · SIAM Review · doi:10.1137/1026034 · Cited by 2,693
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Self-citationA Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
Lee, K., Lee, K., Lee, H., & Shin, J. (2018) · arXiv · doi:10.48550/arxiv.1807.03888 · Cited by 2,450
International Journal of Computer Vision
(2013) · International Journal of Computer Vision · doi:10.1007/11263.1573-1405 · Cited by 2,340
De Maesschalck, R., Jouan-Rimbaud, D., & Massart, D. L. (2000) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/s0169-7439(99)00047-7 · Cited by 2,325
Deep visual domain adaptation: A survey
Wang, M., & Deng, W. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.05.083 · Cited by 2,209
MVTec AD — A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection
Bergmann, P., Fauser, M., Sattlegger, D., & Steger, C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00982 · Cited by 2,203 · cited 5×
“Some AD Methods are Good at Far-OOD Although anomaly detection (AD) methods were originally designed to detect pixel-level appearance differences on the MVTec-AD dataset, they have shown potency in far-OOD detection, such as with DRAEM and CutPaste.”
InfluentialThreat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
Akhtar, N., & Mian, A. (2018) · IEEE Access · doi:10.1109/access.2018.2807385 · Cited by 2,021
A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
Hendrycks, D., & Gimpel, K. (2016) · arXiv · doi:10.48550/arxiv.1610.02136 · Cited by 1,968
Jia, M., Tang, L., Chen, B.-C., Cardie, C., Belongie, S., Hariharan, B., & Lim, S.-N. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19827-4_41 · Cited by 1,965
Gong, D., Liu, L., Le, V., Saha, B., Mansour, M. R., Venkatesh, S., & Van Den Hengel, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00179 · Cited by 1,930
80 Million Tiny Images: A Large Data Set for Nonparametric Object and Scene Recognition
Torralba, A., Fergus, R., & Freeman, W. T. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.128 · Cited by 1,921
Bayesian Learning via Stochastic Gradient Langevin Dynamics
Welling, M., & Teh, Y. W. (2012) · Cited by 1,860
Curiosity-Driven Exploration by Self-Supervised Prediction
Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.70 · Cited by 1,815 · cited 2×
“In fact, novelty detection is also known as “novel class detection” (Markou & Singh, 2003a,b) (Idrees et al., 2018; Diehl & Hampshire, 2002), planetaryexploration(Kerner et al.,2019) and incremental learning (Al-Behadili et al., 2015; Pathak et al., 2017).”
Towards Open Set Deep Networks
Bendale, A., & Boult, T. E. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.173 · Cited by 1,799
Conditional Prompt Learning for Vision-Language Models
Zhou, K., Yang, J., Loy, C. C., & Liu, Z. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01631 · Cited by 1,796
Self-citationRuff, L., Vandermeulen, R. A., Görnitz, N., Deecke, L., Siddiqui, S. A., Binder, A., Müller, E., & Kloft, M. (2018) · International Conference on Machine Learning · Cited by 1,719
Pimentel, M. A. F., Clifton, D. A., Clifton, L., & Tarassenko, L. (2014) · Signal Processing · doi:10.1016/j.sigpro.2013.12.026 · Cited by 1,649 · cited 1×
“…al., 2019a), anomaly detection (AD) (Ruff et al., 2021; Pang et al., 2020; Bulusu et al., 2020; Chalapathy & Chawla, 2019), novelty detection (ND) (Pimentel et al., 2014; Miljkovi´c, 2010; Markou & Singh, 2003a; Markou & Singh, 2003b), and open set recognition (OSR) (Boult et al., 2019; Huang &…”
Scheirer, W. J., de Rezende Rocha, A., Sapkota, A., & Boult, T. E. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.256 · Cited by 1,573
Dataset Shift in Machine Learning
(2008) · doi:10.7551/mitpress/9780262170055.001.0001 · Cited by 1,553 · cited 1×
“Examples of covariate distribution shift on P (X) include adversarial examples [31], [32], domain shift [33], and style changes [34].”
Future Frame Prediction for Anomaly Detection - A New Baseline
Liu, W., Luo, W., Lian, D., & Gao, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00684 · Cited by 1,530
Graph based anomaly detection and description: a survey
Akoglu, L., Tong, H., & Koutra, D. (2015) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-014-0365-y · Cited by 1,504 · cited 1×
“…& Borgwardt, 2013; Orair et al., 2010), DBSCAN clustering (Ester et al., 1996), and graph-based methods (Hautamaki et al., 2004; Muhlenbach et al., 2004; Liu et al., 2010; Akoglu et al., 2015; Noble & Cook, 2003; Kou et al., 2007; Mingqiang et al., 2012; Wu et al., 2021; Yang et al., 2020a).”
Novelty detection: a review—part 1: statistical approaches
Markou, M., & Singh, S. (2003) · Signal Processing · doi:10.1016/j.sigpro.2003.07.018 · Cited by 1,432
Domain Generalization: A Survey
Zhou, K., Liu, Z., Qiao, Y., Xiang, T., & Loy, C. C. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3195549 · Cited by 1,315
Self-citationConcrete Problems in AI Safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016) · arXiv · doi:10.48550/arxiv.1606.06565 · Cited by 1,314 · cited 2×
“A trustworthy visual recognition system should not only produce accurate predictions on known context, but also detect unknown examples and reject them (or hand them over to human users for safe handling) [1], [2], [3], [4], [5], [6], [7], [8].”
(2015) · Cited by 1,279
Variational Inference with Normalizing Flows
Rezende, D. J., & Mohamed, S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.05770 · Cited by 1,253
Deep Learning for Anomaly Detection: A Survey
Chalapathy, R., & Chawla, S. (2019) · arXiv · doi:10.48550/arxiv.1901.03407 · Cited by 1,240 · cited 4×
“Remark: This Survey Highlights OOD Detection Methods Notice the existence of comprehensive surveys on AD, ND, and OD methodologies in recent years [20], [21], [22], [23],”
InfluentialDeeper, Broader and Artier Domain Generalization
Li, D., Yang, Y., Song, Y.-Z., & Hospedales, T. M. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.591 · Cited by 1,237
Normalizing Flows: An Introduction and Review of Current Methods
Kobyzev, I., Prince, S. J. D., & Brubaker, M. A. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2992934 · Cited by 1,235
CLIP-Adapter: Better Vision-Language Models with Feature Adapters
Gao, P., Geng, S., Zhang, R., Ma, T., Fang, R., Zhang, Y., Li, H., & Qiao, Y. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-023-01891-x · Cited by 1,197
Large-Scale Long-Tailed Recognition in an Open World
Liu, Z., Miao, Z., Zhan, X., Wang, J., Gong, B., & Yu, S. X. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00264 · Cited by 1,126
Self-citationMarkov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference, Second Edition
Gamerman, D., & Lopes, H. F. (2006) · doi:10.1201/9781482296426 · Cited by 1,097
IEEE Transactions on Circuits and Systems for Video Technology
(2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.76 · Cited by 1,074
Outlier detection for high dimensional data
Aggarwal, C. C., & Yu, P. S. (2001) · Proceedings of the 2001 ACM SIGMOD international conference on Management of data · doi:10.1145/375663.375668 · Cited by 1,029
Recent Advances in Open Set Recognition: A Survey
Geng, C., Huang, S.-J., & Chen, S. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2981604 · Cited by 1,006
Deep Anomaly Detection with Outlier Exposure
Hendrycks, D., Mazeika, M., & Dietterich, T. (2018) · arXiv · doi:10.48550/arxiv.1812.04606 · Cited by 932
On optimum recognition error and reject tradeoff
Chow, C. (1970) · IEEE Transactions on Information Theory · doi:10.1109/tit.1970.1054406 · Cited by 929
Learning Memory-Guided Normality for Anomaly Detection
Park, H., Noh, J., & Ham, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01438 · Cited by 929
A Unifying Review of Deep and Shallow Anomaly Detection
Ruff, L., Kauffmann, J. R., Vandermeulen, R. A., Montavon, G., Samek, W., Kloft, M., Dietterich, T. G., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3052449 · Cited by 914
Face Spoof Detection With Image Distortion Analysis
Wen, D., Han, H., & Jain, A. K. (2015) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2015.2400395 · Cited by 905
Meta-Learning in Neural Networks: A Survey
Hospedales, T., Antoniou, A., Micaelli, P., & Storkey, A. (2020) · arXiv · doi:10.48550/arxiv.2004.05439 · Cited by 886
Meta-Learning in Neural Networks: A Survey
Hospedales, T. M., Antoniou, A., Micaelli, P., & Storkey, A. J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3079209 · Cited by 886
Standard deviations and standard errors
Altman, D. G., & Bland, J. M. (2005) · BMJ · doi:10.1136/bmj.331.7521.903 · Cited by 878
Novelty detection: a review—part 2:
Markou, M., & Singh, S. (2003) · Signal Processing · doi:10.1016/j.sigpro.2003.07.019 · Cited by 817
Adversarially Learned One-Class Classifier for Novelty Detection
Sabokrou, M., Khalooei, M., Fathy, M., & Adeli, E. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00356 · Cited by 765
Extreme value theory in engineering
Castillo, E. (1988) · Cited by 764
(2005) · doi:10.5555/1046920.1088696 · Cited by 756
Likelihood Ratios for Out-of-Distribution DetectionPreprint
Ren, J., Liu, P. J., Fertig, E., Snoek, J., Poplin, R. E., DePristo, M. A., Dillon, J. V., & Lakshminarayanan, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02845 · Cited by 727
A tutorial on conformal prediction
Shafer, G., & Vovk, V. (2007) · arXiv · doi:10.48550/arxiv.0706.3188 · Cited by 655
Learning from positive and unlabeled data: a survey
Bekker, J., & Davis, J. (2020) · Machine Learning · doi:10.1007/s10994-020-05877-5 · Cited by 651 · cited 2×
“PU learning (Zhang & Zuo, 2008; Bekker & Davis, 2020; Jaskie & Spanias, 2019; Ruff et al., 2020) is proposed for the semi-supervised AD setting where unlabeled data is available in addition to the normal data.”
High-Frequency Component Helps Explain the Generalization of Convolutional Neural Networks
Wang, H., Wu, X., Huang, Z., & Xing, E. P. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00871 · Cited by 638
Probability Models for Open Set Recognition
Scheirer, W. J., Jain, L. P., & Boult, T. E. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2321392 · Cited by 630 · cited 5×
“CAP explicitly models the probability of class membership abating from ID points to OOD points, and EVT focuses on modeling the tail distribution with extreme high/low values.”
InfluentialEnhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
Liang, S., Li, Y., & Srikant, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02690 · Cited by 611
Self-citationBayesian methods for adaptive models
Mackay, D. (1992) · doi:10.7907/h3a1-wm07 · Cited by 610
OCGAN: One-Class Novelty Detection Using GANs With Constrained Latent Representations
Perera, P., Nallapati, R., & Xiang, B. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00301 · Cited by 608 · cited 9×
“Existing works leverage GANs to generate OOD training samples and force the model predictions to be uniform (Lee et al., 2018a), generate boundary samples in the low-density region (Vernekar et al., 2019), or similarly, high-confidence OOD samples (Sricharan et al., 2018), or using meta-learning the update sample gener…”
InfluentialSpatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain
Liu, H., Li, X., Zhou, W., Chen, Y., He, Y., Xue, H., Zhang, W., & Yu, N. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00083 · Cited by 600
Open-Set Recognition: a Good Closed-Set Classifier is All You Need?
Vaze, S., Han, K., Vedaldi, A., & Zisserman, A. (2021) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.48550/arxiv.2110.06207 · Cited by 582
Communications in statistics: theory and methods
Monari, P., Piccolo, D., Rampichini, C., & Salmaso, L. (2014) · Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) · Cited by 581
Busto, P. P., & Gall, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.88 · Cited by 564
Large Graph Construction for Scalable Semi-Supervised Learning
Liu, W., He, J., & Chang, S. (2010) · Cited by 554
Learning from Noisy Labels with Distillation
Li, Y., Yang, J., Song, Y., Cao, L., Luo, J., & Li, L.-J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.211 · Cited by 541
Self-citationHuang, X., Kroening, D., Ruan, W., Sharp, J., Sun, Y., Thamo, E., Wu, M., & Yi, X. (2020) · Computer Science Review · doi:10.1016/j.cosrev.2020.100270 · Cited by 532
Towards Open World Recognition
Bendale, A., & Boult, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298799 · Cited by 529 · cited 2×
“OCC KPCA [142] [125] IsoForest [109] DBSCAN [255] KDE [273] EnergyAD [99]”
A Survey of Zero-Shot Learning: Settings, Methods, and Applications
Wang, W., Zheng, V. W., Yu, H., & Miao, C. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3293318 · Cited by 528 · cited 3×
“Zero-shot Learning Zero-shot learning [98] has a similar goal of novelty discovery, but follows the training-testing scheme.”
Progress in Outlier Detection Techniques: A Survey
Wang, H., Bah, M. J., & Hammad, M. (2019) · IEEE Access · doi:10.1109/access.2019.2932769 · Cited by 523
WILDS: A Benchmark of in-the-Wild Distribution Shifts
Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07421 · Cited by 518
Latent Space Autoregression for Novelty Detection
Abati, D., Porrello, A., Calderara, S., & Cucchiara, R. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00057 · Cited by 508 · cited 5×
“2 can be directly adapted to OOD detection by unifying the ID data as a whole [87], [88], [89], [90], [121].”
BackgroundMethodInfluentialOut-of-Distribution Detection with Deep Nearest NeighborsPreprint
Sun, Y., Ming, Y., Zhu, X., & Li, Y. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.06507 · Cited by 506
Self-citationDetecting multivariate outliers: Use a robust variant of the Mahalanobis distance
Leys, C., Klein, O., Dominicy, Y., & Ley, C. (2018) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.09.011 · Cited by 504
Generalized ODIN: Detecting Out-of-Distribution Image Without Learning From Out-of-Distribution Data
Hsu, Y.-C., Shen, Y., Jin, H., & Kira, Z. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01096 · Cited by 499
Secure Face Unlock: Spoof Detection on Smartphones
Patel, K., Han, H., & Jain, A. K. (2016) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2016.2578288 · Cited by 491
ReAct: Out-of-distribution Detection With Rectified ActivationsPreprint
Sun, Y., Guo, C., & Li, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.12797 · Cited by 484
Self-citationOpenOOD: Benchmarking Generalized Out-Of-Distribution Detection
Yang, J., Wang, P., Zou, D., Zhou, Z., Ding, K., Peng, W., Wang, H., Chen, G., Li, B., Sun, Y., et al. (2022) · neural information processing systems · doi:10.52202/068431-2362 · Cited by 483
Self-citationOpen Set Learning with Counterfactual Images
Neal, L., Olson, M., Fern, X., Wong, W.-K., & Li, F. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01231-1_38 · Cited by 480
Classification-Reconstruction Learning for Open-Set Recognition
Yoshihashi, R., Shao, W., Kawakami, R., You, S., Iida, M., & Naemura, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00414 · Cited by 472
Generative OpenMax for Multi-Class Open Set Classification
Ge, Z., Demyanov, S., & Garnavi, R. (2017) · Procedings of the British Machine Vision Conference 2017 · doi:10.5244/c.31.42 · Cited by 472 · cited 3×
“[160] [180] [181] [153] [154] [155] [173] [183] [184]”
Anomaly Detection over Noisy Data Using Learned Probability Distributions
Eskin, E. (2000) · doi:10.7916/d8c53skf · Cited by 469
Efficient GAN-Based Anomaly Detection
Zenati, H., Foo, C.-S., Lecouat, B., Manek, G., & Chandrasekhar, V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.06222 · Cited by 460
Noble, C. C., & Cook, D. J. (2003) · Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/956750.956831 · Cited by 454
Histogram-based Outlier Score (HBOS): A fast Unsupervised Anomaly Detection Algorithm
Goldstein, M., & Dengel, A. R. (2012) · Cited by 449
Data cleaning: detecting, diagnosing, and editing data abnormalities
Van den Broeck, J., Argeseanu Cunningham, S., Eeckels, R., & Herbst, K. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020267 · Cited by 446 · cited 2×
“Application and Benchmark While mostly applied in data mining tasks [77], [78], [79], outlier detection is also used in the real-world computer vision applications such as video surveillance [80] and dataset cleaning [81], [82], [83].”
A Simple Baseline for Bayesian Uncertainty in Deep Learning
Maddox, W., Garipov, T., Izmailov, P., Vetrov, D., & Wilson, A. G. (2019) · arXiv · doi:10.48550/arxiv.1902.02476 · Cited by 429 · cited 2×
“Recent works attempt several less principled approximations including MC-dropout (Gal & Ghahramani, 2016) and deep ensembles (Dietterich, 2000; Lakshminarayanan et al., 2017; Maddox et al., 2019) for faster and better estimates of uncertainty.”
Hein, M., Andriushchenko, M., & Bitterwolf, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00013 · Cited by 428
VOS: Learning What You Don't Know by Virtual Outlier SynthesisPreprint
Du, X., Wang, Z., Cai, M., & Li, Y. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.01197 · Cited by 424
Self-citationAutoencoder-based network anomaly detection
Chen, Z., Yeo, C. K., Lee, B. S., & Lau, C. T. (2018) · 2018 Wireless Telecommunications Symposium (WTS) · doi:10.1109/wts.2018.8363930 · Cited by 412
Energy-based Out-of-distribution Detection
Liu, W., Wang, X., Owens, J. D., & Li, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.03759 · Cited by 412
Self-citationOn Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks
Thulasidasan, S., Chennupati, G., Bilmes, J., Bhattacharya, T., & Michalak, S. (2019) · doi:10.2172/1525811 · Cited by 410
Using Pre-Training Can Improve Model Robustness and Uncertainty
Hendrycks, D., Lee, K., & Mazeika, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.09960 · Cited by 408
C2AE: Class Conditioned Auto-Encoder for Open-Set Recognition
Oza, P., & Patel, V. M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00241 · Cited by 399
Deep Semi-Supervised Anomaly Detection
Ruff, L., Vandermeulen, R. A., Görnitz, N., Binder, A., Müller, E., Müller, K.-R., & Kloft, M. (2019) · arXiv · doi:10.48550/arxiv.1906.02694 · Cited by 388
Atha, D. J., & Jahanshahi, M. R. (2018) · Structural Health Monitoring · doi:10.1177/1475921717737051 · Cited by 372
Deep Anomaly Detection Using Geometric Transformations
Golan, I., & El‐Yaniv, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.10917 · Cited by 372
Outlier detection using k-nearest neighbour graph
Hautamaki, V., Karkkainen, I., & Franti, P. (2004) · Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004. · doi:10.1109/icpr.2004.1334558 · Cited by 353
Webly Supervised Learning of Convolutional Networks
Chen, X., & Gupta, A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.168 · Cited by 351
CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances
Tack, J., Mo, S., Jeong, J., & Shin, J. (2020) · arXiv · doi:10.48550/arxiv.2007.08176 · Cited by 351
Anomaly Detection in Video via Self-Supervised and Multi-Task Learning
Georgescu, M.-I., Barbalau, A., Ionescu, R. T., Shahbaz Khan, F., Popescu, M., & Shah, M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01255 · Cited by 350
Multi-class Open Set Recognition Using Probability of Inclusion
Jain, L. P., Scheirer, W. J., & Boult, T. E. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_26 · Cited by 340
Predictive Uncertainty Estimation via Prior Networks
Malinin, A., & Gales, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.10501 · Cited by 337
Deep Structured Energy Based Models for Anomaly Detection
Zhai, S., Cheng, Y., Lu, W., & Zhang, Z. (2016) · arXiv · doi:10.48550/arxiv.1605.07717 · Cited by 334 · cited 4×
“OCC KPCA [142] [125] IsoForest [109] DBSCAN [255] KDE [273] EnergyAD [99]”
InfluentialDelving into Out-Of-Distribution Detection with Vision-Language Representations
Ming, Y., Cai, Z., Gu, J., Sun, Y., Li, W., & Li, Y. (2022) · neural information processing systems · doi:10.52202/068431-2543 · Cited by 332
Self-citationDICE: Leveraging Sparsification for Out-of-Distribution Detection
Sun, Y., & Li, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20053-3_40 · Cited by 327
Self-citationRudd, E. M., Jain, L. P., Scheirer, W. J., & Boult, T. E. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2707495 · Cited by 326
Iterative Learning with Open-set Noisy Labels
Wang, Y., Liu, W., Ma, X., Bailey, J., Zha, H., Song, L., & Xia, S.-T. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00906 · Cited by 324
Classification with a Reject Option using a Hinge Loss
Bartlett, P. L., & Wegkamp, M. H. (2008) · QUT ePrints (Queensland University of Technology) · doi:10.5555/1390681.1442792 · Cited by 317
Conditional Gaussian Distribution Learning for Open Set Recognition
Sun, X., Yang, Z., Zhang, C., Ling, K.-V., & Peng, G. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01349 · Cited by 312
SELF: Learning to Filter Noisy Labels with Self-EnsemblingPreprint
Nguyen, D. T., Mummadi, C. K., Ngo, T. P. N., Nguyen, T., Beggel, L., & Brox, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.01842 · Cited by 309
Extremely Simple Activation Shaping for Out-of-Distribution DetectionPreprint
Djurisic, A., Bozanic, N., Ashok, A., & Liu, R. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.09858 · Cited by 308
Learning to Discover Novel Visual Categories via Deep Transfer Clustering
Han, K., Vedaldi, A., & Zisserman, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00849 · Cited by 303
Detecting Out-of-Distribution Examples with Gram Matrices
Sastry, C. S., & Oore, S. (2020) · International Conference on Machine Learning · Cited by 299
Pretrained Transformers Improve Out-of-Distribution Robustness
Hendrycks, D., Liu, X., Wallace, E., Dziedzic, A., Krishnan, R., & Song, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.244 · Cited by 298
ViM: Out-Of-Distribution with Virtual-logit Matching
Wang, H., Li, Z., Feng, L., & Zhang, W. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00487 · Cited by 289
Learning Discriminative Reconstructions for Unsupervised Outlier Removal
Xia, Y., Cao, X., Wen, F., Hua, G., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.177 · Cited by 285
Review Papers: Recent Developments in Nonparametric Density Estimation
Izenman, A. J. (1991) · Journal of the American Statistical Association · doi:10.1080/01621459.1991.10475021 · Cited by 281
Mitigating Neural Network Overconfidence with Logit NormalizationPreprint
Wei, H., Xie, R., Cheng, H., Feng, L., An, B., & Li, Y. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.09310 · Cited by 280
Self-citationObjections to Bayesian statistics
Gelman, A. (2008) · Bayesian Analysis · doi:10.1214/08-ba318 · Cited by 279
Nearest neighbors distance ratio open-set classifier
Mendes Júnior, P. R., de Souza, R. M., Werneck, R. D. O., Stein, B. V., Pazinato, D. V., de Almeida, W. R., Penatti, O. A. B., Torres, R. D. S., & Rocha, A. (2017) · Machine Learning · doi:10.1007/s10994-016-5610-8 · Cited by 276
Uncertainty Estimation Using a Single Deep Deterministic Neural Network
van Amersfoort, J., Smith, L., Teh, Y. W., & Gal, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.02037 · Cited by 263
On-line outlier detection and data cleaning
LIU, H. (2004) · Computers & Chemical Engineering · doi:10.1016/s0098-1354(04)00024-9 · Cited by 256
Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIP
Esmaeilpour, S., Liu, B., Robertson, E., & Shu, L. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i6.20610 · Cited by 256
Generative Probabilistic Novelty Detection with Adversarial Autoencoders
Pidhorskyi, S., Almohsen, R., Adjeroh, D. A., & Doretto, G. (2018) · arXiv · doi:10.48550/arxiv.1807.02588 · Cited by 255 · cited 9×
“4.2 can be directly adapted to OOD detection by unifying the ID data as a whole (Zong et al., 2018; Abati et al., 2019; Pidhorskyi et al., 2018; Deecke et al., 2018; Sabokrou et al., 2018).”
InfluentialLearning Placeholders for Open-Set Recognition
Zhou, D.-W., Ye, H.-J., & Zhan, D.-C. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00438 · Cited by 246
Generalized Category Discovery
Vaze, S., Hant, K., Vedaldi, A., & Zisserman, A. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00734 · Cited by 241
Learning Open Set Network with Discriminative Reciprocal Points
Chen, G., Qiao, L., Shi, Y., Peng, P., Li, J., Huang, T., Pu, S., & Tian, Y. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58580-8_30 · Cited by 239
On the Importance of Gradients for Detecting Distributional Shifts in the WildPreprint
Huang, R., Geng, A., & Li, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.00218 · Cited by 239
Self-citationCLIPN for Zero-Shot OOD Detection: Teaching CLIP to Say No
Wang, H., Li, Y., Yao, H., & Li, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00173 · Cited by 236
Self-citationAutomatic outlier detection for time series: an application to sensor data
Basu, S., & Meckesheimer, M. (2007) · Knowledge and Information Systems · doi:10.1007/s10115-006-0026-6 · Cited by 235
Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoderPreprint
Xiao, Z., Yan, Q. Y., & Amit, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.02977 · Cited by 234
Ben-Gal, I. (2006) · Data Mining and Knowledge Discovery Handbook · doi:10.1007/0-387-25465-x_7 · Cited by 229
Hybrid Models for Open Set Recognition
Zhang, H., Li, A., Guo, J. J., & Guo, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58580-8_7 · Cited by 218
Sparse Representation-Based Open Set Recognition
Zhang, H., & Patel, V. M. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2613924 · Cited by 217
OpenGAN: Open-Set Recognition via Open Data Generation
Kong, S., & Ramanan, D. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00085 · Cited by 216
PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures
Hendrycks, D., Zou, A., Mazeika, M., Tang, L., Li, B., Song, D. X., & Steinhardt, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01628 · Cited by 214
GEN: Pushing the Limits of Softmax-Based Out-of-Distribution Detection
Liu, X., Lochman, Y., & Zach, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02293 · Cited by 211
Ngiam, J., Chen, Z., Koh, P. W., & Ng, A. Y. (2011) · International Conference on Machine Learning · Cited by 204
Image Anomaly Detection with Generative Adversarial Networks
Deecke, L., Vandermeulen, R., Ruff, L., Mandt, S., & Kloft, M. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-10925-7_1 · Cited by 203
Counterfactual Zero-Shot and Open-Set Visual Recognition
Yue, Z., Wang, T., Sun, Q., Hua, X.-S., & Zhang, H. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01515 · Cited by 200
Out-of-Distribution Detection Using an Ensemble of Self Supervised Leave-Out Classifiers
Vyas, A., Jammalamadaka, N., Zhu, X., Das, D., Kaul, B., & Willke, T. L. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01237-3_34 · Cited by 199
Unsupervised Clustering Approach for Network Anomaly Detection
Syarif, I., Prugel-Bennett, A., & Wills, G. (2012) · Communications in Computer and Information Science · doi:10.1007/978-3-642-30507-8_13 · Cited by 195
Support Vector Machines with Embedded Reject Option
Fumera, G., & Roli, F. (2002) · Lecture notes in computer science · doi:10.1007/3-540-45665-1_6 · Cited by 194
Generative-Discriminative Feature Representations for Open-Set Recognition
Perera, P., Morariu, V. I., Jain, R., Manjunatha, V., Wigington, C., Ordonez, V., & Patel, V. M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01183 · Cited by 193
Anomaly Detection Using Self-Organizing Maps-Based K-Nearest Neighbor Algorithm
Tian, J., Azarian, M. H., & PECHT, M. G. (2014) · PHM Society European Conference · doi:10.36001/phme.2014.v2i1.1554 · Cited by 190
OPTIMOL: Automatic Online Picture Collection via Incremental Model Learning
Li, L.-J., & Fei-Fei, L. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0265-6 · Cited by 187
Anomalous Example Detection in Deep Learning: A Survey
Bulusu, S., Kailkhura, B., Li, B., Varshney, P. K., & Song, D. (2020) · IEEE Access · doi:10.1109/access.2020.3010274 · Cited by 186
Bayesian Methods: General Background
Jaynes, E. T. (1986) · Maximum Entropy and Bayesian Methods in Applied Statistics · doi:10.1017/cbo9780511569678.003 · Cited by 185
Chen, G., Peng, P., Ma, L., Li, J., Du, L., & Tian, Y. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00051 · Cited by 183
Learning Semantic Context from Normal Samples for Unsupervised Anomaly Detection
Yan, X., Zhang, H., Xu, X., Hu, X., & Heng, P.-A. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i4.16420 · Cited by 177
Histogram-based traffic anomaly detection
Kind, A., Stoecklin, M., & Dimitropoulos, X. (2009) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2009.090604 · Cited by 176
Learning and the Unknown: Surveying Steps toward Open World Recognition
Boult, T. E., Cruz, S. S., Dhamija, A. R., Günther, M., Henrydoss, J., & Scheirer, W. J. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019801 · Cited by 171
Few-Shot Open-Set Recognition Using Meta-Learning
Liu, B., Kang, H., Li, H., Hua, G., & Vasconcelos, N. M. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00882 · Cited by 166
Semantically Coherent Out-of-Distribution Detection
Yang, J., Wang, H., Feng, L., Yan, X., Zheng, H., Zhang, W., & Liu, Z. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00819 · Cited by 166
Self-citationAnomaly Detection Using Local Kernel Density Estimation and Context-Based Regression
Hu, W., Gao, J. T., Li, B., Wu, O., Du, J., & Maybank, S. J. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2882404 · Cited by 164
MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space
Huang, R., & Li, Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00860 · Cited by 164
Self-citationIdentifying and Handling Mislabelled Instances
Muhlenbach, F., Lallich, S., & Zighed, A. D. (2003) · Journal of Intelligent Information Systems · doi:10.1023/a:1025832930864 · Cited by 163
Outlier exposure with confidence control for out-of-distribution detection
Papadopoulos, A.-A., Rajati, M. R., Shaikh, N., & Wang, J. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.02.007 · Cited by 155
Open-world Machine Learning: Applications, Challenges, and Opportunities
Parmar, J., Chouhan, S., Raychoudhury, V., & Rathore, S. (2023) · ACM Computing Surveys · doi:10.1145/3561381 · Cited by 154
Open Domain Generalization with Domain-Augmented Meta-Learning
Shu, Y., Cao, Z., Wang, C., Wang, J., & Long, M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00950 · Cited by 151
Explorations of the mean field theory learning algorithm
Peterson, C., & Hartman, E. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90045-2 · Cited by 148
Rethinking Reconstruction Autoencoder-Based Out-of-Distribution Detection
Zhou, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00723 · Cited by 148
In or Out? Fixing ImageNet Out-of-Distribution Detection EvaluationPreprint
Bitterwolf, J., Müller, M., & Hein, M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.00826 · Cited by 148
Unknown-Aware Object Detection: Learning What You Don't Know from Videos in the Wild
Du, X., Wang, X., Gozum, G., & Li, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01331 · Cited by 147
Self-citationSelf-Supervised Learning for Generalizable Out-of-Distribution Detection
Mohseni, S., Pitale, M., Yadawa, J., & Wang, Z. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5966 · Cited by 146
Research works printed but not linked: 52
- Angelopoulos, A. N., & Bates, S. (2021). A gentle introduction to conformal prediction and distribution-free uncertainty quantification, arXiv preprint arXiv:2107.07511 · arXiv
- Cao, K., Brbic, M., & Leskovec, J. (2021). Open-world semi-supervised learning, arXiv preprint arXiv:2102.03526 · arXiv
- Chen, J., Li, Y., Wu, X., Liang, Y., & Jha, S. (2020b). Robust out-of-distribution detection for neural networks, arXiv preprint arXiv:2003.09711 · arXiv
- Choi, H., Jang, E., & Alemi, A. A. (2018). Waic, but why? generative ensembles for robust anomaly detection, arXiv preprint arXiv:1810.01392 · arXiv
- Denouden, T., Salay, R., Czarnecki, K., Abdelzad, V., Phan, B., & Vernekar, S. (2018). Improving reconstruction autoencoder out-of-distribution detection with Mahalanobis distance, arXiv preprint arXiv:1812.02765 · arXiv
- DeVries, T., & Taylor, G. W. (2017). Improved regularization of convolutional neural networks with cutout, arXiv preprint arXiv:1708.04552 · arXiv
- DeVries, T., & Taylor, G. W. (2018). Learning confidence for out-of-distribution detection in neural networks, arXiv preprint arXiv:1802.04865 · arXiv
- Dolhansky, B., Howes, R., Pflaum, B., Baram, N., & Ferrer, C. C. (2019). The deepfake detection challenge (dfdc) preview dataset, arXiv preprint arXiv:1910.08854 · arXiv
- Dong, J., Gao, Y., Zhou, H., Cen, J., Yao, Y., Yoon, S., & Sun, P. D. (2023). Towards few-shot out-of-distribution detection, arXiv preprint arXiv:2311.12076 · arXiv
- Dong, X., Guo, J., Li, A., Ting, W.-T., Liu, C., & Kung, H. (2022a). Neural mean discrepancy for efficient out-of-distribution detection. In CVPR. · DOI
- Drummond, N., & Shearer, R. (2006). The open world assumption. In eSI workshop.
- Fang, Z., Li, Y., Lu, J., Dong, J., Han, B., & Liu, F. (2022). Is out-of-distribution detection learnable? In NeurIPS.
- Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative adversarial nets. In NIPS.
- Han, X., Chen, X., & Liu, L.-P. (2020). Gan ensemble for anomaly detection, arXiv preprint arXiv:2012.07988 · arXiv
- Hendrycks, D., Basart, S., Mazeika, M., Mostajabi, M., Steinhardt, J., & Song, D. (2022a) Scaling out-of-distribution detection for real-world settings. In ICML.
- Hendrycks, D., & Mazeika, M. (2022). X-risk analysis for AI research. arXiv preprint, arXiv:2206.05862 · arXiv
- Hendrycks, D., Mu, N., Cubuk, E. D., Zoph, B., Gilmer, J., & Lakshminarayanan, B. (2019c). Augmix: A simple data processing method to improve robustness and uncertainty. arXiv preprint arXiv:1912.02781 · arXiv
- Huang, H., Li, Z., Wang, L., Chen, S., Dong, B., & Zhou, X. (2020a). Feature space singularity for out-of-distribution detection, arXiv preprint arXiv:2011.14654 · arXiv
- Igoe, C., Chung, Y., Char, I., & Schneider, J. (2022). How useful are gradients for ood detection really? arXiv preprint arXiv:2205.10439 · arXiv
- Jang, J., & Kim, C. O. (2020). One-vs-rest network-based deep probability model for open set recognition, arXiv preprint arXiv:2004.08067 · arXiv
- Jiang, D., Sun, S., & Yu, Y. (2021a). Revisiting flow generative models for out-of-distribution detection. In International conference on learning representations.
- Jiang, L., Guo, Z., Wu, W., Liu, Z., Liu, Z., Loy, C.C., Yang, S., Xiong, Y., Xia, W., Chen, B., Zhuang, P., Li, S., Chen, S., Yao, T., Ding, S., Li, J., Huang, F., Cao, L., Ji, R., Lu, C., & Tan, G. (2021c). DeeperForensics Challenge 2020 on real-world face forgery detection: Methods and results, arXiv preprint arXiv:2102.09471 · arXiv
- Jiang, X., Liu, F., Fang, Z., Chen, H., Liu, T., Zheng, F., & Han, B. (2023b). Detecting out-of-distribution data through in-distribution class prior. In International conference on machine learning (pp. 15067–15088). PMLR.
- Kingma, D. P., & Dhariwal, P. (2018). Glow: Generative flow with invertible 1x1 convolutions. In NeurIPS.
- Kirichenko, P., Izmailov, P., & Wilson, A. G. (2020). Why normalizing flows fail to detect out-of-distribution data. in NeurIPS.
- Krizhevsky, A., Nair, V., & Hinton, G. (2009). Cifar-10 and cifar-100 datasets (vol. 6, (no. 1), p. 1). https://www.cs.toronto.edu/kriz/cifar.html
- Li, B., Zhang, Y., Chen, L., Wang, J., Yang, J., & Liu, Z. (2023a). Otter: A multi-modal model with in-context instruction tuning, arXiv preprintarXiv:2305.03726
- Liu, H., Li, C., Wu, Q., & Lee, Y. J. (2023). Visual instruction tuning, arXiv preprint arXiv:2304.08485 · arXiv
- Loureiro, A., Torgo, L., & Soares, C. (2004). Outlier detection using clustering methods: A data cleaning application. In Proceedings of KDNet symposium on knowledge-based systems.
- Lu, F., Zhu, K., Zheng, K., Zhai, W., & Cao, Y. (2023). Likelihood-aware semantic alignment for full-spectrum out-of-distribution detection, arXiv preprint arXiv:2312.01732 · arXiv
- Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2018). Towards deep learning models resistant to adversarial attacks, ICLR.
- Meinke, A., & Hein, M. (2019). Towards neural networks that provably know when they don’t know, arXiv preprint arXiv:1909.12180 · arXiv
- Miljković, D. (2010). Review of novelty detection methods. In MIPRO.
- Ming, Y., Sun, Y., Dia, O., & Li, Y. (2023). Cider: Exploiting hyperspherical embeddings for out-of-distribution detection. In ICLR.
- Miyai, A., Yang, J., Zhang, J., Ming, Y., Yu, Q., Irie, G., Li, Y., Li, H., Liu, Z., & Aizawa, K. (2024). Unsolvable problem detection: Evaluating trustworthiness of vision language models. arXiv preprint, arXiv:2403.20331 · arXiv
- Miyai, A., Yu, Q., Irie, G., & Aizawa, K. (2023a). Can pre-trained networks detect familiar out-of-distribution data? arXiv preprint arXiv:2310.00847 · arXiv
- Miyai, A., Yu, Q., Irie, G., & Aizawa, K. (2023b). Locoop: Few-shot out-of-distribution detection via prompt learning, arXiv preprint arXiv:2306.01293 · arXiv
- Mohseni, S., Wang, H., Yu, Z., Xiao, C., Wang, Z., & Yadawa, J. (2021). Practical machine learning safety: A survey and primer. arXiv preprint, arXiv:2106.04823 · arXiv
- Münz, G., Li, S., & Carle, G. (2007). Traffic anomaly detection using k-means clustering. In GI/ITG workshop MMBnet.
- Nalisnick, E., Matsukawa, A., Teh, Y. W., Gorur, D., & Lakshminarayanan, B. (2018). Do deep generative models know what they don’t know? In NeurIPS.
- Nguyen, V. D. (2022). Out-of-distribution detection for lidar-based 3d object detection, Master’s thesis, University of Waterloo.
- Ren, J., Fort, S., Liu, J., Roy, A. G., Padhy, S., & Lakshminarayanan, B. (2021). A simple fix to Mahalanobis distance for improving near-ood detection, arXiv preprint arXiv:2106.09022 · arXiv
- Sun, X., Ding, H., Zhang, C., Lin, G., & Ling, K.-V. (2021a). M2iosr: Maximal mutual information open set recognition, arXiv preprint arXiv:2108.02373 · arXiv
- Van Oord, A., Kalchbrenner, N., & Kavukcuoglu, K. (2016). Pixel recurrent neural networks. In ICML.
- Vinyals, O., Ewalds, T., Bartunov, S., Georgiev, P., Vezhnevets, A. S., Yeo, M., Makhzani, A., Küttler, H., Agapiou, J., Schrittwieser, J., & Quan, J. (2017). Starcraft II: A new challenge for reinforcement learning, arXiv preprint arXiv:1708.04782 · arXiv
- Wang, H., Liu, W., Bocchieri, A., & Li, Y. (2021). Can multi-label classification networks know what they don’t know? NeurIPS, 34, 29074–29087.
- Wenzel, F., Roth, K., Veeling, B. S., Światkowski, J., Tran, L., Mandt, S., Snoek, J., Salimans, T., Jenatton, R., & Nowozin, S. (2020). How good is the Bayes posterior in deep neural networks really? In ICML.
- Yang, J., Zhou, K., & Liu, Z. (2022b). Full-spectrum out-of-distribution detection, arXiv preprint arXiv:2204.05306 · arXiv
- Yang, Z., Li, L., Lin, K., Wang, J., Lin, C.-C., Liu, Z., & Wang, L. (2023). The dawn of lmms Preliminary explorations with gpt-4v (ision). arXiv preprint, arXiv:2309.17421 · arXiv
- Zhang, J., Fu, Q., Chen, X., Du, L., Li, Z., Wang, G., Han, S., & Zhang, D. (2023a). Out-of-distribution detection based on in-distribution data patterns memorization with modern Hopfield energy. In ICLR.
- Zhang, J., Yang, J., Wang, P., Wang, H., Lin, Y., Zhang, H., Sun, Y., Du, X., Zhou, K., Zhang, W., Li, Y., Liu, Z., Chen, Y., & Li, H. (2023c). Openood v1.5: Enhanced benchmark for out-of-distribution detection. arXiv preprint, arXiv:2306.09301 · arXiv
- Zong, B., Song, Q., Min, M. R., Cheng, W., Lumezanu, C., Cho, D., & Chen, H. (2018). Deep autoencoding Gaussian mixture model for unsupervised anomaly detection. In ICLR.
Cited by
Domain Generalization: A Survey
Zhou, K., Liu, Z., Qiao, Y., Xiang, T., & Loy, C. C. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3195549 · Cited by 1,315
Self-citationTransformers in medical imaging: A survey
Shamshad, F., Khan, S., Zamir, S. W., Khan, M. H., Hayat, M., Khan, F. S., & Fu, H. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102802 · Cited by 1,296
Generalizing to unseen domains: A survey on domain generalization
Wang, J., Lan, C., Liu, C., Ouyang, Y., Qin, T., Lu, W., Chen, Y., Zeng, W., & Yu, P. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3178128 · Cited by 1,046
Díaz-Rodríguez, N., Del Ser, J., Coeckelbergh, M., López de Prado, M., Herrera-Viedma, E., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101896 · Cited by 800
ReAct: Out-of-distribution Detection With Rectified ActivationsPreprint
Sun, Y., Guo, C., & Li, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.12797 · Cited by 484
Self-citation
Metadata
- Authors
- Yang, J., Zhou, K., Li, Y., & Liu, Z.
- Affiliations
- Nanyang Technological University (SG)
- University of Wisconsin–Madison (US)
- Corresponding author
- Yang, Jingkang
- Zhou, Kaiyang
- Li, Yixuan
- Liu, Ziwei
- Venue
- International Journal of Computer Vision
- Published
- 2024-06-23
- Volume
- 132
- Issue
- 12
- Pages
- 5635-5662
- ISSN
- 0920-5691 (ISSN-L)
- 1573-1405
- OpenAlex
- W4399929809
- Cited by
- 884
- References
- 319
- Citations by year
- 2026: 217 · 2025: 283 · 2024: 173
- FWCI
- 99.11
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- International Journal of Computer Vision (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- DR-NTU (Nanyang Technological University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Wisconsin Alumni Research Foundation
- Ministry of Education - Singapore (AcRF‐Tier2, MOET2EP20221-0012, Tier 2, RIE2020)
- Ministry of Education, India
- Division of Graduate Education
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- out-of-distribution detection, anomaly detection, novelty detection, open-set recognition, outlier detection, density-based methods, distance-based methods, autonomous driving safety, unified framework
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.