CoChat
Sign Up
Cited by 5Anomaly Detection Techniques and ApplicationsAdversarial Robustness in Machine LearningAdvanced Neural Network Applications

Detecting semantic anomalies

Open access
Ahmed, Faruk, Courville, Aaron·2020·Proceedings of the AAAI Conference on Artificial Intelligence
DOI 10.1609/aaai.v34i04.5712
Open this paper in CoChat — free account

Abstract

We critically appraise the recent interest in out-of-distribution (OOD) detection and question the practical relevance of existing benchmarks. While the currently prevalent trend is to consider different datasets as OOD, we argue that out-distributions of practical interest are ones where the distinction is semantic in nature for a specified context, and that evaluative tasks should reflect this more closely. Assuming a context of object recognition, we recommend a set of benchmarks, motivated by practical applications. We make progress on these benchmarks by exploring a multi-task learning based approach, showing that auxiliary objectives for improved semantic awareness result in improved semantic anomaly detection, with accompanying generalization benefits.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1609/aaai.v34i04.5712

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Semantic AD studies the many-versus-one setting, which treats 9 classes as normal and the remaining class as anomalous.

    “Semantic AD studies the many-versus-one setting, which treats 9 classes as normal and the remaining class as anomalous.”

    From A Unified Model for Multi-Class Anomaly Detection (2022) · cites this paper as [1]

  • Semantic AD treats one class as anomalous and the remaining classes as normal.

    “Semantic AD treats one class as anomalous and the remaining classes as normal.”

    From A Unified Model for Multi-Class Anomaly Detection (2022) · cites this paper as [1]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

References 51 · Cited by 5

Research works printed but not linked: 35

  1. [3][3] Michael Alcorn, Qi Li, Zhitao Gong, Chengfei Wang, Long Mai, Wei shinn Ku, and Anh Nguyen. Strike (with) a pose: Neural networks are easily fooled by strange poses of familiar objects. CVPR , 2019. -
  2. [4][4] Sara Beery, Grant Van Horn, and Pietro Perona. Recognition in terra incognita. CoRR , 2018. -
  3. [5][5] Dario Amodei, Chris Olah, Jacob Steinhardt, Paul F. Christiano, John Schulman, and Dan Mané. Concrete problems in AI safety. CoRR , abs/1606.06565, 2016. - · arXiv
  4. [6][6] Daniel Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-ofdistribution examples in neural networks. ICLR , 2017. -
  5. [7][7] Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing the reliability of out-of-distribution detection in neural networks. ICLR , 2018. -
  6. [10][10] Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. Do deep generative models know what they don't know? ICLR , 2019. -
  7. [11][11] Jerry A. Fodor and Zenon W. Pylyshyn. Connectionism and cognitive architecture: A critical analysis. Cognition , 28(1):3 - 71, 1988. -
  8. [13][13] Peter Fedor, Jaromír Vanhara, Josef Havel, Igor Malenovsky, and Ian Spellerberg. Artificial intelligence in pest insect monitoring. Systematic Entomology , 34(2):398-400, 2009. -
  9. [14][14] Marco Willi, Ross T. Pitman, Anabelle W. Cardoso, Christina Locke, Alexandra Swanson, Amy Boyer, Marten Veldthuis, and Lucy Fortson. Identifying animal species in camera trap images using deep learning and citizen science. Methods in Ecology and Evolution , 10(1):80-91, 2019. -
  10. [15][15] Jose Carranza-Rojas, Herve Goeau, Pierre Bonnet, Erick Mata-Montero, and Alexis Joly. Going deeper in the automated identification of herbarium specimens. BMC Evolutionary Biology , 17(1):181, Aug 2017. -
  11. [16][16] Joseph Paul Cohen, Paul Bertin, and Vincent Frappier. Chester: A web delivered locally computed chest x-ray disease prediction system. CoRR , abs/1901.11210, 2019. - · arXiv
  12. [17][17] Bartosz Zieli´ nski, Anna Plichta, Krzysztof Misztal, Przemysław Spurek, Monika BrzychczyWłoch, and Dorota Ocho´ nska. Deep learning approach to bacterial colony classification. PLoS One , 12(9), 2017. -
  13. [19][19] Houssam Zenati, Chuan Sheng Foo, Bruno Lecouat, Gaurav Manek, and Vijay Ramaseshan Chandrasekhar. Efficient gan-based anomaly detection. CoRR , abs/1802.06222, 2018. - · arXiv
  14. [22][22] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. IJCV , 2015. -
  15. [23][23] Tom Fawcett. An introduction to roc analysis. Pattern Recognition Letters , 27(8):861 - 874, 2006. -
  16. [24][24] Jesse Davis and Mark Goadrich. The relationship between precision-recall and roc curves. pages 233-240, 2006. -
  17. [25][25] Anand Avati, Kenneth Jung, Stephanie Harman, Lance Downing, Andrew Ng, and Nigam H. Shah. Improving palliative care with deep learning. BMC Medical Informatics and Decision Making , 18(4):122, Dec 2018. -
  18. [26][26] Kendrick Boyd, Kevin H. Eng, and C. David Page. Area under the precision-recall curve: Point estimates and confidence intervals. ECML-PKDD , 2013. -
  19. [27][27] Abhijit Bendale and Terrance E Boult. Towards open set deep networks. ICCV , 2016. -
  20. [28][28] Akshay Raj Dhamija, Manuel Günther, and Terrance Boult. Reducing network agnostophobia. NIPS , 2018. -
  21. [29][29] Terrance DeVries and Graham W. Taylor. Learning confidence for out-of-distribution detection in neural networks. arXiv preprint arXiv:1802.04865 , 2018. - · arXiv
  22. [30][30] Daniel Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. ICLR , 2019. -
  23. [31][31] Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training confidence-calibrated classifiers for detecting out-of-distribution samples. ICLR , 2018. -
  24. [33][33] Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representations. ICML , 2013. -
  25. [34][34] Rich Caruana. Multitask learning: A knowledge-based source of inductive bias. In ICML , 1993. -
  26. [35][35] Carl Doersch, Abhinav Gupta, and Alexei A Efros. Unsupervised visual representation learning by context prediction. ICCV , 2015. -
  27. [36][36] Deepak Pathak, Philipp Krähenbühl, Jeff Donahue, Trevor Darrell, and Alexei Efros. Context encoders: Feature learning by inpainting. CVPR , 2016. -
  28. [37][37] Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. ECCV , 2016. -
  29. [38][38] Richard Zhang, Phillip Isola, and Alexei A Efros. Split-brain autoencoders: Unsupervised learning by cross-channel prediction. CVPR , 2017. -
  30. [40][40] Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. ICLR , 2018. -
  31. [41][41] Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze. Deep clustering for unsupervised learning of visual features. In ECCV , 2018. -
  32. [42][42] Carl Doersch and Andrew Zisserman. Multi-task self-supervised visual learning. In ICCV , 2017. -
  33. [43][43] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC , 2016. -
  34. [44][44] Harm de Vries, Florian Strub, Jérémie Mary, Hugo Larochelle, Olivier Pietquin, and Aaron C. Courville. Modulating early visual processing by language. NIPS , 2017. -
  35. [45][45] Olivier J Hénaff, Ali Razavi, Carl Doersch, SM Eslami, and Aaron van den Oord. Data-efficient image recognition with contrastive predictive coding. arXiv preprint arXiv:1905.09272 , 2019. · arXiv

Authors
Ahmed, F., & Courville, A.
Affiliations
Université de Montréal (CA)
ORCID
Courville, Aaron
Venue
Proceedings of the AAAI Conference on Artificial Intelligence
Publisher
Association for the Advancement of Artificial Intelligence (AAAI)
Published
2020-04-03
Volume
34
Issue
04
Pages
3154-3162
ISSN
  • 2159-5399 (ISSN-L)
  • 2374-3468
DOI
10.1609/aaai.v34i04.5712
OpenAlex
W2968923568
Cited by
5
References
51
Citations by year
2021: 1 · 2020: 3 · 2019: 1
FWCI
0.38
Type
conference paper
Language
en
Open access
Licence
www.aaai.org/ (unspecified)
Locations
  • Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Canadian Institute for Advanced Research
  • Compute Canada
  • Natural Sciences and Engineering Research Council of Canada
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
semantic anomaly detection, out-of-distribution detection, object recognition, multi-task learning, semantic awareness, generalization

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review