Toward Scalable and Unified Example-Based Explanation and Outlier Detection
Open accessAbstract
When neural networks are employed for high-stakes decision-making, it is desirable that they provide explanations for their prediction in order for us to understand the features that have contributed to the decision. At the same time, it is important to flag potential outliers for in-depth verification by domain experts. In this work we propose to unify two differing aspects of explainability with outlier detection. We argue for a broader adoption of prototype-based student networks capable of providing an example-based explanation for their prediction and at the same time identify regions of similarity between the predicted sample and the examples. The examples are real prototypical cases sampled from the training set via a novel iterative prototype replacement algorithm. Furthermore, we propose to use the prototype similarity scores for identifying outliers. We compare performance in terms of the classification, explanation quality and outlier detection of our proposed network with baselines. We show that our prototype-based networks extending beyond similarity kernels deliver meaningful explanations and promising outlier detection results without compromising classification accuracy.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Nearly all deep post-model XAD techniques provide only feature-based explanations; the only exceptions are References [14,48,72], which also produce sample-based explanations.
“Second, nearly all deep post-model XAD techniques provide only feature-based explanations; the only exceptions are References [14,48,72], which also produce sample-based explanations.”
From A Survey on Explainable Anomaly Detection (2024) · cites this paper as [48]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Cortes, C., & Vapnik, V. N. (1995) · Machine Learning · doi:10.1023/a:1022627411411 · Cited by 34,565 · see citing works
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · see citing works
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works
(2017) · Cited by 11,625 · see citing works
Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2008) · 2008 Eighth IEEE International Conference on Data Mining · doi:10.1109/icdm.2008.17 · Cited by 6,601 · see citing works
An Introduction to Kernel and Nearest-Neighbor Nonparametric Regression
Altman, N. (1992) · The American Statistician · doi:10.1080/00031305.1992.10475879 · Cited by 5,721 · see citing works
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · see citing works
LOF: identifying density-based local outliers
Breunig, M. M., Kriegel, H.-P., Ng, R. T., & Sander, J. (2000) · ACM SIGMOD Record · doi:10.1145/335191.335388 · Cited by 5,207 · see citing works
Automated Flower Classification over a Large Number of Classes
Nilsback, M.-E., & Zisserman, A. (2008) · 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing · doi:10.1109/icvgip.2008.47 · Cited by 3,442 · see citing works
Ehteshami Bejnordi, B., Veta, M., Johannes van Diest, P., van Ginneken, B., Karssemeijer, N., Litjens, G., van der Laak, J. A. W. M., and the CAMELYON16 Consortium, Hermsen, M., Manson, Q. F., et al. (2017) · JAMA · doi:10.1001/jama.2017.14585 · Cited by 3,293 · see citing works
A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
Hendrycks, D., & Gimpel, K. (2016) · arXiv · doi:10.48550/arxiv.1610.02136 · Cited by 1,968 · see citing works
Gong, D., Liu, L., Le, V., Saha, B., Mansour, M. R., Venkatesh, S., & Van Den Hengel, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00179 · Cited by 1,930 · see citing works
LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the LoopPreprint
Yu, F., Seff, A., Zhang, Y., Song, S., Funkhouser, T., & Xiao, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03365 · Cited by 1,339 · see citing works
Evaluating the Visualization of What a Deep Neural Network Has Learned
Samek, W., Binder, A., Montavon, G., Lapuschkin, S., & Muller, K.-R. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2599820 · Cited by 1,164 · see citing works
Self-citationEnhancing The Reliability of Out-of-distribution Image Detection in Neural NetworksPreprint
Liang, S., Li, Y., & Srikant, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02690 · Cited by 1,117 · see citing works
Deep Anomaly Detection with Outlier Exposure
Hendrycks, D., Mazeika, M., & Dietterich, T. (2018) · arXiv · doi:10.48550/arxiv.1812.04606 · Cited by 932 · see citing works
Fast Outlier Detection in High Dimensional Spaces
Angiulli, F., & Pizzuti, C. (2002) · Lecture notes in computer science · doi:10.1007/3-540-45681-3_2 · Cited by 880 · see citing works
Layer-Wise Relevance Propagation: An Overview
Montavon, G., Binder, A., Lapuschkin, S., Samek, W., & Müller, K. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-28954-6_10 · Cited by 825 · see citing works
Self-citationExplanations based on the Missing: Towards Contrastive Explanations with Pertinent NegativesPreprint
Dhurandhar, A., Chen, P.-Y., Luss, R., Tu, C.-C., Ting, P.-S., Shanmugam, K., & Das, P. (2018) · Neural Information Processing Systems · doi:10.48550/arxiv.1802.07623 · Cited by 730 · see citing works
This Looks Like That: Deep Learning for Interpretable Image Recognition
Chen, C., Li, O., Tao, C., Barnett, A. J., Su, J., & Rudin, C. (2018) · arXiv · doi:10.48550/arxiv.1806.10574 · Cited by 702 · see citing works
Efficient Additive Kernels via Explicit Feature Maps
Vedaldi, A., & Zisserman, A. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.153 · Cited by 695 · see citing works
(2016) · neural information processing systems · Cited by 556 · see citing works
Generalized ODIN: Detecting Out-of-Distribution Image Without Learning From Out-of-Distribution Data
Hsu, Y.-C., Shen, Y., Jin, H., & Kira, Z. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01096 · Cited by 499 · see citing works
Robust Classification with Convolutional Prototype Learning
Yang, H., Zhang, X.-Y., Yin, F., & Liu, C. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00366 · Cited by 493 · see citing works
Rotation Equivariant CNNs for Digital Pathology
Veeling, B. S., Linmans, J., Winkens, J., Cohen, T., & Welling, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00934-2_24 · Cited by 476 · see citing works
Outlier Detection with Kernel Density Functions
Latecki, L. J., Lazarević, A. M., & Pokrajac, D. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-73499-4_6 · Cited by 302 · see citing works
Neural Prototype Trees for Interpretable Fine-grained Image Recognition
Nauta, M., van Bree, R., & Seifert, C. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01469 · Cited by 269 · see citing works
A baseline for detecting misclassified and out-of-distribution examples in neural networksPreprint
Hendrycks, D. (2025) · arXiv (Cornell University) · doi:10.57702/9gs720u1 · Cited by 196 · see citing works
Li, O., Liu, H., Chen, C., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.04806 · Cited by 158 · see citing works
Efficient Data Representation by Selecting Prototypes with Importance Weights
Gurumoorthy, K. S., Dhurandhar, A., Cecchi, G., & Aggarwal, C. C. (2019) · Proceedings · doi:10.1109/icdm.2019.00036 · Cited by 135 · see citing works
Statistical Analysis of Nearest Neighbor Methods for Anomaly DetectionPreprint
Gu, X., Akoglu, L., & Rinaldo, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.03813 · Cited by 74 · see citing works
Building and Interpreting Deep Similarity Models
Eberle, O., Büttner, J., Kräutli, F., Müller, K., Valleriani, M., & Montavon, G. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3020738 · Cited by 68 · see citing works
Interpretable Image Recognition with Hierarchical PrototypesPreprint
Hase, P., Chen, C., Li, O., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.10651 · Cited by 24 · see citing works
ProtoPShare: Prototype Sharing for Interpretable Image Classification and Similarity DiscoveryPreprint
Rymarczyk, D., Struski, Ł., Tabor, J., & Zieliński, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.14340 · Cited by 17 · see citing works
Similarity Kernels for Nearest Neighbor-Based Outlier Detection
Ramirez-Padron, R., Foregger, D., Manuel, J., et al. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-13062-5_16 · Cited by 8 · see citing works
Prototype based outlier detection
Kim, S., & Cho, S. (2006) · 2022 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2006.246769 · Cited by 7 · see citing works
Object-Level Context Modeling For Scene Classification with Context-CNNPreprint
Javed, S. A., & Nelakanti, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.04358 · Cited by 6 · see citing works
Flagged first, then by citations
Research works printed but not linked: 7
- [7][7] Y. Ming, P. Xu, H. Qu, and L. Ren, 'Interpretable and steerable sequence learning via prototypes,' in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , 2019, pp. 903-913. -
- [14][14] S. O. Arık and T. Pfister, 'Protoattend: Attention-based prototypical learning,' Journal of Machine Learning Research , vol. 21, pp. 1-35, 2020. -
- [16][16] A. Chattopadhay, A. Sarkar, P. Howlader, and V. N. Balasubramanian, 'Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks,' in 2018 IEEE winter conference on applications of computer vision (WACV) . IEEE, 2018, pp. 839-847. -
- [18][18] J. V. Jeyakumar, J. Noor, Y.-H. Cheng, L. Garcia, and M. Srivastava, 'How can i explain this to you? an empirical study of deep neural network explanation methods,' Advances in Neural Information Processing Systems , 2020. -
- [17][17] S. M. Lundberg and S.-I. Lee, 'A unified approach to interpreting model predictions,' in Proceedings of the 31st international conference on neural information processing systems , 2017, pp. 4768-4777. -
- [31][31] S. Bach, A. Binder, G. Montavon, F. Klauschen, K.-R. M¨ uller, and W. Samek, 'On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation,' PloS one , vol. 10, no. 7, p. e0130140, 2015. -
- [36][36] J. Krause, M. Stark, J. Deng, and L. Fei-Fei, '3d object representations for fine-grained categorization,' in 4th International IEEE Workshop on 3D Representation and Recognition (3dRR-13) , Sydney, Australia, 2013. -
Cited by
Sorted by citations
A Survey on Explainable Anomaly Detection
Li, Z., Zhu, Y., & Van Leeuwen, M. (2024) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3609333 · Cited by 139 · see citing works
“Moreover, model-specific techniques such as References [7, 31, 94] generally have consistent Explanation-Definition and Detection-Definition of anomaly as they explore the internal structure of anomaly detection model to generate explanations.”
BackgroundMethodInfluential
- Authors
- Chong, P., Cheung, N.-M., Elovici, Y., & Binder, A.
- Affiliations
- Singapore University of Technology and Design (SG)
- Ben-Gurion University of the Negev (IL)
- University of Oslo (NO)
- Venue
- IEEE Transactions on Image Processing
- Published
- 2022
- Volume
- 31
- Pages
- 525-540
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W3213509545
- Cited by
- 14
- References
- 38
- Citations by year
- 2026: 2 · 2025: 3 · 2024: 2 · 2023: 5 · 2022: 2
- FWCI
- 0.80
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Image Processing (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- ST Engineering Electronics and the National Research Foundation (NRF), Singapore, under its Corporate Laboratory@University Scheme
- Research Council of Norway, via the SFI Visual Intelligence (309439)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- example-based explanations, outlier detection, explainable neural networks, similarity kernels, high-stakes decision making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
