Evaluating the Use of Reconstruction Error for Novelty Localization
Open accessAbstract
The pixelwise reconstruction error of deep autoencoders is often utilized for image novelty detection and localization under the assumption that pixels with high error indicate which parts of the input image are unfamiliar and therefore likely to be novel. This assumed correlation between pixels with high reconstruction error and novel regions of input images has not been verified and may limit the accuracy of these methods. In this paper we utilize saliency maps to evaluate whether this correlation exists. Saliency maps reveal directly how much a change in each input pixel would affect reconstruction loss, while each pixel's reconstruction error may be attributed to many input pixels when layers are fully connected. We compare saliency maps to reconstruction error maps via qualitative visualizations as well as quantitative correspondence between the top K elements of the maps for both novel and normal images. Our results indicate that reconstruction error maps do not closely correlate with the importance of pixels in the input images, making them insufficient for novelty localization.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
PP-Mare assumes a false correlation between pixels with high reconstruction error and novel regions of input images.
“Further, PP-Mare assumes a false correlation between pixels with high reconstruction error and novel regions of input images (Feeney & Hughes, 2021).”
From Novelty Detection in Reinforcement Learning with World Models (2023) · cites this paper as Feeney & Hughes, 2021
The Introduction (p. 2) says that using reconstruction-error maps assumes high-error pixels correspond to novel input pixels. The Discussion (p. 4) finds that these maps are not always representative of novel regions, but the paper does not identify PP-Mare or establish that its particular assumption is categorically false. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 23
- Abati, D., Porrello, A., Calderara, S., and Cucchiara, R. Latent space autoregression for novelty detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2019.
- Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., and Kim, B. Sanity checks for saliency maps. arXiv preprint arXiv:1810.03292 , 2020. · arXiv
- Arun, N. T., Gaw, N., Singh, P., Chang, K., Hoebel, K. V., Patel, J., Gidwani, M., and Kalpathy-Cramer, J. Assessing the validity of saliency maps for abnormality localization in medical imaging. arXiv preprint arXiv:2006.00063 , 2020. · arXiv
- Bergmann, P., Fauser, M., Sattlegger, D., and Steger, C. Uninformed students: Student-teacher anomaly detection with discriminative latent embeddings. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2020.
- Bourlard, H. and Kamp, Y. Auto-association by multilayer perceptrons and singular value decomposition. Biol. Cybern. , 59(4-5):291-294, 1988. URL https: //doi.org/10.1007/BF00332918 . · DOI
- Geng, C., Huang, S.-j., and Chen, S. Recent advances in open set recognition: A survey. IEEE transactions on pattern analysis and machine intelligence , 2020.
- Gong, D., Liu, L., Le, V., Saha, B., Mansour, M. R., Venkatesh, S., and Hengel, A. v. d. Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , October 2019.
- Japkowicz, N., Myers, C., Gluck, M., et al. A novelty detection approach to classification. In IJCAI , volume 1, pp. 518-523. Citeseer, 1995.
- Liang, S., Li, Y., and Srikant, R. Enhancing the reliability of out-of-distribution image detection in neural networks. arXiv preprint arXiv:1706.02690 , 2017. · arXiv
- Luo, W., Liu, W., and Gao, S. A revisit of sparse coding based anomaly detection in stacked rnn framework. In Proceedings of the IEEE International Conference on Computer Vision , pp. 341-349, 2017.
- Oza, P. and Patel, V. M. C2ae: Class conditioned autoencoder for open-set recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2019.
- Perera, P., Nallapati, R., and Xiang, B. Ocgan: One-class novelty detection using gans with constrained latent representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 28982906, 2019.
- Ren, J., Liu, P. J., Fertig, E., Snoek, J., Poplin, R., DePristo, M. A., Dillon, J. V., and Lakshminarayanan, B. Likelihood ratios for out-of-distribution detection. arXiv preprint arXiv:1906.02845 , 2019. · arXiv
- Sabokrou, M., Fayyaz, M., Fathy, M., Moayed, Z., and Klette, R. Deep-anomaly: Fully convolutional neural network for fast anomaly detection in crowded scenes. Computer Vision and Image Understanding , 172:88-97, 2018. ISSN 10773142. doi: https://doi.org/10.1016/j.cviu.2018.02. 006. URL https://www.sciencedirect.com/ science/article/pii/S1077314218300249 . · DOI
- Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) , Oct 2017.
- Shrikumar, A., Greenside, P., Shcherbina, A., and Kundaje, A. Not just a black box: Learning important features through propagating activation differences. arXiv preprint arXiv:1605.01713 , 2017. · arXiv
- Simonyan, K., Vedaldi, A., and Zisserman, A. Deep inside convolutional networks: Visualising image classification models and saliency maps. In In Workshop at International Conference on Learning Representations . Citeseer, 2014.
- Smilkov, D., Thorat, N., Kim, B., Vi´ egas, F., and Wattenberg, M. Smoothgrad: removing noise by adding noise. arXiv preprint arXiv:1706.03825 , 2017. · arXiv
- Springenberg, J. T., Dosovitskiy, A., Brox, T., and Riedmiller, M. Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806 , 2015. · arXiv
- Sundararajan, M., Taly, A., and Yan, Q. Axiomatic attribution for deep networks. In Precup, D. and Teh, Y. W. (eds.), Proceedings of the 34th International Conference on Machine Learning , volume 70 of Proceedings of Machine Learning Research , pp. 3319-3328. PMLR, 0611 Aug 2017. URL http://proceedings.mlr. press/v70/sundararajan17a.html .
- Vincent, P., Larochelle, H., Bengio, Y., and Manzagol, P.-A. Extracting and composing robust features with denoising autoencoders. In International Conference on Machine Learning , 2008. URL https://doi.org/10. 1145/1390156.1390294 .
- Zimmerer, D., Kohl, S. A. A., Petersen, J., Isensee, F., and Maier-Hein, K. H. Context-encoding variational autoencoder for unsupervised anomaly detection. arXiv preprint arXiv:1812.05941 , 2018. · arXiv
- Zong, B., Song, Q., Min, M. R., Cheng, W., Lumezanu, C., Cho, D., and Chen, H. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In International Conference on Learning Representations , 2018.
Cited by
Sorted by citations
Novelty Detection in Reinforcement Learning with World Models
Zollicoffer, G., Eaton, K., Balloch, J., Kim, J., Zhou, W., Wright, R., & Riedl, M. O. (2023) · arXiv · doi:10.48550/arxiv.2310.08731
- Authors
- Feeney, P., & Hughes, M. C.
- ORCID
- Hughes, Michael C.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2107.13379
- OpenAlex
- W3185710536
- Cited by
- 1
- References
- 0
- Citations by year
- 2023: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Keywords
- reconstruction error, deep autoencoder, saliency maps, reconstruction loss, anomaly detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
