A Diffusion-Based Framework for Multi-Class Anomaly Detection
Open accessThe DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
Reconstruction-based approaches have achieved remarkable outcomes in anomaly detection. The exceptional image reconstruction capabilities of recently popular diffusion models have sparked research efforts to utilize them for enhanced reconstruction of anomalous images. Nonetheless, these methods might face challenges related to the preservation of image categories and pixel-wise structural integrity in the more practical multi-class setting. To solve the above problems, we propose a Difusion-based Anomaly Detection (DiAD) framework for multi-class anomaly detection, which consists of a pixel-space autoencoder, a latent-space Semantic-Guided (SG) network with a connection to the stable diffusion’s denoising network, and a feature-space pre-trained feature extractor. Firstly, The SG network is proposed for reconstructing anomalous regions while preserving the original image’s semantic information. Secondly, we introduce Spatial-aware Feature Fusion (SFF) block to maximize reconstruction accuracy when dealing with extensively reconstructed areas. Thirdly, the input and reconstructed images are processed by a pre-trained feature extractor to generate anomaly maps based on features extracted at different scales. Experiments on MVTec-AD and VisA datasets demonstrate the effectiveness of our approach which surpasses the state-of-the-art methods, e.g., achieving 96.8/52.6 and 97.2/99.0 (AUROC/AP) for localization and detection respectively on multi-class MVTec-AD dataset. Code will be available at https://lewandofskee.github.io/projects/diad.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072 · cited 1×
“Most methods employ networks (He et al. 2016; Tan and Le 2019; Zhang et al. 2022, 2023c; Wu et al. 2023) pre-trained on ImageNet (Deng et al. 2009) for feature extraction.”
MethodDecoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
Denoising Diffusion Probabilistic Models
Ho, J., Jain, A., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11239 · Cited by 8,712
Zagoruyko, S., & Komodakis, N. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.87 · Cited by 7,244
Adding Conditional Control to Text-to-Image Diffusion Models
Zhang, L., Rao, A., & Agrawala, M. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00355 · Cited by 4,547
Denoising Diffusion Implicit Models
Song, J., Meng, C., & Ermon, S. (2020) · arXiv · doi:10.48550/arxiv.2010.02502 · Cited by 2,507 · cited 1×
“We use DDIM (Song, Meng, and Ermon 2021) as the sampler with 10 steps by default.”
MethodMVTec AD — A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection
Bergmann, P., Fauser, M., Sattlegger, D., & Steger, C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00982 · Cited by 2,203 · cited 1×
“MVTec-AD (Bergmann et al. 2019) dataset simulates real-world industrial production scenarios, filling the gap in unsupervised anomaly detection.”
MethodSigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning
Elfwing, S., Uchibe, E., & Doya, K. (2018) · Neural Networks · doi:10.1016/j.neunet.2017.12.012 · Cited by 2,119
Towards Total Recall in Industrial Anomaly Detection
Roth, K., Pemula, L., Zepeda, J., Scholkopf, B., Brox, T., & Gehler, P. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01392 · Cited by 1,810 · cited 2×
“2) Embedding-based methods encode the original image’s three-dimensional information into a multidimensional feature space (Roth et al. 2022; Cao et al. 2022; Gu et al. 2023).”
MethodPaDiM: A Patch Distribution Modeling Framework for Anomaly Detection and Localization
Defard, T., Setkov, A., Loesch, A., & Audigier, R. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-68799-1_35 · Cited by 1,415
Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit
Hahnloser, R. H. R., Sarpeshkar, R., Mahowald, M. A., Douglas, R. J., & Seung, H. S. (2000) · Nature · doi:10.1038/35016072 · Cited by 1,293
DRÆM – A discriminatively trained reconstruction embedding for surface anomaly detection
Zavrtanik, V., Kristan, M., & Skocaj, D. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00822 · Cited by 1,086
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2021) · arXiv · doi:10.48550/arxiv.2112.10752 · Cited by 737
DiffusionDet: Diffusion Model for Object Detection
Chen, S., Sun, P., Song, Y., & Luo, P. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01816 · Cited by 657
SPot-the-Difference Self-supervised Pre-training for Anomaly Detection and Segmentation
Zou, Y., Jeong, J., Pemula, L., Zhang, D., & Dabeer, O. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20056-4_23 · Cited by 651
WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation
Jeong, J., Zou, Y., Kim, T., Zhang, D., Ravichandran, A., & Dabeer, O. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01878 · Cited by 585
Patch SVDD: Patch-Level SVDD for Anomaly Detection and Segmentation
Yi, J., & Yoon, S. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-69544-6_23 · Cited by 551
Deep Industrial Image Anomaly Detection: A Survey
Liu, J., Xie, G., Wang, J., Li, S., Wang, C., Zheng, F., & Jin, Y. (2024) · Machine Intelligence Research · doi:10.1007/s11633-023-1459-z · Cited by 463 · cited 2×
“Anomaly detection is a crucial task in computer vision and industrial applications (Tao et al. 2022; Salehi et al. 2022; Liu et al. 2023), which goal of visual anomaly detection is to determine anomalous images and locate the regions of anomaly accurately.”
BackgroundMethodSelf-citationImagen Video: High Definition Video Generation with Diffusion Models
Ho, J., Chan, W., Saharia, C., Whang, J., Gao, R., Gritsenko, A., Kingma, D. P., Poole, B., Norouzi, M., Fleet, D. J., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.02303 · Cited by 399
A Unified Model for Multi-Class Anomaly Detection
You, Z., Cui, L., Shen, Y., Yang, K., Lu, X., Zheng, Y., & Le, X. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-0330 · Cited by 303
Multimodal Industrial Anomaly Detection via Hybrid Fusion
Wang, Y., Peng, J., Zhang, J., Yi, R., Wang, Y., & Wang, C. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00776 · Cited by 246
Self-citationDeep Learning for Unsupervised Anomaly Localization in Industrial Images: A Survey
Tao, X., Gong, X., Zhang, X., Yan, S., & Adak, C. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3196436 · Cited by 228
Registration Based Few-Shot Anomaly Detection
Huang, C., Guan, H., Jiang, A., Zhang, Y., Spratling, M., & Wang, Y.-F. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20053-3_18 · Cited by 216
Self-citationFastFlow: Unsupervised Anomaly Detection and Localization via 2D Normalizing Flows
Yu, J., Zheng, Y., Wang, X., Li, W., Wu, Y., Zhao, R., & Wu, L. (2021) · arXiv · doi:10.48550/arxiv.2111.07677 · Cited by 210
Adding Conditional Control to Text-to-Image Diffusion Models
Zhang, L., Rao, A., & Agrawala, M. (2023) · arXiv · doi:10.48550/arxiv.2302.05543 · Cited by 201
Catching Both Gray and Black Swans: Open-set Supervised Anomaly Detection
Ding, C., Pang, G., & Shen, C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00724 · Cited by 197 · cited 1×
“(Ding, Pang, and Shen 2022), noisy learning (Tan et al. 2021; Yoon et al. 2022), zero-/few-shot (Huang et al. 2022; Jeong et al. 2023; Cao et al. 2023; Chen, Han, and Zhang 2023; Chen et al. 2023b; Zhang et al. 2023b), 3D AD (Wang et al. 2023; Chen et al. 2023a), etc .”
BackgroundExplainable Deep One-Class Classification
Liznerski, P., Ruff, L., Vandermeulen, R. A., Franks, B. J., Kloft, M., & Müller, K.-R. (2020) · arXiv · doi:10.48550/arxiv.2007.01760 · Cited by 185
SegDiff: Image Segmentation with Diffusion Probabilistic Models
Amit, T., Shaharbany, T., Nachmani, E., & Wolf, L. (2021) · arXiv · doi:10.48550/arxiv.2112.00390 · Cited by 179
Informative knowledge distillation for image anomaly segmentation
Cao, Y., Wan, Q., Shen, W., & Gao, L. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.108846 · Cited by 130
Salehi, M., Mirzaei, H., Hendrycks, D., Li, Y., Rohban, M. H., & Sabokrou, M. (2021) · arXiv · doi:10.48550/arxiv.2110.14051 · Cited by 118
Notice of Removal: Multiresolution Knowledge Distillation for Anomaly Detection
Salehi, M., Sadjadi, N., Baselizadeh, S., Rohban, M. H., & Rabiee, H. R. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01466 · Cited by 97
Cited by
Unbiased Missing-Modality Multimodal Learning
Dai, R., Li, C., Yan, Y., Mo, L., Qin, K., & He, T. (2025) · 2025 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02272 · Cited by 11
Metadata
- Authors
- He, H., Zhang, J., Chen, H., Chen, X., Li, Z., Chen, X., Wang, Y., Wang, C., & Xie, L.
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2024-03-24
- Volume
- 38
- Issue
- 8
- Pages
- 8472-8480
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W4393147759
- Cited by
- 212
- References
- 36
- FWCI
- 64.49
- Type
- conference paper
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.