Multimodal Industrial Anomaly Detection via Hybrid Fusion
Open accessAbstract
2D-based Industrial Anomaly Detection has been widely discussed, however, multimodal industrial anomaly detection based on 3D point clouds and RGB images still has many untouched fields. Existing multimodal industrial anomaly detection methods directly concatenate the multimodal features, which leads to a strong disturbance between features and harms the detection performance. In this paper, we propose Multi-3D-Memory (M3DM), a novel multimodal anomaly detection method with hybrid fusion scheme: firstly, we design an unsupervised feature fusion with patch-wise contrastive learning to encourage the interaction of different modal features; secondly, we use a decision layer fusion with multiple memory banks to avoid loss of information and additional novelty classifiers to make the final decision. We further propose a point feature alignment operation to better align the point cloud and RGB features. Extensive experiments show that our multi-modal industrial anomaly detection model outperforms the state-of-the-art (SOTA) methods on both detection and segmentation precision on MVTec-3D AD dataset. Code at github.com/nomewang/M3DM.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 21,793 · see citing works
Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2008) · 2008 Eighth IEEE International Conference on Data Mining · doi:10.1109/icdm.2008.17 · Cited by 6,601 · see citing works
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242 · see citing works
Emerging Properties in Self-Supervised Vision Transformers
Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., & Joulin, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00951 · Cited by 5,638 · cited 5× · see citing works
“We use 2 Transformer-based feature extractors to separately extract the RGB feature and point clouds feature: 1) For the RGB feature, we use a Vision Transformer (ViT) [11] to directly extract each patch feature, and in order to adapt to the anomaly detection task, we use a ViT-B/8 architecture for both efficiency and…”
MethodInfluentialTowards Total Recall in Industrial Anomaly Detection
Roth, K., Pemula, L., Zepeda, J., Scholkopf, B., Brox, T., & Gehler, P. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01392 · Cited by 1,810 · cited 10× · see citing works
“Although PatchCore + FPFH method gets a high P-AUROC score, its I-AUROC is much lower than the other two.”
BackgroundMethodInfluentialPaDiM: A Patch Distribution Modeling Framework for Anomaly Detection and Localization
Defard, T., Setkov, A., Loesch, A., & Audigier, R. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-68799-1_35 · Cited by 1,415 · see citing works
FastFlow: Unsupervised Anomaly Detection and Localization via 2D Normalizing Flows
Yu, J., Zheng, Y., Wang, X., Li, W., Wu, Y., Zhao, R., & Wu, L. (2021) · arXiv · doi:10.48550/arxiv.2111.07677 · Cited by 210 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Multi-Feature Hybrid Anomaly Detection in ICS: An Integration of ML, DL, and Statistical Techniques
Aslam, M. M., Tufail, A., De Silva, L. C., & Apong, R. A. A. H. M. (2025) · Proceedings of the 3rd ACM Workshop on Secure and Trustworthy Deep Learning Systems · doi:10.1145/3709021.3737669 · Cited by 4 · see citing works
ZUMA: Training-Free Zero-Shot Unified Multimodal Anomaly Detection
Ma, Y., Liu, M., Jiang, S., Zhou, J., Bian, Y., Wang, X., & Wang, Y. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3658856 · Cited by 4 · see citing works
Self-citation
- Authors
- Wang, Y., Peng, J., Zhang, J., Yi, R., Wang, Y., & Wang, C.
- Affiliations
- Shanghai Jiao Tong University (CN)
- Tencent (China) (CN)
- Venue
- 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2023-06
- Pages
- 8032-8041
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4386065775
- Cited by
- 246
- References
- 7
- Citations by year
- 2026: 92 · 2025: 102 · 2024: 43 · 2023: 9
- FWCI
- 106.70
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (72192821,61972157,62272447)
- Shanghai Municipal Science and Technology (2021SHZDZX0102,21511101200)
- CCF-Tencent Open Research Fund (RAGR20220121)
- Young Elite Scientists Sponsorship Program by CAST (2022QNRC001)
- Topics
- Physical Sciences › Engineering › Industrial and Manufacturing Engineering › Industrial Vision Systems and Defect Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Physical Sciences › Engineering › Computational Mechanics › 3D Shape Modeling and Analysis
- Keywords
- industrial anomaly detection, multimodal fusion, 3D point clouds, RGB images, hybrid fusion, memory bank, novelty detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
