IterMask3D: Unsupervised anomaly detection and segmentation with test-time iterative mask refinement in 3D brain MRI
Open accessAbstract
Unsupervised anomaly detection and segmentation methods train a model to learn the training distribution as 'normal'. In the testing phase, they identify patterns that deviate from this normal distribution as 'anomalies'. To learn the 'normal' distribution, prevailing methods corrupt the images and train a model to reconstruct them. During testing, the model attempts to reconstruct corrupted inputs based on the learned 'normal' distribution. Deviations from this distribution lead to high reconstruction errors, which indicate potential anomalies. However, corrupting an input image inevitably causes information loss even in normal regions, leading to suboptimal reconstruction and an increased risk of false positives. To alleviate this, we propose IterMask3D, an iterative spatial mask-refining strategy designed for 3D brain MRI. We iteratively spatially mask areas of the image as corruption and reconstruct them, then shrink the mask based on reconstruction error. This process iteratively unmasks 'normal' areas to the model, whose information further guides reconstruction of 'normal' patterns under the mask to be reconstructed accurately, reducing false positives. In addition, to achieve better reconstruction performance, we also propose using high-frequency image content as additional structural information to guide the reconstruction of the masked area. Extensive experiments on the detection of both synthetic and real-world imaging artifacts, as well as segmentation of various pathological lesions across multiple MRI sequences, consistently demonstrate the effectiveness of our proposed method. Code is available at https://github.com/ZiyunLiang/IterMask3D.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Denoising Diffusion Probabilistic Models
Ho, J., Jain, A., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11239 · Cited by 8,712 · see citing works
A nonparametric method for automatic correction of intensity nonuniformity in MRI data
Sled, J. G., Zijdenbos, A. P., & Evans, A. C. (1998) · IEEE Transactions on Medical Imaging · doi:10.1109/42.668698 · Cited by 4,731 · see citing works
Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation
Kamnitsas, K., Ledig, C., Newcombe, V. F. J., Simpson, J. P., Kane, A. D., Menon, D. K., Rueckert, D., & Glocker, B. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.10.004 · Cited by 3,498 · see citing works
Self-citationBakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J. S., Freymann, J. B., Farahani, K., & Davatzikos, C. (2017) · Scientific Data · doi:10.1038/sdata.2017.117 · Cited by 3,049 · see citing works
The importance of phase in signals
Oppenheim, A. V., & Lim, J. S. (1981) · Proceedings of the IEEE · doi:10.1109/proc.1981.12022 · Cited by 1,875 · see citing works
f-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks
Schlegl, T., Seeböck, P., Waldstein, S. M., Langs, G., & Schmidt-Erfurth, U. M. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.01.010 · Cited by 1,561 · see citing works
The Alzheimer's Disease Neuroimaging Initiative
Mueller, S. G., Weiner, M. W., Thal, L. J., Petersen, R. C., Jack, C., Jagust, W., Trojanowski, J. Q., Toga, A. W., & Beckett, L. (2005) · Neuroimaging Clinics of North America · doi:10.1016/j.nic.2005.09.008 · Cited by 1,525 · see citing works
Automated model-based tissue classification of MR images of the brain
Van Leemput, K., Maes, F., Vandermeulen, D., & Suetens, P. (1999) · IEEE Transactions on Medical Imaging · doi:10.1109/42.811270 · Cited by 1,024 · see citing works
LaMontagne, P. J., Benzinger, T. L., Morris, J. C., Keefe, S., Hornbeck, R., Xiong, C., Grant, E., Hassenstab, J., Moulder, K., Vlassenko, A. G., et al. (2019) · medRxiv · doi:10.1101/2019.12.13.19014902 · Cited by 779 · see citing works
Pérez‐García, F., Sparks, R. E., & Sebastien Ourselin, A. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106236 · Cited by 618 · see citing works
Robust Brain Extraction Across Datasets and Comparison With Publicly Available Methods
Iglesias, J. E., Liu, C., Thompson, P. M., & Tu, Z. (2011) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2011.2138152 · Cited by 561 · see citing works
Maier, O., Menze, B. H., von der Gablentz, J., Häni, L., Heinrich, M. P., Liebrand, M., Winzeck, S., Basit, A., Bentley, P., Chen, L., et al. (2016) · Medical Image Analysis · doi:10.1016/j.media.2016.07.009 · Cited by 528 · see citing works
Self-citationnnU-Net for Brain Tumor Segmentation
Isensee, F., Jaeger, P. F., Full, P. M., Vollmuth, P., & Maier-Hein, K. H. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-72087-2_11 · Cited by 512 · see citing works
SynthStrip: skull stripping for any brain image
Hoopes, A., Mora, J., Dalca, A. V., Fischl, B. R., & Hoffmann, M. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119474 · Cited by 502 · see citing works
Autoencoders for unsupervised anomaly segmentation in brain MR images: A comparative study
Baur, C., Denner, S., Wiestler, B., Navab, N., & Albarqouni, S. (2021) · Medical Image Analysis · doi:10.1016/j.media.2020.101952 · Cited by 397 · see citing works
Denoising diffusion probabilistic models for 3D medical image generation
Khader, F., Müller‐Franzes, G. A., Arasteh, S. T., Han, T., Haarburger, C., Schulze-Hagen, M. F., Schad, P., Engelhardt, S., Baeßler, B., Foersch, S., et al. (2023) · Scientific Reports · doi:10.1038/s41598-023-34341-2 · Cited by 372 · see citing works
Image segmentations produced by BAMF under the AIMI Annotations initiative
Van Oss, J., Murugesan, G. K., McCrumb, D., et al. (2024) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.12734644 · Cited by 367 · see citing works
Diffusion Models for Medical Anomaly Detection
Wolleb, J., Bieder, F., Sandkühler, R., & Cattin, P. C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16452-1_4 · Cited by 349 · see citing works
Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation
Stollenga, M. F., Byeon, W., Liwicki, M., & Schmidhuber, J. (2015) · arXiv · doi:10.48550/arxiv.1506.07452 · Cited by 244 · see citing works
Unsupervised brain imaging 3D anomaly detection and segmentation with transformers
Pinaya, W. H. L., Tudosiu, P.-D., Gray, R., Rees, G. E., Nachev, P., Sebastien Ourselin, A., & Cardoso, M. J. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102475 · Cited by 203 · see citing works
Unsupervised Anomaly Localization Using Variational Auto-Encoders
Zimmerer, D., Isensee, F., Petersen, J., et al. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32251-9_32 · Cited by 156 · see citing works
Fast Unsupervised Brain Anomaly Detection and Segmentation with Diffusion Models
Pinaya, W. H. L., Graham, M. S., Gray, R., da Costa, P. F., Tudosiu, P.-D., Wright, P. K., Mah, Y., MacKinnon, A. D., Teo, J., Jäger, H. R., et al. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16452-1_67 · Cited by 117 · see citing works
Oppenheim, A. V., Lim, J. S., Kopec, G. E., et al. (2005) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1979.1170798 · Cited by 85 · see citing works
k-Space tutorial: an MRI educational tool for a better understanding of k-space
Moratal, D., Valles-Luch, A., Martí‐Bonmatí, L., et al. (2008) · Biomedical Imaging and Intervention Journal · doi:10.2349/biij.4.1.e15 · Cited by 75 · see citing works
Unsupervised brain lesion segmentation from MRI using a convolutional autoencoder
Atlason, H. E., Löve, Á., Sigurdsson, S. O., Gudnason, V. G., & Ellingsen, L. M. (2019) · SPIE Medical Imaging: Image Processing · doi:10.1117/12.2512953 · Cited by 46 · see citing works
Incidental Findings from 16,400 Brain MRI Examinations of Research Volunteers
Rowley, P. A., Paukner, M. J., Eisenmenger, L. B., Field, A. S., Davidson, R. J., Johnson, S. C., Asthana, S., Chin, N. A., Prabhakaran, V., Bendlin, B. B., et al. (2023) · American Journal of Neuroradiology · doi:10.3174/ajnr.a7821 · Cited by 24 · see citing works
Rethinking Autoencoders for Medical Anomaly Detection from A Theoretical Perspective
Cai, Y., Chen, H., & Cheng, K.-T. T. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72120-5_51 · Cited by 23 · see citing works
deepbet: Fast brain extraction of T1-weighted MRI using Convolutional Neural Networks
Fisch, L., Zumdick, S., Barkhau, C., Emden, D., Ernsting, J., Leenings, R., Sarink, K., Winter, N. R., Risse, B., Dannlowski, U., et al. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108845 · Cited by 18 · see citing works
Modality Cycles with Masked Conditional Diffusion for Unsupervised Anomaly Segmentation in MRI
Liang, Z., Anthony, H., Wagner, F., & Kamnitsas, K. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-47425-5_16 · Cited by 17 · see citing works
Self-citationLatent Transformer Models for out-of-distribution detection
Graham, M., Tudosiu, P.-D., Wright, P., Pinaya, W. H. L., Teikari, P., Patel, A., U-King-im, J., Mah, Y., Teo, J., Jäger, H., et al. (2023) · Medical Image Anal. · doi:10.1016/j.media.2023.102967 · Cited by 16 · see citing works
Bercea, C. I., Neumayr, M., Rueckert, D., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.19643 · Cited by 15 · see citing works
A systematic review of (semi-)automatic quality control of T1-weighted MRI scans
Hendriks, J., Mutsaerts, H. J. M. M., Joules, R., Peña‐Nogales, Ó., Rodrigues, P., Wolz, R., Burchell, G. L., Barkhof, F., & Schrantee, A. (2023) · Neuroradiology · doi:10.1007/s00234-023-03256-0 · Cited by 15 · see citing works
Xu, W.-T., Moffat, M., Seale, T., Liang, Z.-Y., Wagner, F., Whitehouse, D., Menon, D., Newcombe, V. F. J., Voets, N., Banerjee, A., et al. (2024) · International Conference on Medical Imaging with Deep Learning · doi:10.48550/arxiv.2405.18511 · Cited by 15 · see citing works
Self-citationWhat Do AEs Learn? Challenging Common Assumptions in Unsupervised Anomaly Detection
Bercea, C. I., Rueckert, D., & Schnabel, J. A. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43904-9_30 · Cited by 15 · see citing works
Ravì, D., Barkhof, F., Alexander, D. C., et al. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.103033 · Cited by 11 · see citing works
Liang, Z., Guo, X., Noble, J. A., & Kamnitsas, K. (2024) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-72111-3_32 · Cited by 10 · see citing works
Self-citationPrinciples of Medical Imaging for Engineers
Chappell, M. A. (2019) · Springer eBooks · doi:10.1007/978-3-030-30511-6 · Cited by 6 · see citing works
Zheng, J.-Q., Mo, Y., Sun, Y., et al. (2026) · Medical Image Analysis · doi:10.1016/j.media.2026.103987 · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Still being checked.
- Authors
- Liang, Z., Guo, X., Xu, W., Ibrahim, Y., Voets, N., Pretorius, P. M., Noble, J. A., & Kamnitsas, K.
- Affiliations
- University of Oxford (GB)
- Hong Kong Baptist University (HK)
- Corresponding author
- Kamnitsas, Konstantinos
- ORCID
- Venue
- Medical Image Analysis
- Publisher
- Elsevier BV
- Published
- 2026-01
- Volume
- 107
- Issue
- Pt A
- Pages
- 103763
- ISSN
- 1361-8415 (ISSN-L)
- OpenAlex
- W4413787196
- Cited by
- 3
- References
- 38
- Citations by year
- 2026: 3
- FWCI
- 1.08
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Medical Image Analysis (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Engineering and Physical Sciences Research Council (EP/S02428X/1, EP/W524311/1, EP/T028572/1)
- Wellcome Trust (203139/A/16/Z, 203139/Z/16/Z)
- NIHR Oxford Biomedical Research Centre
- UK Research and Innovation (EP/X040186/1)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Life Sciences › Neuroscience › Neurology › Brain Tumor Detection and Classification
- Keywords
- unsupervised anomaly detection, brain MRI, 3D MRI, spatial masking, reconstruction-based anomaly detection, false positive reduction, image artifacts
- MeSH terms
- Unsupervised Machine Learning, Algorithms, Brain, Humans, Image Interpretation, Computer-Assisted, Magnetic Resonance Imaging, Imaging, Three-Dimensional
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
