Cited by 575Domain Adaptation and Few-Shot LearningMedical Image Segmentation TechniquesArtificial Intelligence in Healthcare and Education

The DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index; none of the 23 sources its claims cite is checked for support yet.

Self-supervised learning for medical image analysis using image context restoration

Open access

Abstract

Machine learning, particularly deep learning has boosted medical image analysis over the past years. Training a good model based on deep learning requires large amount of labelled data. However, it is often difficult to obtain a sufficient number of labelled images for training. In many scenarios the dataset in question consists of more unlabelled images than labelled ones. Therefore, boosting the performance of machine learning models by using unlabelled as well as labelled data is an important but challenging problem. Self-supervised learning presents one possible solution to this problem. However, existing self-supervised learning strategies applicable to medical images cannot result in significant performance improvement. Therefore, they often lead to only marginal improvements. In this paper, we propose a novel self-supervised learning strategy based on context restoration in order to better exploit unlabelled images. The context restoration strategy has three major features: 1) it learns semantic image features; 2) these image features are useful for different types of subsequent image analysis tasks; and 3) its implementation is simple. We validate the context restoration strategy in three common problems in medical imaging: classification, localization, and segmentation. For classification, we apply and test it to scan plane detection in fetal 2D ultrasound images; to localise abdominal organs in CT images; and to segment brain tumours in multi-modal MR images. In all three cases, self-supervised learning based on context restoration learns useful semantic features and lead to improved machine learning models for the above tasks.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

The paper proposes context restoration as a self-supervised pretraining task for medical images: randomly swap image patches and train a CNN to restore the original context. Across fetal ultrasound classification, abdominal CT organ localization, and brain MR tumor segmentation, context restoration pretraining outperformed autoencoder, relative-position, context-prediction, and no self-supervision.

Key findings

  • Context restoration pretraining improved CNNs in all three evaluated tasks: fetal standard scan plane classification, abdominal multi-organ localization, and brain tumor segmentation. It outperformed other self-supervised strategies and no self-supervision in the reported comparisons.
  • For fetal ultrasound classification, when using only a quarter of the training data, context restoration pretraining improved the SonoNet performance the most among the compared pretraining strategies.
  • For abdominal CT localization, context restoration pretraining produced more localization improvements than other pretraining methods; in some cases it was comparable to or better than no pretraining on more annotated training data.
  • For BraTS 2017 brain tumor segmentation, U-Nets initialized by context restoration pretraining achieved the best performance in total; using 50% of the training set gave similar performance to using the whole training set, with Dice in enhanced tumor core, sensitivity in non-enhanced and enhanced tumor cores, and Hausdorff distances even slightly better.

Methods

  • Self-supervised task: given an unlabelled image, randomly select two isolated patches and swap them; repeat T times to disorder context, then train a CNN to restore the original image using an L2 loss.
  • Evaluated on three datasets/tasks: fetal 2D ultrasound classification with 2694 examinations and SonoNet-64; abdominal CT multi-organ localization with 150 subjects and a SonoNet-like CNN; BraTS 2017 brain tumor segmentation with 285 subjects, 142 for training/validation and 143 for testing, using a 2D U-Net on 64×64 patches.
  • Compared pretraining strategies: autoencoder, relative position prediction, local context prediction, and context restoration; subsequent tasks used whole, half, or quarter of the training set. Training details include 58K iterations for ultrasound classification, 55K for CT localization, and 50K for segmentation.

Limitations

  • The paper states that the L2 loss used for context restoration results in blurry restored images.
  • The paper reports that data augmentation did not consistently improve performance and sometimes decreased segmentation accuracy; it also notes that self-supervised pretraining was not compared against supervised pretraining because supervised pretraining is difficult in medical imaging due to annotation requirements.
  • Evident from the text: evaluation is limited to three medical imaging tasks and datasets, and the paper notes that in brain tumor segmentation the performance differences among self-supervision strategies were not always significant.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/j.media.2019.101539

Cites

74 of 74 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

26 Own interpretation · 25 Own result · 21 No basis given · 20 Not checked yet

Verify all 23 references — free account

Show all 54 claims

  • It may state two separate claims as one. 18
  • The quote only partly states this claim. 1
  • Probably not a substantial claim. 21
38 of 38
  • The paper relies on this without support.

    Training CNNs only using a small number of labelled images cannot always achieve satisfactory results and does not exploit the potentially large number of unlabelled images that may be available.

    “Training CNNs only using the small number of labelled images cannot always achieve satisfactory results and does not exploit the potentially large number of unlabelled images that may be available.”
  • The paper relies on this without support.

    It is usually easier to access a large number of unlabelled medical images rather than a large number of annotated images.

    “This means it is usually easier to access a large number of unlabelled medical images rather than a large number of annotated images.”
  • The paper relies on this without support.

    In medical imaging, most images are in greyscale so that no colour information is available.

    “In medical imaging, most images are in greyscale so that no colour information is available.”
  • The paper relies on this without support.

    The key challenge for self-supervised learning is identifying a suitable self-supervision task, i.e. generating input and output instance pairs from data.

    “The key challenge for self-supervised learning is identifying a suitable self supervision task, i.e. generating input and out-put instance pairs from data.”
  • The paper relies on this without support.

    Manual cropping requires expert knowledge and is costly.

    “However, manual cropping requires expert knowledge and costly.”
  • The paper relies on this without support.

    Not all patients have two kidneys.

    “This is because not all patients have two kidneys.”
  • The paper relies on this without support.

    Gliomas are the major brain tumours occurring in adults.

    “Gliomas are the major brain tumours occurring in adults.”
  • The paper relies on this without support.

    Accurate segmentation of gliomas on MR image is a key step for quantification.

    “Accurate segmentation of gliomas on MR image is a key step for quantification.”
  • The paper relies on this without support.

    In medical image analysis, the image context is the common feature for classification, localization/detection, and segmentation tasks.

    “Particularly, in medical image analysis, the image context is the common feature for classification, localization/detection, and segmentation tasks.”
9 of 9

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 73 references

Cited by

Sorted by citations

Items 501–573 of 573

Metadata

Authors
Chen, L., Bentley, P., Mori, K., Misawa, K., Fujiwara, M., & Rueckert, D.
Affiliations
  • Imperial College London (GB)
  • Nagoya University (JP)
  • Aichi Cancer Center (JP)
  • Nagoya University Hospital (JP)
Corresponding author
Chen, Liang
Venue
Medical Image Analysis
Publisher
Elsevier BV
Published
2019-12
Volume
58
Pages
101539
ISSN
1361-8415 (ISSN-L)
OpenAlex
W2964744899
Cited by
575
References
73
Citations by year
2026: 44 · 2025: 85 · 2024: 99 · 2023: 133 · 2022: 105 · 2021: 74 · 2020: 34 · 2019: 1
FWCI
24.53
Type
journal article
Language
en
Open access
Locations
  • Medical Image Analysis (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
  • Spiral (Imperial College London) (submittedVersion, cc-by-nc-nd)
Funders
Wellcome Trust (102431)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
self-supervised learning, unlabeled data, CT images, brain tumor segmentation, multimodal MRI, image classification, image localization, image segmentation
MeSH terms
Abdomen, Supervised Machine Learning, Brain Neoplasms, Humans, Magnetic Resonance Imaging, Tomography, X-Ray Computed, Ultrasonography, Prenatal

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review