Student or educator? Sign up with your school email and get 6 months of Pro free.
Cited by 88Advanced Neural Network ApplicationsDomain Adaptation and Few-Shot LearningMedical Image Segmentation Techniques

The DOI matches its Crossref record; 143 of the 156 references it cites were found in a registry or an index; 10 of the 19 sources its claims cite were checked for support.

Large-Scale Unsupervised Semantic Segmentation

Open access
Gao, Shanghua, Li, Zhong-Yu, Yang, Ming-Hsuan, Cheng, Ming-Ming, Han, Junwei, Torr, Philip·2023·IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract

Empowered by large datasets, e.g., ImageNet and MS COCO, unsupervised learning on large-scale data has enabled significant advances for classification tasks. However, whether the large-scale unsupervised semantic segmentation can be achieved remains unknown. There are two major challenges: i) we need a large-scale benchmark for assessing algorithms; ii) we need to develop methods to simultaneously learn category and shape representation in an unsupervised manner. In this work, we propose a new problem of large-scale unsupervised semantic segmentation (LUSS) with a newly created benchmark dataset to help the research progress. Building on the ImageNet dataset, we propose the ImageNet-S dataset with 1.2 million training images and 50k high-quality semantic segmentation annotations for evaluation. Our benchmark has a high data diversity and a clear task objective. We also present a simple yet effective method that works surprisingly well for LUSS. In addition, we benchmark related un/weakly/fully supervised methods accordingly, identifying the challenges and possible directions of LUSS. The benchmark and source code is publicly available at https://github.com/LUSSeg.

Overview

CoChat Summary

TL;DR

The paper proposes large-scale unsupervised semantic segmentation (LUSS) and introduces ImageNet-S, a benchmark built from ImageNet with 919 categories and nearly 50k pixel-level annotated images. It also presents a simple LUSS method using enhanced representation learning and pixel-attention, and benchmarks related un/weakly/fully supervised approaches.

Key findings

  • ImageNet-S contains 1,183,322 training, 12,419 validation, and 27,423 testing images from 919 categories; 39,842 validation/testing images and 9,190 training images are annotated with precise pixel-level masks.
  • The dataset includes ImageNet-S50 (50 categories; 64,431 train / 752 val / 1,682 test) and ImageNet-S300 (300 categories; 384,862 train / 4,097 val / 9,088 test) in addition to the full ImageNet-S.
  • Most images contain one category, while 8.6% of images have more than one category.
  • The authors correct 296 mislabeled images and find 942 images with missing labels during annotation.

Methods

  • The paper defines LUSS as assigning labels to pixels from large-scale data without human-annotation supervision, requiring simultaneous learning of category and shape representations.
  • The proposed method includes unsupervised representation learning with a non-contrastive pixel-to-pixel representation alignment strategy, a deep-to-shallow supervision strategy, and a pixel-attention scheme for label generation and fine-tuning.
  • The benchmark is built from ImageNet by removing unsegmentable categories, annotating pixel-level masks, and evaluating related unsupervised, weakly supervised, and fully supervised methods.

Limitations

  • The paper states that existing unsupervised semantic segmentation methods focus on small datasets and few easy categories, and some use supervised prior knowledge such as pre-trained weights, edge detection, or saliency detection, making evaluation difficult.
  • It identifies LUSS challenges including simultaneous shape and category representation learning, semantic clustering at large scale, and high computational cost.
  • The provided text is truncated before the experimental results and any explicit limitations section, so quantitative LUSS performance and author-stated limitations beyond the benchmark discussion are not available.

Model-written summary of a shortened version of the paper’s full text (it is longer than we read). Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/tpami.2022.3218275

Cites

143 of 143 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

4 Valid · 2 Not valid · 2 Not checked · 18 Own results and interpretations

2 more claims

Create a free account to see all 26 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 15
  • Probably not a substantial claim. 6

Create a free account to see these claims.

Citations

Cites

Show all 156 references

Research works printed but not linked: 4

  1. 10.5555/3524938.3525087 · doi:10.5555/3524938.3525087
  2. 10.5555/3495724.3497510 · doi:10.5555/3495724.3497510
  3. Russell, Segmenting scenes by matching image composites, 1580
  4. Li, Prototypical contrastive learning of unsupervised representations

Cited by

See all 88

Metadata

Authors
Gao, S., Li, Z.-Y., Yang, M.-H., Cheng, M.-M., Han, J., & Torr, P.
Affiliations
  • Nankai University (CN)
  • University of California, Merced (US)
  • Northwestern Polytechnical University (CN)
  • University of Oxford (GB)
Venue
IEEE Transactions on Pattern Analysis and Machine Intelligence
Published
2023-06-01
Volume
45
Issue
6
Pages
7457-7476
ISSN
  • 0162-8828 (ISSN-L)
  • 2160-9292
  • 1939-3539
OpenAlex
W4307844773
Cited by
88
References
156
Citations by year
2026: 10 · 2025: 33 · 2024: 26 · 2023: 14
FWCI
6.30
Type
journal article
Language
en
Open access
Locations
  • IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Key Research and Development Program of China (2018AAA0100400)
  • National Natural Science Foundation of China (62225604)
  • Fundamental Research Funds for the Central Universities (63223050)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
Keywords
unsupervised semantic segmentation, benchmark dataset, weakly supervised segmentation, ImageNet

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Large-Scale Unsupervised Semantic Segmentation. https://app.cochat.ai/papers/10.1109/tpami.2022.3218275
The paper
Gao, S., Li, Z.-Y., Yang, M.-H., Cheng, M.-M., Han, J., & Torr, P. (2023). Large-Scale Unsupervised Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/tpami.2022.3218275

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.