Cited by 3,565Advanced Neural Network ApplicationsDomain Adaptation and Few-Shot LearningVisual Attention and Saliency Detection

Masked-attention Mask Transformer for Universal Image Segmentation

Open access
Cheng, Bowen, Misra, Ishan, Schwing, Alexander G., Kirillov, Alexander, Girdhar, Rohit·2022·2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Abstract

Image segmentation groups pixels with different semantics, e.g., category or instance membership. Each choice of semantics defines a task. While only the semantics of each task differ, current research focuses on designing spe-cialized architectures for each task. We present Masked- attention Mask Transformer (Mask2Former), a new archi-tecture capable of addressing any image segmentation task (panoptic, instance or semantic). Its key components in-clude masked attention, which extracts localized features by constraining cross-attention within predicted mask regions. In addition to reducing the research effort by at least three times, it outperforms the best specialized architectures by a significant margin on four popular datasets. Most no-tably, Mask2Former sets a new state-of-the-art for panoptic segmentation (57.8 PQ on COCO), instance segmentation (50.1 AP on COCO) and semantic segmentation (57.7 mIoU onADE20K).

Integrity signals

Registry

Crossref · 10.1109/cvpr52688.2022.00135

Cites

41 of 41 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • CLFT uses a bidirectional network and cross-fusion strategy to integrate data at the Transformer decoder layer, enhancing accuracy and robustness of semantic segmentation in autonomous driving.

    “The model utilizes a bidirectional network and a cross-fusion strategy to integrate data from both sensors at the Transformer decoder layer, enhancing the accuracy and robustness of semantic segmentation in autonomous driving.”
  • CLFT fuses camera and LiDAR data by projecting LiDAR point clouds onto the camera plane, upsampling them into 2D feature maps, and inputting them with camera images into a Transformer-based network.

    “The CLFT [61] model achieves the fusion of camera and LiDAR data by projecting Li-DAR point clouds onto the camera plane, upsampling them into 2D feature maps, and then inputting them together with camera images into a Transformer-based network.”

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Cheng, B., Misra, I., Schwing, A. G., Kirillov, A., & Girdhar, R.
Affiliations
  • Meta (United States) (US)
  • University of Illinois Urbana-Champaign (US)
Venue
2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2022-06
Pages
1280-1289
ISSN
1063-6919 (ISSN-L)
OpenAlex
W4226069476
Cited by
3,565
References
41
Citations by year
2026: 907 · 2025: 1179 · 2024: 973 · 2023: 456 · 2022: 49 · 2019: 1
FWCI
240.91
Type
conference paper
Language
en
Open access
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • NSF (1718221,2008387,2045586,2106825)
  • MRI (1725729)
  • NIFA (2020-67021-32799)
  • Cisco Systems Inc. (1377144)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
Keywords
Mask2Former, attention masking, cross-attention, panoptic segmentation, instance segmentation, semantic segmentation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review