CoChat
Sign Up
PreprintCited by 619Domain Adaptation and Few-Shot LearningAdvanced Neural Network ApplicationsMachine Learning and Algorithms

The DOI matches its DataCite record.

Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentation

Open access
Yang, Lihe, Qi, Lei, Feng, Litong, Zhang, Wayne, Shi, Yinghuan·2022·arXiv
DOI 10.48550/arxiv.2208.09910arXiv 2208.09910
Open this paper in CoChat — free accountOpen PDF

Abstract

In this work, we revisit the weak-to-strong consistency framework, popularized by FixMatch from semi-supervised classification, where the prediction of a weakly perturbed image serves as supervision for its strongly perturbed version. Intriguingly, we observe that such a simple pipeline already achieves competitive results against recent advanced works, when transferred to our segmentation scenario. Its success heavily relies on the manual design of strong data augmentations, however, which may be limited and inadequate to explore a broader perturbation space. Motivated by this, we propose an auxiliary feature perturbation stream as a supplement, leading to an expanded perturbation space. On the other, to sufficiently probe original image-level augmentations, we present a dual-stream perturbation technique, enabling two strong views to be simultaneously guided by a common weak view. Consequently, our overall Unified Dual-Stream Perturbations approach (UniMatch) surpasses all existing methods significantly across all evaluation protocols on the Pascal, Cityscapes, and COCO benchmarks. Its superiority is also demonstrated in remote sensing interpretation and medical image analysis. We hope our reproduced FixMatch and our results can inspire more future works. Code and logs are available at https://github.com/LiheYoung/UniMatch.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2208.09910

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 74

  1. Inigo Alonso, Alberto Sabater, David Ferstl, Luis Montesano, and Ana C Murillo. Semi-supervised semantic segmenta- tion with pixel-level contrastive learning from a class-wise memory bank. In ICCV, 2021. 3
  2. Wele Gedara Chaminda Bandara and Vishal M Patel. Revis- iting consistency regularization for semi-supervised change detection in remote sensing images. arXiv:2204.08454, 2022. · arXiv
  3. David Berthelot, Nicholas Carlini, Ekin D Cubuk, Alex Ku- rakin, Kihyuk Sohn, Han Zhang, and Colin Raffel. Remix- match: Semi-supervised learning with distribution alignment and augmentation anchoring. In ICLR, 2020. 2, 4, 10
  4. David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin Raffel. Mixmatch: A holistic approach to semi-supervised learning. In NeurIPS, 2019. 2, 10
  5. Avrim Blum and Tom Mitchell. Combining labeled and unlabeled data with co-training. In COLT, 1998. 3
  6. Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learn- ing of visual features by contrasting cluster assignments. In NeurIPS, 2020. 4
  7. Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In ECCV, 2018. 5
  8. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geof- frey Hinton. A simple framework for contrastive learning of visual representations. In ICML, 2020. 2, 5
  9. Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In CVPR, 2021. 5
  10. Xiaokang Chen, Yuhui Yuan, Gang Zeng, and Jingdong Wang. Semi-supervised semantic segmentation with cross pseudo supervision. In CVPR, 2021. 1, 3, 5, 6, 8
  11. François Chollet. Xception: Deep learning with depthwise separable convolutions. In CVPR, 2017. 6
  12. Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In CVPR, 2016. 6
  13. Terrance DeVries and Graham W Taylor. Improved reg- ularization of convolutional neural networks with cutout. arXiv:1708.04552, 2017. 3 · arXiv
  14. Mark Everingham, SM Ali Eslami, Luc Van Gool, Christo- pher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes challenge: A retrospective. IJCV, 2015. 5
  15. Geoff French, Timo Aila, Samuli Laine, Michal Mackiewicz, and Graham Finlayson. Semi-supervised semantic segmenta- tion needs strong, high-dimensional perturbations. In BMVC, 2020. 1, 3
  16. Chengyue Gong, Dilin Wang, and Qiang Liu. Alphamatch: Improving consistency for semi-supervised learning with alpha-divergence. In CVPR, 2021. 2
  17. Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. In NeurIPS, 2014. 1, 3
  18. Yves Grandvalet and Yoshua Bengio. Semi-supervised learn- ing by entropy minimization. In NeurIPS, 2005. 2
  19. Dayan Guan, Jiaxing Huang, Aoran Xiao, and Shijian Lu. Un- biased subclass regularization for semi-supervised semantic segmentation. In CVPR, 2022. 1, 3
  20. Bharath Hariharan, Pablo Arbeláez, Lubomir Bourdev, Subhransu Maji, and Jitendra Malik. Semantic contours from inverse detectors. In ICCV, 2011. 5
  21. Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual repre- sentation learning. In CVPR, 2020. 2, 5
  22. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016. 5
  23. Ruifei He, Jihan Yang, and Xiaojuan Qi. Re-distributing biased pseudo labels for semi-supervised semantic segmenta- tion: A baseline investigation. In ICCV, 2021. 1, 3
  24. Hanzhe Hu, Fangyun Wei, Han Hu, Qiwei Ye, Jinshi Cui, and Liwei Wang. Semi-supervised semantic segmentation via adaptive equalization learning. In NeurIPS, 2021. 1, 3, 6
  25. Xinyue Huo, Lingxi Xie, Jianzhong He, Zijie Yang, and Qi Tian. Atso: Asynchronous teacher-student optimization for semi-supervised image segmentation. In CVPR, 2021. 1
  26. Jisoo Jeong, Seungeui Lee, Jeesoo Kim, and Nojun Kwak. Consistency-based semi-supervised learning for object detec- tion. In NeurIPS, 2019. 2
  27. Zhanghan Ke, Di Qiu, Kaican Li, Qiong Yan, and Ryn- son WH Lau. Guided collaborative training for pixel-wise semi-supervised learning. In ECCV, 2020. 3
  28. Chia-Wen Kuo, Chih-Yao Ma, Jia-Bin Huang, and Zsolt Kira. Featmatch: Feature-based augmentation for semi-supervised learning. In ECCV, 2020. 2
  29. Donghyeon Kwon and Suha Kwak. Semi-supervised semantic segmentation with error localization network. In CVPR, 2022. 3
  30. Xin Lai, Zhuotao Tian, Li Jiang, Shu Liu, Hengshuang Zhao, Liwei Wang, and Jiaya Jia. Semi-supervised semantic seg- mentation with directional context-aware consistency. In CVPR, 2021. 3, 5
  31. Samuli Laine and Timo Aila. Temporal ensembling for semi- supervised learning. In ICLR, 2017. 2
  32. Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In ICML Workshop, 2013. 2
  33. Junnan Li, Caiming Xiong, and Steven CH Hoi. Comatch: Semi-supervised learning with contrastive graph regulariza- tion. In ICCV, 2021. 2
  34. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, 2014. 6
  35. Shikun Liu, Shuaifeng Zhi, Edward Johns, and Andrew J Davison. Bootstrapping semantic segmentation with regional contrast. In ICLR, 2022. 2, 3, 5
  36. Yuyuan Liu, Yu Tian, Yuanhong Chen, Fengbei Liu, Vasileios Belagiannis, and Gustavo Carneiro. Perturbed and strict mean teachers for semi-supervised semantic segmentation. In CVPR, 2022. 2, 3, 4, 5, 6, 7
  37. Yen-Cheng Liu, Chih-Yao Ma, Zijian He, Chia-Wen Kuo, Kan Chen, Peizhao Zhang, Bichen Wu, Zsolt Kira, and Peter Vajda. Unbiased teacher for semi-supervised object detection. In ICLR, 2021. 1, 3
  38. Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In CVPR, 2015. 1
  39. Luke Melas-Kyriazi and Arjun K Manrai. Pixmatch: Unsuper- vised domain adaptation via pixelwise consistency training. In CVPR, 2021. 1, 3
  40. Robert Mendel, Luis Antonio de Souza, David Rauber, João Paulo Papa, and Christoph Palm. Semi-supervised seg- mentation based on error-correcting supervision. In ECCV, 2020. 3
  41. Sudhanshu Mittal, Maxim Tatarchenko, and Thomas Brox. Semi-supervised semantic segmentation with high-and low- level consistency. TPAMI, 2019. 1, 3, 8
  42. Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. Virtual adversarial training: a regularization method for supervised and semi-supervised learning. TPAMI, 2018. 2, 7
  43. Yassine Ouali, Céline Hudelot, and Myriam Tami. Semi- supervised semantic segmentation with cross-consistency training. In CVPR, 2020. 1, 3, 7
  44. Hieu Pham, Zihang Dai, Qizhe Xie, and Quoc V Le. Meta pseudo labels. In CVPR, 2021. 2
  45. Siyuan Qiao, Wei Shen, Zhishuai Zhang, Bo Wang, and Alan Yuille. Deep co-training for semi-supervised image recogni- tion. In ECCV, 2018. 3
  46. Chuck Rosenberg, Martial Hebert, and Henry Schneiderman. Semi-supervised self-training of object detection models. In WACV/MOTION, 2005. 2
  47. Mehdi Sajjadi, Mehran Javanmardi, and Tolga Tasdizen. Reg- ularization with stochastic transformations and perturbations for deep semi-supervised learning. In NeurIPS, 2016. 2
  48. Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. In NeurIPS, 2020. 1, 2, 3, 10
  49. Nasim Souly, Concetto Spampinato, and Mubarak Shah. Semi supervised semantic segmentation using generative adversar- ial network. In ICCV, 2017. 1, 3
  50. Yihe Tang, Weifeng Chen, Yijun Luo, and Yuting Zhang. Humble teachers teach better students for semi-supervised object detection. In CVPR, 2021. 1, 3
  51. Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In NeurIPS, 2017. 2
  52. Aaron Van den Oord, Yazhe Li, and Oriol Vinyals. Rep- resentation learning with contrastive predictive coding. arXiv:1807.03748, 2018. 5 · arXiv
  53. Yidong Wang, Hao Chen, Qiang Heng, Wenxin Hou, Marios Savvides, Takahiro Shinozaki, Bhiksha Raj, Zhen Wu, and Jindong Wang. Freematch: Self-adaptive thresholding for semi-supervised learning. arXiv:2205.07246, 2022. 2 · arXiv
  54. Yuchao Wang, Haochen Wang, Yujun Shen, Jingjing Fei, Wei Li, Guoqiang Jin, Liwei Wu, Rui Zhao, and Xinyi Le. Semi- supervised semantic segmentation using unreliable pseudo- labels. In CVPR, 2022. 1, 2, 3, 5, 6
  55. Junfei Xiao, Longlong Jing, Lin Zhang, Ju He, Qi She, Zong- wei Zhou, Alan Yuille, and Yingwei Li. Learning from tempo- ral gradient for semi-supervised action recognition. In CVPR, 2022. 1, 3
  56. Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Luong, and Quoc V Le. Unsupervised data augmentation for consis- tency training. In NeurIPS, 2020. 2, 10
  57. Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoc V Le. Self-training with noisy student improves imagenet clas- sification. In CVPR, 2020. 2
  58. Hai-Ming Xu, Lingqiao Liu, Qiuchen Bian, and Zhen Yang. Semi-supervised semantic segmentation with prototype-based consistency regularization. In NeurIPS, 2022. 5, 6
  59. Mengde Xu, Zheng Zhang, Han Hu, Jianfeng Wang, Lijuan Wang, Fangyun Wei, Xiang Bai, and Zicheng Liu. End-to-end semi-supervised object detection with soft teacher. In ICCV, 2021. 1, 3
  60. Yinghao Xu, Fangyun Wei, Xiao Sun, Ceyuan Yang, Yujun Shen, Bo Dai, Bolei Zhou, and Stephen Lin. Cross-model pseudo-labeling for semi-supervised action recognition. In CVPR, 2022. 1, 3
  61. Lihe Yang, Wei Zhuo, Lei Qi, Yinghuan Shi, and Yang Gao. St++: Make self-training work better for semi-supervised semantic segmentation. In CVPR, 2022. 1, 2, 3, 5
  62. Lequan Yu, Shujun Wang, Xiaomeng Li, Chi-Wing Fu, and Pheng-Ann Heng. Uncertainty-aware self-ensembling model for semi-supervised 3d left atrium segmentation. In MICCAI, 2019. 8
  63. Jianlong Yuan, Yifan Liu, Chunhua Shen, Zhibin Wang, and Hao Li. A simple baseline for semi-supervised semantic segmentation with strong data augmentation. In ICCV, 2021. 1, 3
  64. Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regu- larization strategy to train strong classifiers with localizable features. In ICCV, 2019. 2, 3, 5
  65. Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu, Jindong Wang, Manabu Okumura, and Takahiro Shinozaki. Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling. In NeurIPS, 2021. 2, 10
  66. Hang Zhang, Kristin Dana, Jianping Shi, Zhongyue Zhang, Xiaogang Wang, Ambrish Tyagi, and Amit Agrawal. Context encoding for semantic segmentation. In CVPR, 2018. 1
  67. Jianrong Zhang, Tianyi Wu, Chuanghao Ding, Hongwei Zhao, and Guodong Guo. Region-level contrastive and consistency learning for semi-supervised semantic segmentation. In IJCAI, 2022. 3
  68. Pan Zhang, Bo Zhang, Ting Zhang, Dong Chen, and Fang Wen. Robust mutual learning for semi-supervised semantic segmentation. arXiv:2106.00609, 2021. 3 · arXiv
  69. Ying Zhang, Tao Xiang, Timothy M Hospedales, and Huchuan Lu. Deep mutual learning. In CVPR, 2018. 3
  70. Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. Pyramid scene parsing network. In CVPR, 2017. 1
  71. Yuanyi Zhong, Bodi Yuan, Hong Wu, Zhiqiang Yuan, Jian Peng, and Yu-Xiong Wang. Pixel contrastive-consistent semi- supervised semantic segmentation. In ICCV, 2021. 3, 5, 6
  72. Yanning Zhou, Hang Xu, Wei Zhang, Bin Gao, and Pheng- Ann Heng. C3-semiseg: Contrastive semi-supervised seg- mentation via cross-set learning and dynamic class-balancing. In ICCV, 2021. 3
  73. Barret Zoph, Golnaz Ghiasi, Tsung-Yi Lin, Yin Cui, Hanxiao Liu, Ekin D Cubuk, and Quoc V Le. Rethinking pre-training and self-training. In NeurIPS, 2020. 2, 10
  74. Yuliang Zou, Zizhao Zhang, Han Zhang, Chun-Liang Li, Xiao Bian, Jia-Bin Huang, and Tomas Pfister. Pseudoseg: Designing pseudo labels for semantic segmentation. In ICLR, 2021. 1, 3, 6

Cited by

Sorted by citations

See all 619

Metadata

Authors
Yang, L., Qi, L., Feng, L., Zhang, W., & Shi, Y.
Affiliations
  • Nanjing University (CN)
  • Southeast University (CN)
  • Sensetime (China) (CN)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2022
Pages
7236-7246
ISSN
1063-6919 (ISSN-L)
DOI
10.48550/arxiv.2208.09910
arXiv
2208.09910
OpenAlex
W4386075981
Cited by
619
References
0
Citations by year
2026: 171 · 2025: 238 · 2024: 177 · 2023: 33
FWCI
109.58
Type
conference paper
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • University of Sydney
  • National Natural Science Foundation of China (62222604,62206052,62192783, 62192783, BK20210224, 62222604, 62206052)
  • China Postdoctoral Science Foundation (2021M690609)
  • Natural Science Foundation of Jiangsu Province (BK20210224)
  • SenseTime Group
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
Keywords
semi-supervised semantic segmentation, FixMatch, feature perturbation, cityscape, COCO, remote sensing interpretation, medical image analysis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review