The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Cross-Image Relational Knowledge Distillation for Semantic Segmentation
Open accessAbstract
Current Knowledge Distillation (KD) methods for semantic segmentation often guide the student to mimic the teacher's structured information generated from individual data samples. However, they ignore the global semantic relations among pixels across various images that are valuable for KD. This paper proposes a novel Cross-Image Relational KD (CIRKD), which focuses on transferring structured pixel-to-pixel and pixel-to-region relations among the whole images. The motivation is that a good teacher network could construct a well-structured feature space in terms of global pixel dependencies. CIRKD makes the student mimic better structured semantic relations from the teacher, thus improving the segmentation performance. Experimental results over Cityscapes, CamVid and Pascal VOC datasets demonstrate the effectiveness of our proposed approach against state-of-the-art distillation methods. The code is available at https://github.com/winycg/CIRKD.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982
Rethinking Atrous Convolution for Semantic Image Segmentation
Chen, L.-C., Papandreou, G., Schroff, F., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1706.05587 · Cited by 7,517
SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J. M., & Luo, P. (2021) · arXiv · doi:10.48550/arxiv.2105.15203 · Cited by 5,090
Unsupervised Feature Learning via Non-parametric Instance Discrimination
Wu, Z., Xiong, Y., Yu, S. X., & Lin, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00393 · Cited by 3,664
RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation
Lin, G., Milan, A., Shen, C., & Reid, I. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.549 · Cited by 3,287 · cited 1×
“RefineNet [19] preserves high-resolution predictions by long-range residual connections for the down-sampling process.”
BackgroundBiSeNet: Bilateral Segmentation Network for Real-Time Semantic Segmentation
Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., & Sang, N. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_20 · Cited by 2,968
FitNets: Hints for Thin Deep Nets
Romero, A., Ballas, N., Kahou, S. E., Chassang, A., Gatta, C., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.6550 · Cited by 2,859
Zagoruyko, S., & Komodakis, N. (2016) · arXiv · doi:10.48550/arxiv.1612.03928 · Cited by 2,027
Tian, Y., Krishnan, D., & Isola, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_45 · Cited by 1,997
ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
Paszke, A., Chaurasia, A., Kim, S., & Culurciello, E. (2016) · arXiv · doi:10.48550/arxiv.1606.02147 · Cited by 1,826
ICNet for Real-Time Semantic Segmentation on High-Resolution Images
Zhao, H., Qi, X., Shen, X., Shi, J., & Jia, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_25 · Cited by 1,709
Relational Knowledge Distillation
Park, W., Kim, D., Lu, Y., & Cho, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00409 · Cited by 1,681
Semantic contours from inverse detectors
Hariharan, B., Arbelaez, P., Bourdev, L., Maji, S., & Malik, J. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126343 · Cited by 1,624
Object-Contextual Representations for Semantic Segmentation
Yuan, Y., Chen, X., & Wang, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58539-6_11 · Cited by 1,285
Quantized Convolutional Neural Networks for Mobile Devices
Wu, J., Leng, C., Wang, Y., Hu, Q., & Cheng, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.521 · Cited by 1,212
Similarity-Preserving Knowledge Distillation
Tung, F., & Mori, G. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00145 · Cited by 1,079
Segmentation and Recognition Using Structure from Motion Point Clouds
Brostow, G. J., Shotton, J., Fauqueur, J., & Cipolla, R. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88682-2_5 · Cited by 1,050
ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation
Mehta, S., Rastegari, M., Caspi, A., Shapiro, L., & Hajishirzi, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_34 · Cited by 1,029
A Comprehensive Overhaul of Feature Distillation
Heo, B., Kim, J., Yun, S., Park, H., Kwak, N., & Choi, J. Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00201 · Cited by 706
Structured Knowledge Distillation for Semantic Segmentation
Liu, Y., Chen, K., Liu, C., Qin, Z., Luo, Z., & Wang, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00271 · Cited by 681
Exploring Cross-Image Pixel Contrast for Semantic Segmentation
Wang, W., Zhou, T., Yu, F., Dai, J., Konukoglu, E., & Gool, L. V. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00721 · Cited by 604
Correlation Congruence for Knowledge Distillation
Peng, B., Jin, X., Li, D., Zhou, S., Wu, Y., Liu, J., Zhang, Z., & Liu, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00511 · Cited by 570
Channel-wise Knowledge Distillation for Dense Prediction
Shu, C., Liu, Y., Gao, J., Yan, Z., & Shen, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00526 · Cited by 452
Mimicking Very Efficient Network for Object Detection
Li, Q., Jin, S., & Yan, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.776 · Cited by 379
Cross-Batch Memory for Embedding Learning
Wang, X., Zhang, H., Huang, W., & Scott, M. R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00642 · Cited by 254
Knowledge Adaptation for Efficient Semantic Segmentation
He, T., Shen, C., Tian, Z., Gong, D., Sun, C., & Yan, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00067 · Cited by 223
Structured Knowledge Distillation for Dense Prediction
Liu, Y., Shu, C., Wang, J., & Shen, C. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3001940 · Cited by 197
BiSeNet: Bilateral Segmentation Network for Real-time Semantic SegmentationPreprint
Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., & Sang, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.00897 · Cited by 179
Intra-class Feature Variation Distillation for Semantic Segmentation
Wang, Y., Zhou, W., Jiang, T., Bai, X., & Xu, Y. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58571-6_21 · Cited by 164
Self-citationRethinking Soft Labels for Knowledge Distillation: A Bias-Variance Tradeoff PerspectivePreprint
Zhou, H., Song, L., Chen, J., Zhou, Y., Wang, G., Yuan, J., & Zhang, Q. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.00650 · Cited by 161
Self-citationRegion-aware Contrastive Learning for Semantic Segmentation
Hu, H., Cui, J., & Wang, L. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01598 · Cited by 148
SEED: Self-supervised Distillation For Visual RepresentationPreprint
Fang, Z., Wang, J., Wang, L., Zhang, L., Yang, Y., & Liu, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.04731 · Cited by 144
Efficient Semantic Video Segmentation with Per-Frame Inference
Liu, Y., Shen, C., Yu, C., & Wang, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58607-2_21 · Cited by 139
Improving Fast Segmentation With Teacher-student LearningPreprint
Xie, J., Shuai, B., Hu, J., Lin, J., & Zheng, W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.08476 · Cited by 97
Mutual Contrastive Learning for Visual Representation Learning
Yang, C., An, Z., Cai, L., & Xu, Y. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20211 · Cited by 95
Self-citationHierarchical Self-supervised Augmented Knowledge Distillation
Yang, C., An, Z., Cai, L., & Xu, Y. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/168 · Cited by 94
Self-citationGated Convolutional Networks with Hybrid Connectivity for Image Classification
Yang, C., An, Z., Zhu, H., Hu, X., Zhang, K., Xu, K., Li, C., & Xu, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6948 · Cited by 68
Self-citationPrior Gradient Mask Guided Pruning-Aware Fine-Tuning
Cai, L., An, Z., Yang, C., Yan, Y., & Xu, Y. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.19888 · Cited by 40
Self-citationMulti-objective Pruning for CNNs Using Genetic Algorithm
Yang, C., An, Z., Li, C., Diao, B., & Xu, Y. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-30484-3_25 · Cited by 39
Self-citationMulti-View Contrastive Learning for Online Knowledge Distillation
Yang, C., An, Z., & Xu, Y. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414664 · Cited by 26
Self-citation
Metadata
- Authors
- Yang, C., Zhou, H., An, Z., Jiang, X., Xu, Y., & Zhang, Q.
- Venue
- 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2022-06
- Pages
- 12309-12318
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4312309807
- Cited by
- 265
- References
- 54
- FWCI
- 21.05
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.