Perspective-Adaptive Convolutions for Scene Parsing
The DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index.
Abstract
Many existing scene parsing methods adopt Convolutional Neural Networks with receptive fields of fixed sizes and shapes, which frequently results in inconsistent predictions of large objects and invisibility of small objects. To tackle this issue, we propose perspective-adaptive convolutions to acquire receptive fields of flexible sizes and shapes during scene parsing. Through adding a new perspective regression layer, we can dynamically infer the position-adaptive perspective coefficient vectors utilized to reshape the convolutional patches. Consequently, the receptive fields can be adjusted automatically according to the various sizes and perspective deformations of the objects in scene images. Our proposed convolutions are differentiable to learn the convolutional parameters and perspective coefficients in an end-to-end way without any extra training supervision of object sizes. Furthermore, considering that the standard convolutions lack contextual information and spatial dependencies, we propose a context adaptive bias to capture both local and global contextual information through average pooling on the local feature patches and global feature maps, followed by flexible attentive summing to the convolutional results. The attentive weights are position-adaptive and context-aware, and can be learned through adding an additional context regression layer. Experiments on Cityscapes and ADE20K datasets well demonstrate the effectiveness of the proposed methods.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 3×
“This network transfers the ResNet101 model [3] pre-trained on ImageNet dataset [4] to the convolu-tional-deconvolutional structure.”
BackgroundMethodFully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433 · cited 3×
“Then we perform experiments on two challenging scene parsing benchmarks, including Cityscapes dataset [10] and ADE20K dataset [11].”
MethodSpeech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · cited 1×
“Both global and local contextual information are exploited in recent approaches through graphical models [17], [35] or Recurrent Neural Networks (RNNs) [36], [37].”
BackgroundMulti-scale context aggregation by dilated convolutions
Yu, F., & Koltun, V. (2015) · arXiv · doi:10.48550/arxiv.1511.07122 · Cited by 7,738 · cited 3×
“Some other methods also leverage a sequence of CNN layers with large receptive fields to capture contextual information [20], [44].”
MethodInfluentialDeformable Convolutional Networks
Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., & Wei, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.89 · Cited by 7,513 · cited 4×
“As shown in Table 2, the reproduced Deformable ConvNets achieves 76.6 percent in Mean IoU, which is higher than 75.2 percent reported in [34].”
BackgroundMethodResultInfluentialSelective Search for Object Recognition
Uijlings, J. R. R., van de Sande, K. E. A., Gevers, T., & Smeulders, A. W. M. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0620-5 · Cited by 5,980
Jaderberg, M., Simonyan, K., Zisserman, A., & Kavukcuoglu, K. (2015) · arXiv · doi:10.48550/arxiv.1506.02025 · Cited by 5,137
Fully-Convolutional Siamese Networks for Object Tracking
Bertinetto, L., Valmadre, J., Henriques, J. F., Vedaldi, A., & Torr, P. H. S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-48881-3_56 · Cited by 4,546 · cited 1×
“For example: (1) in tasks related to the objects of various sizes, such as object detection [27], object tracking [46] and image caption [47], the perspective-adaptive convolu-”
MethodLearning Deconvolution Network for Semantic Segmentation
Noh, H., Hong, S., & Han, B. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.178 · Cited by 4,422
Scene Parsing through ADE20K Dataset
Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., & Torralba, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.544 · Cited by 3,326
RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation
Lin, G., Milan, A., Shen, C., & Reid, I. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.549 · Cited by 3,287 · cited 2×
“RefineNet(ResNet101) [52] 40.”
Holistically-Nested Edge Detection
Xie, S., & Tu, Z. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.164 · Cited by 3,096
Edge Boxes: Locating Object Proposals from Edges
Zitnick, C. L., & Dollár, P. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_26 · Cited by 2,656
Conditional Random Fields as Recurrent Neural Networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., & Torr, P. H. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.179 · Cited by 2,386 · cited 2×
“Some approaches [17], [35], [38], [39] formulate graphical models as special layers and insert these layers into FCN based frameworks, thus”
Understanding Convolution for Semantic Segmentation
Wang, P., Chen, P., Yuan, Y., Liu, D., Huang, Z., Hou, X., & Cottrell, G. (2018) · 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2018.00163 · Cited by 1,957
MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
Chen, T., Li, M., Li, Y., Lin, M., Wang, N., Wang, M., Xiao, T., Xu, B., Zhang, C., & Zhang, Z. (2015) · arXiv · doi:10.48550/arxiv.1512.01274 · Cited by 1,819
Hypercolumns for object segmentation and fine-grained localization
Hariharan, B., Arbelaez, P., Girshick, R., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298642 · Cited by 1,511
Attention to Scale: Scale-Aware Semantic Image Segmentation
Chen, L.-C., Yang, Y., Wang, J., Xu, W., & Yuille, A. L. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.396 · Cited by 1,377
Instance-aware Semantic Segmentation via Multi-task Network Cascades
Dai, J., He, K., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.04412 · Cited by 1,273
Instance-Aware Semantic Segmentation via Multi-task Network Cascades
Dai, J., He, K., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.343 · Cited by 1,244
Adaptive deconvolutional networks for mid and high level feature learning
Zeiler, M. D., Taylor, G. W., & Fergus, R. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126474 · Cited by 1,217
DenseCap: Fully Convolutional Localization Networks for Dense Captioning
Johnson, J., Karpathy, A., & Fei-Fei, L. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.494 · Cited by 1,137
Fully Convolutional Instance-Aware Semantic Segmentation
Li, Y., Qi, H., Dai, J., Ji, X., & Wei, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.472 · Cited by 1,116 · cited 1×
“Some other approaches [25], [26] insert the Region Proposal Networks (RPNs) [27] into CNNs to learn object proposals and resize the feature patches of proposals through the ROI-pooling operator.”
MethodSimultaneous Detection and Segmentation
Hariharan, B., Arbeláez, P., Girshick, R., & Malik, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10584-0_20 · Cited by 1,085
Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation
Lin, G., Shen, C., van den Hengel, A., & Reid, I. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.348 · Cited by 862
Scene Text Recognition using Higher Order Language Priors
Mishra, A., Alahari, K., & Jawahar, C. (2012) · Procedings of the British Machine Vision Conference 2012 · doi:10.5244/c.26.127 · Cited by 831
Putting Objects in Perspective
Hoiem, D., Efros, A. A., & Hebert, M. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-008-0137-5 · Cited by 805
Semantic Image Segmentation via Deep Parsing Network
Liu, Z., Li, X., Luo, P., Loy, C.-C., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.162 · Cited by 675
Geometric context from a single image
Hoiem, D., Efros, A. A., & Hebert, M. (2005) · Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 · doi:10.1109/iccv.2005.107 · Cited by 662
Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation
Ghiasi, G., & Fowlkes, C. C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_32 · Cited by 502
Pulling Things out of Perspective
Ladicky, L., Shi, J., & Pollefeys, M. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.19 · Cited by 457
Feedforward semantic segmentation with zoom-out features
Mostajabi, M., Yadollahpour, P., & Shakhnarovich, G. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298959 · Cited by 436
Scene labeling with LSTM recurrent neural networks
Byeon, W., Breuel, T. M., Raue, F., & Liwicki, M. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298977 · Cited by 375
Semantic Object Parsing with Graph LSTM
Liang, X., Shen, X., Feng, J., Lin, L., & Yan, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_8 · Cited by 352
Self-citationMulti-dimensional Recurrent Neural Networks
Graves, A., Fernández, S., & Schmidhuber, J. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-74690-4_56 · Cited by 303 · cited 1×
“Both global and local contextual information are exploited in recent approaches through graphical models [17], [35] or Recurrent Neural Networks (RNNs) [36], [37].”
BackgroundHuman Parsing with Contextualized Convolutional Neural Network
Liang, X., Xu, C., Shen, X., Yang, J., Liu, S., Tang, J., Lin, L., & Yan, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.163 · Cited by 245 · cited 2×
“Multi-scale features or predictions learned from both of the three types of approaches are aggregated with feature concatenation [13], [15], [19], weighted summation [6], [14], [20] or attention models [12].”
BackgroundSelf-citationWider or Deeper: Revisiting the ResNet Model for Visual Recognition
Wu, Z., Shen, C., & van den Hengel, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.10080 · Cited by 198
Scale-Adaptive Convolutions for Scene Parsing
Zhang, R., Tang, S., Zhang, Y., Li, J., & Yan, S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.224 · Cited by 195 · cited 6×
“Apreliminary version of this paper has appeared as a full paper in the 2017 IEEE International Conference on Computer Vision (ICCV) [9].”
BackgroundMethodInfluentialSelf-citationCombining Appearance and Structure from Motion Features for Road Scene Understanding
Sturgess, P., Alahari, K., Ladicky, L., & Torr, P. H. S. (2009) · Procedings of the British Machine Vision Conference 2009 · doi:10.5244/c.23.62 · Cited by 189
Monocular Object Instance Segmentation and Depth Ordering with CNNs
Zhang, Z., Schwing, A. G., Fidler, S., & Urtasun, R. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.300 · Cited by 176
Semantic Object Parsing with Local-Global Long Short-Term Memory
Liang, X., Shen, X., Xiang, D., Feng, J., Lin, L., & Yan, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.347 · Cited by 175
Self-citationDAG-Recurrent Neural Networks for Scene Labeling
Shuai, B., Zuo, Z., Wang, B., & Wang, G. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.394 · Cited by 151
Gaussian Conditional Random Field Network for Semantic Segmentation
Vemulapalli, R., Tuzel, O., Liu, M.-Y., & Chellappa, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.351 · Cited by 150 · cited 1×
“Some approaches [17], [35], [38], [39] formulate graphical models as special layers and insert these layers into FCN based frameworks, thus”
MethodVideo Scene Parsing with Predictive Feature Learning
Jin, X., Li, X., Xiao, H., Shen, X., Lin, Z., Yang, J., Chen, Y., Dong, J., Liu, L., Jie, Z., et al. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.595 · Cited by 103
Self-citationFoveaNet: Perspective-Aware Urban Scene Parsing
Li, X. Y., Jie, Z., Wang, W., Liu, C., Yang, J., Shen, X., Lin, Z., Chen, Q., Yan, S., & Feng, J. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.91 · Cited by 61
Self-citationGlobal-residual and Local-boundary Refinement Networks for Rectifying Scene Parsing Predictions
Zhang, R., Tang, S., Lin, M., Li, J., & Yan, S. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/479 · Cited by 37
Self-citationMulti-Path Feedback Recurrent Neural Network for Scene ParsingPreprint
Jin, X., Chen, Y., Feng, J., Jie, Z., & Yan, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.07706 · Cited by 23
Self-citationHigh Resolution Feature Recovering for Accelerating Urban Scene Parsing
Zhang, R., Tang, S., Liu, L., Zhang, Y., Li, J., & Yan, S. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/161 · Cited by 16
Self-citation
Cited by
CGNet: A Light-Weight Context Guided Network for Semantic Segmentation
Wu, T., Tang, S., Zhang, R., Cao, J., & Zhang, Y. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3042065 · Cited by 859
Self-citationDomain-Aware Visual Bias Eliminating for Generalized Zero-Shot Learning
Min, S., Yao, H., Xie, H., Wang, C., Zha, Z.-J., & Zhang, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01268 · Cited by 181
Self-citationBoundary Perception Guidance: A Scribble-Supervised Semantic Segmentation Approach
Wang, B., Qi, G.-J., Tang, S., Zhang, T., Wei, Y., Li, L., & Zhang, Y. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/508 · Cited by 86
Self-citationPale Transformer: A General Vision Transformer Backbone with Pale-Shaped Attention
Wu, S., Wu, T., Tan, H., & Guo, G. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20176 · Cited by 110
Few-Shot Structured Domain Adaptation for Virtual-to-Real Scene Parsing
Zhang, J., Chen, Z., Huang, J., Lin, L., & Zhang, D. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00008 · Cited by 22
Metadata
- Authors
- Zhang, R., Tang, S., Zhang, Y., Li, J., & Yan, S.
- Affiliations
- Chinese Academy of Sciences (CN)
- Qihoo 360 (China) (CN)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2020-04-01
- Volume
- 42
- Issue
- 4
- Pages
- 909-924
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2907052985
- Cited by
- 25
- References
- 58
- Citations by year
- 2025: 2 · 2023: 2 · 2022: 5 · 2021: 10 · 2020: 4
- FWCI
- 2.32
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Natural Science Foundation of China (61525206, 61572472)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Remote-Sensing Image Classification
- Keywords
- scene parsing, semantic segmentation, receptive field adaptation, attention mechanism, small object detection, cityscape, ADE20K
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.