CoChat
Sign Up
PreprintCited by 7,738Advanced Neural Network ApplicationsDomain Adaptation and Few-Shot LearningRemote-Sensing Image Classification

Multi-scale context aggregation by dilated convolutions

Open access
Yu, Fisher, Koltun, Vladlen·2015·arXiv
DOI 10.48550/arxiv.1511.07122arXiv 1511.07122
Open this paper in CoChat — free accountOpen PDF

Abstract

State-of-the-art models for semantic segmentation are based on adaptations of convolutional networks that had originally been designed for image classification. However, dense prediction and image classification are structurally different. In this work, we develop a new convolutional network module that is specifically designed for dense prediction. The presented module uses dilated convolutions to systematically aggregate multi-scale contextual information without losing resolution. The architecture is based on the fact that dilated convolutions support exponential expansion of the receptive field without loss of resolution or coverage. We show that the presented context module increases the accuracy of state-of-the-art semantic segmentation systems. In addition, we examine the adaptation of image classification networks to dense prediction and show that simplifying the adapted network can increase accuracy.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1511.07122

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Dilated convolution captures multi-scale context with the same amount of parameters as standard convolutions.

    “Moreover, dilated convolution [36, 37] is proposed to capture the multi-scale context with the same amount of parameters as standard convolutions.”

    From DilatedFormer: dilated granularity transformer network for placental maturity grading in ultrasound (2023) · cites this paper as [36]

  • Based upon ResNet, many studies have managed to improve the performance of the original ResNet, such as pre-activation ResNet, wide ResNet, stochastic depth ResNets (SDR), and ResNet in ResNet (RiR).

    “Based upon ResNet, many studies have managed to improve the performance of the original ResNet, such as pre-activation ResNet [38], wide ResNet [39], stochastic depth ResNets (SDR) [40], and ResNet in ResNet (RiR) [41].”

    From A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects (2022) · cites this paper as [40]

    The Introduction (pp. 1–2) discusses adapting image-classification networks for dense prediction, and the Conclusion (p. 6) reports gains from dilated convolutions in semantic segmentation. Neither addresses pre-activation ResNet, wide ResNet, stochastic-depth ResNets, or ResNet in ResNet; searches for those variants and for ResNet performance found no supporting passage. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

  • Dilated convolution preserves the resolution of feature maps and enlarges the receptive field to incorporate larger contextual information.

    “To overcome the drawbacks of successive max-pooling and downsampling that lead to a loss of feature map resolution, dilated convolution [22], [23] was proposed to preserve the resolution of feature maps and enlarge the receptive field to incorporate larger contextual information.”

    From DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation (2019) · cites this paper as [23]

  • Multi-scale features extracted from CNNs have been shown to be effective for improving segmentation accuracy.

    “Multi-scale features extracted from CNNs have been shown to be effective for improving segmentation accuracy [11], [22], [23].”

    From DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation (2019) · cites this paper as [23]

  • The dilated network has become the de facto paradigm for semantic segmentation.

    “The dilated network has become the de facto paradigm for semantic segmentation.”

    From Unified Perceptual Parsing for Scene Understanding (2018) · cites this paper as [14]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 33

  1. Badrinarayanan, Vijay, Handa, Ankur, and Cipolla, Roberto. SegNet: A deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling. arXiv:1505.07293 , 2015. · arXiv
  2. Brostow, Gabriel J., Fauqueur, Julien, and Cipolla, Roberto. Semantic object classes in video: A high-definition ground truth database. Pattern Recognition Letters , 30(2), 2009.
  3. Chen, Liang-Chieh, Papandreou, George, Kokkinos, Iasonas, Murphy, Kevin, and Yuille, Alan L. Semantic image segmentation with deep convolutional nets and fully connected CRFs. In ICLR , 2015a.
  4. Chen, Liang-Chieh, Yang, Yi, Wang, Jiang, Xu, Wei, and Yuille, Alan L. Attention to scale: Scale-aware semantic image segmentation. arXiv:1511.03339 , 2015b. · arXiv
  5. Cordts, Marius, Omran, Mohamed, Ramos, Sebastian, Rehfeld, Timo, Enzweiler, Markus, Benenson, Rodrigo, Franke, Uwe, Roth, Stefan, and Schiele, Bernt. The Cityscapes dataset for semantic urban scene understanding. In CVPR , 2016.
  6. Everingham, Mark, Gool, Luc J. Van, Williams, Christopher K. I., Winn, John M., and Zisserman, Andrew. The Pascal visual object classes (VOC) challenge. IJCV , 88(2), 2010.
  7. Fischer, Philipp, Dosovitskiy, Alexey, Ilg, Eddy, H¨ ausser, Philip, Hazrba, Caner, Golkov, Vladimir, van der Smagt, Patrick, Cremers, Daniel, and Brox, Thomas. Learning optical flow with convolutional neural networks. In ICCV , 2015.
  8. Galleguillos, Carolina and Belongie, Serge J. Context based object categorization: A critical survey. Computer Vision and Image Understanding , 114(6), 2010.
  9. Geiger, Andreas, Lenz, Philip, Stiller, Christoph, and Urtasun, Raquel. Vision meets robotics: The KITTI dataset. International Journal of Robotics Research , 32(11), 2013.
  10. Glorot, Xavier and Bengio, Yoshua. Understanding the difficulty of training deep feedforward neural networks. In AISTATS , 2010.
  11. Hariharan, Bharath, Arbelaez, Pablo, Bourdev, Lubomir D., Maji, Subhransu, and Malik, Jitendra. Semantic contours from inverse detectors. In ICCV , 2011.
  12. He, Xuming, Zemel, Richard S., and Carreira-Perpi˜ n´ an, Miguel ´ A. Multiscale conditional random fields for image labeling. In CVPR , 2004.
  13. Holschneider, M., Kronland-Martinet, R., Morlet, J., and Tchamitchian, Ph. A real-time algorithm for signal analysis with the help of the wavelet transform. In Wavelets: Time-Frequency Methods and Phase Space. Proceedings of the International Conference , 1987.
  14. Jia, Yangqing, Shelhamer, Evan, Donahue, Jeff, Karayev, Sergey, Long, Jonathan, Girshick, Ross B., Guadarrama, Sergio, and Darrell, Trevor. Caffe: Convolutional architecture for fast feature embedding. In Proc. ACM Multimedia , 2014.
  15. Kohli, Pushmeet, Ladicky, Lubor, and Torr, Philip H. S. Robust higher order potentials for enforcing label consistency. IJCV , 82(3), 2009.
  16. Kr¨ ahenb¨ uhl, Philipp and Koltun, Vladlen. Efficient inference in fully connected CRFs with Gaussian edge potentials. In NIPS , 2011.
  17. Krizhevsky, Alex, Sutskever, Ilya, and Hinton, Geoffrey E. ImageNet classification with deep convolutional neural networks. In NIPS , 2012.
  18. Kundu, Abhijit, Vineet, Vibhav, and Koltun, Vladlen. Feature space optimization for semantic video segmentation. In CVPR , 2016.
  19. Ladicky, Lubor, Russell, Christopher, Kohli, Pushmeet, and Torr, Philip H. S. Associative hierarchical CRFs for object class image segmentation. In ICCV , 2009.
  20. Le, Quoc V., Jaitly, Navdeep, and Hinton, Geoffrey E. A simple way to initialize recurrent networks of rectified linear units. arXiv:1504.00941 , 2015. · arXiv
  21. LeCun, Yann, Boser, Bernhard, Denker, John S., Henderson, Donnie, Howard, Richard E., Hubbard, Wayne, and Jackel, Lawrence D. Backpropagation applied to handwritten zip code recognition. Neural Computation , 1(4), 1989.
  22. Lin, Guosheng, Shen, Chunhua, Reid, Ian, and van dan Hengel, Anton. Efficient piecewise training of deep structured models for semantic segmentation. arXiv:1504.01013 , 2015. · arXiv
  23. Lin, Tsung-Yi, Maire, Michael, Belongie, Serge, Hays, James, Perona, Pietro, Ramanan, Deva, Doll´ ar, Piotr, and Zitnick, C. Lawrence. Microsoft COCO: Common objects in context. In ECCV , 2014.
  24. Liu, Buyu and He, Xuming. Multiclass semantic video segmentation with object-level active inference. In CVPR , 2015.
  25. Long, Jonathan, Shelhamer, Evan, and Darrell, Trevor. Fully convolutional networks for semantic segmentation. In CVPR , 2015.
  26. Noh, Hyeonwoo, Hong, Seunghoon, and Han, Bohyung. Learning deconvolution network for semantic segmentation. In ICCV , 2015.
  27. Ros, Germ´ an, Ramos, Sebastian, Granados, Manuel, Bakhtiary, Amir, V´ azquez, David, and L´ opez, Antonio Manuel. Vision-based offline-online perception paradigm for autonomous driving. In WACV , 2015.
  28. Shensa, Mark J. The discrete wavelet transform: wedding the ` a trous and Mallat algorithms. IEEE Transactions on Signal Processing , 40(10), 1992.
  29. Shotton, Jamie, Winn, John M., Rother, Carsten, and Criminisi, Antonio. TextonBoost for image understanding: Multi-class object recognition and segmentation by jointly modeling texture, layout, and context. IJCV , 81 (1), 2009.
  30. Simonyan, Karen and Zisserman, Andrew. Very deep convolutional networks for large-scale image recognition. In ICLR , 2015.
  31. Sturgess, Paul, Alahari, Karteek, Ladicky, Lubor, and Torr, Philip H. S. Combining appearance and structure from motion features for road scene understanding. In BMVC , 2009.
  32. Tighe, Joseph and Lazebnik, Svetlana. Superparsing - scalable nonparametric image parsing with superpixels. IJCV , 101(2), 2013.
  33. Zheng, Shuai, Jayasumana, Sadeep, Romera-Paredes, Bernardino, Vineet, Vibhav, Su, Zhizhong, Du, Dalong, Huang, Chang, and Torr, Philip. Conditional random fields as recurrent neural networks. In ICCV , 2015.

Cited by

Sorted by citations

Authors
Yu, F., & Koltun, V.
Affiliations
  • Princeton University (US)
  • Intel (United States) (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2015
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1511.07122
arXiv
1511.07122
OpenAlex
W2286929393
Cited by
7,738
References
1
Citations by year
2026: 55 · 2025: 446 · 2024: 752 · 2023: 862 · 2022: 863 · 2021: 1415 · 2020: 1148 · 2019: 1141 · 2018: 667 · 2017: 319 · 2016: 68 · 2015: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Engineering › Media Technology › Remote-Sensing Image Classification
Keywords
semantic segmentation, dilated convolution, multi-scale context aggregation, dense prediction, receptive field expansion, network simplification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review