Learning Dual Multi-Scale Manifold Ranking for Semantic Segmentation of High-Resolution Images
Open accessPublished version of a preprint
The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
Semantic image segmentation has recently witnessed considerable progress by training deep convolutional neural networks (CNNs). The core issue of this technique is the limited capacity of CNNs to depict visual objects. Existing approaches tend to utilize approximate inference in a discrete domain or additional aides and do not have a global optimum guarantee. We propose the use of the multi-label manifold ranking (MR) method in solving the linear objective energy function in a continuous domain to delineate visual objects and solve these problems. We present a novel embedded single stream optimization method based on the MR model to avoid approximations without sacrificing expressive power. In addition, we propose a novel network, which we refer to as dual multi-scale manifold ranking (DMSMR) network, that combines the dilated, multi-scale strategies with the single stream MR optimization method in the deep learning architecture to further improve the performance. Experiments on high resolution images, including close-range and remote sensing datasets, demonstrate that the proposed approach can achieve competitive accuracy without additional aides in an end-to-end manner.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
An overview of gradient descent optimization algorithms
Ruder, S. (2016) · arXiv · doi:10.48550/arxiv.1609.04747 · Cited by 4,962
RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation
Lin, G., Milan, A., Shen, C., & Reid, I. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.549 · Cited by 3,287
Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
Krähenbühl, P., & Koltun, V. (2012) · arXiv · doi:10.48550/arxiv.1210.5644 · Cited by 2,908
Eigen, D., & Fergus, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.304 · Cited by 2,862
Learning Hierarchical Features for Scene Labeling
Farabet, C., Couprie, C., Najman, L., & LeCun, Y. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.231 · Cited by 2,620
Saliency Detection via Graph-Based Manifold Ranking
Yang, C., Zhang, L., Lu, H., Ruan, X., & Yang, M.-H. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.407 · Cited by 2,554 · cited 1×
“Moreover, the MR method has been applied to estimate the status of many other complex low-level vision tasks, such as saliency detection [38,64], image retrieval [65,66] and visual tracking [67].”
MethodPoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization
Kendall, A., Grimes, M., & Cipolla, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.336 · Cited by 2,489
Conditional Random Fields as Recurrent Neural Networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., & Torr, P. H. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.179 · Cited by 2,386
Research works printed but not linked: 69
- Ladicky, L., Torr, P., and Zisserman, A. (2013, January 23–28). Human Pose Estimation using a Joint Pixel-wise and Part-wise Formulation. Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA. · DOI
- Romera, E., Bergasa, L., and Arroyo, R. (arXiv, 2016). Can we unify monocular detectors for autonomous driving by using the pixel-wise semantic segmentation of CNNs?, arXiv.
- Barrnes, D., Maddern, W., and Posner, I. (arXiv, 2016). Find Your Own Way: Weakly-Supervised Segmentation of Path Proposals for Urban Autonomy, arXiv. · DOI · Paper page
- Xiao, J., and Quan, L. (October, January 29). Multiple View Semantic Segmentation for Street View Images. Proceedings of the 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan.
- Floros, G., and Leibe, B. (2012, January 16–21). Joint 2D-3D Temporally Consistent Semantic Segmentation of Street Scenes. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA. · DOI
- Toshev, A., and Szegedy, C. (arXiv, 2014). DeepPose: Human Pose Estimation via Deep Neural Networks, arXiv. · DOI · Paper page
- Tompson, J.J., Jain, A., LeCun, Y., and Bregler, C. (arXiv, 2014). Joint training of a convolutional network and a graphical model for human pose estimation, arXiv.
- Jackson, A., Valstar, M., and Tzimiropoulos, G. (arXiv, 2016). A CNN Cascade for Landmark Guided Semantic Part Segmentation, arXiv. · DOI
- Maggiori, E., Tarabalka, Y., Charpiat, G., and Alliez, P. (arXiv, 2016). High-Resolution Semantic Labeling with Convolutional Neural Networks, arXiv. · DOI
- Kampffmeyer, M., Salberg, A.B., and Jenssen, R. (July, January 26). Semantic segmentation of small objects and modeling of uncertainty in urban remote sensing images using deep convolutional neural networks. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Las Vegas, NV, USA. · DOI · Paper page
- Audebert, N., Saux, B.L., and Lefèvre, S. (arXiv, 2016). Semantic Segmentation of Earth Observation Data Using Multimodal and Multi-scale Deep Networks, arXiv. · DOI · Paper page
- Längkvist, M., Kiselev, A., Alirezaie, M., and Loutfi, A. (2016). Classification and segmentation of satellite orthoimagery using convolutional neural networks. Remote Sens., 8. · DOI · Paper page
- Muruganandham, S. (2016). Semantic Segmentation of Satellite Images Using Deep Learning. [Master’s Thesis, Department of Computer Science, Electrical and Space Engineering, Luleå University of Technology].
- Barron, J.T., and Poole, B. (arXiv, 2016). The fast bilateral solver, arXiv. · DOI · Paper page
- Mostajabi, M., Yadollahpour, P., and Shakhnarovich, G. (arXiv, 2015). Feedforward semantic segmentation with zoom-out features, arXiv. · DOI · Paper page
- Shelhamer, E., Long, J., and Darrell, T. (arXiv, 2015). Fully Convolutional Networks for Semantic Segmentation, arXiv.
- Yu, F., and Koltun, V. (arXiv, 2016). Multi-Scale Context Aggregation by Dilated Convolutions, arXiv.
- Chen, L., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A. (arXiv, 2015). DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs, arXiv.
- Chandra, S., and Kokkinos, I. (arXiv, 2016). Fast, Exact and Multi-Scale Inference for Semantic Image Segmentation with Deep Gaussian CRFs, arXiv. · DOI · Paper page
- Badrinarayanan, V., Handa, A., and Cipolla, R. (arXiv, 2015). SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation, arXiv.
- Hyeonwoo, N., Hong, S., and Han, B. (arXiv, 2015). Learning deconvolution network for semantic segmentation, arXiv.
- Chen, L., Schwing, A., Yuille, A., and Urtasun, R. (arXiv, 2015). Learning Deep Structured Models, arXiv.
- Chen, L., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A. (arXiv, 2015). Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs, arXiv.
- Arnab, A., Jayasumana, S., Zheng, S., and Torr, P. (arXiv, 2016). Higher Order Conditional Random Fields in Deep Neural Networks, arXiv. · DOI · Paper page
- Zhou, D., Weston, J., Gretton, A., Bousquent, O., and Scholkopf, B. (2003, January 9–11). Ranking on data manifolds. Proceedings of the 16th International Conference on Neural Information Processing Systems, Whistler, BC, Canada.
- Zhou, D., Bousquent, O., Lal, T., Weston, J., and Scholkopf, B. (2003, January 9–11). Learning with Local and Global Consistency. Proceedings of the 16th International Conference on Neural Information Processing Systems, Whistler, BC, Canada.
- Bencherif, 2015, Fusion of Extreme Learning Machine and Graph-Based Optimization Methods for Active Classification of Remote Sensing Images, IEEE Geosci. Remote Sens. Lett., 12, 527, 10.1109/LGRS.2014.2349538 · DOI
- Krähenbühl, P., and Koltun, V. (2013, January 16–21). Parameter Learning and Convergent Inference for Dense Random Fields. Proceedings of the 30th International Conference on Machine Learning, Atlanta, GA, USA.
- Hariharan, B., Arbeláez, P., Girshick, R., and Malik, J. (arXiv, 2014). Simultaneous Detection and Segmentation, arXiv. · DOI · Paper page
- Dai, J., He, K., and Sun, J. (arXiv, 2015). Convolutional Feature Masking for Joint Object and Stuff Segmentation, arXiv. · DOI · Paper page
- Chen, L., Yang, Y., Wang, J., Xu, W., and Yuille, A. (arXiv, 2016). Attention to Scale: Scale-aware Semantic Image Segmentation, arXiv. · DOI · Paper page
- Bearman, A., Russakovsky, O., Ferrari, V., and Li, F.F. (arXiv, 2016). What’s the Point: Semantic Segmentation with Point Supervision, arXiv. · DOI · Paper page
- Romero, 2015, Unsupervised Deep Feature Extraction for Remote Sensing Image Classification, IEEE Trans. Geosci. Remote Sens., 54, 1349, 10.1109/TGRS.2015.2478379 · DOI · Paper page
- Gatta, 2016, Processing of Extremely High-Resolution LiDAR and RGB Data: Outcome of the 2015 IEEE GRSS Data Fusion Contest–Part A: 2-D Contest, IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 9, 5547, 10.1109/JSTARS.2016.2569162 · DOI
- Tschannen, M., Cavigelli, L., Mentzer, F., Wiatowski, T., and Benini, L. (arXiv, 2016). Deep Structured Features for Semantic Segmentation, arXiv. · DOI
- Piramanayagam, S., Schwartzkopf, W., Koehler, F.W., and Saber, E. (2016). Classification of remote sensed images using random forests and deep learning framework. SPIE Remote Sens. Int. Soc. Opt. Photonics. · DOI
- Marcu, A., and Leordeanu, M. (arXiv, 2016). Dual Local-Global Contextual Pathways for Recognition in Aerial Imagery, arXiv.
- Yuan, 2016, Dual-clustering-based hyperspectral band selection by contextual analysis, IEEE Trans. Geosci. Remote Sens., 54, 1431, 10.1109/TGRS.2015.2480866 · DOI
- Simonyan, K., and Zisserman, A. (arXiv, 2015). Very Deep Convolutional Networks for Large-Scale Image Recognition, arXiv.
- Hong, S., Noh, H., and Han, B. (arXiv, 2015). Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation, arXiv.
- Audebert, N., Saux, B.L., and Lefèvre, S. (2017). Segment-before-Detect: Vehicle Detection and Classification through Semantic Segmentation of Aerial Images. Remote Sens., 9. · DOI · Paper page
- Huang, Z., Cheng, G., Wang, H., Li, H., Shi, L., and Pan, C. (2016, January 10–15). Building extraction from multi-source remote sensing images via deep deconvolution neural networks. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Beijing, China. · DOI
- Audebert, N., Boulch, A., Lagrange, A., Le Saux, B., and Lefevre, S. (2016). Deep Learning for Remote Sensing, ONERA The French Aerospace Lab, DTIM & Univ. Bretagne-Sud & ENSTA ParisTech. Technical Report. · DOI
- Paisitkriangkrai, S., Sherrah, J., Janney, P., and Hengel, V.D. (2015, January 7–12). Effective semantic pixel labelling with convolutional networks and conditional random fields. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Boston, MA, USA. · DOI · Paper page
- Alam, F.I., Zhou, J., Liew, A.W.C., and Jia, X. (2016, January 10–15). CRF learning with CNN features for hyperspectral image segmentation. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium, Beijing, China. · DOI
- He, X., Cai, D., and Niyogi, P. (2005, January 5–8). Laplacian Score for Feature Selection. Proceedings of the 18th International Conference on Neural Information Processing Systems, Vancouver, BC, Canada.
- Quan, R., Han, J., Zhang, D., and Nie, F. (2016, January 27–30). Object co-segmentation via graph optimized-flexible manifold ranking. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA. · DOI
- Wang, 2016, Salient band selection for hyperspectral image classification via manifold ranking, IEEE Trans. Neural Netw. Learn. Syst., 27, 1279, 10.1109/TNNLS.2015.2477537 · DOI · Paper page
- Yang, 2013, Graph-regularized saliency detection with convex-hull-based center prior, IEEE Signal Process. Lett., 20, 637, 10.1109/LSP.2013.2260737 · DOI
- Xu, B., Bu, J., Chen, C., Cai, D., He, X., Liu, W., and Luo, J. (2011, January 24–28). Efficient Manifold Ranking for Image Retrieval. Proceedings of the 34th international ACM SIGIR Conference on Research and Development in Information Retrieval, Beijing, China. · DOI
- Hsieh, C., Han, C., Shih, J., Lee, C., and Fan, K. (2015, January 24–26). 3D Model Retrieval Using Multiple Features and Manifold Ranking. Proceedings of the 2015 8th International Conference on Ubi-Media Computing (UMEDIA), Colombo, Sri Lanka. · doi:10.1109/umedia.2015.7297419
- Zhou, 2015, Robust visual tracking via efficient manifold ranking with low-dimensional compressive features, Pattern Recognit., 48, 2459, 10.1016/j.patcog.2015.03.008 · DOI
- Brostow, 2009, Semantic object classes in video: A high-definition ground truth database, Pattern Recognit. Lett., 30, 88, 10.1016/j.patrec.2008.04.005 · DOI · Paper page
- Everingham, 2010, The Pascal Visual Object Classes (VOC) Challenge, Int. J. Comput. Vis., 88, 303, 10.1007/s11263-009-0275-4 · DOI · Paper page
- Glorot, X., Bordes, A., and Bengio, Y. (2011, January 11–13). Deep Sparse Rectifier Neural Networks. Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA.
- Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., and Darrell, T. (2014, January 3–7). Caffe: Convolutional Architecture for Fast Feature Embedding. Proceedings of the 22nd ACM International Conference on Multimedia, Orlando, FL, USA. · DOI · Paper page
- Marmanis, 2016, Semantic Segmentation of Aerial Images with an Ensemble of CNSS, ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci., 3, 473, 10.5194/isprs-annals-III-3-473-2016 · DOI · Paper page
- Hariharan, B., Arbeláez, P., Bourdev, L., Maji, S., and Malik, J. (2011, January 6–13). Semantic Contours from Inverse Detectors. Proceedings of the 2011 IEEE International Conference on Computer Vision (ICCV), Barcelona, Spain. · DOI · Paper page
- Lin, T., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C., and Dollár, P. (arXiv, 2014). Microsoft coco: Common objects in context, arXiv. · DOI · Paper page
- Kohli, 2009, Robust higher order potentials for enforcing label consistency, Int. J. Comput. Vis., 82, 302, 10.1007/s11263-008-0202-0 · DOI · Paper page
- Quang, N.T., Thuy, N.T., Sang, D.V., and Binh, H.T.T. (2015, January 3–4). An efficient framework for pixel-wise building segmentation from aerial images. Proceedings of the Sixth International Symposium on Information and Communication Technology, Hue City, Vietnam. · DOI
- Sherrah, J. (arXiv, 2016). Fully convolutional networks for dense semantic labelling of high-resolution aerial imagery, arXiv.
- Immitzer, 2012, Tree species classification with random forest using very high spatial resolution 8-band WorldView-2 satellite data, Remote Sens., 4, 2661, 10.3390/rs4092661 · DOI · Paper page
- Dribault, 2012, Monitoring seasonal hydrological dynamics of minerotrophic peatlands using multi-date GeoEye-1 very high resolution imagery and object-based classification, Remote Sens., 4, 1887, 10.3390/rs4071887 · doi:10.3390/rs4071887
- Onojeghuo, 2011, Mapping reedbed habitats using texture-based classification of QuickBird imagery, Int. J. Remote Sens., 32, 8121, 10.1080/01431161.2010.532822 · doi:10.1080/01431161.2010.532822
- Junwei, 2016, Comparison between GF-1 and Landsat-8 images in land cover classification, Prog. Geogr., 35, 255
- Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative adversarial nets. Advances in Neural Information Processing Systems, The MIT Press.
- Mirza, M., and Osindero, S. (arXiv, 2014). Conditional generative adversarial nets, arXiv.
- Luc, P., Couprie, C., Chintala, S., and Verbeek, J. (arXiv, 2016). Semantic Segmentation using Adversarial Networks, arXiv.
Cited by
Deep learning in remote sensing applications: A meta-analysis and review
Ma, L., Liu, Y., Zhang, X., Ye, Y., Yin, G., & Johnson, B. A. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.04.015 · Cited by 2,456
Algorithms for semantic segmentation of multispectral remote sensing imagery using deep learning
Kemker, R., Salvaggio, C., & Kanan, C. (2018) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2018.04.014 · Cited by 558
“Newer datasets, such as Zurich (Volpi and Ferrari, 2015) and EvLab-SS (Zhang et al., 2017), have sacrificed some spatial resolution but include additional labeled classes.”
MethodSwin Transformer Embedding UNet for Remote Sensing Image Semantic Segmentation
He, X., Zhou, Y., Zhao, J., Zhang, D., Yao, R., & Xue, Y. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3144165 · Cited by 550
Himeur, Y., Rimal, B., Tiwary, A., & Amira, A. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.06.003 · Cited by 288
On Creating Benchmark Dataset for Aerial Image Interpretation: Reviews, Guidances, and Million-AID
Long, Y., Xia, G.-S., Li, S., Yang, W., Yang, M. Y., Zhu, X. X., Zhang, L., & Li, D. (2021) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2021.3070368 · Cited by 250
Hidden Path Selection Network for Semantic Segmentation of Remote Sensing Images
Yang, K., Tong, X.-Y., Xia, G.-S., Shen, W., & Zhang, L. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3197334 · Cited by 22
From Difference to Similarity: A Manifold Ranking-Based Hyperspectral Anomaly Detection Framework
Huang, Z., & Li, S. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2918342 · Cited by 18
“Moreover, deep neural network (DNN)-based techniques [48]–[51] also have been an interesting research topic in”
Background
Metadata
- Authors
- Zhang, M., Hu, X., Zhao, L., Lv, Y., Luo, M., & Pang, S.
- Venue
- Remote Sensing
- Publisher
- MDPI AG
- Published
- 2017-05-19
- Volume
- 9
- Issue
- 5
- Pages
- 500
- ISSN
- 2072-4292 (ISSN-L)
- OpenAlex
- W2616727008
- Cited by
- 57
- References
- 22
- FWCI
- 2.13
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.