The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Predicting Ground-Level Scene Layout from Aerial Imagery
Abstract
We introduce a novel strategy for learning to extract semantically meaningful features from aerial imagery. Instead of manually labeling the aerial imagery, we propose to predict (noisy) semantic features automatically extracted from co-located ground imagery. Our network architecture takes an aerial image as input, extracts features using a convolutional neural network, and then applies an adaptive transformation to map these features into the ground-level perspective. We use an end-to-end learning approach to minimize the difference between the semantic segmentation extracted directly from the ground image and the semantic segmentation predicted solely based on the aerial image. We show that a model learned using this strategy, with no additional training, is already capable of rough semantic labeling of aerial imagery. Furthermore, we demonstrate that by finetuning this model we can achieve more accurate semantic segmentation than two baseline initialization strategies. We use our network to address the task of estimating the geolocation and geo-orientation of a ground image. Finally, we show how features extracted from an aerial image can be used to hallucinate a plausible ground-level panorama.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
(2010) · Cited by 14,134
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
A survey on object detection in optical remote sensing images
Cheng, G., & Han, J. (2016) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2016.03.014 · Cited by 1,475
Visual Domain Adaptation: A survey of recent advances
Patel, V. M., Gopalan, R., Li, R., & Chellappa, R. (2015) · IEEE Signal Processing Magazine · doi:10.1109/msp.2014.2347059 · Cited by 899
Domain Adaptation for Statistical Classifiers
Daume III, H., & Marcu, D. (2006) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1872 · Cited by 816
Learning to Detect Roads in High-Resolution Aerial Images
Mnih, V., & Hinton, G. E. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15567-3_16 · Cited by 624
View Synthesis by Appearance Flow
Zhou, T., Tulsiani, S., Sun, W., Malik, J., & Efros, A. A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_18 · Cited by 579
Learning deep representations for ground-to-aerial geolocalization
Lin, T.-Y., Cui, Y., Belongie, S., & Hays, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299135 · Cited by 401
Wide-Area Image Geolocalization with Aerial Reference Imagery
Workman, S., Souvenir, R., & Jacobs, N. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.451 · Cited by 400
Self-citationLearning to Label Aerial Images from Noisy Data
Mnih, V., & Hinton, G. E. (2012) · Cited by 364
Robust visual domain adaptation with low-rank reconstruction
Jhuo, I.-H., Liu, D., Lee, D. T., & Chang, S.-F. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247924 · Cited by 308
Detecting buildings in aerial images
Huertas, A., & Nevatia, R. (1988) · Computer Vision, Graphics, and Image Processing · doi:10.1016/0734-189x(88)90016-3 · Cited by 293
Cross-View Image Geolocalization
Lin, T.-Y., Belongie, S., & Hays, J. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.120 · Cited by 291
Localizing and Orienting Street Views Using Overhead Imagery
Vo, N. N., & Hays, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_30 · Cited by 283
Effective semantic pixel labelling with convolutional networks and Conditional Random Fields
Paisitkriangkrai, S., Sherrah, J., Janney, P., & Van-Den Hengel, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301381 · Cited by 238
Cataloging Public Objects Using Aerial and Street-Level Images — Urban Trees
Wegner, J. D., Branson, S., Hall, D., Schindler, K., & Perona, P. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.647 · Cited by 175
HD Maps: Fine-Grained Road Segmentation by Parsing Ground and Aerial Images
Máttyus, G., Wang, S., Fidler, S., & Urtasun, R. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.393 · Cited by 142
On the location dependence of convolutional neural network features
Workman, S., & Jacobs, N. (2015) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301385 · Cited by 140
Self-citationDeep Directed Generative Models with Energy-Based Probability EstimationPreprint
Kim, T., & Bengio, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.03439 · Cited by 92
Event recognition: viewing the world with a third eye
Luo, J., Yu, J., Joshi, D., & Hao, W. (2008) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1459359.1459574 · Cited by 69
PixelNet: Towards a General Pixel-level ArchitecturePreprint
Bansal, A., Chen, X., Russell, B. C., Gupta, A. K., & Ramanan, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.06694 · Cited by 62
Seitz, S. M., & Baker, S. (2009) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459155 · Cited by 59
Unsupervised Cross-View Semantic Transfer for Remote Sensing Image Classification
Sun, H., Liu, S., Zhou, S., & Zou, H. (2015) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2015.2491605 · Cited by 52
Semantic Classification in Aerial Imagery by Integrating Appearance and Height Information
Kluckner, S., Mauthner, T., Roth, P. M., & Bischof, H. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-12304-7_45 · Cited by 43
Coupling ground-level panoramas and aerial imagery for change detection
Ghouaiel, N., & Lefèvre, S. (2016) · Geo-spatial Information Science · doi:10.1080/10095020.2016.1244998 · Cited by 24
Cited by
University-1652: A Multi-view Multi-source Benchmark for Drone-based Geo-localization
Zheng, Z., Wei, Y., & Yang, Y. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413896 · Cited by 402
Rescaling egocentric vision: Collection, pipeline and challenges for epic-kitchens-100
Damen, D., Doughty, H., Farinella, G. M., Furnari, A., Kazakos, E., Ma, J., Moltisanti, D., Munro, J., Perrett, T., Price, W., et al. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-021-01531-2 · Cited by 400
CVM-Net: Cross-View Matching Network for Image-Based Ground-to-Aerial Geo-Localization
Hu, S., Feng, M., Nguyen, R. M. H., & Lee, G. H. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00758 · Cited by 394
Lending Orientation to Neural Networks for Cross-View Geo-Localization
Liu, L., & Li, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00577 · Cited by 336
Each Part Matters: Local Patterns Facilitate Cross-View Geo-Localization
Wang, T., Zheng, Z., Yan, C., Zhang, J., Sun, Y., Zheng, B., & Yang, Y. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3061265 · Cited by 306
Metadata
- Authors
- Zhai, M., Bessinger, Z., Workman, S., & Jacobs, N.
- Venue
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2017-07
- Pages
- 4132-4140
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2572697301
- Cited by
- 265
- References
- 32
- FWCI
- 4.13
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.