Semi-Supervised Deep Learning for Monocular Depth Map Prediction
Abstract
Supervised deep learning often suffers from the lack of sufficient training data. Specifically in the context of monocular depth map prediction, it is barely possible to determine dense ground truth depth images in realistic dynamic outdoor environments. When using LiDAR sensors, for instance, noise is present in the distance measurements, the calibration between sensors cannot be perfect, and the measurements are typically much sparser than the camera images. In this paper, we propose a novel approach to depth map prediction from monocular images that learns in a semi-supervised way. While we use sparse ground-truth depth for supervised learning, we also enforce our deep network to produce photoconsistent dense depth maps in a stereo setup using a direct image alignment loss. In experiments we demonstrate superior performance in depth map prediction from single images compared to the state-of-the-art methods.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Progressive expansion and contraction in a fully invertible manner, without discarding uninformative input variability, increases performance across many tasks.
“In fact, progressive expansion and contraction in a fully invertible manner, without discarding”
From 3D Packing for Self-Supervised Monocular Depth Estimation (2020) · cites this paper as [11]
Reflection padding significantly reduces border artifacts found in existing approaches.
“We found that this significantly reduces the border artifacts found in existing approaches, e.g.[15].”
From Digging Into Self-Supervised Monocular Depth Estimation (2019) · cites this paper as [15]
The method outperforms all methods that do not use depth supervision on Make3D.
“We outperform all methods that do not use depth supervision, with the evaluation criteria from [15].”
From Digging Into Self-Supervised Monocular Depth Estimation (2019) · cites this paper as [15]
Stereo-based training has known scale, so no additional scaling is required during evaluation.
“This is in contrast to stereo based training where the scale is known and as a result no additional scaling is required during the evaluation e.g.[12,15].”
From Digging Into Self-Supervised Monocular Depth Estimation (2019) · cites this paper as [15]
Post-processing, introduced by [15], is a technique to improve test time results on stereo-trained monocular depth estimation methods by running each test image through the network twice, once unflipped and then flipped.
“estimation methods by running each test image through the network twice, once unflipped and then flipped.”
From Digging Into Self-Supervised Monocular Depth Estimation (2019) · cites this paper as [15]
Our models generate higher quality outputs and do not produce holes in the depth maps or border artifacts that can be seen in many existing baselines.
“We can see that our models generate higher quality outputs and do not produce 'holes' in the depth maps or border artifacts that can be seen in many existing baselines e.g.[76,51,15,73].”
From Digging Into Self-Supervised Monocular Depth Estimation (2019) · cites this paper as [15]
Post-processing has been shown to bring significant gains in accuracy for stereo results, at the expense of requiring two forward-passes through the network at test time.
“This has been shown to bring significant gains in accuracy for stereo results, at the expense of requiring two forward-passes through the network at test time [15,50].”
From Digging Into Self-Supervised Monocular Depth Estimation (2019) · cites this paper as [15]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · see citing works
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · see citing works
(2010) · Cited by 14,134 · see citing works
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433 · see citing works
Flownet: Learning optical flow with convolutional networks
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.316 · Cited by 4,313 · see citing works
Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
Eigen, D., Puhrsch, C., & Fergus, R. (2014) · arXiv · doi:10.48550/arxiv.1406.2283 · Cited by 3,442 · see citing works
Unsupervised Monocular Depth Estimation with Left-Right Consistency
Godard, C., Aodha, O. M., & Brostow, G. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.699 · Cited by 3,270 · see citing works
Eigen, D., & Fergus, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.304 · Cited by 2,862 · see citing works
Deeper Depth Prediction with Fully Convolutional Residual Networks
Laina, I., Rupprecht, C., Belagiannis, V., Tombari, F., & Navab, N. (2016) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.32 · Cited by 2,061 · see citing works
Make3D: Learning 3D Scene Structure from a Single Still Image
Saxena, A., Sun, M., & Ng, A. Y. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.132 · Cited by 1,738 · see citing works
Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
Garg, R., B.G., V. K., Carneiro, G., & Reid, I. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_45 · Cited by 1,632 · see citing works
(2005) · Cited by 981 · see citing works
Deep convolutional neural fields for depth estimation from a single image
Liu, F., Shen, C., & Lin, G. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299152 · Cited by 892 · see citing works
Li, B., Shen, C., Dai, Y., van den Hengel, A., & He, M. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298715 · Cited by 621 · see citing works
3-D Depth Reconstruction from a Single Still Image
Saxena, A., Chung, S. H., & Ng, A. Y. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-007-0071-y · Cited by 600 · see citing works
Single image depth estimation from predicted semantic labels
Liu, B., Gould, S. J., & Koller, D. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5539823 · Cited by 484 · see citing works
Pulling Things out of Perspective
Ladický, Ľ., Shi, J., & Pollefeys, M. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.19 · Cited by 463 · see citing works
Towards unified depth and semantic prediction from a single image
Wang, P., Shen, X., Lin, Z., Cohen, S. D., Price, B. L., & Yuille, A. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298897 · Cited by 455 · see citing works
(2016) · doi:10.1007/978-3-319-46493-0_51 · Cited by 411 · see citing works
Discrete-Continuous Depth Estimation from a Single Image
Liu, M., Salzmann, M., & He, X. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.97 · Cited by 381 · see citing works
Depth Extraction from Video Using Non-parametric Sampling
Karsch, K., Liu, C., & Kang, S. B. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33715-4_56 · Cited by 269 · see citing works
2D-to-3D image conversion by learning depth from examples
Konrad, J., Wang, M., & Ishwar, P. (2012) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2012.6238903 · Cited by 145 · see citing works
Toward Holistic Scene Understanding: Feedback Enabled Cascaded Classification Models
Li, C., Kowdle, A., Saxena, A., & Chen, T. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.232 · Cited by 90 · see citing works
Unsupervised Monocular Depth Estimation with Left-Right Consistency
Godard, C., Mac Aodha, O., & Brostow, G. J. (2016) · arXiv · doi:10.48550/arxiv.1609.03677 · Cited by 28 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Unsupervised Learning of Depth and Ego-Motion from Video
Zhou, T., Brown, M., Snavely, N., & Lowe, D. G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.700 · Cited by 2,910 · see citing works
Digging Into Self-Supervised Monocular Depth Estimation
Godard, C., Aodha, O. M., Firman, M., & Brostow, G. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00393 · Cited by 2,561 · see citing works
Deep Ordinal Regression Network for Monocular Depth Estimation
Fu, H., Gong, M., Wang, C., Batmanghelich, K., & Tao, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00214 · Cited by 1,991 · see citing works
3D Packing for Self-Supervised Monocular Depth Estimation
Guizilini, V., Ambrus, R., Pillai, S., Raventos, A., & Gaidon, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00256 · Cited by 741 · see citing works
Zhan, H., Garg, R., Weerasekera, C. S., Li, K., Agarwal, H., & Reid, I. M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00043 · Cited by 705 · see citing works
Learning Depth from Monocular Videos Using Direct Methods
Wang, C., Buenaposada, J. M., Zhu, R., & Lucey, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00216 · Cited by 667 · see citing works
D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry
Yang, N., von Stumberg, L., Wang, R., & Cremers, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00136 · Cited by 438 · see citing works
Learning Depth with Convolutional Spatial Propagation Network
Cheng, X., Wang, P., & Yang, R. (2018) · arXiv · doi:10.48550/arxiv.1810.02695 · Cited by 345 · see citing works
Self-Supervised Monocular Depth Hints
Watson, J., Firman, M., Brostow, G., & Turmukhambetov, D. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00225 · Cited by 275 · see citing works
Luo, Y., Ren, J., Lin, M., Pang, J., Sun, W., Li, H., & Lin, L. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00024 · Cited by 205 · see citing works
Unsupervised Adversarial Depth Estimation Using Cycled Generative Networks
Pilzer, A., Xu, D., Puscas, M., Ricci, E., & Sebe, N. (2018) · 2018 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00073 · Cited by 192 · see citing works
Monocular depth estimation with hierarchical fusion of dilated CNNs and soft-weighted-sum inference
Li, B., Dai, Y., & He, M. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.05.029 · Cited by 142 · see citing works
CAM-Convs: Camera-Aware Multi-Scale Convolutions for Single-View Depth
Facil, J. M., Ummenhofer, B., Zhou, H., Montesano, L., Brox, T., & Civera, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01210 · Cited by 130 · see citing works
Single image depth estimation: An overview
Mertan, A., Duff, D. J., & Unal, G. (2022) · Digital Signal Processing · doi:10.1016/j.dsp.2022.103441 · Cited by 121 · see citing works
Visual SLAM for Automated Driving: Exploring the Applications of Deep Learning
Milz, S., Arbeiter, G., Witt, C., Abdallah, B., & Yogamani, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00062 · Cited by 107 · see citing works
Web Stereo Video Supervision for Depth Prediction from Dynamic Scenes
Wang, C., Lucey, S., Perazzi, F., & Wang, O. (2019) · 2019 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2019.00046 · Cited by 103 · see citing works
Smolyanskiy, N., Kamenev, A., & Birchfield, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00147 · Cited by 104 · see citing works
Duran, O., & Turan, B. (2022) · Measurement · doi:10.1016/j.measurement.2022.110732 · Cited by 25 · see citing works
- Authors
- Kuznietsov, Y., Stuckler, J., & Leibe, B.
- Affiliations
- RWTH Aachen University (DE)
- ORCID
- Leibe, Bastian
- Venue
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2017-07
- Pages
- 2215-2223
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2593414960
- Cited by
- 680
- References
- 27
- Citations by year
- 2026: 8 · 2025: 22 · 2024: 58 · 2023: 65 · 2022: 75 · 2021: 116 · 2020: 128 · 2019: 139 · 2018: 61 · 2017: 8
- FWCI
- 26.86
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- OPUS (Augsburg University) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Engineering › Aerospace Engineering › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- monocular depth estimation, semi-supervised learning, dense depth maps, LiDAR-camera calibration, outdoor environments
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review