The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Depth From Videos in the Wild: Unsupervised Monocular Depth Learning From Unknown Cameras
Open accessAbstract
We present a novel method for simultaneous learning of depth, egomotion, object motion, and camera intrinsics from monocular videos, using only consistency across neighboring video frames as supervision signal. Similarly to prior work, our method learns by applying differentiable warping to frames and comparing the result to adjacent ones, but it provides several improvements: We address occlusions geometrically and differentiably, directly using the depth maps as predicted during training. We introduce randomized layer normalization, a novel powerful regularizer, and we account for object motion relative to the scene. To the best of our knowledge, our work is the first to learn the camera intrinsic parameters, including lens distortion, from video in an unsupervised manner, thereby allowing us to extract accurate depth and motion from arbitrary videos of unknown origin at scale. We evaluate our results on the Cityscapes, KITTI and EuRoC datasets, establishing new state of the art on depth prediction and odometry, and demonstrate qualitatively that depth prediction can be learned from a collection of YouTube videos. The code will be open sourced once anonymity is lifted.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In contrast to traditional matching and optimization-based methods, current research on this topic mainly focuses on deep learning in an unsupervised or self-supervised way [132].”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [132]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
(2014) · Cited by 33,419
Layer NormalizationPreprint
Ba, J., Kiros, J., & Hinton, G. E. (2016) · arXiv.org · doi:10.48550/arxiv.1607.06450 · Cited by 13,762
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433
Vision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913
Structure-from-Motion Revisited
Schonberger, J. L., & Frahm, J.-M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.445 · Cited by 6,433
Flownet: Learning optical flow with convolutional networks
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.316 · Cited by 4,313
Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
Eigen, D., Puhrsch, C., & Fergus, R. (2014) · arXiv · doi:10.48550/arxiv.1406.2283 · Cited by 3,442
Unsupervised Monocular Depth Estimation with Left-Right Consistency
Godard, C., Aodha, O. M., & Brostow, G. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.699 · Cited by 3,270
Unsupervised Learning of Depth and Ego-Motion from Video
Zhou, T., Brown, M., Snavely, N., & Lowe, D. G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.700 · Cited by 2,910
Eigen, D., & Fergus, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.304 · Cited by 2,862
Digging Into Self-Supervised Monocular Depth Estimation
Godard, C., Aodha, O. M., Firman, M., & Brostow, G. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00393 · Cited by 2,561
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
Goyal, P., Dollár, P., Girshick, R., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y., & He, K. (2017) · arXiv · doi:10.48550/arxiv.1706.02677 · Cited by 2,294
Deeper Depth Prediction with Fully Convolutional Residual Networks
Laina, I., Rupprecht, C., Belagiannis, V., Tombari, F., & Navab, N. (2016) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.32 · Cited by 2,105
The EuRoC micro aerial vehicle datasets
Burri, M., Nikolic, J., Gohl, P., Schneider, T., Rehder, J., Omari, S., Achtelik, M., & Siegwart, R. (2016) · The International Journal of Robotics Research · doi:10.1177/0278364915620033 · Cited by 2,086
Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
Garg, R., B.G., V. K., Carneiro, G., & Reid, I. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_45 · Cited by 1,632
Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
Garg, R., BG, V. K., Carneiro, G., & Reid, I. (2016) · arXiv · doi:10.48550/arxiv.1603.04992 · Cited by 1,632
MVSNet: Depth Inference for Unstructured Multi-view Stereo
Yao, Y., Luo, Z., Li, S., Fang, T., & Quan, L. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_47 · Cited by 1,619
End-to-End Learning of Geometry and Context for Deep Stereo Regression
Kendall, A., Martirosyan, H., Dasgupta, S., Henry, P., Kennedy, R., Bachrach, A., & Bry, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.17 · Cited by 1,531
YouTube-8M: A Large-Scale Video Classification BenchmarkPreprint
Abu-El-Haija, S., Kothari, N., Lee, J., Natsev, P., Toderici, G. D., Varadarajan, B., & Vijayanarasimhan, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.08675 · Cited by 1,382
GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose
Yin, Z., & Shi, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00212 · Cited by 1,280
MegaDepth: Learning Single-View Depth Prediction from Internet Photos
Li, Z., & Snavely, N. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00218 · Cited by 1,244
Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields
Liu, F., Shen, C., Lin, G., & Reid, I. D. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2505283 · Cited by 1,231
Agarwal, S., Snavely, N., Simon, I., Seitz, S. M., & Szeliski, R. (2009) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459148 · Cited by 1,066
Kato, H., Ushiku, Y., & Harada, T. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00411 · Cited by 954
Deep convolutional neural fields for depth estimation from a single image
Liu, F., Shen, C., & Lin, G. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299152 · Cited by 892
Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints
Mahjourian, R., Wicke, M., & Angelova, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00594 · Cited by 775
Self-citationDeMoN: Depth and Motion Network for Learning Monocular Stereo
Ummenhofer, B., Zhou, H., Uhrig, J., Mayer, N., Ilg, E., Dosovitskiy, A., & Brox, T. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.596 · Cited by 712
Zhan, H., Garg, R., Weerasekera, C. S., Li, K., Agarwal, H., & Reid, I. M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00043 · Cited by 705
Learning Depth from Monocular Videos Using Direct Methods
Wang, C., Buenaposada, J. M., Zhu, R., & Lucey, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00216 · Cited by 667
Adding Gradient Noise Improves Learning for Very Deep NetworksPreprint
Neelakantan, A., Vilnis, L., Le, Q. V., Sutskever, I., Kaiser, Ł., Kurach, K., & Martens, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06807 · Cited by 570
Multi-View Supervision for Single-View Reconstruction via Differentiable Ray Consistency
Tulsiani, S., Zhou, T., Efros, A. A., & Malik, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/cvpr.2017.30 · Cited by 531
Cao, Y., Wu, Z., & Shen, C. (2017) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2740321 · Cited by 467
StereoNet: Guided Hierarchical Refinement for Real-Time Edge-Aware Depth Prediction
Khamis, S., Fanello, S., Rhemann, C., Kowdle, A., Valentin, J., & Izadi, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01267-0_35 · Cited by 418
Designing deep networks for surface normal estimation
Wang, X., Fouhey, D. F., & Gupta, A. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298652 · Cited by 348
Occlusion Aware Unsupervised Learning of Optical Flow
Wang, Y., Yang, Y., Yang, Z., Zhao, L., Wang, P., & Xu, W. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00513 · Cited by 331
SfM-Net: Learning of Structure and Motion from VideoPreprint
Vijayanarasimhan, S., Ricco, S., Schmid, C., Sukthankar, R., & Fragkiadaki, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.07804 · Cited by 327
End-to-End Learning of Geometry and Context for Deep Stereo RegressionPreprint
Kendall, A., Martirosyan, H., Dasgupta, S., Henry, P., Kennedy, R., Bachrach, A., & Bry, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.04309 · Cited by 287
Depth estimation using monocular and stereo cues
Saxena, A., Schulte, J., & Ng, A. Y. (2007) · Cited by 211
Unsupervised Learning of Multi-Frame Optical Flow with Occlusions
Janai, J., Güney, F., Ranjan, A., Black, M. J., & Geiger, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01270-0_42 · Cited by 203
Unsupervised Adversarial Depth Estimation Using Cycled Generative Networks
Pilzer, A., Xu, D., Puscas, M., Ricci, E., & Sebe, N. (2018) · 2018 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00073 · Cited by 192
LEGO: Learning Edge with Geometry all at Once by Watching Videos
Yang, Z., Wang, P., Wang, Y., Xu, W. H., & Nevatia, R. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00031 · Cited by 191
Bogdan, O., Eckstein, V., Rameau, F., & Bazin, J. (2018) · Conference on Visual Media Production · doi:10.1145/3278471.3278479 · Cited by 124
Every Pixel Counts: Unsupervised Geometry Learning with Holistic 3D Motion Understanding
Yang, Z., Wang, P., Wang, Y., Xu, W., & Nevatia, R. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11021-5_43 · Cited by 109
RenderNet: A deep convolutional network for differentiable rendering from 3D shapesPreprint
Nguyen-Phuoc, T., Li, C., Balaban, S., & Yang, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.06575 · Cited by 106
Unsupervised Learning of Geometry with Edge-aware Depth-Normal ConsistencyPreprint
Yang, Z., Wang, P., Xu, W. H., Zhao, L., & Nevatia, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.03665 · Cited by 104
DEEPFOCAL: A method for direct focal length estimation
Workman, S., Greenwell, C., Zhai, M., Baltenberger, R., & Jacobs, N. (2015) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2015.7351024 · Cited by 94
Cao, Y., Wu, Z., & Shen, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.02305 · Cited by 44
Shamwell, E. J., Leung, S., & Nothwang, W. D. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593573 · Cited by 41
Continual Occlusion and Optical Flow Estimation
Neoral, M., Šochman, J., & Matas, J. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20870-7_10 · Cited by 36
Every Pixel Counts: Unsupervised Geometry Learning with Holistic 3D Motion UnderstandingPreprint
Yang, Z., Wang, P., Wang, Y., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.10556 · Cited by 10
Focal length estimation guided with object distribution on FocaLens dataset
Han, Y., Zhang, Y., Zhang, S., Zhao, S., & Zhang, L. (2017) · Journal of Electronic Imaging · doi:10.1117/1.jei.26.3.033018 · Cited by 9
Continual Occlusions and Optical Flow EstimationPreprint
Neoral, M., Šochman, J., & Matas, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.01602 · Cited by 3
Flagged first, then by citations
Research works printed but not linked: 3
- schonberger, 2016, Structurefrom-motion revisited, CVPR
- godard, 2017, Unsupervised monocular depth estimation with leftright consistency, CVPR
- casser, 2019, Unsupervised learning of depth and ego-motion: A structured approach, AAAI-19
Cited by
Sorted by citations
Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer
Ranftl, R., Lasinger, K., Hafner, D., Konrad Schindler, B., & Koltun, V. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3019967 · Cited by 2,279
Zhang, J., & Tao, D. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3039359 · Cited by 774
D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry
Yang, N., von Stumberg, L., Wang, R., & Cremers, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00136 · Cited by 438
Self-supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidance
Klingner, M., Termöhlen, J.-A., Mikolajczyk, J., & Fingscheidt, T. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58565-5_35 · Cited by 398
Consistent video depth estimation
Luo, X., Huang, J.-B., Szeliski, R., Matzen, K., & Kopf, J. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392377 · Cited by 322
Metadata
- Authors
- Gordon, A., Li, H., Jonschkowski, R., & Angelova, A.
- Affiliations
- Google (United States) (US)
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 8976-8985
- OpenAlex
- W2935920407
- Cited by
- 389
- References
- 52
- Citations by year
- 2026: 17 · 2025: 21 · 2024: 51 · 2023: 72 · 2022: 73 · 2021: 86 · 2020: 60 · 2019: 9
- FWCI
- 18.60
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- camera motion estimation, object motion estimation, lens distortion, occlusion handling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
