The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
DeMoN: Depth and Motion Network for Learning Monocular Stereo
Open accessAbstract
In this paper we formulate structure from motion as a learning problem. We train a convolutional network end-to-end to compute depth and camera motion from successive, unconstrained image pairs. The architecture is composed of multiple stacked encoder-decoder networks, the core part being an iterative network that is able to improve its own predictions. The network estimates not only depth and motion, but additionally surface normals, optical flow between the images and confidence of the matching. A crucial component of the approach is a training loss based on spatial relative differences. Compared to traditional two-frame structure from motion methods, results are more accurate and more robust. In contrast to the popular depth-from-single-image networks, DeMoN learns the concept of matching and, thus, better generalizes to structures not seen during training.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Fischler, M. A., & Bolles, R. C. (1981) · Communications of the ACM · doi:10.1145/358669.358692 · Cited by 25,893
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070
(1981) · Cited by 11,963
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Shi, J., & Tomasi (1994) · Proceedings of IEEE Conference on Computer Vision and Pattern Recognition CVPR-94 · doi:10.1109/cvpr.1994.323794 · Cited by 7,724 · cited 1×
“(“Base-Matlab”) uses Matlab implementations of the KLT algorithm [38, 27, 35] for correspondence search while (“Base-Mat-F”) uses the predicted flow from DeMoN.”
MethodStructure-from-Motion Revisited
Schonberger, J. L., & Frahm, J.-M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.445 · Cited by 6,433
Indoor segmentation and support inference from rgbd images
Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_54 · Cited by 5,760
Research works printed but not linked: 4
- triggs, 2000, Bundle Adjustment A Modern Synthesis Vision Algorithms: Theory and Practice, Vision Algorithms Theory and Practice, 1883, 153
- fuhrmann, 2014, Mve-a multiview reconstruction environment, Proceedings of the Eurographics Workshop on Graphics and Cultural Heritage (GCH), 6, 8
- jayaraman, 2015, Learning image representations tied to egomotion, ICCV
- agarwal, 0, Ceres Solver
Cited by
SuperPoint: Self-Supervised Interest Point Detection and Description
DeTone, D., Malisiewicz, T., & Rabinovich, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00060 · Cited by 3,355
Unsupervised Learning of Depth and Ego-Motion from Video
Zhou, T., Brown, M., Snavely, N., & Lowe, D. G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.700 · Cited by 2,910
Digging Into Self-Supervised Monocular Depth Estimation
Godard, C., Aodha, O. M., Firman, M., & Brostow, G. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00393 · Cited by 2,561
Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer
Ranftl, R., Lasinger, K., Hafner, D., Schindler, K., & Koltun, V. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3019967 · Cited by 1,618
GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose
Yin, Z., & Shi, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00212 · Cited by 1,256
Densely Connected Pyramid Dehazing Network
Zhang, H., & Patel, V. M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00337 · Cited by 1,223
On the Continuity of Rotation Representations in Neural Networks
Zhou, Y., Barnes, C., Lu, J., Yang, J., & Li, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00589 · Cited by 1,255
Uhrig, J., Schneider, N., Schneider, L., Franke, U., Brox, T., & Geiger, A. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00012 · Cited by 924
Self-citationUnsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints
Mahjourian, R., Wicke, M., & Angelova, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00594 · Cited by 775
NICE-SLAM: Neural Implicit Scalable Encoding for SLAM
Zhu, Z., Peng, S., Larsson, V., Xu, W., Bao, H., Cui, Z., Oswald, M. R., & Pollefeys, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01245 · Cited by 825
Artificial intelligence in the creative industries: a review
Anantrasirichai, N., & Bull, D. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10039-7 · Cited by 793
3D Packing for Self-Supervised Monocular Depth Estimation
Guizilini, V., Ambrus, R., Pillai, S., Raventos, A., & Gaidon, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00256 · Cited by 741
AdaBins: Depth Estimation Using Adaptive Bins
Farooq Bhat, S., Alhashim, I., & Wonka, P. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00400 · Cited by 726
Zhan, H., Garg, R., Weerasekera, C. S., Li, K., Agarwal, H., & Reid, I. M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00043 · Cited by 705
The ApolloScape Dataset for Autonomous Driving
Huang, X., Cheng, X., Geng, Q., Cao, B., Zhou, D., Wang, P., Lin, Y., & Yang, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00141 · Cited by 679
Ranjan, A., Jampani, V., Balles, L., Kim, K., Sun, D., Wulff, J., & Black, M. J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01252 · Cited by 632
UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction
Oechsle, M., Peng, S., & Geiger, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00554 · Cited by 622
The ApolloScape Open Dataset for Autonomous Driving and Its Application
Huang, X., Wang, P., Cheng, X., Zhou, D., Geng, Q., & Yang, R. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2926463 · Cited by 615
The ApolloScape Open Dataset for Autonomous Driving and its Application
Huang, X., Wang, P., Cheng, X., Zhou, D., Geng, Q., & Yang, R. (2018) · arXiv · doi:10.48550/arxiv.1803.06184 · Cited by 615
Learning to Find Good Correspondences
Yi, K. M., Trulls, E., Ono, Y., Lepetit, V., Salzmann, M., & Fua, P. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00282 · Cited by 552
DeepMVS: Learning Multi-view Stereopsis
Huang, P.-H., Matzen, K., Kopf, J., Ahuja, N., & Huang, J.-B. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00298 · Cited by 493
Deep learning for monocular depth estimation: A review
Ming, Y., Meng, X., Fan, C., & Yu, H. (2021) · Neurocomputing · doi:10.1016/j.neucom.2020.12.089 · Cited by 454
D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry
Yang, N., von Stumberg, L., Wang, R., & Cremers, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00136 · Cited by 438
Understanding the Limitations of CNN-Based Absolute Camera Pose Regression
Sattler, T., Zhou, Q., Pollefeys, M., & Leal-Taixe, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00342 · Cited by 418
Hu, J., Ozay, M., Zhang, Y., & Okatani, T. (2019) · 2019 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2019.00116 · Cited by 395
Depth From Videos in the Wild: Unsupervised Monocular Depth Learning From Unknown Cameras
Gordon, A., Li, H., Jonschkowski, R., & Angelova, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00907 · Cited by 389
FastDepth: Fast Monocular Depth Estimation on Embedded Systems
Wofk, D., Ma, F., Yang, T.-J., Karaman, S., & Sze, V. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8794182 · Cited by 384
Deep Stereo Using Adaptive Thin Volume Representation With Uncertainty Awareness
Cheng, S., Xu, Z., Zhu, S., Li, Z., Li, L. E., Ramamoorthi, R., & Su, H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00260 · Cited by 385
A Survey on Deep Learning Techniques for Stereo-Based Depth Estimation
Laga, H., Jospin, L. V., Boussaid, F., & Bennamoun, M. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3032602 · Cited by 336
Is Pseudo-Lidar needed for Monocular 3D Object detection?
Park, D., Ambrus, R., Guizilini, V., Li, J., & Gaidon, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00313 · Cited by 334
Single-View View Synthesis With Multiplane Images
Tucker, R., & Snavely, N. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00063 · Cited by 324
Consistent video depth estimation
Luo, X., Huang, J.-B., Szeliski, R., Matzen, K., & Kopf, J. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392377 · Cited by 322
Anipose: A toolkit for robust markerless 3D pose estimation
Karashchuk, P., Rupp, K. L., Dickinson, E. S., Walling-Bell, S., Sanders, E., Azim, E., Brunton, B. W., & Tuthill, J. C. (2021) · Cell Reports · doi:10.1016/j.celrep.2021.109730 · Cited by 324
NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video
Sun, J., Xie, Y., Chen, L., Zhou, X., & Bao, H. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01534 · Cited by 313
Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation
Hur, J., & Roth, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00590 · Cited by 288
COTR: Correspondence Transformer for Matching Across Images
Jiang, W., Trulls, E., Hosang, J., Tagliasacchi, A., & Yi, K. M. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00615 · Cited by 282
DeepTAM: Deep Tracking and Mapping
Zhou, H., Ummenhofer, B., & Brox, T. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01270-0_50 · Cited by 226
Self-citationVisual Odometry Revisited: What Should Be Learnt?
Zhan, H., Weerasekera, C. S., Bian, J.-W., & Reid, I. (2020) · 2020 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra40945.2020.9197374 · Cited by 201
Deep semantic segmentation for automated driving: Taxonomy, roadmap and challenges
Siam, M., Elkerdawy, S., Jagersand, M., & Yogamani, S. (2017) · 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2017.8317714 · Cited by 178
Monocular depth estimation with hierarchical fusion of dilated CNNs and soft-weighted-sum inference
Li, B., Dai, Y., & He, M. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.05.029 · Cited by 142
CAM-Convs: Camera-Aware Multi-Scale Convolutions for Single-View Depth
Facil, J. M., Ummenhofer, B., Zhou, H., Montesano, L., Brox, T., & Civera, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01210 · Cited by 130
“As [38], we enforce the network to predict a confidence map for every depth prediction. n : Surface normals.”
BackgroundMethodInfluentialSelf-citationComputer vision in automated parking systems: Design, implementation and challenges
Heimberger, M., Horgan, J., Hughes, C., McDonald, J., & Yogamani, S. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2017.07.002 · Cited by 126
Wang, K., Ma, S., Chen, J., Ren, F., & Lu, J. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2020.3038898 · Cited by 114
“based methods, most of them use depth and optical flow to assist the training process [96], [104]–[106].”
MethodVisual SLAM for Automated Driving: Exploring the Applications of Deep Learning
Milz, S., Arbeiter, G., Witt, C., Abdallah, B., & Yogamani, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00062 · Cited by 107
Web Stereo Video Supervision for Depth Prediction from Dynamic Scenes
Wang, C., Lucey, S., Perazzi, F., & Wang, O. (2019) · 2019 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2019.00046 · Cited by 103
Deep Learning for Visual Localization and Mapping: A Survey
Chen, C., Wang, B., Lu, C. X., Trigoni, N., & Markham, A. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3309809 · Cited by 97
DeepMapping: Unsupervised Map Estimation From Multiple Point Clouds
Ding, L., & Feng, C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00885 · Cited by 79
Motion and Depth Augmented Semantic Segmentation for Autonomous Navigation
Rashed, H., El Sallab, A., Yogamani, S., & ElHelw, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00049 · Cited by 38
Self-supervised Learning for Single View Depth and Surface Normal Estimation
Zhan, H., Weerasekera, C. S., Garg, R., & Reid, I. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8793984 · Cited by 31
DF-VO: What Should Be Learnt for Visual Odometry?
Zhan, H., Weerasekera, C. S., Bian, J.-W., Garg, R., & Reid, I. (2021) · arXiv · doi:10.48550/arxiv.2103.00933 · Cited by 31
Metadata
- Authors
- Ummenhofer, B., Zhou, H., Uhrig, J., Mayer, N., Ilg, E., Dosovitskiy, A., & Brox, T.
- Venue
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2017-07
- Pages
- 5622-5631
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2561074213
- Cited by
- 692
- References
- 46
- FWCI
- 27.59
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.