REMODE: Probabilistic, monocular dense reconstruction in real time
Open accessThe DOI matches its Crossref record; 25 of the 26 references it cites were found in a registry or an index.
Abstract
In this paper, we solve the problem of estimating dense and accurate depth maps from a single moving camera. A probabilistic depth measurement is carried out in real time on a per-pixel basis and the computed uncertainty is used to reject erroneous estimations and provide live feedback on the reconstruction progress. Our contribution is a novel approach to depth map computation that combines Bayesian estimation and recent development on convex optimization for image processing. We demonstrate that our method outperforms state-of-the-art techniques in terms of accuracy, while exhibiting high efficiency in memory usage and computing power. We call our approach REMODE (REgularized MOnocular Depth Estimation). Our CUDA-based implementation runs at 30Hz on a laptop computer and is released as open-source software.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These representations have recently gained popularity in monocular SLAM, in conjunction with the use of direct methods [118, 184, 203], which estimate the trajectory of the robot and a 3D model directly from the intensity values of all the image pixels.”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [203]
Citations
Cites
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070
Nonlinear total variation based noise removal algorithms
Rudin, L. I., Osher, S., & Fatemi, E. (1992) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(92)90242-f · Cited by 15,704
A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging
Chambolle, A., & Pock, T. (2011) · Journal of Mathematical Imaging and Vision · doi:10.1007/s10851-010-0251-1 · Cited by 4,336
Accurate, Dense, and Robust Multiview Stereopsis
Furukawa, Y., & Ponce, J. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.161 · Cited by 3,019
A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms
Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.19 · Cited by 2,457 · cited 1×
“The growing interest for dense reconstructions has renewed the attention in multi-view stereo techniques [3], [4], [5], [6], where the involved computational complexity used to prevent applications in robot perception.”
BackgroundSVO: Fast semi-direct monocular visual odometry
Forster, C., Pizzoli, M., & Scaramuzza, D. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6906584 · Cited by 2,205
Self-citationDTAM: Dense tracking and mapping in real-time
Newcombe, R. A., Lovegrove, S. J., & Davison, A. J. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126513 · Cited by 2,078 · cited 4×
“If, on one hand, estimating the depth independently for every pixel leads to efficient, parallel implementations, on the other hand the authors of [10], [11], [12] argued that, similar to other computer vision problems, such as image de-noising [13] and optical flow estimation [14], a smoothing step is required in ord…”
BackgroundMethodInfluentialReal-time simultaneous localisation and mapping with a single camera
Davison (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238654 · Cited by 1,733
Fast Global Minimization of the Active Contour/Snake Model
Bresson, X., Esedoḡlu, S., Vandergheynst, P., Thiran, J.-P., & Osher, S. (2007) · Journal of Mathematical Imaging and Vision · doi:10.1007/s10851-007-0002-0 · Cited by 857
Inverse Depth Parametrization for Monocular SLAM
Civera, J., Davison, A. J., & Montiel, J. (2008) · IEEE Transactions on Robotics · doi:10.1109/tro.2008.2003276 · Cited by 779
Evaluation of Stereo Matching Costs on Images with Radiometric Differences
Hirschmuller, H., & Scharstein, D. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.221 · Cited by 721
Handling occlusions in dense multi-view stereo
Kang, S. B., Szeliski, R., & Chai, J. (2005) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 · doi:10.1109/cvpr.2001.990462 · Cited by 353
Robust real-time visual odometry for dense RGB-D mapping
Whelan, T., Johannsson, H., Kaess, M., Leonard, J. J., & McDonald, J. (2013) · 2013 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2013.6631400 · Cited by 344
Real-Time Plane-Sweeping Stereo with Multiple Sweeping Directions
Gallup, D., Frahm, J.-M., Mordohai, P., Yang, Q., & Pollefeys, M. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383245 · Cited by 322
Motion estimation with non-local total variation regularization
Werlberger, M., Pock, T., & Bischof, H. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539945 · Cited by 216
Real-Time Dense Geometry from a Handheld Camera
Stühmer, J., Gumhold, S., & Cremers, D. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15986-2_2 · Cited by 205
Video-based, real-time multi-view stereo
Vogiatzis, G., & Hernández, C. (2011) · Image and Vision Computing · doi:10.1016/j.imavis.2011.01.006 · Cited by 175
Real-Time Camera Tracking: When is High Frame-Rate Best?
Handa, A., Newcombe, R. A., Angeli, A., & Davison, A. J. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33786-4_17 · Cited by 159
Fast and High Quality Fusion of Depth Maps
Zach, C. (2008) · Cited by 120
Sampling the disparity space image
Szeliski, R., & Scharstein, D. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.1262341 · Cited by 89
Air-ground localization and map augmentation using monocular dense reconstruction
Förster, C., Pizzoli, M., & Scaramuzza, D. (2013) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2013.6696924 · Cited by 84
Self-citationOnline 3D reconstruction using convex optimization
Graber, G., Pock, T., & Bischof, H. (2011) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2011.6130318 · Cited by 79
Incremental estimation of dense depth maps from image sequences
Matthies, L. H., Szeliski, R., & Takeo Kanade, A. (2003) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.1988.196261 · Cited by 55
The City of Sights: Design, construction, and measurement of an Augmented Reality stage set
Gruber, L., Gauglitz, S., Ventura, J., Zollmann, S., Huber, M., Schlegel, M., Klinker, G., Schmalstieg, D., & Höllerer, T. (2010) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2010.5643564 · Cited by 35
Super-resolution 3D tracking and mapping
Meilland, M., & Comport, A. I. (2013) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2013.6631399 · Cited by 31
Research works printed but not linked: 1
- werlberger, 0, Motion estimation with nonlocal total variation regularization, Proc IEEE Conf on Computer Vision and Pattern Recognition 2010
Cited by
LSD-SLAM: Large-scale direct monocular SLAM
Engel, J., Schöps, T., & Cremers, D. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10605-2_54 · Cited by 3,851
“While direct image alignment is well-established for RGB-D or stereo sensors [14,4], only recently monocular direct VO algorithms have been proposed: In [24,20,21], accurate and fully dense depth maps are computed using a variational formulation, which however is computationally demanding and requires”
MethodPast, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
“These representations have recently gained popularity in monocular SLAM, in conjunction with the use of direct methods [118], [184], [203],”
Self-citationPast, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
Self-citationEngel, J., Koltun, V., & Cremers, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2658577 · Cited by 2,924
SVO: Fast semi-direct monocular visual odometry
Forster, C., Pizzoli, M., & Scaramuzza, D. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6906584 · Cited by 2,205
“In [29] we demonstrate how the same approach can be used for dense mapping.”
BackgroundMethodSelf-citation
Metadata
- Authors
- Pizzoli, M., Forster, C., & Scaramuzza, D.
- Affiliations
- University of Zurich (CH)
- ORCID
- Scaramuzza, Davide
- Venue
- 2014 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2014-05
- Pages
- 2609-2616
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2093659073
- Cited by
- 330
- References
- 26
- Citations by year
- 2026: 3 · 2025: 11 · 2024: 10 · 2023: 16 · 2022: 26 · 2021: 33 · 2020: 36 · 2019: 48 · 2018: 39 · 2017: 47 · 2016: 33 · 2015: 20 · 2014: 8
- FWCI
- 19.36
- Type
- conference paper
- Language
- en
- Open access
- green
- www.zora.uzh.ch
- Licence
- other-oa
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, other-oa)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion, other-oa)
- Zurich Open Repository and Archive (University of Zurich)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Engineering › Media Technology › Image Processing Techniques and Applications
- Keywords
- monocular depth estimation, dense reconstruction, uncertainty estimation, Bayesian estimation, convex optimization, real-time depth estimation, CUDA
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.