REMODE: Probabilistic, monocular dense reconstruction in real time
Open accessThe DOI matches its Crossref record; 25 of the 26 references it cites were found in a registry or an index.
Abstract
In this paper, we solve the problem of estimating dense and accurate depth maps from a single moving camera. A probabilistic depth measurement is carried out in real time on a per-pixel basis and the computed uncertainty is used to reject erroneous estimations and provide live feedback on the reconstruction progress. Our contribution is a novel approach to depth map computation that combines Bayesian estimation and recent development on convex optimization for image processing. We demonstrate that our method outperforms state-of-the-art techniques in terms of accuracy, while exhibiting high efficiency in memory usage and computing power. We call our approach REMODE (REgularized MOnocular Depth Estimation). Our CUDA-based implementation runs at 30Hz on a laptop computer and is released as open-source software.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These representations have recently gained popularity in monocular SLAM, in conjunction with the use of direct methods [118, 184, 203], which estimate the trajectory of the robot and a 3D model directly from the intensity values of all the image pixels.”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [203]
Citations
Cites
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070
Nonlinear total variation based noise removal algorithms
Rudin, L. I., Osher, S., & Fatemi, E. (1992) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(92)90242-f · Cited by 15,704
A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging
Chambolle, A., & Pock, T. (2011) · Journal of Mathematical Imaging and Vision · doi:10.1007/s10851-010-0251-1 · Cited by 4,336
Accurate, Dense, and Robust Multiview Stereopsis
Furukawa, Y., & Ponce, J. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.161 · Cited by 3,019
A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms
Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.19 · Cited by 2,457 · cited 1×
“The growing interest for dense reconstructions has renewed the attention in multi-view stereo techniques [3], [4], [5], [6], where the involved computational complexity used to prevent applications in robot perception.”
BackgroundSVO: Fast semi-direct monocular visual odometry
Forster, C., Pizzoli, M., & Scaramuzza, D. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6906584 · Cited by 2,205
Self-citationDTAM: Dense tracking and mapping in real-time
Newcombe, R. A., Lovegrove, S. J., & Davison, A. J. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126513 · Cited by 2,078 · cited 4×
“If, on one hand, estimating the depth independently for every pixel leads to efficient, parallel implementations, on the other hand the authors of [10], [11], [12] argued that, similar to other computer vision problems, such as image de-noising [13] and optical flow estimation [14], a smoothing step is required in ord…”
BackgroundMethodInfluentialReal-time simultaneous localisation and mapping with a single camera
Davison (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238654 · Cited by 1,733
Fast Global Minimization of the Active Contour/Snake Model
Bresson, X., Esedoḡlu, S., Vandergheynst, P., Thiran, J.-P., & Osher, S. (2007) · Journal of Mathematical Imaging and Vision · doi:10.1007/s10851-007-0002-0 · Cited by 857
Inverse Depth Parametrization for Monocular SLAM
Civera, J., Davison, A. J., & Montiel, J. (2008) · IEEE Transactions on Robotics · doi:10.1109/tro.2008.2003276 · Cited by 779
Research works printed but not linked: 1
- werlberger, 0, Motion estimation with nonlocal total variation regularization, Proc IEEE Conf on Computer Vision and Pattern Recognition 2010
Cited by
LSD-SLAM: Large-scale direct monocular SLAM
Engel, J., Schöps, T., & Cremers, D. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10605-2_54 · Cited by 3,851
“While direct image alignment is well-established for RGB-D or stereo sensors [14,4], only recently monocular direct VO algorithms have been proposed: In [24,20,21], accurate and fully dense depth maps are computed using a variational formulation, which however is computationally demanding and requires”
MethodPast, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
“These representations have recently gained popularity in monocular SLAM, in conjunction with the use of direct methods [118], [184], [203],”
Self-citationPast, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
Self-citationEngel, J., Koltun, V., & Cremers, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2658577 · Cited by 2,924
SVO: Fast semi-direct monocular visual odometry
Forster, C., Pizzoli, M., & Scaramuzza, D. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6906584 · Cited by 2,205
“In [29] we demonstrate how the same approach can be used for dense mapping.”
BackgroundMethodSelf-citationLabbé, M., & Michaud, F. (2019) · Journal of Field Robotics · doi:10.1002/rob.21831 · Cited by 989
SVO: Semidirect Visual Odometry for Monocular and Multicamera Systems
Forster, C., Zhang, Z., Gassner, M., Werlberger, M., & Scaramuzza, D. (2017) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2623335 · Cited by 922
“When a depth filter’s uncertainty becomes small enough, a new 3D point is inserted in the map and is immediately used for motion estimation.”
BackgroundMethodSelf-citationCNN-SLAM: Real-Time Dense Monocular SLAM with Learned Depth Prediction
Tateno, K., Tombari, F., Laina, I., & Navab, N. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.695 · Cited by 800
“As for REMODE, since it requires as input the camera pose estimation at each frame, we use the trajectory and key-frames estimated by LSD-SLAM with bootstrapping.”
MethodInfluentialPL-SLAM: A Stereo SLAM System Through the Combination of Points and Line Segments
Gomez-Ojeda, R., Moreno, F.-A., Zuniga-Noel, D., Scaramuzza, D., & Gonzalez-Jimenez, J. (2019) · IEEE Transactions on Robotics · doi:10.1109/tro.2019.2899783 · Cited by 560
Self-citationPL-SLAM: Real-time monocular visual SLAM with points and lines
Pumarola, A., Vakhitov, A., Agudo, A., Sanfeliu, A., & Moreno-Noguer, F. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989522 · Cited by 482
“To solve this, dense and direct methods can be applied, even though they are likely to be computationally expensive [19], [21], and require dedicated GPU-implementations to achieve real-time performance.”
MethodDeep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry
Yang, N., Wang, R., Stückler, J., & Cremers, D. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01237-3_50 · Cited by 346
Survey on Computer Vision for UAVs: Current Developments and Trends
Kanellakis, C., & Nikolakopoulos, G. (2017) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-017-0483-z · Cited by 345
“In this work a monocular approach uses only an onboard smartphone processor for semi direct visual odometry [54], multi sensor fusion [56] and a modified version of Regularized Modular Depth Estimation (REMODE) [57].”
MethodNeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video
Sun, J., Xie, Y., Chen, L., Zhou, X., & Bao, H. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01534 · Cited by 313
González-García, J., Gómez-Espinosa, A., Cuan-Urquizo, E., García-Valdovinos, L. G., Salgado-Jiménez, T., & Cabello, J. A. E. (2020) · Applied Sciences · doi:10.3390/app10041256 · Cited by 283
“Obstacle avoidance and visualization [77].”
EMVS: Event-Based Multi-View Stereo—3D Reconstruction with an Event Camera in Real-Time
Rebecq, H., Gallego, G., Mueggler, E., & Scaramuzza, D. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-017-1050-6 · Cited by 278
Self-citationRecent Trends in Underwater Visible Light Communication (UVLC) Systems
Ali, M. F., Jayakody, D. N. K., & Li, Y. (2022) · IEEE Access · doi:10.1109/access.2022.3150093 · Cited by 258
Autonomous, Vision‐based Flight and Live Dense 3D Mapping with a Quadrotor Micro Aerial Vehicle
Faessler, M., Fontana, F., Forster, C., Mueggler, E., Pizzoli, M., & Scaramuzza, D. (2016) · Journal of Field Robotics · doi:10.1002/rob.21581 · Cited by 247
“Therefore, we use the REMODE (“Regularized Monocular Depth”) algorithm that we proposed in Pizzoli et al. (2014) and that we summarize in the following.”
MethodInfluentialSelf-citationAutonomous aerial navigation using monocular visual‐inertial fusion
Lin, Y., Gao, F., Qin, T., Gao, W., Liu, T., Wu, W., Yang, Z., & Shen, S. (2018) · Journal of Field Robotics · doi:10.1002/rob.21732 · Cited by 237
“By demonstrating challenging indoor environment experiments, the F IGURE 1 A synthetic and closeup of our aerial robot platform: (1) Perception core; (2) DJI Lightbridge 2; (3) DJI Phantom 4 Pro battery; (4) DJI Takyon Z425-M ESC; (5) DJI E310motors; (6) iPad Pro; (7) radio controller authors give a well thought out o…”
MethodSmolyanskiy, N., Kamenev, A., Smith, J., & Birchfield, S. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206285 · Cited by 222
“Other approaches have been developed that estimate depth using a monocular video stream [23], [34].”
BackgroundEvent-Based, 6-DOF Camera Tracking from Photometric Depth Maps
Gallego, G., Lund, J. E. A., Mueggler, E., Rebecq, H., Delbruck, T., & Scaramuzza, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2769655 · Cited by 211
“These can be obtained from a previous mapping stage by means of classical dense reconstruction approaches using standard cameras (e.g., DTAM [16] or REMODE [17]) or even using a DVS (future research).”
BackgroundSelf-citationComparison of Various SLAM Systems for Mobile Robot in an Indoor Environment
Filipenko, M., & Afanasyev, I. (2018) · 2018 International Conference on Intelligent Systems (IS) · doi:10.1109/is.2018.8710464 · Cited by 199
“Over the past decade, numerous methods have been proposed, including Parallel Tracking and Mapping (PTAM, [17], [18]), REgularized MOnocular Depth Estimation (REMODE, [19]), Oriented FAST [20] and Rotated BRIEF [21] (ORB-SLAM, [22], [23]), Dense Tracking and Mapping in Real-Time (DTAM, [24]), Large-Scale Direct monocu…”
Robotic weed control using automated weed and crop classification
Wu, X., Aravecchia, S., Lottes, P., Stachniss, C., & Pradalier, C. (2020) · Journal of Field Robotics · doi:10.1002/rob.21938 · Cited by 196
“For direct dense mapping (Newcombe et al., 2011; Pizzoli et al., 2014; Stühmer, Gumhold, & Cremers, 2010), the photometric error with spacial geometry prior formulations are developed to reconstruct a dense depth map and corresponding camera motion accurately.”
BackgroundFalanga, D., Mueggler, E., Faessler, M., & Scaramuzza, D. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989679 · Cited by 185
“A valid alternative to cope with real-world gaps would be to use monocular densereconstruction methods, such as REMODE [13]; however, they require more computing power (GPUs).”
MethodSelf-citationDeepFactors: Real-Time Probabilistic Dense Monocular SLAM
Czarnowski, J., Laidlow, T., Clark, R., & Davison, A. J. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2965415 · Cited by 169
“[8], [9]) to purely end-to-end learned systems that directly predict the camera poses and the geometry of the scene (e.”
Semi-dense visual odometry for AR on a smartphone
Schops, T., Engel, J., & Cremers, D. (2014) · 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) · doi:10.1109/ismar.2014.6948420 · Cited by 158
A Review on Viewpoints and Path Planning for UAV-Based 3-D Reconstruction
Maboudi, M., Homaei, M., Song, S., Malihi, S., Saadatseresht, M., & Gerke, M. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3276427 · Cited by 158
“it utilizes a monocular mapping algorithm called REMODE [153].”
MethodInfluentialKeyframe-based monocular SLAM: design, survey, and future directions
Younes, G., Asmar, D., Shammas, E., & Zelek, J. (2017) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2017.09.010 · Cited by 191
An information gain formulation for active volumetric 3D reconstruction
Isler, S., Sabzevari, R., Delmerico, J., & Scaramuzza, D. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487527 · Cited by 232
“To retrieve depth information from a given view, the robot moves the camera around the view that serves as key frame until REMODE reports successful convergence of depth estimates.”
MethodInfluentialSelf-citationActive Autonomous Aerial Exploration for Ground Robot Path Planning
Delmerico, J., Mueggler, E., Nitsch, J., & Scaramuzza, D. (2017) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2017.2651163 · Cited by 160
Self-citationA Photometrically Calibrated Benchmark For Monocular Visual OdometryPreprint
Engel, J. J., Usenko, V., & Cremers, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.02555 · Cited by 209
“…cameras (which often do not allow to either read or set parameters like the exposure time), we argue that for the above-mentioned use cases, this ultimately is the wrong approach: sensors – including cameras – can, and will be designed to fit the needs of the algorithms processing their data.”
BackgroundDPPTAM: Dense piecewise planar tracking and mapping from a monocular sequence
Concha, A., & Civera, J. (2015) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7354184 · Cited by 157
“REMODE [16] (standing for Regularized Monocular Dense reconstruction) propose to integrate a Bayesian estimation of the inverse depth into the variational formulation.”
BackgroundRobust Dense Mapping for Large-Scale Dynamic Environments
Bârsan, I. A., Liu, P., Pollefeys, M., & Geiger, A. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8462974 · Cited by 152
“Map representations range from the conventional point cloud [2], [17], [21] and mesh-based [29] approaches to more sophisticated ones based on, e.”
BackgroundMVDepthNet: Real-Time Multiview Depth Estimation Neural Network
Wang, K., & Shen, S. (2018) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00037 · Cited by 157
Cvišić, I., Ćesić, J., Marković, I., & Petrović, I. (2018) · Journal of Field Robotics · doi:10.1002/rob.21762 · Cited by 133
State of the Art of Automated Buses
Ainsalu, J., Arffman, V., Bellone, M., Ellner, M., Haapamäki, T., Haavisto, N., Josefson, E., Ismailogullari, A., Lee, B., Madland, O., et al. (2018) · Sustainability · doi:10.3390/su10093118 · Cited by 128
“Considering that an autonomous vehicle would be moving, it is possible estimate the distance of objects by measuring the variations between two consecutive frames acquired from the same (moving) camera [44,45].”
A review of monocular visual odometry
He, M., Zhu, C., Huang, Q., Ren, B., & Liu, J. (2019) · The Visual Computer · doi:10.1007/s00371-019-01714-6 · Cited by 140
“However, computing dense depths from a monocular vision requires substantial computing power, typically using GPU parallel operations, such as open-source REMODE [52,53].”
MethodProbabilistic Semi-Dense Mapping from Highly Accurate Feature-Based Monocular SLAM
Mur-Artal, R., & Tardós, J. D. (2015) · Robotics: Science and Systems · doi:10.15607/rss.2015.xi.041 · Cited by 130
“One of the main novelties of our semi-dense mapping method is that instead of using many subsequent frames to filter the inverse depth of a reference frame [2, 3, 15], we perform the reconstruction over keyframes, which are very well localised by local bundle adjustment, and pose graph optimization after a loop closur…”
MethodRecent developments and applications of simultaneous localization and mapping in agriculture
Ding, H., Zhang, B., Zhou, J. C., Yan, Y., Tian, G., & Gu, B. (2022) · Journal of Field Robotics · doi:10.1002/rob.22077 · Cited by 146
“Monocular and binocular cameras obtain depth indirectly through some geometric relations (Pizzoli et al., 2014), while RGB‐D cameras can directly measure depth by computing the time to receive reflected infrared light.”
BackgroundDirect Visual SLAM Using Sparse Depth for Camera-LiDAR System
Shin, Y.-S., Park, Y. S., & Kim, A. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8461102 · Cited by 115
“Instead of using a feature-based reprojection error model, they mainly use a photometric error model that considers the intensity difference of a pixel [4]–[6], [9], [16].”
Method3D Modeling on the Go: Interactive 3D Reconstruction of Large-Scale Scenes on Mobile Devices
Schöps, T., Sattler, T., Häne, C., & Pollefeys, M. (2015) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2015.40 · Cited by 114
“While [19, 21] employ regularization to fill in missing regions, [4,24,28,32] filter out unreliable points based on estimates of their uncertainty.”
BackgroundDVL-SLAM: sparse depth enhanced direct visual-LiDAR SLAM
Shin, Y.-S., Park, Y. S., & Kim, A. (2019) · Autonomous Robots · doi:10.1007/s10514-019-09881-0 · Cited by 120
“Instead of using the reprojection errors of correspondences, the direct method considers the intensity difference of a pixel as a photometric error model (Engel et al., 2014, 2017; Forster et al., 2014; Newcombe et al., 2011b; Pizzoli et al., 2014).”
MethodNeRFusion: Fusing Radiance Fields for Large-Scale Scene Reconstruction
Zhang, X., Bi, S., Sunkavalli, K., Su, H., & Xu, Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00537 · Cited by 154
“This fusion process is similar to previous 3D reconstruction pipelines [20, 33, 39] that focus on geometry reconstruction; in contrast, we instead reconstruct neural feature volumes to represent neural radiance fields for volume rendering, leading to photo-realistic novel view synthesis.”
Förster, C., Faessler, M., Fontana, F., Werlberger, M., & Scaramuzza, D. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7138988 · Cited by 116
“Using the zero-mean sum of squared differences (ZMSSD) score on a 8×8 patch, we search the pixel u′i on the epipolar line segment l that has highest correlation with the reference pixel ui.”
MethodInfluentialSelf-citationMonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera
Wimbauer, F., Yang, N., von Stumberg, L., Zeller, N., & Cremers, D. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00605 · Cited by 128
“[38, 41, 2] and saw a great improvement through the use of convolutional neural networks (CNNs) [24, 55, 51].”
BackgroundMethodGood features to track for visual SLAM
Zhang, G., & Vela, P. (2015) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2015.7298743 · Cited by 85
“It is equally important when considering visual SLAM systems developed for large-scale/dense reconstruction with massive data processing needs [27, 8, 12, 19, 29].”
BackgroundFast depth densification for occlusion-aware augmented reality
Holynski, A., & Kopf, J. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275083 · Cited by 92
“Some SLAM methods attempt to reconstruct a dense depth map that covers all pixels in the video [M. Pizzoli 2014; New-combe et al. 2011].”
MethodKeyframe‐based thermal–inertial odometry
Khattak, S., Papachristos, C., & Alexis, K. (2019) · Journal of Field Robotics · doi:10.1002/rob.21932 · Cited by 101
“…visual odometry methods that rely on the minimization of photometric error, such as methods discussed in Newcombe, Lovegrove, and Davison (2011) and Pizzoli, Forster, and Scaramuzza (2014), typically require the optimization to be performed on a large number of points and thus become…”
MethodA review of visual inertial odometry from filtering and optimisation perspectives
Gui, J., Gu, D., Wang, S., & Hu, H. (2015) · Advanced Robotics · doi:10.1080/01691864.2015.1057616 · Cited by 110
“Estimating the depth information in a monocular camera is also conducted via a Bayesian filter followed by an optimisation process to smooth the depth map in [28].”
MethodSimultaneous Localization and Mapping in the Epoch of Semantics: A Survey
Sualeh, M., & Kim, G.-W. (2018) · International Journal of Control Automation and Systems · doi:10.1007/s12555-018-0130-x · Cited by 94
“REMODE [111] 14 Mo D Gradient - + - - + RGB-D SLAM [112] 14 R F Combination PG G2O - + - - + SlamDunk [113] 14 R|Mu F SIFT/SURF BA G2O - + + + + -”
Keyframe-based Direct Thermal–Inertial Odometry
Khattak, S., Papachristos, C., & Alexis, K. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793927 · Cited by 81
“Historically, direct visual odometry methods based on the minimization of photometric error such as [20–22], require optimization to be performed on a large number of points and thus becoming computationally expensive.”
Perception-aware Receding Horizon Navigation for MAVs
Zhang, Z., & Scaramuzza, D. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8461133 · Cited by 80
“REMODE, on the other hand, is originally designed to run on a Graphic Processing Unit (GPU).”
MethodInfluentialSelf-citationEye in the Sky: Drone-Based Object Tracking and 3D Localization
Zhang, H., Wang, G., Lei, Z., & Hwang, J.-N. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3350933 · Cited by 97
“Based on the sparse depth map values estimated from SVO, we further formulate the dense depth calculation as a Gaussian estimation problem [26] so as to estimate the depth map values surrounding the initialized SIFT points based on multiple frames of a monocular video.”
MethodRealtime Edge-Based Visual Odometry for a Monocular Camera
Tarrio, J., & Pedre, S. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.87 · Cited by 88
“Even though some of these algorithms are able to run in real time in modern cpu+gpu equipment [19] [18] their computational cost is still too high for embedded solutions, in which the processing power is limited by weight (in the case of light aerial vehicles), power consumption or simply by commercial price when thin…”
BackgroundMobile3DRecon: Real-time Monocular 3D Reconstruction on a Mobile Phone
Yang, X., Zhou, L., Jiang, H., Tang, Z.-L., Wang, Y., Bao, H., & Zhang, G. (2020) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2020.3023634 · Cited by 89
“Table 1: We report RMSEs and MAEs of the depth and surface mesh results by our Mobile3DRecon and [11,27,40] on our five experimental sequences captured by OPPO R17 Pro with ToF depth measurements as GT.”
BackgroundMethodInfluentialSimultaneous Localization And Mapping: Present, Future, and the Robust-Perception Age.Preprint
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · arXiv (Cornell University) · Cited by 90
Self-citationFrom SLAM to Situational Awareness: Challenges and Survey
Bavle, H., Sánchez-López, J. L., Cimarelli, C., Tourani, A., & Voos, H. (2023) · Sensors · doi:10.3390/s23104849 · Cited by 108
“Similar extensions to the PTAM framework are ORB-SLAM [124], and REMODE [171] creating a semi-dense 3D geometric map of the environment while estimating the camera trajectory.”
MethodLarge-scale outdoor 3D reconstruction on a mobile device
Schöps, T., Sattler, T., Häne, C., & Pollefeys, M. (2016) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2016.09.007 · Cited by 74
Shariq, M. H., & Hughes, B. R. (2020) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2020.109979 · Cited by 86
Direct-PoseNet: Absolute Pose Regression with Photometric Consistency
Chen, S., Wang, Z., & Prisacariu, V. A. (2021) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00125 · Cited by 102
“Direct matching is a common approach in traditional SLAM and VO systems [10, 11, 13, 14, 36, 42].”
MethodAerial-guided navigation of a ground robot among movable obstacles
Mueggler, E., Faessler, M., Fontana, F., & Scaramuzza, D. (2014) · IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR) · doi:10.1109/ssrr.2014.7017662 · Cited by 80
“We plan to overcome these limitations by building a 3D map using the images from the aerial robot, e.g., using our real-time dense reconstruction algorithm (REMODE) [23].”
MethodSelf-citationToward Domain Independence for Learning-Based Monocular Depth Estimation
Mancini, M., Costante, G., Valigi, P., Ciarfuglia, T. A., Delmerico, J., & Scaramuzza, D. (2017) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2017.2657002 · Cited by 81
“Despite the impressive results achieved by state-of-the-art approaches [14]–[16], the performance of their reconstruction routines drops during high-speed motion, as dense alignment becomes extremely challenging.”
BackgroundSelf-citationDirect Sparse OdometryPreprint
Engel, J. J., Koltun, V., & Cremers, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.02565 · Cited by 70
“of an additional log-likelihood energy term [21], [22], [26].”
BackgroundMethodDRE-SLAM: Dynamic RGB-D Encoder SLAM for a Differential-Drive Robot
Yang, D., Bi, S., Wang, W., Chang, Y., Wang, W., Qi, X., & Cai, Y. (2019) · Remote Sensing · doi:10.3390/rs11040380 · Cited by 75
“Point cloud [12], surfel [2], and volumetric [13,14] representations are three commonly used dense environment representations [4].”
BackgroundReal-time monocular dense mapping on aerial robots using visual-inertial fusion
Yang, Z., Gao, F., & Shen, S. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989529 · Cited by 71
“However, the convex optimization used in DTAM and REMODE requires a desktop GPU.”
MethodLoosely-Coupled Semi-Direct Monocular SLAM
Lee, S. H., & Civera, J. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2889156 · Cited by 81
“It was shown in [23] that SVO could also be used for dense mapping.”
BackgroundLoosely-Coupled Semi-Direct Monocular SLAM
Lee, S. H., & Civera, J. (2019) · Zaguan (University of Zaragoza Repository) · Cited by 63
Towards the Internet of Flying Robots: A Survey
Huang, H., & Savkin, A. V. (2018) · Sensors · doi:10.3390/s18114038 · Cited by 68
“This system uses an “on-the-spot training” approach to classify terrains [91] and REMODE for 3D reconstruction [92].”
MethodObstacle Avoidance Strategy using Onboard Stereo Vision on a Flapping Wing MAV
Tijmons, S., de Croon, G., Remes, B. D. W., Wagter, C. D., & Mulder, M. (2017) · IEEE Transactions on Robotics · doi:10.1109/tro.2017.2683530 · Cited by 74
“passive, are commonly used in combination with simultaneous localization and mapping (SLAM) methods [5]–[7], dense reconstruction methods [8], or visual odometry methods [9] which”
MethodComparative analysis of ROS-based monocular SLAM methods for indoor navigation
Buyval, A., Afanasyev, I. M., & Magid, E. (2017) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2268809 · Cited by 59
“REMODE uses a depth map computation approach, which combines Bayesian estimation and convex optimization for image processing, executing a camera pose estimation by Semi-Direct Monocular Visual Odometry (SVO) [5].”
MethodLaidlow, T., Czarnowski, J., & Leutenegger, S. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793527 · Cited by 74
“Following the evaluation procedure of CNN-SLAM, we also compare our results to the depth maps produced by a sparse feature-based monocular system (ORB-SLAM2 [3]), a semidense geometric system (LSD-SLAM [11]), a fully dense reconstruction method (REMODE [5]), and a pure deep learning approach (Laina, et al.”
BackgroundMethodInfluentialTian, F., Gao, Y., Fang, Z., Fang, Y., Gu, J., Fujita, H., & Hwang, J. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3080928 · Cited by 61
View Path Planning via Online Multiview Stereo for 3-D Modeling of Large-Scale Structures
Song, S., Kim, D., & Choi, S. (2021) · IEEE Transactions on Robotics · doi:10.1109/tro.2021.3083197 · Cited by 68
J-MOD2: Joint Monocular Obstacle Detection and Depth Estimation
Mancini, M., Costante, G., Valigi, P., & Ciarfuglia, T. A. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2800083 · Cited by 63
“Monocular obstacle detection can be achieved by dense 3D map reconstruction via SLAM or Structure from Motion (SFM) based procedures [6], [15], [16].”
MethodVisual industrial inspection using aerial robots
Omari, S., Gohl, P., Burri, M., Achtelik, M., & Siegwart, R. (2014) · International Conference on Applied Robotics for the Power Industry (CARPI) · doi:10.1109/carpi.2014.7030056 · Cited by 61
“Lastly, we evaluate the navigation system in a set of experiments.”
MethodPolarimetric Dense Monocular SLAM
Yang, L., Tan, F., Li, A., Cui, Z., Furukawa, Y., & Tan, P. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00406 · Cited by 61
“We modified the code shared by the authors of Remode to use DSO [10] for camera tracking to ensure a fair comparison.”
BackgroundResultInfluentialFu, C., Carrio, A., & Campoy, P. (2015) · International Conference on Unmanned Aircraft Systems (ICUAS) · doi:10.1109/icuas.2015.7152384 · Cited by 55
“For monocular camera, M. Pizzoli et al [1] et al recently presented a real-time probabilistic monocular pose estimation method for 3D dense reconstruction.”
MethodCombining Feature-Based and Direct Methods for Semi-dense Real-Time Stereo Visual Odometry
Krombach, N., Droeschel, D., & Behnke, S. (2017) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-48036-7_62 · Cited by 60
“They often need to use GPUs to achieve real-time performance [16, 19].”
BackgroundPothole 3D Reconstruction With a Novel Imaging System and Structure From Motion Techniques
Ahmed, A., Ashfaque, M., Haq, M. U. U., Mathavan, S., Kamal, K., & Rahman, M. M. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3054026 · Cited by 52
“In inertial measurement unit (IMU) supported reconstruction methods [33]–[37], feature based tracking and 3D map building is assisted by inertial sensing in relative position and orientation estimation.”
MethodMulti-level mapping: Real-time dense monocular SLAM
Greene, W. N., Ok, K., Lommel, P., & Roy, N. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487213 · Cited by 55
“While this approach is typically implemented on a GPU [8], [10], [11], we find that running in a separate thread and operating on our quadtree-compressed depthmap, our version runs sufficiently fast for real-time operation on a CPU.”
MethodEffective Exploration for MAVs Based on the Expected Information Gain
Palazzolo, E., & Stachniss, C. (2018) · Drones · doi:10.3390/drones2010009 · Cited by 53
“[37], we compute the norm of x+ from the two angles β+ and γ and the base vector b (see Figure 3).”
MethodInfluentialSurvey on path and view planning for UAVs
Zhou, X., Yi, Z., Liu, Y., Huang, K., & Huang, H. F. (2020) · Virtual Reality & Intelligent Hardware · doi:10.1016/j.vrih.2019.12.004 · Cited by 56
Leveraging Planar Regularities for Point Line Visual-Inertial Odometry
Li, X., He, Y., Lin, J., & Liu, X. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341278 · Cited by 54
“In [4], [5] dense mapping can be realized based on monocular vision and pixel-wise reconstruction, which is time-consuming and highly relies on Graphics Processing Unit (GPU) to guarantee real-time performance.”
BackgroundNeuromorphic vision: From sensors to event‐based algorithms
Annamalai, L., Chakraborty, A., & Thakur, C. S. (2019) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1310 · Cited by 59
“These maps were obtained by a previous run of dense standard reconstruction approaches of CMOS camera such as DTAM (Newcombe et al., 2011) or REMODE (Pizzoli et al., 2014).”
MethodAppearance-based Active, Monocular, Dense Reconstruction for Micro Aerial Vehicles
Förster, C., Pizzoli, M., & Scaramuzza, D. (2014) · Robotics: Science and Systems · doi:10.15607/rss.2014.x.029 · Cited by 53
“Furthermore, in [19], we showed how regularity in the depth map can be enforced by making use of a smoothness prior in regions characterized by high uncertainty.”
BackgroundSelf-citationShu, F., Lesur, P., Xie, Y., Pagani, A., & Stricker, D. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00180 · Cited by 58
“However, the employed real-time algorithm REMODE estimates depth based on depth filter, which approximates the mean and variance of the depth at each pixel’s position and updates the depth uncertainty when there is a new measurement (new image captured from the camera).”
MethodInfluentialGe, Y., Zhang, L., Wu, Y., & Hu, D. (2024) · IEEE Transactions on Robotics · doi:10.1109/tro.2024.3366815 · Cited by 52
“D points [6], [7].”
Fast and effective online pose estimation and mapping for UAVs
Schneider, J., Eling, C., Klingbeil, L., Kuhlmann, H., Förstner, W., & Stachniss, C. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487682 · Cited by 42
“As our evaluation shows, we provide accurate pose estimates at the given frequencies, we can handle underconstrained GPS situations well, and the resulting 3D models are accurate and georeferenced.”
MethodAutonomous aerial robot using dual‐fisheye cameras
Gao, W., Wang, K., Ding, W., Gao, F., Qin, T., & Shen, S. (2020) · Journal of Field Robotics · doi:10.1002/rob.21946 · Cited by 51
Visual Sensing and Depth Perception for Welding Robots and Their Industrial Applications
Wang, J., Li, L., & Xu, P. (2023) · Sensors · doi:10.3390/s23249700 · Cited by 51
“To improve the accuracy and stability of depth estimation, some algorithms [94,95] employ depth regularization and convolutional neural networks for monocular depth estimation.”
MethodProgressive Prioritized Multi-view Stereo
Locher, A., Perďoch, M., & Gool, L. V. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.353 · Cited by 41
“Coarse visualization methods such as triangulation of the Structure-from-Motion (SfM) points (e.g. Bodis et al. [2]) or low resolution filter based methods (e.g. Pizzoli et al. [13]) give a good insight into the currently reconstructed scene.”
MethodCollaborative localization of aerial and ground robots through elevation maps
Käslin, R., Fankhauser, P., Stumm, E. S., Taylor, Z., Mueggler, E., Delmerico, J., Scaramuzza, D., Siegwart, R., & Hutter, M. (2016) · IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR) · doi:10.1109/ssrr.2016.7784317 · Cited by 43
Self-citationTowards Autonomous Navigation of Miniature UAV
Brockers, R., Humenberger, M., Weiß, S., & Matthies, L. H. (2014) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2014.98 · Cited by 49
“Since t can only be estimated up to scale from a monocular setup, we use the pose information from our pose estimator to scale the translation vector.”
MethodImproving Monocular Visual Odometry Using Learned Depth
Sun, L., Yin, W., Xie, E., Li, Z., Sun, C., & Shen, C. (2022) · IEEE Transactions on Robotics · doi:10.1109/tro.2022.3164834 · Cited by 48
“Previous geometry-based dense mapping methods [69] usually use a depth filter to obtain depth maps for mapping.”
DRM-SLAM: Towards dense reconstruction of monocular SLAM with scene depth fusion
Ye, X., Ji, X., Sun, B., Chen, S., Wang, Z., & Li, H. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.02.044 · Cited by 41
OpenREALM: Real-time Mapping for Unmanned Aerial Vehicles
Kern, A., Bobbe, M., Khedar, Y., & Bestmann, U. (2020) · International Conference on Unmanned Aircraft Systems (ICUAS) · doi:10.1109/icuas48674.2020.9213960 · Cited by 50
“Their framework ’OpenREMODE’ [13] provides distance weighting on the camera baseline and a Bayesian formulation on depth uncertainty to minimize noise.”
BackgroundFeature-based visual odometry prior for real-time semi-dense stereo SLAM
Krombach, N., Droeschel, D., Houben, S., & Behnke, S. (2018) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2018.08.002 · Cited by 43
“proach for dense image registration and combine it with a sparse feature matcher in order to compute visual odometry [21]. Dense direct methods often need to use GPUs to achieve real-time performance [22, 23]. By using only pixels with sucient gradient, LSD-SLAM [1] reduces the computational demand and real-time semi-…”
Herrera-Granda, E. P., Torres, J. C., & Peluffo-Ordóñez, D. H. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e37356 · Cited by 49
Exploiting Photometric Information for Planning Under Uncertainty
Costante, G., Delmerico, J., Werlberger, M., Valigi, P., & Scaramuzza, D. (2017) · Springer proceedings in advanced robotics · doi:10.1007/978-3-319-51532-8_7 · Cited by 38
“The computed pose Tk,w and the relative image are then fed into the dense map reconstruction module (REMODE [28], a probabilistic, pixelwise depth estimate to compute dense depthmaps).”
MethodSelf-citationDesaraju, V. R., Michael, N., Humenberger, M., Brockers, R., Weiß, S., Nash, J., & Matthies, L. H. (2015) · Autonomous Robots · doi:10.1007/s10514-015-9456-x · Cited by 42
“The work of Pizzoli et al (2014) presents a monocular reconstruction engine using Bayesian estimation and convex optimization that employs the approach from Forster et al (2014) to estimate camera pose.”
MethodReal-Time Dense 3D Mapping of Underwater Environments
Wang, W., Joshi, B., Burgdorfer, N., Batsosc, K., Li, A. Q., Mordohaia, P., & Rekleitis, I. M. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160266 · Cited by 32
Metadata
- Authors
- Pizzoli, M., Forster, C., & Scaramuzza, D.
- Affiliations
- University of Zurich (CH)
- ORCID
- Scaramuzza, Davide
- Venue
- 2014 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2014-05
- Pages
- 2609-2616
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2093659073
- Cited by
- 330
- References
- 26
- Citations by year
- 2026: 3 · 2025: 11 · 2024: 10 · 2023: 16 · 2022: 26 · 2021: 33 · 2020: 36 · 2019: 48 · 2018: 39 · 2017: 47 · 2016: 33 · 2015: 20 · 2014: 8
- FWCI
- 19.36
- Type
- conference paper
- Language
- en
- Open access
- green
- www.zora.uzh.ch
- Licence
- other-oa
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, other-oa)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion, other-oa)
- Zurich Open Repository and Archive (University of Zurich)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Engineering › Media Technology › Image Processing Techniques and Applications
- Keywords
- monocular depth estimation, dense reconstruction, uncertainty estimation, Bayesian estimation, convex optimization, real-time depth estimation, CUDA
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.