Direct semi-dense SLAM for rolling shutter cameras
The DOI matches its Crossref record; 21 of the 22 references it cites were found in a registry or an index.
Abstract
In this paper, we present a monocular Direct and Semi-dense SLAM (Simultaneous Localization And Mapping) system for rolling shutter cameras. In a rolling shutter camera, the pose is different for each row of each image, and this yields poor pose estimates and poor structure estimates when using a state-of-the-art semi-dense direct method designed for global shutter cameras. To address this issue in tracking, we model the smooth and continuous camera trajectory using a B-spline curve of degree k??1 for poses in the Lie algebra, se(3).We solve for the camera poses at each row-time by a direct optimisation of photometric error as a function of the control points of the spline. Likewise for mapping, we develop generalised epipolar geometry for the rolling shutter case and solve for point depths using photometric error. Although each of these issues has been previously tackled, to the best of our knowledge ours is the first full solution to monocular, direct (feature-less) SLAM. We benchmark our method for pose accuracy and map accuracy against the state-of-the-art semi-dense SLAM system, LSD-SLAM, demonstrating the improved efficacy of our approach when using rolling shutter cameras via synthetic sequences with known ground-truth and real sequences.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Parallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346
A benchmark for the evaluation of RGB-D SLAM systems
Sturm, J., Engelhard, N., Endres, F., Burgard, W., & Cremers, D. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6385773 · Cited by 3,976
LSD-SLAM: Large-scale direct monocular SLAM
Engel, J., Schöps, T., & Cremers, D. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10605-2_54 · Cited by 3,851 · cited 3×
“Although each of these changes is foreshadowed in previous work, it remains a significant challenge to bring these components together to produce a full working system.”
BackgroundMethodDTAM: Dense tracking and mapping in real-time
Newcombe, R. A., Lovegrove, S. J., & Davison, A. J. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126513 · Cited by 2,078 · cited 1×
“Direct SLAM approaches, which use dense textured maps and photometric based image warping (in contrast to landmark based approaches) have been introduced [17], [4].”
BackgroundA benchmark for RGB-D visual odometry, 3D reconstruction and SLAM
Handa, A., Whelan, T., McDonald, J., & Davison, A. J. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6907054 · Cited by 906
Parallel Tracking and Mapping on a camera phone
Klein, G., & Murray, D. (2009) · 2009 8th IEEE International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2009.5336495 · Cited by 522
Semi-dense Visual Odometry for a Monocular Camera
Engel, J., Sturm, J., & Cremers, D. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.183 · Cited by 517
Sliding window filter with application to planetary landing
Sibley, G., Matthies, L., & Sukhatme, G. (2010) · Journal of Field Robotics · doi:10.1002/rob.20360 · Cited by 259
Geometric Models of Rolling-Shutter CamerasPreprint
Meingast, M., Geyer, C., & Sastry, S. (2005) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0503076 · Cited by 165
A linear approach to motion estimation using generalized camera models
Li, H., Hartley, R. I., & Kim, J.-H. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587545 · Cited by 152
Self-citationRolling Shutter Camera Calibration
Oth, L., Furgale, P., Kneip, L., & Siegwart, R. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.179 · Cited by 150
Rolling shutter bundle adjustment
Hedborg, J., Forssen, P.-E., Felsberg, M., & Ringaby, E. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247831 · Cited by 129
Lovegrove, S., Patron-Perez, A., & Sibley, G. (2013) · Procedings of the British Machine Vision Conference 2013 · doi:10.5244/c.27.93 · Cited by 106
General matrix representations for B-splines
Qin, K. (2000) · The Visual Computer · doi:10.1007/s003710050206 · Cited by 95
Dense Continuous-Time Tracking and Mapping with Rolling Shutter RGB-D Cameras
Kerl, C., Stückler, J., & Cremers, D. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.261 · Cited by 93
Continuous-time batch trajectory estimation using temporal basis functions
Furgale, P., Tong, C. H., Barfoot, T. D., & Sibley, G. (2015) · The International Journal of Robotics Research · doi:10.1177/0278364915585860 · Cited by 79
Saurer, O., Köser, K., Bouguet, J.-Y., & Pollefeys, M. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.64 · Cited by 77
Simultaneous Localisation and Mapping in Dynamic Environments (SLAMIDE) with Reversible Data Associa
Bibby, C., & Reid, I. (2007) · Robotics: Science and Systems III · doi:10.15607/rss.2007.iii.014 · Cited by 75
Self-citationMotion Estimation for Nonoverlapping Multicamera Rigs: Linear Algebraic and L∞Geometric Solutions
Kim, J.-H., Li, H., & Hartley, R. I. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.82 · Cited by 59
Self-citationA hybrid SLAM representation for dynamic marine environments
Bibby, C., & Reid, I. (2010) · 2010 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2010.5509262 · Cited by 53 · cited 1×
“More recent efforts, inspired by the work of [3] who proposed a planar (3-DOF) ego-motion trajectory representation using cubic B-splines, have considered full 6-DOF representations using splines [15], [11].”
BackgroundSelf-citationA C/sup 2/-continuous B-spline quaternion curve interpolating a given sequence of solid orientations
Kim, M.-J., Kim, M.-S., & Shin, S. Y. (2002) · Computer Animation (CA) · doi:10.1109/ca.1995.393545 · Cited by 47
Research works printed but not linked: 1
- kim, 1995, A C2-continuous B-spline quaternion curve interpolating a given sequece of solid orientations, Proceedings of Computer Animation
Cited by
Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
“If SLAM has to work “out of the box” in arbitrary scenarios, methods for automatic tuning of the involved parameters need to be considered.”
Self-citationPast, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
Self-citationVisual SLAM algorithms: a survey from 2010 to 2016
Taketomi, T., Uchiyama, H., & Ikeda, S. (2017) · IPSJ Transactions on Computer Vision and Applications · doi:10.1186/s41074-017-0027-2 · Cited by 725
Stereo DSO: Large-Scale Direct Sparse Visual Odometry with Stereo Cameras
Wang, R., Schworer, M., & Cremers, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.421 · Cited by 401
Elastic LiDAR Fusion: Dense Map-Centric Continuous-Time SLAM
Park, C., Moghadam, P., Kim, S., Elfes, A., Fookes, C., & Sridharan, S. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8462915 · Cited by 126
Metadata
- Authors
- Kim, J.-H., Cadena, C., & Reid, I.
- Affiliations
- The University of Adelaide (AU)
- Venue
- 2016 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2016-05
- Pages
- 1308-1315
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2419416064
- Cited by
- 46
- References
- 22
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 2 · 2023: 6 · 2022: 6 · 2021: 6 · 2020: 4 · 2019: 4 · 2018: 8 · 2017: 6 · 2016: 2
- FWCI
- 11.66
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
- Funders
- Technische Universität München
- Australian Centre for Robotic Vision
- Australian Research Council
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Keywords
- rolling shutter camera, monocular SLAM, photometric errors, depth estimation, camera pose estimation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.