Direct semi-dense SLAM for rolling shutter cameras
The DOI matches its Crossref record; 21 of the 22 references it cites were found in a registry or an index.
Abstract
In this paper, we present a monocular Direct and Semi-dense SLAM (Simultaneous Localization And Mapping) system for rolling shutter cameras. In a rolling shutter camera, the pose is different for each row of each image, and this yields poor pose estimates and poor structure estimates when using a state-of-the-art semi-dense direct method designed for global shutter cameras. To address this issue in tracking, we model the smooth and continuous camera trajectory using a B-spline curve of degree k??1 for poses in the Lie algebra, se(3).We solve for the camera poses at each row-time by a direct optimisation of photometric error as a function of the control points of the spline. Likewise for mapping, we develop generalised epipolar geometry for the rolling shutter case and solve for point depths using photometric error. Although each of these issues has been previously tackled, to the best of our knowledge ours is the first full solution to monocular, direct (feature-less) SLAM. We benchmark our method for pose accuracy and map accuracy against the state-of-the-art semi-dense SLAM system, LSD-SLAM, demonstrating the improved efficacy of our approach when using rolling shutter cameras via synthetic sequences with known ground-truth and real sequences.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Parallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346
A benchmark for the evaluation of RGB-D SLAM systems
Sturm, J., Engelhard, N., Endres, F., Burgard, W., & Cremers, D. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6385773 · Cited by 3,976
LSD-SLAM: Large-scale direct monocular SLAM
Engel, J., Schöps, T., & Cremers, D. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10605-2_54 · Cited by 3,851 · cited 3×
“Although each of these changes is foreshadowed in previous work, it remains a significant challenge to bring these components together to produce a full working system.”
BackgroundMethodDTAM: Dense tracking and mapping in real-time
Newcombe, R. A., Lovegrove, S. J., & Davison, A. J. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126513 · Cited by 2,078 · cited 1×
“Direct SLAM approaches, which use dense textured maps and photometric based image warping (in contrast to landmark based approaches) have been introduced [17], [4].”
BackgroundA benchmark for RGB-D visual odometry, 3D reconstruction and SLAM
Handa, A., Whelan, T., McDonald, J., & Davison, A. J. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6907054 · Cited by 906
Parallel Tracking and Mapping on a camera phone
Klein, G., & Murray, D. (2009) · 2009 8th IEEE International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2009.5336495 · Cited by 522
Semi-dense Visual Odometry for a Monocular Camera
Engel, J., Sturm, J., & Cremers, D. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.183 · Cited by 517
Sliding window filter with application to planetary landing
Sibley, G., Matthies, L., & Sukhatme, G. (2010) · Journal of Field Robotics · doi:10.1002/rob.20360 · Cited by 259
Geometric Models of Rolling-Shutter CamerasPreprint
Meingast, M., Geyer, C., & Sastry, S. (2005) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0503076 · Cited by 165
A linear approach to motion estimation using generalized camera models
Li, H., Hartley, R. I., & Kim, J.-H. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587545 · Cited by 152
Self-citation
Research works printed but not linked: 1
- kim, 1995, A C2-continuous B-spline quaternion curve interpolating a given sequece of solid orientations, Proceedings of Computer Animation
Cited by
Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
“If SLAM has to work “out of the box” in arbitrary scenarios, methods for automatic tuning of the involved parameters need to be considered.”
Self-citationPast, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
Self-citationVisual SLAM algorithms: a survey from 2010 to 2016
Taketomi, T., Uchiyama, H., & Ikeda, S. (2017) · IPSJ Transactions on Computer Vision and Applications · doi:10.1186/s41074-017-0027-2 · Cited by 725
Stereo DSO: Large-Scale Direct Sparse Visual Odometry with Stereo Cameras
Wang, R., Schworer, M., & Cremers, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.421 · Cited by 401
Elastic LiDAR Fusion: Dense Map-Centric Continuous-Time SLAM
Park, C., Moghadam, P., Kim, S., Elfes, A., Fookes, C., & Sridharan, S. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8462915 · Cited by 126
Survey and Evaluation of RGB-D SLAM
Zhang, S., Zheng, L., & Tao, W. (2021) · IEEE Access · doi:10.1109/access.2021.3053188 · Cited by 91
Elasticity Meets Continuous-Time: Map-Centric Dense 3D LiDAR SLAM
Park, C., Moghadam, P., Williams, J., Kim, S., Sridharan, S., & Fookes, C. (2021) · IEEE Transactions on Robotics · doi:10.1109/tro.2021.3096650 · Cited by 76
Research on Multi-Sensor Fusion SLAM Algorithm Based on Improved Gmapping
Tian, C., Liu, H., Liu, Z., Li, H., & Wang, Y. (2023) · IEEE Access · doi:10.1109/access.2023.3243633 · Cited by 65
Direct Sparse Odometry with Rolling Shutter
Schubert, D., Demmel, N., Usenko, V., Stückler, J., & Cremers, D. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_42 · Cited by 50
Continuous-Time State Estimation Methods in Robotics: A Survey
Talbot, W., Nubert, J., Tuna, T., Cadena, C., Dümbgen, F., Tordesillas, J., Barfoot, T. D., & Hutter, M. (2025) · IEEE Transactions on Robotics · doi:10.1109/tro.2025.3593079 · Cited by 59
Self-citationRolling-Shutter Modelling for Direct Visual-Inertial Odometry
Schubert, D., Demmel, N., von Stumberg, L., Usenko, V., & Cremers, D. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968539 · Cited by 46
Trajectory representation and landmark projection for continuous-time structure from motion
Ovrén, H., & Forssén, P.-E. (2019) · The International Journal of Robotics Research · doi:10.1177/0278364919839765 · Cited by 38
Yang, A., Cui, C., Bârsan, I. A., Urtasun, R., & Wang, S. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561481 · Cited by 37
Ctrl-VIO: Continuous-Time Visual-Inertial Odometry for Rolling Shutter Cameras
Lang, X., Lv, J., Huang, J., Ma, Y., Liu, Y., & Zuo, X. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3202349 · Cited by 38
Direct visual-inertial navigation with analytical preintegration
Eckenhoff, K., Geneva, P., & Huang, G. P. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989171 · Cited by 29
Rolling Shutter Camera: Modeling, Optimization and Learning
Fan, B., Dai, Y., & He, M. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1399-z · Cited by 22
Robustified Structure from Motion with rolling-shutter camera using straightness constraint
Lao, Y., Ait-Aider, O., & Araújo, H. J. (2018) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.04.004 · Cited by 18
Revisiting Rolling Shutter Bundle Adjustment: Toward Accurate and Fast Solution
Liao, B., Qu, D., Xue, Y., Zhang, H., & Lao, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00471 · Cited by 16
Solving Rolling Shutter 3D Vision Problems using Analogies with Non-rigidity
Lao, Y., Ait-Aider, O., & Bartoli, A. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01368-1 · Cited by 16
PairCon-SLAM: Distributed, Online, and Real-Time RGBD-SLAM in Large Scenarios
Zhu, D., Xu, G., Wang, X., Xiao-gang, L., & Tian, D. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3116288 · Cited by 20
DyStSLAM: an efficient stereo vision SLAM system in dynamic environment
Li, X., Shen, Y., Lu, J., Jiang, Q., Xie, O., Yang, Y., & Zhu, Q. (2022) · Measurement Science and Technology · doi:10.1088/1361-6501/ac97b1 · Cited by 17
Fast Rolling Shutter Correction in the Wild
Qu, D., Liao, B., Zhang, H., Ait-Aider, O., & Lao, Y. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3284847 · Cited by 15
Rolling Shutter Pose and Ego-Motion Estimation Using Shape-from-Template
Lao, Y., Ait-Aider, O., & Bartoli, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01216-8_29 · Cited by 15
Object Distance Estimation Using a Single Image Taken from a Moving Rolling Shutter Camera
Kim, N., Bae, J., Kim, C., Park, S., & Sohn, H. (2020) · Sensors · doi:10.3390/s20143860 · Cited by 15
Rolling-Shutter-stereo-aware motion estimation and image correction
Fan, B., Dai, Y., & Wang, K. (2021) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2021.103296 · Cited by 14
Multical: Spatiotemporal Calibration for Multiple IMUs, Cameras and LiDARs
Zhi, X., Hou, J., Lu, Y., Kneip, L., & Schwertfeger, S. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9982031 · Cited by 19
Towards Nonlinear-Motion-Aware and Occlusion-Robust Rolling Shutter Correction
Qu, D., Lao, Y., Wang, Z., Wang, D., Zhao, B. H., & Li, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00980 · Cited by 11
Rolling Shutter and Radial Distortion are Features for High Frame Rate Multi-camera Tracking
Bapat, A., Price, T., & Frahm, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00507 · Cited by 11
Differential SfM and image correction for a rolling shutter stereo rig
Fan, B., Dai, Y., Zhang, Z., & Wang, K. (2022) · Image and Vision Computing · doi:10.1016/j.imavis.2022.104492 · Cited by 11
Scanline Homographies for Rolling-Shutter Plane Absolute Pose
Bai, F., Sengupta, A., & Bartoli, A. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00879 · Cited by 9
RRD-SLAM: Radial-distorted rolling-shutter direct SLAM
Kim, J.-H., Latif, Y., & Reid, I. D. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989602 · Cited by 8
Self-citationRolling Shutter Correction with Intermediate Distortion Flow Estimation
Cao, M., Yang, S., Yang, Y., & Zheng, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02394 · Cited by 10
A direct formulation for camera calibration
Rehder, J., Nikolic, J., Schneider, T., & Siegwart, R. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989765 · Cited by 7
Fisher, M. H., French, G., Gorpincenko, A., Holah, H., Clayton, L., Skirrow, R., & Mackiewicz, M. (2022) · IET Image Processing · doi:10.1049/ipr2.12636 · Cited by 6
Gyroscope-aided Relative Pose Estimation for Rolling Shutter CamerasPreprint
Lee, C., Yoon, J. H., Park, M.-G., & Yoon, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.06770 · Cited by 7
Motion transformation solutions based on Euler angle perturbation model
Nie, B., Cai, Z., Zhao, J. B., & Wang, Y. (2024) · Measurement · doi:10.1016/j.measurement.2024.115631 · Cited by 3
Point feature correction based rolling shutter modeling for EKF-based visual-inertial odometry
Zhang, K., & Zhang, M. (2023) · Measurement Science and Technology · doi:10.1088/1361-6501/ad044e · Cited by 3
Inertial-aided Rolling Shutter Relative Pose EstimationPreprint
Lee, C., & Yoon, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.00184 · Cited by 2
Combining SLAM with muti-spectral photometric stereo for real-time dense 3D reconstructionPreprint
Xu, Y., Dong, P., Dong, J., & Qi, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.02294 · Cited by 2
Basics and Advances in Monocular vSLAM
Uchiyama, H., Taketomi, T., Ikeda, S., & Mori, S. (2020) · Springer eBooks · doi:10.1007/978-3-030-42234-9_5 · Cited by 1
Xu, Y., Dong, P., Lu, L., Dong, J., & Qi, L. (2018) · International Conference on Communication and Information Processing (ICCIP) · doi:10.1145/3290420.3290458
Rolling-Shutter Correction for Monocular Vision-Based Obstacle Mapping
Dunbabin, O., & Johnson, E. N. (2021) · AIAA SciTech Forum · doi:10.2514/6.2021-2014
Gohard, P.-A. (2018) · theses.fr (ABES) · doi:10.70675/d90fc3e3zd301z43d6zac80z5a16f803431f · Cited by 1
3D Vision Geometry for Rolling Shutter Cameras
Lao, Y. (2019) · theses.fr (ABES) · doi:10.70675/9c6f6b63z06f6z4e91z9a8bz9509605f1ef0
Billy, A. (2020) · theses.fr (ABES) · doi:10.70675/7303fcc2z7adez4463z8f16z9281bfcbc850
Rolling Shutter Camera Self-calibration
Zhang, Y., Rabbani, N., Liao, B., Wang, C., Lao, Y., & Bartoli, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37550-6_6
Rolling-Shutter Aware LiDAR-Inertial-Visual Odometry
Zhu, H., Hong, X., Zheng, C., Li, H., & Zhang, F. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3747596
Metadata
- Authors
- Kim, J.-H., Cadena, C., & Reid, I.
- Affiliations
- The University of Adelaide (AU)
- Venue
- 2016 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2016-05
- Pages
- 1308-1315
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2419416064
- Cited by
- 46
- References
- 22
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 2 · 2023: 6 · 2022: 6 · 2021: 6 · 2020: 4 · 2019: 4 · 2018: 8 · 2017: 6 · 2016: 2
- FWCI
- 11.66
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
- Funders
- Technische Universität München
- Australian Centre for Robotic Vision
- Australian Research Council
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Keywords
- rolling shutter camera, monocular SLAM, photometric errors, depth estimation, camera pose estimation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.