DynaSLAM II: Tightly-Coupled Multi-Object Tracking and SLAM
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
The assumption of scene rigidity is common in visual SLAM algorithms. However, it limits their applicability in populated real-world environments. Furthermore, most scenarios including autonomous driving, multi-robot collaboration and augmented/virtual reality, require explicit motion information of the surroundings to help with decision making and scene understanding. We present in this paper DynaSLAM II, a visual SLAM system for stereo and RGB-D camera configurations that tightly integrates the multi-object tracking capability. DynaSLAM II makes use of instance semantic segmentation and ORB features to track dynamic objects. The structures of the static scene and the dynamic objects are optimized jointly with the trajectories of both the camera and the moving agents within a novel bundle adjustment proposal. The 3D bounding boxes of the objects are also estimated and loosely optimized within a fixed temporal window. We demonstrate that tracking dynamic objects does not only provide rich clues for scene understanding but can be also beneficial for camera tracking.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ORB: An efficient alternative to SIFT or SURF
Rublee, E., Rabaud, V., Konolige, K., & Bradski, G. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126544 · Cited by 11,037 · cited 1×
“For each incoming frame, pixel-wise semantic segmentation is computed and ORB features [26] are extracted and matched across stereo image pairs.”
MethodVision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913
ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras
Mur-Artal, R., & Tardos, J. D. (2017) · IEEE Transactions on Robotics · doi:10.1109/tro.2017.2705103 · Cited by 6,898 · cited 2×
“DynaSLAM II builds on the popular ORB-SLAM2 [1].”
BackgroundAlgorithms for the Assignment and Transportation Problems
Munkres, J. (1957) · Journal of the Society for Industrial and Applied Mathematics · doi:10.1137/0105003 · Cited by 3,986
Engel, J., Koltun, V., & Cremers, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2658577 · Cited by 2,924
Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics
Bernardin, K., & Stiefelhagen, R. (2008) · EURASIP Journal on Image and Video Processing · doi:10.1155/2008/246309 · Cited by 2,918 · cited 1×
“In the last decade Bernardin et al. [27] introduced the CLEAR MOT metrics to allow for objective comparison of tracker characteristics, focusing on their precision in estimating object locations, their accuracy in recognizing object configurations and their ability to consistently label [12] and…”
BackgroundYOLACT: Real-Time Instance Segmentation
Bolya, D., Zhou, C., Xiao, F., & Lee, Y. J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00925 · Cited by 2,536 · cited 1×
“Algorithms such as YOLACT [29] can run in real time and provide high-quality instance masks.”
BackgroundSVO: Fast semi-direct monocular visual odometry
Forster, C., Pizzoli, M., & Scaramuzza, D. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6906584 · Cited by 2,205
DynaSLAM: Tracking, Mapping, and Inpainting in Dynamic Scenes
Bescos, B., Facil, J. M., Civera, J., & Neira, J. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2860039 · Cited by 1,263
Self-citationDynaSLAM: Tracking, Mapping and Inpainting in Dynamic Scenes
Bescos, B., Fácil, J. M., Civera, J., & Neira, J. (2018) · Zaguan (Universidad de Zaragoza) · Cited by 893
Self-citationSimultaneous Localization, Mapping and Moving Object Tracking
Wang, C.-C., Thorpe, C., Thrun, S., Hebert, M., & Durrant-Whyte, H. (2007) · The International Journal of Robotics Research · doi:10.1177/0278364907081229 · Cited by 596 · cited 1×
“The idea of simultaneously estimating camera motion and multiple moving objects motion originated from the SLAM-MOT work [20].”
MethodCubeSLAM: Monocular 3-D Object SLAM
Yang, S., & Scherer, S. (2019) · IEEE Transactions on Robotics · doi:10.1109/tro.2019.2909168 · Cited by 454
3D Object Proposals Using Stereo Imagery for Accurate Object Class Detection
Chen, X., Kundu, K., Zhu, Y., Ma, H., Fidler, S., & Urtasun, R. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2706685 · Cited by 421
MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects
Runz, M., Buffier, M., & Agapito, L. (2018) · 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) · doi:10.1109/ismar.2018.00024 · Cited by 415
Improving RGB-D SLAM in dynamic environments: A motion removal approach
Sun, Y., Liu, M., & Meng, M. Q.-H. (2017) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2016.11.012 · Cited by 385
Xiao, L., Wang, J., Qiu, X., Rong, Z., & Zou, X. (2019) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2019.03.012 · Cited by 348
Wang, C.-C., Thorpe, C., & Thrun, S. (2004) · 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422) · doi:10.1109/robot.2003.1241698 · Cited by 316
VDO-SLAM: A Visual Dynamic Object-aware SLAM SystemPreprint
Zhang, J., Henein, M., Mahony, R. E., & Ila, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.11052 · Cited by 235
Co-fusion: Real-time segmentation, tracking and fusion of multiple objects
Runz, M., & Agapito, L. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989518 · Cited by 227
MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAM
Xu, B., Li, W., Tzoumanikas, D., Bloesch, M., Davison, A., & Leutenegger, S. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8794371 · Cited by 217
3D Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humans
Rosinol, A., Gupta, A., Abate, M., Shi, J., & Carlone, L. (2020) · Robotics: Science and Systems XVI · doi:10.15607/rss.2020.xvi.079 · Cited by 201
Stereo Vision-Based Semantic 3D Object and Ego-Motion Tracking for Autonomous Driving
Li, P., Qin, T., & Shen, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01216-8_40 · Cited by 158
Robust Dense Mapping for Large-Scale Dynamic Environments
Bârsan, I. A., Liu, P., Pollefeys, M., & Geiger, A. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8462974 · Cited by 152
ClusterVO: Clustering Moving Instances and Estimating Visual Odometry for Self and Surroundings
Huang, J., Yang, S. Y., Mu, T., & Hu, S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00224 · Cited by 133
Real-Time Monocular Object-Model Aware Sparse SLAM
Hosseinzadeh, M., Li, K., Latif, Y., & Reid, I. D. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793728 · Cited by 94
ClusterSLAM: A SLAM Backend for Simultaneous Rigid Body Clustering and Motion Estimation
Huang, J., Yang, S. Y., Zhao, Z., Lai, Y., & Hu, S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00597 · Cited by 94
Avoiding moving outliers in visual SLAM by tracking moving objects
Wangsiripitak, S., & Murray, D. W. (2009) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2009.5152290 · Cited by 78
Multimotion Visual Odometry (MVO): Simultaneous Estimation of Camera and Third-Party Motions
Judd, K. M., Gammell, J. D., & Newman, P. M. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594213 · Cited by 59
Dynamic Object Removal and Spatio-Temporal RGB-D Inpainting via Geometry-Aware Adversarial Learning
Bešić, B., & Valada, A. (2022) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3140654 · Cited by 42
Empty Cities: A Dynamic-Object-Invariant Space for Visual SLAM
Bescos, B., Cadena, C., & Neira, J. (2020) · IEEE Transactions on Robotics · doi:10.1109/tro.2020.3031267 · Cited by 41
Self-citationEmpty Cities: Image Inpainting for a Dynamic-Object-Invariant Space
Bescos, B., Neira, J., Siegwart, R., & Cadena, C. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794417 · Cited by 35
Self-citationSLAM with Expectation Maximization for moveable object tracking
Rogers, J. G., Trevor, A. J. B., Nieto-Granda, C., & Christensen, H. I. (2010) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2010.5652091 · Cited by 25
Cited by
A Comprehensive Survey of Visual SLAM Algorithms
Macario Barros, A., Michel, M., Moline, Y., Corre, G., & Carrel, F. (2022) · Robotics · doi:10.3390/robotics11010024 · Cited by 429
Autonomous driving system: A comprehensive survey
Zhao, J., Zhao, W., Deng, B., Wang, Z., Zhang, F., Zheng, W., Cao, W., Nan, J., Lian, Y., & Burke, A. F. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122836 · Cited by 361
RDS-SLAM: Real-Time Dynamic SLAM Using Semantic Segmentation Methods
Liu, Y., & Miura, J. (2021) · IEEE Access · doi:10.1109/access.2021.3050617 · Cited by 324
Kimera: From SLAM to spatial perception with 3D dynamic scene graphs
Rosinol, A., Violette, A., Abate, M., Hughes, N., Chang, Y., Shi, J., Gupta, A., & Carlone, L. (2021) · The International Journal of Robotics Research · doi:10.1177/02783649211056674 · Cited by 271
RD-VIO: Robust visual-inertial odometry for mobile augmented reality in dynamic environments
Li, J., Pan, X., Huang, G., Zhang, Z., Wang, N., Bao, H., & Zhang, G. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3353263 · Cited by 62
Metadata
- Authors
- Bescos, B., Campos, C., Tardos, J. D., & Neira, J.
- Venue
- IEEE Robotics and Automation Letters
- Published
- 2021-07
- Volume
- 6
- Issue
- 3
- Pages
- 5191-5198
- ISSN
- 2377-3766 (ISSN-L)
- 2377-3774
- OpenAlex
- W3093324477
- Cited by
- 335
- References
- 32
- FWCI
- 26.37
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.