A Tightly-coupled Semantic SLAM System with Visual, Inertial and Surround-view Sensors for Autonomous Indoor Parking
Abstract
The semantic SLAM (simultaneous localization and mapping) system is an indispensable module for autonomous indoor parking. Monocular and binocular visual cameras constitute the basic configuration to build such a system. Features used in existing SLAM systems are often dynamically movable, blurred and repetitively textured. By contrast, semantic features on the ground are more stable and consistent in the indoor parking environment. Due to their inabilities to perceive salient features on the ground, existing SLAM systems are prone to tracking loss during navigation. Therefore, a surround-view camera system capturing images from a top-down viewpoint is necessarily called for. To this end, this paper proposes a novel tightly-coupled semantic SLAM system by integrating Visual, Inertial, and Surround-view sensors, VIS SLAM for short, for autonomous indoor parking. In VIS SLAM, apart from low-level visual features and IMU (inertial measurement unit) motion data, parking-slots in surround-view images are also detected and geometrically associated, forming semantic constraints. Specifically, each parking-slot can impose a surround-view constraint that can be split into an adjacency term and a registration term. The former pre-defines the position of each individual parking-slot subject to whether it has an adjacent neighbor. The latter further constrains by registering between each observed parking-slot and its position in the world coordinate system. To validate the effectiveness and efficiency of VIS SLAM, a large-scale dataset composed of synchronous multi-sensor data collected from typical indoor parking sites is established, which is the first of its kind. The collected dataset has been made publicly available at https://cslinzhang.github.io/VISSLAM/.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Depth estimation, pose estimation, VO/VIO, and SLAM constitute the important 3D perceiving ability of AIoT and could be very useful in smart transportation, smart industry, smart agriculture, smart cities, and smart homes.
“Depth estimation, pose estimation, VO/VIO, and SLAM constitute the important 3D perceiving ability of AIoT, which could be very useful in smart transportation [137], smart industry [138], smart agriculture [139], [140], [141], smart cities and homes [142], [143], [144].”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [137]
SLAM can be used for lane departure warning, lane-keeping, and high-precision map construction and update.
“Besides, SLAM can be used for lane departure warning, lanekeeping, high-precision map construction and update [137].”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [137]
Cites
A flexible new technique for camera calibration
Zhang, Z. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.888718 · Cited by 15,004 · see citing works
A Combined Corner and Edge Detector
Harris, C., & Stephens, M. (1988) · Procedings of the Alvey Vision Conference 1988 · doi:10.5244/c.2.23 · Cited by 12,901 · see citing works
ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras
Mur-Artal, R., & Tardos, J. D. (2017) · IEEE Transactions on Robotics · doi:10.1109/tro.2017.2705103 · Cited by 6,898 · see citing works
ORB-SLAM: A Versatile and Accurate Monocular SLAM System
Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2463671 · Cited by 6,778 · see citing works
VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator
Qin, T., Li, P., & Shen, S. (2018) · IEEE Transactions on Robotics · doi:10.1109/tro.2018.2853729 · Cited by 4,669 · see citing works
Simultaneous localization and mapping: part I
Durrant-Whyte, H., & Bailey, T. (2006) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2006.1638022 · Cited by 4,199 · see citing works
Lucas-Kanade 20 Years On: A Unifying Framework
Baker, S., & Matthews, I. A. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000011205.11775.fd · Cited by 3,159 · see citing works
ORB-SLAM: A Versatile and Accurate Monocular SLAM System
Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015) · Zaguan (Universidad de Zaragoza) · Cited by 3,153 · see citing works
International Journal of Computer Vision
(2013) · International Journal of Computer Vision · doi:10.1007/11263.1573-1405 · Cited by 2,338 · see citing works
A Multi-State Constraint Kalman Filter for Vision-aided Inertial Navigation
Mourikis, A. I., & Roumeliotis, S. I. (2007) · Proceedings 2007 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2007.364024 · Cited by 1,951 · see citing works
Bags of Binary Words for Fast Place Recognition in Image Sequences
Gálvez‐López, D., & Tardós, J. D. (2012) · IEEE Transactions on Robotics · doi:10.1109/tro.2012.2197158 · Cited by 1,890 · see citing works
Faster and Better: A Machine Learning Approach to Corner Detection
Rosten, E., Porter, R. B., & Drummond, T. W. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.275 · Cited by 1,852 · see citing works
Keyframe-based visual–inertial odometry using nonlinear optimization
Leutenegger, S., Lynen, S., Bosse, M. C., Siegwart, R., & Furgale, P. (2014) · The International Journal of Robotics Research · doi:10.1177/0278364914554813 · Cited by 1,718 · see citing works
Unified temporal and spatial calibration for multi-sensor systems
Furgale, P., Rehder, J., & Siegwart, R. (2013) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2013.6696514 · Cited by 834 · see citing works
CNN-SLAM: Real-Time Dense Monocular SLAM with Learned Depth Prediction
Tateno, K., Tombari, F., Laina, I., & Navab, N. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.695 · Cited by 800 · see citing works
An introduction to inertial navigation
Woodman, O. J. (2021) · doi:10.48456/tr-696 · Cited by 797 · see citing works
Visual-Inertial Monocular SLAM With Map Reuse
Mur-Artal, R., & Tardós, J. D. (2017) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2017.2653359 · Cited by 790 · see citing works
Robust Stereo Visual Inertial Odometry for Fast Autonomous Flight
Sun, K., Mohta, K., Pfrommer, B., Watterson, M., Liu, S., Mulgaonkar, Y., Taylor, C. J., & Kumar, V. R. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2793349 · Cited by 542 · see citing works
Probabilistic data association for semantic SLAM
Bowman, S. L., Atanasov, N., Daniilidis, K., & Pappas, G. J. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989203 · Cited by 464 · see citing works
Fast relocalisation and loop closing in keyframe-based SLAM
Mur-Artal, R., & Tardós, J. D. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6906953 · Cited by 262 · see citing works
Online Temporal Calibration for Monocular Visual-Inertial Systems
Qin, T., & Shen, S. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593603 · Cited by 244 · see citing works
Real-time metric state estimation for modular vision-inertial systems
Weiß, S., & Siegwart, R. (2011) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2011.5979982 · Cited by 221 · see citing works
Towards semantic SLAM using a monocular camera
Civera, J., Galvez-Lopez, D., Riazuelo, L., Tardos, J. D., & Montiel, J. M. M. (2011) · 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2011.6094648 · Cited by 172 · see citing works
Vision-Based Parking-Slot Detection: A DCNN-Based Approach and a Large-Scale Benchmark Dataset
Zhang, L., Huang, J., Li, X., & Xiong, L. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2857407 · Cited by 138 · see citing works
Self-citationDevelopment of advanced parking assistance system
WADA, M., Yoon, K. S., & Hashimoto, H. (2003) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2002.807690 · Cited by 112 · see citing works
VINet: Visual-Inertial Odometry as a Sequence-to-Sequence Learning Problem
Clark, R., Wang, S., Wen, H., Markham, A., & Trigoni, N. (2017) · Warwick Research Archive Portal (University of Warwick) · doi:10.5555/3298023.3298149 · Cited by 102 · see citing works
PIRVS: An Advanced Visual-Inertial SLAM System with Flexible Sensor Fusion and Hardware Co-Design
Zhang, Z., Liu, S., Tsai, G., Hu, H., Chu, C.-C., & Zheng, F. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460672 · Cited by 54 · see citing works
Visual Semantic Landmark-Based Robust Mapping and Localization for Autonomous Indoor Parking
Zhao, J., Huang, Y., He, X., Zhang, S., Ye, C., Feng, T., & Xiong, L. (2019) · Sensors · doi:10.3390/s19010161 · Cited by 26 · see citing works
A Visual-Aided Inertial Navigation and Mapping System
Munguía, R., Nuño, E., Aldana, C. I., & Urzua, S. (2016) · International Journal of Advanced Robotic Systems · doi:10.5772/64011 · Cited by 13 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Zhang, J., & Tao, D. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3039359 · Cited by 774 · see citing works
- Authors
- Shao, X., Zhang, L., Zhang, T., Shen, Y., Li, H., & Zhou, Y.
- Affiliations
- Tongji University (CN)
- Tongdun AI Institute, Shanghai, China
- University of Macau (MO)
- Venue
- Proceedings of the 28th ACM International Conference on Multimedia
- Publisher
- ACM
- Published
- 2020-10-12
- Pages
- 2691-2699
- OpenAlex
- W3092848371
- Cited by
- 23
- References
- 29
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 7 · 2023: 1 · 2022: 1 · 2021: 6 · 2020: 1
- FWCI
- 3.32
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Conference on Multimedia (ACM MM) (publishedVersion)
- Funders
- National Natural Science Foundation of China (61672380, 61936014, 61973235, 61972285)
- Natural Science Foundation of Shanghai (19ZR1461300)
- Topics
- Physical Sciences › Engineering › Aerospace Engineering › Robotics and Sensor-Based Localization
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Keywords
- semantic SLAM, surround-view cameras, visual-inertial fusion, parking slot detection, semantic constraints, tightly coupled sensor fusion, tracking loss
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
