Towards semantic SLAM using a monocular camera
The DOI matches its Crossref record; 32 of the 34 references it cites were found in a registry or an index.
Abstract
Monocular SLAM systems have been mainly focused on producing geometric maps just composed of points or edges; but without any associated meaning or semantic content. In this paper, we propose a semantic SLAM algorithm that merges in the estimated map traditional meaningless points with known objects. The non-annotated map is built using only the information extracted from a monocular image sequence. The known object models are automatically computed from a sparse set of images gathered by cameras that may be different from the SLAM camera. The models include both visual appearance and tridimensional information. The semantic or annotated part of the map -the objects- are estimated using the information in the image sequence and the precomputed object models. The proposed algorithm runs an EKF monocular SLAM parallel to an object recognition thread. This latest one informs of the presence of an object in the sequence by searching for SURF correspondences and checking afterwards their geometric compatibility. When an object is recognized it is inserted in the SLAM map, being its position measured and hence refined by the SLAM algorithm in subsequent frames. Experimental results show real-time performance for a hand held camera imaging a desktop environment and for a camera mounted in a robot moving in a room-sized scenario.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“First attempts were done in small scale by Castle et al. [44] and by Civera et al. [50] with a monocular SLAM with sparse features”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [50]
Citations
Cites
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070
SURF: Speeded Up Robust Features
Bay, H., Tuytelaars, T., & Van Gool, L. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744023_32 · Cited by 14,955
Speeded-Up Robust Features (SURF)
Bay, H., Ess, A., Tuytelaars, T., & Van Gool, L. (2008) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2007.09.014 · Cited by 13,557
Simultaneous localization and mapping: part I
Durrant-Whyte, H., & Bailey, T. (2006) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2006.1638022 · Cited by 4,199
Accurate, Dense, and Robust Multiview Stereopsis
Furukawa, Y., & Ponce, J. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.161 · Cited by 3,019 · cited 1×
“Multi- View Stereo algorithms (specifically the software package PMVS [11]) are used for this purpose.”
MethodFAST APPROXIMATE NEAREST NEIGHBORS WITH AUTOMATIC ALGORITHM CONFIGURATION
(2009) · Proceedings of the Fourth International Conference on Computer Vision Theory and Applications · doi:10.5220/0001787803310340 · Cited by 2,640 · cited 1×
“For each object O in the database, correspondences are calculated between I k−m and its faces F by applying the distance ratio (NNDR) to fast approximate nearest neighbors [19].”
MethodSimultaneous localization and mapping (SLAM): part II
Bailey, T., & Durrant-Whyte, H. (2006) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2006.1678144 · Cited by 2,554
Modeling the World from Internet Photo Collections
Snavely, N., Seitz, S. M., & Szeliski, R. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0107-3 · Cited by 2,155 · cited 2×
“A modern software library called Bundler [24] has been used in this paper for constructing this geometric part of the model.”
MethodFAB-MAP: Probabilistic Localization and Mapping in the Space of Appearance
Cummins, M., & Newman, P. (2008) · The International Journal of Robotics Research · doi:10.1177/0278364908090961 · Cited by 1,472 · cited 1×
“Visual recognition has been used in robotic SLAM mostly for scene recognition, for example loop closing in [7], [2], relocalization in [28] and place classification in a semantic context in [22].”
MethodInverse Depth Parametrization for Monocular SLAM
Civera, J., Davison, A. J., & Montiel, J. (2008) · IEEE Transactions on Robotics · doi:10.1109/tro.2008.2003276 · Cited by 779 · cited 1×
“Point features y i are initialized using inverse depth parametrization and are converted to Euclidean coordinates y i = (X Y Z) when projection equation shows a high degree of linearity [5] .”
MethodSelf-citation
Cited by
Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
“[51] with a monocular SLAM with sparse features, and by Dame et al.”
Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
Simultaneous Localization and Mapping: A Survey of Current Trends in Autonomous Driving
Bresson, G., Alsayed, Z., Yu, L., & Glaser, S. (2017) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2017.2749181 · Cited by 885
“In [278], the authors integrate well known objects inside a monocular SLAM.”
Geometry-Aware Learning of Maps for Camera Localization
Brahmbhatt, S., Gu, J., Kim, K., Hays, J., & Kautz, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00277 · Cited by 530
Probabilistic data association for semantic SLAM
Bowman, S. L., Atanasov, N., Daniilidis, K., & Pappas, G. J. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989203 · Cited by 462
“Many other approaches [14]–[19] extract both metric and semantic information but typically the two processes are carried out separately and the results are merged afterwards.”
BackgroundMaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects
Runz, M., Buffier, M., & Agapito, L. (2018) · 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) · doi:10.1109/ismar.2018.00024 · Cited by 415
Semantic mapping for mobile robotics tasks: A survey
Kostavelis, I., & Gasteratos, A. (2015) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2014.12.006 · Cited by 376
“Additionally, Civera et al. [45] employed an object recognition strategy which informs for the presence of an object in the sequence by searching for SURF correspondences and checking their geometric compatibility .”
BackgroundOpen-vocabulary Queryable Scene Representations for Real World Planning
Chen, B., Xia, F., Ichter, B., Rao, K., Gopalakrishnan, K., Ryoo, M. S., Stone, A., & Kappler, D. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10161534 · Cited by 143
“Semantic SLAM [9]–[11] is an augmentation over traditional SLAM, it assigns semantic features over geometric features provided by SLAM (points, lines, planes).”
BackgroundC2TAM: A Cloud framework for cooperative tracking and mapping
Riazuelo, L., Civera, J., & Montiel, J. M. M. (2014) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2013.11.007 · Cited by 254
“Additionally, other background processes at different time scales can handlemapmanagement operations aiming at life-longmapping [5], semanticmapping [6,7], layout estimation [8] or computing free space for navigation [9]; • the centralized map building also allows a straightforward massive data storage of robotic sequ…”
BackgroundSelf-citationActive Mapping and Robot Exploration: A Survey
Lluvia, I., Lazkano, E., & Ansuategi, A. (2021) · Sensors · doi:10.3390/s21072445 · Cited by 202
Perception and Navigation in Autonomous Systems in the Era of Learning: A Survey
Tang, Y. S., Zhao, C., Wang, J., Zhang, C., Sun, Q., Zheng, W. X., Du, W., Qian, F., & Kurths, J. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3167688 · Cited by 212
“Salas-Moreno et al. [210] extended this work in [209].”
BackgroundMethodStereo Vision-Based Semantic 3D Object and Ego-Motion Tracking for Autonomous Driving
Li, P., Qin, T., & Shen, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01216-8_40 · Cited by 158
Choudhary, S., Carlone, L., Nieto-Granda, C., Rogers, J. G., Christensen, H. I., & Dellaert, F. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364917732640 · Cited by 163
“Civera et al. (2011) propose a semantic SLAM algorithm that annotates the low-level 3D point based maps with precomputed object models.”
MethodLianos, K.-N., Schönberger, J. L., Pollefeys, M., & Sattler, T. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01225-0_15 · Cited by 157
“While some approaches rely on a database of specific objects that are detected online [7,15,43], others use generic object detectors [3,5,16].”
BackgroundMethodReal-time monocular object SLAM
Gálvez‐López, D., Salas, M., Tardós, J. D., & Montiel, J. M. M. (2015) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2015.08.009 · Cited by 153
Self-citationCombining Optimal Control and Learning for Visual Navigation in Novel EnvironmentsPreprint
Bansal, S., Tolani, V., Gupta, S., Malik, J., & Tomlin, C. J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.02531 · Cited by 152
“To overcome these limitations, task planning approaches have been proposed that leverage object and region specific semantics during planning [4, 7, 9, 13, 33]; however, often hand-tuned heuristics are used to leverage these semantics in planning.”
A Review of SLAM Techniques and Security in Autonomous Driving
Singandhupe, A., & La, H. M. (2019) · 2019 Third IEEE International Conference on Robotic Computing (IRC) · doi:10.1109/irc.2019.00122 · Cited by 147
“The problem of consistently fusing it, still persists [7], [8].”
Recent developments and applications of simultaneous localization and mapping in agriculture
Ding, H., Zhang, B., Zhou, J. C., Yan, Y., Tian, G., & Gu, B. (2022) · Journal of Field Robotics · doi:10.1002/rob.22077 · Cited by 146
“With the rapid development of pattern recognition, the addition of semantic content to a geometric map (Civera et al., 2011; Tateno et al., 2017) has been proposed by some researchers.”
BackgroundFrom Machine Learning to Robotics: Challenges and Opportunities for Embodied IntelligencePreprint
Roy, N., Posner, I., Barfoot, T., Beaudoin, P., Bengio, Y., Bohg, J., Brock, O., Depatie, I., Fox, D., Koditschek, D. E., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.15245 · Cited by 150
“Only recently have results begun to emerge that use so-called “semantic information” for navigation (Civera et al., 2011; Atanasov et al., 2014; Kostavelis et al., 2016) supported without a dense geometric model.”
Visual SLAM Integration With Semantic Segmentation and Deep Learning: A Review
Pu, H., Luo, J., Wang, G., Huang, T., Liu, H., & Luo, J. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3306371 · Cited by 132
RoboEarth Semantic Mapping: A Cloud Enabled Knowledge-Based Approach
Riazuelo, L., Tenorth, M., Marco, D. D., Salas, M., Gálvez‐López, D., Mösenlechner, L., Kunze, L., Beetz, M., Tardós, J. D., Montano, L., et al. (2015) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2014.2377791 · Cited by 129
“Combining visual SLAM with object recognition to produce maps of objects has recently gained more attention for pure visual RGB sensors in [11] and [12], and for RGBD in [13].”
BackgroundSelf-citationA review of visual SLAM for robotics: evolution, properties, and future applications
Al-Tawil, B., Hempel, T., Abdelrahman, A. A., & Al-Hamadi, A. K. (2024) · Frontiers in Robotics and AI · doi:10.3389/frobt.2024.1347985 · Cited by 126
Monocular SLAM Supported Object Recognition
Pillai, S., & Leonard, J. (2015) · Robotics: Science and Systems XI · doi:10.15607/rss.2015.xi.034 · Cited by 118
“Other works [4, 7, 10, 20, 30] have also investigated object-based SLAM, SLAM-aware, and 3D object recognition architectures, however they have a few of glaring concerns: either (i) the system cannot scale beyond a finite set of object instances (generally limited to less than 10), or (ii) they…”
BackgroundAutomatic dense visual semantic mapping from street-level imagery
Sengupta, S., Sturgess, P., Ladicky, L., & Torr, P. H. S. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6385958 · Cited by 90
“tasks such as determining drivable regions for autonomous vehicles [7], navigating structured environments [25], and interacting with objects [5].”
BackgroundAdvances in Inference and Representation for Simultaneous Localization and Mapping
Rosen, D. M., Doherty, K., Espinoza, A. T., & Leonard, J. J. (2021) · Annual Review of Control Robotics and Autonomous Systems · doi:10.1146/annurev-control-072720-082553 · Cited by 106
“Since the early work on these representations (104, 105, 106), the community has largely shifted away from a priori known object models toward the use of learned perception models for object detection, recognition, and pose estimation.”
BackgroundEdge Assisted Mobile Semantic Visual SLAM
Xu, J., Cao, H., Li, D., Huang, K., Qian, C., Shangguan, L., & Yang, Z. (2020) · IEEE INFOCOM 2020 - IEEE Conference on Computer Communications · doi:10.1109/infocom41043.2020.9155438 · Cited by 101
“Latest researches [42], [43] attempted to incorporate semantics into visual SLAM for better robustness in time-varying environments.”
BackgroundSimultaneous Localization And Mapping: Present, Future, and the Robust-Perception Age.Preprint
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · arXiv (Cornell University) · Cited by 90
SLAM with objects using a nonparametric pose graph
Mu, B., Liu, S.-Y., Paull, L., Leonard, J. J., & How, J. P. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759677 · Cited by 89
“These data associations by front-end procedures are taken as reliable and true, and then passed to a SLAM solver [17, 18].”
MethodRobust Object-based SLAM for High-speed Autonomous Navigation
Ok, K., Liu, K., Frey, K. M., How, J. P., & Roy, N. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794344 · Cited by 85
“Utilizing such inexpensive object detectors, some previous work [13]–[15] detects and explicitly models an object of interest as a single entity, relaxing the constraint”
MethodDense real-time mapping of object-class semantics from RGB-D video
Stückler, J., Waldvogel, B., Schulz, H., & Behnke, S. (2013) · Journal of Real-Time Image Processing · doi:10.1007/s11554-013-0379-5 · Cited by 85
“Keywords Semantic maps Simultaneous localization and semantic mapping Object-class segmentation Random decision forests”
BackgroundLi, G., Yu, L., & Fei, S. (2020) · Measurement · doi:10.1016/j.measurement.2020.108403 · Cited by 83
Localization from semantic observations via the matrix permanent
Atanasov, N., Zhu, M., Daniilidis, K., & Pappas, G. J. (2015) · The International Journal of Robotics Research · doi:10.1177/0278364915596589 · Cited by 80
“Anati et al. (2012) match histogram-of-gradient-energies and quantized-colors features to the expected features from a prior semantic map.”
BackgroundGarg, S., Sünderhauf, N., & Milford, M. J. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8461051 · Cited by 80
Using superpixels in monocular SLAM
Concha, A., & Civera, J. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6906883 · Cited by 78
“In the visual SLAM domain, [15] is able to recognize and register a set of known objects in a monocular SLAM map through the local features in them.”
BackgroundSelf-citationSalientDSO: Bringing Attention to Direct Sparse Odometry
Liang, H.-J., Sanket, N. J., Fermuller, C. M., & Aloimonos, Y. (2019) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2019.2900980 · Cited by 74
“At the same time, very recently, this philosophy of fixation and attention has started to gain popularity in the robot navigation community [11], [19]–[21].”
BackgroundSensors, SLAM and Long-term Autonomy: A Review
Zaffar, M., Ehsan, S., Stolkin, R., & Maier, K. M. (2018) · NASA/ESA Conference on Adaptive Hardware and Systems (AHS) · doi:10.1109/ahs.2018.8541483 · Cited by 72
“A semantic SLAM system for a monocular camera is proposed in [35].”
Semantic Localization Via the Matrix Permanent
Atanasov, N., Zhu, M., Daniilidis, K., & Pappas, G. J. (2014) · Robotics: Science and Systems · doi:10.15607/rss.2014.x.043 · Cited by 70
“Maps can also be constructed via the semantic mapping approaches which received significant attention in recent years [18, 12, 32, 29, 7].”
BackgroundRecovering Stable Scale in Monocular SLAM Using Object-Supplemented Bundle Adjustment
Frost, D., Prisacariu, V. A., & Murray, D. W. (2018) · IEEE Transactions on Robotics · doi:10.1109/tro.2018.2820722 · Cited by 70
“Civera et al. [33] also used an EKF and augmented their map with full three-dimensional (3-”
MethodObject-aware bundle adjustment for correcting monocular scale drift
Frost, D., Kähler, O., & Murray, D. W. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487680 · Cited by 67
“Civera et al. [29] also used an EKF, and augmented their map with full three-dimensional objects; but they too considered particular objects rather than object classes.”
MethodReal-Time HD Map Change Detection for Crowdsourcing Update Based on Mid-to-High-End Sensors
Zhang, P., Zhang, M., & Liu, J. (2021) · Sensors · doi:10.3390/s21072477 · Cited by 65
“Vision SLAM can produce geometric maps composed of points or edges but without any associated meaning or semantic content [18].”
BackgroundTextSLAM: Visual SLAM with Planar Text Features
Li, B., Zou, D., Sartori, D., Pei, L., & Yu, W. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197233 · Cited by 64
“Therefore texts are naturally good planar features that can be easily detected by text detectors. b) Object features: Integrating object-level features into visual SLAM systems has been receiving increasing interest in recent years [24][25][26].”
BackgroundFeature-based visual simultaneous localization and mapping: a survey
Azzam, R., Taha, T., Huang, S., & Zweiri, Y. (2020) · SN Applied Sciences · doi:10.1007/s42452-020-2001-3 · Cited by 63
“Furthermore, inverse depth parameterization was used in [26] to recover the scale of the scene.”
BackgroundMethodSemantic Visual Simultaneous Localization and Mapping: A Survey
Chen, K., Xiao, J., Liu, J., Tong, Q., Zhang, H., Liu, R., Zhang, J., Ajoudani, A., & Chen, S. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3556928 · Cited by 61
“Civera et al. [33] and Salas-Moreno et al. [34] pioneer approaches that involve constructing a library of object models, against which extracted image features are matched to identify semantic objects.”
DF-SLAM: A Deep-Learning Enhanced Visual SLAM System based on Deep Local FeaturesPreprint
Kang, R., Shi, J., Li, X., Liu, Y., & Liu, X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.07223 · Cited by 60
“Early studies operate semantic and geometric modules separately and merge the results afterward[8, 34].”
Joint Detection, Tracking and Mapping by Semantic Bundle Adjustment
Fioraio, N., & Stefano, L. D. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.202 · Cited by 58
“Recently, the advent of the Kinect, a 30Hz, low-cost, low-range RGB-D sensor, has fostered the SLAM community towards novel approaches that would optimally exploit the device.”
BackgroundMethodBarnes, D., Maddern, W., Pascoe, G., & Posner, I. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460564 · Cited by 57
Semantic mapping using object-class segmentation of RGB-D images
Stückler, J., Biresev, N., & Behnke, S. (2012) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2012.6385983 · Cited by 55
“While most approaches utilize SLAM as a front-end to obtain a sensor trajectory estimate [5], [6], [7], [8], [9], [10], some methods also incorporate the spatial relation of objects into SLAM.”
BackgroundMethodQuadruped Robots: Bridging Mechanical Design, Control, and Applications
Li, Q., Cicirelli, F., Vinci, A., Guerrieri, A., Qi, W., & Fortino, G. (2025) · Robotics · doi:10.3390/robotics14050057 · Cited by 53
A Right Invariant Extended Kalman Filter for Object Based SLAM
Song, Y., Zhang, Z., Wu, J., Wang, Y., Zhao, L., & Huang, S. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3139370 · Cited by 51
“However, most of these works only utilize lowlevel features such as points [1], lines [4] or planes [5] and neglect the high-level features such as objects which contain strong geometric constraints [6].”
Pair-Navi: Peer-to-Peer Indoor Navigation with Mobile Visual SLAM
Dong, E., Xu, J., Wu, C., Liu, Y., & Yang, Z. (2019) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2019.8737640 · Cited by 50
“Latest researches [20], [21] attempted to incorporate semantics into visual SLAM for better robustness in time-varying environments.”
BackgroundCorrecting Scale Drift by Object Recognition in Single-Camera SLAM
Botterill, T., Mills, S. J., & Green, R. (2012) · IEEE Transactions on Cybernetics · doi:10.1109/tsmcb.2012.2230164 · Cited by 50
Robot localization using soft object detection
Anati, R., Scaramuzza, D., Derpanis, K. G., & Daniilidis, K. (2012) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2012.6225216 · Cited by 47
“[13] apply appearance and geometry based recognition to annotate feature maps established with monocular SLAM.”
MethodSemantic segmentation–aided visual odometry for urban autonomous driving
An, L., Zhang, X., Gao, H., & Liu, Y. (2017) · International Journal of Advanced Robotic Systems · doi:10.1177/1729881417735667 · Cited by 44
“proposed a semantic SLAM using a monocular extended kalman filter (EKF) SLAM and inserted 3-D objects into geometric map.(35) Anand et al.”
MethodDynamic objects elimination in SLAM based on image fusion
Fan, Y., Han, H. W., Tang, Y., & Zhi, T. (2018) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.10.024 · Cited by 45
Visual Vocabulary with a Semantic Twist
Arandjelović, R., & Zisserman, A. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16865-4_12 · Cited by 44
“in stereo correspondence [25], 3D reconstruction [26] and Semantic SLAM [27, 28].”
Edge Assisted Mobile Semantic Visual SLAM
Cao, H., Xu, J., Li, D., Shangguan, L., Liu, Y., & Yang, Z. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3201000 · Cited by 43
“Recent studies [33], [34], [35] have attempted to incorporate semantics into visual SLAM to improve robustness in time-varying environments, such as moving objects and pedestrians.”
BackgroundMonocular Vehicle Self-localization method based on Compact Semantic Map
Xiao, Z., Jiang, K., Xie, S., Wen, T., Yu, C., & Yang, D. (2018) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2018.8569274 · Cited by 43
Semantic loop closure detection based on graph matching in multi-objects scenes
Qin, C., Zhang, Y., Liu, Y., & Lv, G. (2021) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2021.103072 · Cited by 42
Open-Vocabulary Online Semantic Mapping for SLAM
Martins, T. B., Oswald, M. R., & Civera, J. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3617736 · Cited by 41
“Traditionally, semantic 3D reconstruction has relied on a closed-set approach in both offline [1, 2] and online [3, 4] settings, including integrations into semantic Simultaneous Localization and Mapping (SLAM) systems [5–7].”
Self-citationInferring robot goals from violations of semantic knowledge
Galindo, C., & Saffiotti, A. (2013) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2012.12.007 · Cited by 40
Semantic SLAM With More Accurate Point Cloud Map in Dynamic Environments
Fan, Y., Zhang, Q., Liu, S., Tang, Y., Jing, X., Yao, J., & Han, H. W. (2020) · IEEE Access · doi:10.1109/access.2020.3003160 · Cited by 40
“With the development of deep learning, great progresses have been made in target recognition and segmentation, so some researchers have combined the semantic information of the objects in the environment with SLAM [11]–[15].”
MethodHu, Y.-T., Wang, J., Yeh, R. A., & Schwing, A. G. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00147 · Cited by 39
“Closer to the goal of our approach, SLAM-based methods have been proposed for object-level multi-view reconstruction [13, 26, 64, 70].”
MethodSemantic Mapping with Simultaneous Object Detection and Localization
Zeng, Z., Zhou, Y., Jenkins, O. C., & Desingh, K. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594205 · Cited by 39
Semantic Mapping in Indoor Embodied AI -- A Survey on Advances, Challenges, and Future Directions
Raychaudhuri, S., & Chang, A. X. (2025) · Transactions on machine learning research · doi:10.48550/arxiv.2501.05750 · Cited by 41
Semantic visual SLAM in populated environments
Riazuelo, L., Montano, L., & Montiel, J. M. M. (2017) · European Conference on Mobile Robots · doi:10.1109/ecmr.2017.8098697 · Cited by 38
“[7] and [8] merged a monocular SLAM system and object recognition for enriching the map.”
BackgroundSelf-citationAlamdari, A. G., Zade, F. A., & Ebrahimkhanlou, A. (2025) · Sensors · doi:10.3390/s25030712 · Cited by 37
EgoMap: Projective Mapping and Structured Egocentric Memory for Deep RL
Beeching, E., Dibangoye, J., Simonin, O., & Wolf, C. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-67661-2_31 · Cited by 37
OrcVIO: Object residual constrained Visual-Inertial Odometry
Shan, M., Feng, Q., & Atanasov, N. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341660 · Cited by 36
“In Table III, we compare the Precision and Recall of OrcVIO on the KITTI raw sequences (2011 09 26 00XX, XX = [01, 19, 22, 23, 35, 36, 39, 61, 64, 93]) against a single-view, end-to-end object estimation approach (SubCNN [91]), and a visual-inertial object detector (VIS-FNL [52]).”
Shao, X., Zhang, L., Zhang, T., Shen, Y., & Zhou, Y. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3135041 · Cited by 36
Fast 3D modeling in complex environments using a single Kinect sensor
Yue, H., Chen, W., Wu, X., & Liu, J. (2013) · Optics and Lasers in Engineering · doi:10.1016/j.optlaseng.2013.08.009 · Cited by 35
Semantic parsing for priming object detection in indoors RGB-D scenes
Cadena, C., & Košecká, J. (2014) · The International Journal of Robotics Research · doi:10.1177/0278364914549488 · Cited by 34
“An online SLAM system recognizing objects is proposed by [7], where in parallel to a monocular EKF SLAM thread there is a thread comparing visual features with the visual features of objects in the database.”
MethodLin, S., Miao, A. J., Lu, J., Yu, S., Chiu, Z.-Y., Richter, F., & Yip, M. C. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160746 · Cited by 34
Extracting Semantic Information from Visual Data: A Survey
Liu, Q., Li, R., Hu, H., & Gu, D. (2016) · Robotics · doi:10.3390/robotics5010008 · Cited by 34
“Wu et al. [12] and Neves dos Santos et al. [13] employed 2D visual data to tackle the problem of place recognition in semantic mapping, and object recognition with monocular cameras was presented by Civera et al. [14] and Riazuelo et al. [15].”
BackgroundIncremental scene understanding on dense SLAM
Li, C., Xiao, H., Tateno, K., Tombari, F., Navab, N., & Hager, G. D. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759111 · Cited by 32
“For example, [11], [12], [13] refine single-view object localization in a weakly cluttered background via a multi-view reconstructed scene model.”
BackgroundShape priors for real-time monocular object localization in dynamic environments
Jatavallabhula, K. M., Sharma, S., & Krishna, K. M. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8205990 · Cited by 31
From Points to Planes - Adding Planar Constraints to Monocular SLAM Factor Graphs
Arndt, C., Sabzevari, R., & Civera, J. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9340805 · Cited by 29
“Over the years there have been significant efforts to use semantic information in Structure from Motion [4], [5] and mapping [6] or to incorporate elements like lines [7], [8] or objects [9]–[11] into monocular SLAM systems.”
BackgroundSelf-citationGaussian Processes Semantic Map RepresentationPreprint
Jadidi, M. G., Gan, L., Parkison, S. A., Li, J., & Eustice, R. M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.01532 · Cited by 28
“[10] extend this approach to arbitrary three-dimensional (3D) geometries.”
BackgroundSemantic mapping techniques for indoor mobile robots: Review and prospect
Xu, S., Liang, X., & Huaidong, Z. (2024) · Measurement and Control · doi:10.1177/00202940241259903 · Cited by 27
“22,23 However, this method relies on prior knowledge and limits the application scenarios of semantic maps.”
Deep Reinforcement Learning on a Budget: 3D Control and Reasoning Without a Supercomputer
Beeching, E., Debangoye, J., Simonin, O., & Wolf, C. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412212 · Cited by 28
“Discover semantics from interactions — while solutions exist for semantic mapping and semantic SLAM [6], we are interested in tasks where the semantics of objects and their affordances are not supervised, but defined through the task and thus learned from reward.”
MethodAccurate Object Association and Pose Updating for Semantic SLAM
Chen, K., Liu, J., Chen, Q., Wang, Z., & Zhang, J. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3136918 · Cited by 26
“Civera et al. [20] propose a semantic SLAM algorithm that merges traditional meaningless points with known objects in the estimated map.”
Semi-supervised Semantic Mapping Through Label Propagation with Semantic Texture Meshes
Rosu, R. A., Quenzel, J., & Behnke, S. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01187-z · Cited by 26
“Civera et al. (2011), for example, paved the way towards a semantic SLAM system by presenting an object reasoning system, able to learn object models using feature descriptors in an offline step and then recognizing and registering them to the map at run time.”
BackgroundRan, T., Yuan, L., Zhang, J., He, L., Huang, R., & Mei, J. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3074954 · Cited by 25
“Civera et al. [25] merged the semantic objects with metric map under the EKF-Mono SLAM framework, which can be regarded as map annotation.”
BackgroundVisual-Inertial Object Detection and Mapping
Fei, X., & Soatto, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_19 · Cited by 24
Multiple Hypothesis Semantic Mapping for Robust Data Association
Bernreiter, L., Gawel, A., Sommer, H., Nieto, J. I., Siegwart, R., & Cadena, C. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2925756 · Cited by 24
Shao, X., Zhang, L., Zhang, T., Shen, Y., Li, H., & Zhou, Y. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413867 · Cited by 23
“…and eight existing representative SLAM systems from the viewpoint of three aspects, sensor modalities used (’S’ V (Visual) Semantic × Civera et al. [3] V Semantic × Mur-Artal et al. [15] V + I (IMU) Geometric × Qin et al. [17] V + I Geometric × Tateno et al. [21] V Semantic × Yang et al. [25] V…”
MethodChen, W., Chen, S., Leng, J., Wang, J., Guan, Y., Meng, M. Q. ‐., & Zhang, H. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3438165 · Cited by 23
“In [72], leveraging Internet big data is proposed to aid object detection in diverse environments through a cloud-edge collaboration communication method.”
Large-scale 3D Semantic Mapping Using Stereo Vision
Yang, Y. M., Qiu, F., Li, H., Zhang, L., Wang, M., & Fu, M. (2018) · Machine Intelligence Research · doi:10.1007/s11633-018-1118-y · Cited by 21
“A common way to construct map online is simultaneous localization and mapping (LAM) [15] consisting of sparse feature points, but it does not contain most of the environment details such as traffic lights and telegraph poles.”
BackgroundMO-SLAM: Multi object SLAM with run-time object discovery through duplicates
Dharmasiri, T., Lui, V., & Drummond, T. W. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759203 · Cited by 20
“Similar to conventional visual SLAM systems, we have a tracking thread which estimates the pose of the camera for every incoming video frame, and a mapping thread which optimizes the map as the camera explores the environment.”
Transforming Architectural Digitisation: Advancements in AI-Driven 3D Reality-Based Modelling
Zhang, K., & Fassi, F. (2025) · Heritage · doi:10.3390/heritage8020081 · Cited by 20
Active information acquisition with mobile robots
Atanasov, N. (2015) · ScholarlyCommons (University of Pennsylvania) · Cited by 20
See-your-room: indoor localization with camera vision
Sun, M., Zhang, L., Liu, Y., Miao, X., & Ding, X. X. (2019) · ACM Turing Celebration Conference - China (ACM TURC) · doi:10.1145/3321408.3321581 · Cited by 20
“Existing camera vision based approaches use vision to match pictures or features which are already known to figure out where people who are using this mobile equipment are [1, 5, 9, 10, 12, 17].”
BackgroundBundle Adjustment for Monocular Visual Odometry Based on Detections of Traffic Signs
Zhang, Y., Zhang, H., Wang, G., Yang, J., & Hwang, J. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2954876 · Cited by 20
“[43] propose a framework to run an extended Kalman filter (EKF) based monocular SLAM parallel to an object recognition thread, which assigns original meaningless points with known objects in the estimated map.”
MethodTwistSLAM++: Fusing Multiple Modalities for Accurate Dynamic Semantic SLAM
Gonzalez, M., Marchand, É., Kacete, A., & Royan, J. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341786 · Cited by 19
“The first object based SLAM systems such as [12], [15] require a specific object pose estimation algorithm [16], [17] which limits their applicability in real world scenarios.”
BackgroundOpti-Acoustic Semantic SLAM with Unknown Objects in Underwater Environments
Singh, K., Hong, J., Rypkema, N. R., & Leonard, J. J. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802819 · Cited by 19
“…SLAM has been thoroughly explored with applications in, for example, dynamic environments (Cui et al. [34]), monocular cameras (Civera et al. [35]), lidar (Chen et al. [36]), probabilistic data association (Bowman et al. [37]), and scene graphs (Rosinol et al. [1]), far fewer such works…”
Visual SLAM based on instance segmentation in dynamic scenes
Yan, Z., Chu, S., & Deng, L. (2021) · Measurement Science and Technology · doi:10.1088/1361-6501/abfceb · Cited by 19
“With the rapid development of deep learning, research into the application of deep learning of SLAM [ 14 ] has gained popularity in recent years, and SLAM based on semantic segmentation [ 15 – 18 ] is favored by researchers.”
Deep-SLAM++: Object-level RGBD SLAM based on class-specific deep shape priorsPreprint
Hu, L., Xu, W., Huang, K., & Kneip, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.09691 · Cited by 18
“Similar to the work of Salas-Moreno et al, Civera et al.[3] propose a semantic SLAM algorithm which extracts SURF features to recognize specific objects within a previously defined collection, and transfer the point cloud of a matched object as”
ResultJin, S., Chen, L., Sun, R., & McLoone, S. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106153 · Cited by 18
“The 2D or 3D map constructed by 56 the existing vSLAM systems can only be used to avoid obstacles and plan paths, which 57 is not enough in more advanced tasks [4].”
MethodLocalization of Classified Objects in SLAM using Nonparametric Statistics and Clustering
Iqbal, A., & Gans, N. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593541 · Cited by 18
“lines without any semantic meaning or information [6], [7].”
BackgroundSmartphone-Based Indoor Visual Navigation with Leader-Follower Mode
Xu, J., Dong, E., Ma, Q., Wu, C., & Yang, Z. (2021) · ACM Transactions on Sensor Networks · doi:10.1145/3448417 · Cited by 18
“Some works attempted to incorporate semantics into monocular visual SLAM [6, 8] for better robustness in time-varying environments.”
BackgroundAasim, W. F. A. W., Okasha, M., & Faris, W. F. (2022) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-022-01643-y · Cited by 18
“A variety of methods have been proposed to resolve this issue which utilises sensor fusion [95, 96], two view initializations [97], distance to vanishing points in a scene [98] and inverse depth parameterization [99].”
Method
Metadata
- Authors
- Civera, J., Galvez-Lopez, D., Riazuelo, L., Tardos, J. D., & Montiel, J. M. M.
- Affiliations
- Universidad de Zaragoza (ES)
- Venue
- 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems
- Publisher
- IEEE
- Published
- 2011-09
- Pages
- 1277-1284
- ISSN
- 2153-0858 (ISSN-L)
- OpenAlex
- W2162731263
- Cited by
- 172
- References
- 34
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 11 · 2023: 8 · 2022: 10 · 2021: 21 · 2020: 17 · 2019: 21 · 2018: 24 · 2017: 14 · 2016: 14 · 2015: 6 · 2014: 8 · 2013: 8 · 2012: 6
- FWCI
- 13.60
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- semantic SLAM, monocular SLAM, object recognition, object model, SURF features, extended Kalman filter, map annotation, robot navigation, real-time mapping, handheld camera
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.