Towards semantic SLAM using a monocular camera
The DOI matches its Crossref record; 32 of the 34 references it cites were found in a registry or an index.
Abstract
Monocular SLAM systems have been mainly focused on producing geometric maps just composed of points or edges; but without any associated meaning or semantic content. In this paper, we propose a semantic SLAM algorithm that merges in the estimated map traditional meaningless points with known objects. The non-annotated map is built using only the information extracted from a monocular image sequence. The known object models are automatically computed from a sparse set of images gathered by cameras that may be different from the SLAM camera. The models include both visual appearance and tridimensional information. The semantic or annotated part of the map -the objects- are estimated using the information in the image sequence and the precomputed object models. The proposed algorithm runs an EKF monocular SLAM parallel to an object recognition thread. This latest one informs of the presence of an object in the sequence by searching for SURF correspondences and checking afterwards their geometric compatibility. When an object is recognized it is inserted in the SLAM map, being its position measured and hence refined by the SLAM algorithm in subsequent frames. Experimental results show real-time performance for a hand held camera imaging a desktop environment and for a camera mounted in a robot moving in a room-sized scenario.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“First attempts were done in small scale by Castle et al. [44] and by Civera et al. [50] with a monocular SLAM with sparse features”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [50]
Citations
Cites
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070
SURF: Speeded Up Robust Features
Bay, H., Tuytelaars, T., & Van Gool, L. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744023_32 · Cited by 14,955
Speeded-Up Robust Features (SURF)
Bay, H., Ess, A., Tuytelaars, T., & Van Gool, L. (2008) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2007.09.014 · Cited by 13,557
Simultaneous localization and mapping: part I
Durrant-Whyte, H., & Bailey, T. (2006) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2006.1638022 · Cited by 4,199
Accurate, Dense, and Robust Multiview Stereopsis
Furukawa, Y., & Ponce, J. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.161 · Cited by 3,019 · cited 1×
“Multi- View Stereo algorithms (specifically the software package PMVS [11]) are used for this purpose.”
MethodFAST APPROXIMATE NEAREST NEIGHBORS WITH AUTOMATIC ALGORITHM CONFIGURATION
(2009) · Proceedings of the Fourth International Conference on Computer Vision Theory and Applications · doi:10.5220/0001787803310340 · Cited by 2,640 · cited 1×
“For each object O in the database, correspondences are calculated between I k−m and its faces F by applying the distance ratio (NNDR) to fast approximate nearest neighbors [19].”
MethodSimultaneous localization and mapping (SLAM): part II
Bailey, T., & Durrant-Whyte, H. (2006) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2006.1678144 · Cited by 2,554
Modeling the World from Internet Photo Collections
Snavely, N., Seitz, S. M., & Szeliski, R. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0107-3 · Cited by 2,155 · cited 2×
“A modern software library called Bundler [24] has been used in this paper for constructing this geometric part of the model.”
MethodFAB-MAP: Probabilistic Localization and Mapping in the Space of Appearance
Cummins, M., & Newman, P. (2008) · The International Journal of Robotics Research · doi:10.1177/0278364908090961 · Cited by 1,472 · cited 1×
“Visual recognition has been used in robotic SLAM mostly for scene recognition, for example loop closing in [7], [2], relocalization in [28] and place classification in a semantic context in [22].”
MethodInverse Depth Parametrization for Monocular SLAM
Civera, J., Davison, A. J., & Montiel, J. (2008) · IEEE Transactions on Robotics · doi:10.1109/tro.2008.2003276 · Cited by 779 · cited 1×
“Point features y i are initialized using inverse depth parametrization and are converted to Euclidean coordinates y i = (X Y Z) when projection equation shows a high degree of linearity [5] .”
MethodSelf-citationDecomposing a scene into geometric and semantically consistent regions
Gould, S., Fulton, R., & Koller, D. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459211 · Cited by 672 · cited 1×
“or image segmentation [13] could also help to augment the density of the annotated objects.”
BackgroundWaibel, M., Beetz, M., Civera, J., D'Andrea, R., Elfring, J., Gálvez-López, D., Häussermann, K., Janssen, R., Montiel, J. M. M., Perzylo, A., et al. (2011) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2011.941632 · Cited by 429
Self-citationLive dense reconstruction with a single moving camera
Newcombe, R. A., & Davison, A. J. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539794 · Cited by 366 · cited 1×
“Second, monocular SLAM algorithms providing with denser geometric maps [21], [25] could be used in order to help robotic tasks like navigation or path planning.”
MethodUsing the Forest to See the Trees: A Graphical Model Relating Features, Objects, and Scenes
Murphy, K. P., Torralba, A. B., & Freeman, W. T. (2003) · neural information processing systems · Cited by 352
Scale Drift-Aware Large Scale Monocular SLAM
Strasdat, H., M. M. Montiel, J., & Davison, A. (2010) · Robotics: Science and Systems VI · doi:10.15607/rss.2010.vi.010 · Cited by 323 · cited 1×
“Second, monocular SLAM algorithms providing with denser geometric maps [21], [25] could be used in order to help robotic tasks like navigation or path planning.”
MethodSelf-citationScale Drift-Aware Large Scale Monocular SLAM
Strasdat, H., Montiel, J. M. M., & Davison, A. J. (2011) · Robotics · doi:10.7551/mitpress/9123.003.0014 · Cited by 320
Self-citationMulti-hierarchical semantic maps for mobile robotics
Galindo, C., Saffiotti, A., Coradeschi, S., Buschka, P., Fernandez-Madrigal, J. A., & Gonzalez, J. (2005) · 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2005.1545511 · Cited by 308 · cited 1×
“[12] is one of the few examples, using a map hierarchy together with object recognition in order to obtain additional information to classify the places where the objects are found in.”
Conceptual spatial representations for indoor mobile robots
Zender, H., Martínez Mozos, O., Jensfelt, P., Kruijff, G.-J. M., & Burgard, W. (2008) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2008.03.007 · Cited by 300 · cited 1×
“Image-based recognition was also used in [9], [17], [29] to register known objects into a laser SLAM map.”
MethodAccurate Non-Iterative O(n) Solution to the PnP Problem
Moreno-Noguer, F., Lepetit, V., & Fua, P. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4409116 · Cited by 295 · cited 1×
“The Perspective-n-Point (PnP) problem [18] is used to estimate the translation t C k−m F and orientation q C k−m F that define this transformation from the image features and 3D points.”
MethodFrom Images to Shape Models for Object Detection
Ferrari, V., Jurie, F., & Schmid, C. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0270-9 · Cited by 295 · cited 1×
“For example, the classical object recognition algorithms used in this paper could be upgraded to the most recent category recognition algorithms [10].”
MethodCivera, J., Grasa, O. G., Davison, A. J., & Montiel, J. M. M. (2010) · Journal of Field Robotics · doi:10.1002/rob.20345 · Cited by 261
Self-citationDominant orientation templates for real-time detection of texture-less objects
Hinterstoisser, S., Lepetit, V., Ilic, S., Fua, P., & Navab, N. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539908 · Cited by 233 · cited 1×
“Third, visual recognition allows to detect the presence of the object in an image stream [15], [16].”
BackgroundCognitive maps for mobile robots—an object based approach
Vasudevan, S., Gächter, S., Nguyen, V., & Siegwart, R. (2007) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2006.12.008 · Cited by 225 · cited 1×
“A similar idea is presented in [26], where the objects are located in the 3D space by means of a stereo camera.”
BackgroundWilliams, B., Klein, G., & Reid, I. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4409115 · Cited by 224 · cited 1×
“Visual recognition has been used in robotic SLAM mostly for scene recognition, for example loop closing in [7], [2], relocalization in [28] and place classification in a semantic context in [22].”
MethodCurious George: An attentive semantic robot
Meger, D., Forssén, P.-E., Lai, K., Helmer, S., McCann, S., Southey, T., Baumann, M., Little, J. J., & Lowe, D. G. (2008) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2008.03.008 · Cited by 167 · cited 1×
“Image-based recognition was also used in [9], [17], [29] to register known objects into a laser SLAM map.”
MethodMulti-modal Semantic Place Classification
Pronobis, A., Martínez Mozos, O., Caputo, B., & Jensfelt, P. (2010) · The International Journal of Robotics Research · doi:10.1177/0278364909356483 · Cited by 149 · cited 1×
“Visual recognition has been used in robotic SLAM mostly for scene recognition, for example loop closing in [7], [2], relocalization in [28] and place classification in a semantic context in [ 22 ].”
MethodSemantic Modeling of Places using Objects
Ranganathan, A., & Dellaert, F. (2007) · Robotics: Science and Systems · doi:10.15607/rss.2007.iii.001 · Cited by 105 · cited 1×
“Although we find it a promising line of research, we do not consider here the relations between objects in a scene (as done, for example, in [23]).”
BackgroundMOPED: A scalable and low latency object recognition and pose estimation system
Martinez, M., Collet, A., & Srinivasa, S. S. (2010) · 2010 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2010.5509801 · Cited by 97 · cited 1×
“Third, visual recognition allows to detect the presence of the object in an image stream [15], [16].”
BackgroundCombining monoSLAM with object recognition for scene augmentation using a wearable camera
Castle, R. O., Klein, G., & Murray, D. W. (2010) · Image and Vision Computing · doi:10.1016/j.imavis.2010.03.009 · Cited by 81 · cited 1×
“The closest approach to this paper is the work by Castle et al. [3], [4], which registers planar objects into an EKF-based monocular SLAM.”
MethodIntegrating Active Mobile Robot Object Recognition and SLAM in Natural Environments
Ekvall, S., Jensfelt, P., & Kragić, D. (2006) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2006.282389 · Cited by 77 · cited 1×
“Image-based recognition was also used in [9], [17], [29] to register known objects into a laser SLAM map.”
MethodTowards simultaneous recognition, localization and mapping for hand-held and wearable cameras
Castle, R. O., Gawley, D. J., Klein, G., & Murray, D. W. (2007) · Proceedings 2007 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2007.364109 · Cited by 68 · cited 1×
“The closest approach to this paper is the work by Castle et al. [3], [4], which registers planar objects into an EKF-based monocular SLAM.”
MethodRobust place recognition with stereo cameras
Cadena, C., Gálvez-López, D., Ramos, F., Tardós, J. D., & Neira, J. (2010) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2010.5650234 · Cited by 54 · cited 1×
“Visual recognition has been used in robotic SLAM mostly for scene recognition, for example loop closing in [7], [2], relocalization in [28] and place classification in a semantic context in [22].”
MethodSelf-citation
Cited by
Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
“[51] with a monocular SLAM with sparse features, and by Dame et al.”
Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
Simultaneous Localization and Mapping: A Survey of Current Trends in Autonomous Driving
Bresson, G., Alsayed, Z., Yu, L., & Glaser, S. (2017) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2017.2749181 · Cited by 885
“In [278], the authors integrate well known objects inside a monocular SLAM.”
Geometry-Aware Learning of Maps for Camera Localization
Brahmbhatt, S., Gu, J., Kim, K., Hays, J., & Kautz, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00277 · Cited by 530
Probabilistic data association for semantic SLAM
Bowman, S. L., Atanasov, N., Daniilidis, K., & Pappas, G. J. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989203 · Cited by 462
“Many other approaches [14]–[19] extract both metric and semantic information but typically the two processes are carried out separately and the results are merged afterwards.”
Background
Metadata
- Authors
- Civera, J., Galvez-Lopez, D., Riazuelo, L., Tardos, J. D., & Montiel, J. M. M.
- Affiliations
- Universidad de Zaragoza (ES)
- Venue
- 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems
- Publisher
- IEEE
- Published
- 2011-09
- Pages
- 1277-1284
- ISSN
- 2153-0858 (ISSN-L)
- OpenAlex
- W2162731263
- Cited by
- 172
- References
- 34
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 11 · 2023: 8 · 2022: 10 · 2021: 21 · 2020: 17 · 2019: 21 · 2018: 24 · 2017: 14 · 2016: 14 · 2015: 6 · 2014: 8 · 2013: 8 · 2012: 6
- FWCI
- 13.60
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- semantic SLAM, monocular SLAM, object recognition, object model, SURF features, extended Kalman filter, map annotation, robot navigation, real-time mapping, handheld camera
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.