Large-scale semantic mapping and reasoning with heterogeneous modalities
The DOI matches its Crossref record; 24 of the 27 references it cites were found in a registry or an index.
Abstract
This paper presents a probabilistic framework combining heterogeneous, uncertain, information such as object observations, shape, size, appearance of rooms and human input for semantic mapping. It abstracts multi-modal sensory information and integrates it with conceptual common-sense knowledge in a fully probabilistic fashion. It relies on the concept of spatial properties which make the semantic map more descriptive, and the system more scalable and better adapted for human interaction. A probabilistic graphical model, a chaingraph, is used to represent the conceptual information and perform spatial reasoning. Experimental results from online system tests in a large unstructured office environment highlight the system's ability to infer semantic room categories, predict existence of objects and values of other spatial properties as well as reason about unexplored space.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Speeded-Up Robust Features (SURF)
Bay, H., Ess, A., Tuytelaars, T., & Van Gool, L. (2008) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2007.09.014 · Cited by 13,557
Context-based vision system for place and object recognition
Torralba, Murphy, Freeman, & Rubin (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238354 · Cited by 867
The Spatial Semantic Hierarchy
Kuipers, B. (2000) · Artificial Intelligence · doi:10.1016/s0004-3702(00)00017-5 · Cited by 726
Towards semantic maps for mobile robots
Nüchter, A., & Hertzberg, J. (2008) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2008.08.001 · Cited by 406
Multi-hierarchical semantic maps for mobile robotics
Galindo, C., Saffiotti, A., Coradeschi, S., Buschka, P., Fernandez-Madrigal, J. A., & Gonzalez, J. (2005) · 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2005.1545511 · Cited by 308
Conceptual spatial representations for indoor mobile robots
Zender, H., Martínez Mozos, O., Jensfelt, P., Kruijff, G.-J. M., & Burgard, W. (2008) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2008.03.007 · Cited by 300
Self-citationHighly scalable appearance-only SLAM - FAB-MAP 2.0
Cummins, M., & Newman, P. (2009) · Robotics: Science and Systems V · doi:10.15607/rss.2009.v.039 · Cited by 282
Chain Graph Models and their Causal Interpretations
Lauritzen, S. L., & Richardson, T. S. (2002) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00340 · Cited by 265
libDAI: A Free and Open Source C++ Library for Discrete Approximate Inference in Graphical Models
Mooij, J. M. (2010) · Max Planck Institute for Plasma Physics · doi:10.5555/1756006.1859925 · Cited by 227
Curious George: An attentive semantic robot
Meger, D., Forssén, P.-E., Lai, K., Helmer, S., McCann, S., Southey, T., Baumann, M., Little, J. J., & Lowe, D. G. (2008) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2008.03.008 · Cited by 167
Research works printed but not linked: 1
- mo?rwald, 0, BLORT - The blocks world robotic vision toolbox, ICRA Workshop Best Practice in 3D Perc and Model for Mobile Manipul 2010
Cited by
Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
“Pronobis and Jensfelt [221] label different rooms, while Pillai and Leonard [214] segment several known objects in the map.”
Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
Visual Place Recognition: A Survey
Lowry, S., Sünderhauf, N., Newman, P., Leonard, J. J., Cox, D., Corke, P., & Milford, M. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2496823 · Cited by 1,079
Semantic mapping for mobile robotics tasks: A survey
Kostavelis, I., & Gasteratos, A. (2015) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2014.12.006 · Cited by 376
“Moreover, the (a) (b) Figure 8: Topological maps -including probabilistic assertions-of the environment associated with a metric map [2] authors in [46] employed hidden Markov models (HMM) to semantically infer about the nodes in the topological graph.”
BackgroundMethodInfluentialKimera: From SLAM to spatial perception with 3D dynamic scene graphs
Rosinol, A., Violette, A., Abate, M., Hughes, N., Chang, Y., Shi, J., Gupta, A., & Carlone, L. (2021) · The International Journal of Robotics Research · doi:10.1177/02783649211056674 · Cited by 271
“Pronobis and Jensfelt (2012) use a Markov Random Field to segment a 2D grid map.”
MethodMeaningful maps – Object-oriented semantic mapping
Sünderhauf, N., Pham, T. T., Latif, Y., Milford, M. J., & Reid, I. D. (2017) · QUT ePrints (Queensland University of Technology) · Cited by 227
Tellex, S., Gopalan, N., Kress-Gazit, H., & Matuszek, C. (2020) · Annual Review of Control, Robotics, and Autonomous Systems · doi:10.1146/annurev-control-101119-071628 · Cited by 220
3D Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humans
Rosinol, A., Gupta, A., Abate, M., Shi, J., & Carlone, L. (2020) · Robotics: Science and Systems XVI · doi:10.15607/rss.2020.xvi.079 · Cited by 201
Place categorization and semantic mapping on a mobile robot
Sünderhauf, N., Dayoub, F., McMahon, S., Talbot, B., Schulz, R., Corke, P., Wyeth, G. F., Upcroft, B., & Milford, M. J. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487796 · Cited by 173
Robot task planning and explanation in open and uncertain worlds
Hanheide, M., Göbelbecker, M., Horn, G. S., Pronobis, A., Sjöö, K., Aydemir, A., Jensfelt, P., Gretton, C., Dearden, R., Janicek, M., et al. (2017) · Artificial Intelligence · doi:10.1016/j.artint.2015.08.008 · Cited by 145
“Following Pronobis and Jensfelt [35], a small set of views is acquired at each place, and two types of low-level features are extracted: (a) geometric features from laser range data which are indicative of room shape and size, and (b) global appearance features based on composed receptive field histograms obtained fro…”
BackgroundMethodInfluentialSelf-citationChoudhary, S., Carlone, L., Nieto-Granda, C., Rogers, J. G., Christensen, H. I., & Dellaert, F. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364917732640 · Cited by 158
“Related work in dense semantic mapping include Nüchter and Hertzberg (2008); Koppula et al. (2011); Pronobis and Jensfelt (2012); Finman et al. (2013); Kundu et al. (2014); Vineet et al. (2015); Valentin et al. (2015); McCormac et al. (2016) and the references therein.”
BackgroundST-HMP: Unsupervised Spatio-Temporal feature learning for tactile data
Madry, M., Bo, L., Kragić, D., & Fox, D. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6907172 · Cited by 115
“tactile sensing is paramount for the interaction of the robot with the real world: while visual input serves as a natural source of information for scene understanding, segmentation and object detection [1]–[5], tactile information is crucial for object grasping and manipulation.”
BackgroundSemantics for Robotic Mapping, Perception and Interaction: A Survey
Garg, S., Sünderhauf, N., Dayoub, F., Morrison, D., Cosgun, A., Carneiro, G., Wu, Q., Chin, T.-J., Reid, I., Gould, S., et al. (2020) · Foundations and Trends® in Robotics · doi:10.1561/2300000059 · Cited by 105
Active Visual Object Search in Unknown Environments Using Uncertain Semantics
Aydemir, A., Pronobis, A., Gobelbecker, M., & Jensfelt, P. (2013) · IEEE Transactions on Robotics · doi:10.1109/tro.2013.2256686 · Cited by 106
“To this end, we have recently proposed a probabilistic semantic mapping framework [1].”
BackgroundMethodInfluentialSelf-citationSemantic Information for Robot Navigation: A Survey
Crespo, J., Castillo, J. C., Mozos, Ó. M., & Barber, R. (2020) · Applied Sciences · doi:10.3390/app10020497 · Cited by 140
“This is the case of Pronobis and Jensfelt that propose a spatial representation in a layered structure [17] as shown in Figure 3.”
BackgroundMethodInfluentialBuilding Multiversal Semantic Maps for Mobile Robot Operation
Ruiz-Sarmiento, J.-R., Galindo, C., & González-Jiménez, J. (2016) · Knowledge-Based Systems · doi:10.1016/j.knosys.2016.12.016 · Cited by 104
Learning Semantic Maps from Natural Language Descriptions
Walter, M., Hemachandra, S., Homberg, B., Tellex, S., & Teller, S. (2013) · Robotics: Science and Systems IX · doi:10.15607/rss.2013.ix.004 · Cited by 84
“Pronobis and Jensfelt [8] describe a multi-modal probabilistic framework incorporating semantic information from a wide variety of modalities including detected objects, place appearance, and human-provided information.”
BackgroundSimultaneous Localization And Mapping: Present, Future, and the Robust-Perception Age.Preprint
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · arXiv (Cornell University) · Cited by 90
Robot navigation via spatial and temporal coherent semantic maps
Kostavelis, I., Charalampous, K., Gasteratos, A. C., & Tsotsos, J. K. (2015) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2015.11.004 · Cited by 79
“In more sophisticated works such as those reported in Pronobis and Jensfelt (2011, 2012) the authors utilize multiple sensors to deduce about different features of the scene.”
BackgroundA review of spatial reasoning and interaction for real-world robotics
Landsiedel, C., Rieser, V., Walter, M. R., & Wollherr, D. (2017) · Advanced Robotics · doi:10.1080/01691864.2016.1277554 · Cited by 72
Interactive 3D Graph SLAM for Map Correction
Koide, K., Miura, J., Yokozuka, M., Oishi, S., & Banno, A. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3028828 · Cited by 59
“They typically project point clouds into a 2D image form (e.g., spherical image [12], [13], histogram image [14], and ScanContext image [15]) and apply convolutional filters to extract discriminative features.”
Autonomous indoor robot navigation using a sketch interface for drawing maps and routes
Boniardi, F., Valada, A., Burgard, W., & Tipaldi, G. D. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487453 · Cited by 53
“In the context of robot navigation, significant amount of research has been devoted to develop natural and human-friendly means for transferring spatial information from users to robots as well as to enhance the robot’s cognition about the surrounding environment [21], [20], [6], [11].”
BackgroundHemachandra, S. M., Walter, M. R., Tellex, S., & Teller, S. J. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6907235 · Cited by 58
“Pronobis and Jensfelt [10] describe a multi-modal probabilistic framework incorporating semantic information from a wide variety of modalities including object detections, place appearance, and human-provided information.”
BackgroundInfluentialEfficient Object Search With Belief Road Map Using Mobile Robot
Wang, C., Cheng, J., Wang, J., Li, X., & Meng, M. Q. ‐. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2849610 · Cited by 59
Seeing the Un-Scene: Learning Amodal Semantic Maps for Room Navigation
Narasimhan, M., Wijmans, E., Chen, X., Darrell, T., Batra, D., Parikh, D., & Singh, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58523-5_30 · Cited by 65
Autonomous Navigation Framework for Intelligent Robots Based on a Semantic Environment Modeling
Joo, S.-H., Manzoor, S., Rocha, Y. G., Bae, S.-H., Lee, K.-H., Kuc, T., & Kim, M. (2020) · Applied Sciences · doi:10.3390/app10093219 · Cited by 59
“[26] proposed a probabilistic system for extracting semantic information from different heterogeneous modalities and combined it with common-sense conceptual knowledge.”
BackgroundErsen, M., Öztop, E., & Sarıel, S. (2017) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2016.2616538 · Cited by 55
“An example semantic map representing beliefs about room categories and existing objects in a probabilistic topological graph aligned with the metric map of the environment [48].”
BackgroundFollowing High-level Navigation Instructions on a Simulated Quadcopter with Imitation Learning
Blukis, V., Brukhim, N., Bennett, A., Knepper, R. A., & Artzi, Y. (2018) · Robotics: Science and Systems · doi:10.15607/rss.2018.xiv.066 · Cited by 68
CAGE: Context-Aware Grasping Engine
Liu, W., Daruna, A., & Chernova, S. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197289 · Cited by 62
“including task planning [10], plan repair [11], and semantic localization [12].”
SLAM with object discovery, modeling and mapping
Choudhary, S., Trevor, A. J. B., Christensen, H. I., & Dellaert, F. (2014) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2014.6942683 · Cited by 53
“[21] proposed a complete and efficient representation of indoor spaces including semantic information.”
BackgroundMultimodal learning and inference from visual and remotely sensed data
Rao, D., Deuge, M. D., Nourani‐Vatani, N., Williams, S. B., & Pizarro, O. R. (2016) · The International Journal of Robotics Research · doi:10.1177/0278364916679892 · Cited by 47
“This work is then extended using a chain-graph to incorporate contextual information, such as adjacent class labels, into the process, allowing the robot to reason about unexplored areas (Pronobis and Jensfelt, 2012).”
BackgroundIntrospective classification for robot perception
Grimmett, H., Triebel, R., Paul, R. S., & Posner, I. (2015) · The International Journal of Robotics Research · doi:10.1177/0278364915587924 · Cited by 53
“A rich body of work now exists in which semantic labels are generated based on a variety of sensor modalities and classification frameworks (e.g. Anguelov et al., 2005; Douillard et al., 2008; Martı́nez-Mozos et al., 2007; Paul et al., 2012; Posner et al., 2009; Pronobis and Jensfelt, 2012; Sengupta et al., 2012).”
BackgroundLiving with robots: Interactive environmental knowledge acquisition
Gemignani, G., Capobianco, R., Bastianelli, E., Bloisi, D. D., Iocchi, L., & Nardi, D. (2016) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2015.11.001 · Cited by 44
“In [15] the authors present a multi-layered semantic mapping algorithm that combines information about the existence of objects and semantic properties about the space, such as room size, share, and appearance.”
BackgroundBastianelli, E., Bloisi, D. D., Capobianco, R., Cossu, F., Gemignani, G., Iocchi, L., & Nardi, D. (2013) · International Conference on Advanced Robotics (ICAR) · doi:10.1109/icar.2013.6766501 · Cited by 38
“The use of heterogeneous modalities for a comprehensive multi-layered semantic mapping algorithm, aiming at place categorization and topological map construction, is also developed in [16].”
BackgroundDynamic Bayesian network for semantic place classification in mobile robotics
Premebida, C., Faria, D. R., & Nunes, U. (2016) · Autonomous Robots · doi:10.1007/s10514-016-9600-2 · Cited by 37
“The process of semantic place recognition, or categorization (Jung et al. 2016), incorporated in a map building process is known as semantic mapping (Shi et al. 2012; Pronobis and Jensfelt 2012; Jung et al. 2014; Shi et al. 2013).”
BackgroundImproved and scalable online learning of spatial concepts and language models with mapping
Taniguchi, A., Hagiwara, Y., Taniguchi, T., & Inamura, T. (2020) · Autonomous Robots · doi:10.1007/s10514-020-09905-0 · Cited by 42
“nce, thus it is desirable for robots to autonomously acquire spatial concepts and vocabulary based on their own experiences. Related research in the fields of semantic mapping and place categorization [1]–[4] have attracted considerable interests in recent years. However, the conventional approaches of most of these st…”
BackgroundTowards the creation of tactile maps for robots and their use in robot contact motion control
Denei, S., Mastrogiovanni, F., & Cannata, G. (2014) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2014.09.011 · Cited by 40
Semantic maps from multiple visual cues
Kostavelis, I., & Gasteratos, A. C. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.10.014 · Cited by 37
Knowing when we don't know: Introspective classification for mission-critical decision making
Grimmett, H., Paul, R. S., Triebel, R., & Posner, I. (2013) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2013.6631221 · Cited by 37
Data association for semantic world modeling from partial views
Wong, L. L. S., Kaelbling, L. P., & Lozano‐Pérez, T. (2015) · The International Journal of Robotics Research · doi:10.1177/0278364914559754 · Cited by 42
Grounding semantic maps in spatial databases
Deeken, H., Wiemann, T., & Hertzberg, J. (2018) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2018.03.011 · Cited by 38
Semantic mapping extension for OpenStreetMap applied to indoor robot navigation
Naik, L., Blumenthal, S., Huebel, N., Bruyninckx, H., & Prassler, E. A. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793641 · Cited by 34
Unsupervised semantic clustering and localization for mobile robotics tasks
Balaska, V. E., Bampis, L., Boudourides, M. A., & Gasteratos, A. C. (2020) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2020.103567 · Cited by 37
AOMD: An analogy-aware approach to offensive meme detection on social media
Shang, L., Zhang, Y., Zha, Y., Chen, Y., Youn, C., & Wang, D. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102664 · Cited by 48
Relational Model for Robotic Semantic Navigation in Indoor Environments
Crespo, J., Barber, R., & Mozos, Ó. M. (2017) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-017-0469-x · Cited by 33
“In [31] they take into account all the information sources that have been published.”
BackgroundA framework for learning semantic maps from grounded natural language descriptions
Walter, M. R., Hemachandra, S. M., Homberg, B. S., Tellex, S., & Teller, S. J. (2014) · The International Journal of Robotics Research · doi:10.1177/0278364914537359 · Cited by 39
“…that would be diminished with a more sophisticated segmentation strategy that takes into account local appearance (Brunskill et al., 2007; Ranganathan and Dellaert, 2009; Pronobis and Jensfelt, 2012; Hemachandra et al., 2014) and semantic (Martínez Mozos et al., 2007) properties of the environment.”
BackgroundMethodInfluentialLearning deep generative spatial models for mobile robots
Pronobis, A., & Rao, R. P. N. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202235 · Cited by 32
“Traditionally, robotic systems utilize an assembly of independent spatial models [2], which exchange information in a limited fashion.”
BackgroundMethodInfluentialSelf-citationMartins, R., Bersan, D., Campos, M. F. M., & Nascimento, E. R. (2020) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-019-01136-5 · Cited by 39
“ountain. Moreover, this higher level representation can provide awareness of dangerous situations (such as the presence ahead of steps, stairs) and of other people for safe navigation and interaction [4, 5]. Recent advances of data-driven machine learning techniques and the increased computing capability of daily-use…”
Semantic Mapping for Mobile Robots in Indoor Scenes: A Survey
Han, X., Li, S., Wang, X., & Zhou, W. (2021) · Information · doi:10.3390/info12020092 · Cited by 42
“While in other works [11], the topological structure is extracted from a metric map to store geometric information of the environment in a hybrid way.”
BackgroundMethodInfluentialReconstructing Interactive 3D Scenes by Panoptic Mapping and CAD Model Alignments
Han, M., Zhang, Z., Jiao, Z., Xie, X., Zhu, Y., Zhu, S. C., & Liu, H. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561546 · Cited by 39
The semantic PHD filter for multi-class target tracking: From theory to practice
Chen, J., Xie, Z., & Dames, P. (2021) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2021.103947 · Cited by 34
“While real-time machine vision algorithms are reaching high levels of object classi-fication accuracy [1, 2, 3, 4], there may still be significant uncertainty in the semantic label and it is often at the expense of high false positive rates.”
BackgroundYang, C., Wang, D., Zeng, Y., Yue, Y., & Siritanawan, P. (2018) · Information Fusion · doi:10.1016/j.inffus.2018.10.007 · Cited by 31
Nyga, D., Bálint-Benczédi, F., & Beetz, M. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6907427 · Cited by 29
“[18] describe a framework for semantic mapping based on a combination of object attributes, room appearance and human input.”
BackgroundPredicting the global structure of indoor environments: A constructive machine learning approach
Luperto, M., & Amigoni, F. (2018) · Autonomous Robots · doi:10.1007/s10514-018-9732-7 · Cited by 33
“For example, the system presented by Pronobis and Jensfelt (2012) is a probabilistic semantic mapping frame-4 work that applies semantic labels according to six kinds of environmental features (objects, doorways, room shapes, room size, appearance, and associated spaces). to The semantic map is…”
BackgroundMethodScene Reconstruction with Functional Objects for Robot Autonomy
Han, M., Zhang, Z., Jiao, Z., Xie, X., Zhu, Y., Zhu, S. C., & Liu, H. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01670-0 · Cited by 43
“Existing SLAM and semantic mapping approaches reviewed above oftentimes represent a reconstructed scene and its entities as sparse landmarks (Pronobis and Jensfelt, 2012; Yang and Scherer, 2019a), surfels (McCormac et al.”
BackgroundPlace Classification With a Graph Regularized Deep Neural Network
Liao, Y., Kodagoda, S., Wang, Y., Shi, L., & Liu, Y. (2016) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2016.2586183 · Cited by 28
“tion and many studies were carried out with consideration of the local consistency [3], [11], [34]–[36].”
A Semantic Classification Approach for Indoor Robot Navigation
Alenzi, Z., Alenzi, E., Alqasir, M., Alruwaili, M., Alhmiedat, T. A., & Alia, O. M. (2022) · Electronics · doi:10.3390/electronics11132063 · Cited by 31
A hybrid topological mapping and navigation method for large area robot mapping
Ravankar, A. A., Ravankar, A. A., Emaru, T., & Kobayashi, Y. (2017) · Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE) · doi:10.23919/sice.2017.8105770 · Cited by 28
“al in [10] and [11] have extensively worked on the problem of fusing topological features into SLAM.”
BackgroundLearning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps
Zheng, K., Pronobis, A., & Rao, R. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11743 · Cited by 35
“In order to integrate the collected spatial knowledge, resolve ambiguities, and make predictions about unobserved places, semantic mapping frameworks often employ structured prediction algorithms (Friedman, Pasula, and Fox 2007; Mozos et al. 2007; Pronobis and Jensfelt 2012).”
BackgroundMethodInfluentialSelf-citationMurray, M. H., & Çakmak, M. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3178804 · Cited by 42
A transfer learning approach for multi-cue semantic place recognition
Costante, G., Ciarfuglia, T. A., Valigi, P., & Ricci, E. (2013) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2013.6696653 · Cited by 25
“Recent works on visual place recognition [1], [2], [3], [4] have demonstrated that good performance can be obtained in many real world indoor settings, even in challenging scenarios (e.g. varying illuminations conditions).”
BackgroundSemantic Labeling of Indoor Environments from 3D RGB Maps
Brucker, M., Durner, M., Ambruş, R., Márton, Z.-C., Wendt, A., Jensfelt, P., Arras, K. O., & Triebel, R. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8462922 · Cited by 30
“[15] propose a probabilistic framework for combining various properties (shape, appearance), the presence of objects and doorways for semantically labeling a 2D map.”
BackgroundSelf-citationCollaborative Mobile Robotics for Semantic Mapping: A Survey
Achour, A., Al-Assaad, H., Dupuis, Y., & Zaher, M. E. (2022) · Applied Sciences · doi:10.3390/app122010316 · Cited by 34
Integration of CNN into a Robotic Architecture to Build Semantic Maps of Indoor Environments
Chaves, D., Ruiz-Sarmiento, J.-R., Petkov, N., & González-Jiménez, J. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20518-8_27 · Cited by 24
Meaningful maps with object-oriented semantic mapping
Sünderhauf, N., Pham, T. T., Latif, Y., Milford, M. J., & Reid, I. D. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206392 · Cited by 126
“[11], [13] proposed an online system to build a semantic map of the environment using laser as well as cameras.”
BackgroundHu, X., Fan, H., Noskov, A., Wang, Z., Zipf, A., Gu, F., & Shang, J. (2020) · Transactions in GIS · doi:10.1111/tgis.12664 · Cited by 26
“Pronobis and Jensfelt (2012) used a graph to represent the indoor environment, where rooms are regarded as the nodes of the graph.”
BackgroundViMantic, a distributed robotic architecture for semantic mapping in indoor environments
Fernandez-Chaves, D., Ruiz-Sarmiento, J.-R., Petkov, N., & González-Jiménez, J. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107440 · Cited by 24
Zhao, W., Lin, R., Dong, S., & Cheng, Y. (2022) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2022.3198801 · Cited by 23
Applying probabilistic Mixture Models to semantic place classification in mobile robotics
Premebida, C., Faria, D. R., Souza, F. A. A., & Nunes, U. (2015) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7353981 · Cited by 22
“The process of applying a semantic term to some place of the map is also known as semantic mapping [3], [4], [5], [6].”
MethodHierarchical Spatial Concept Formation Based on Multimodal Information for Human Support Robots
Hagiwara, Y., Inoue, M., Kobayashi, H., & Taniguchi, T. (2018) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2018.00011 · Cited by 22
“Inthe field of navigation tasks with human-robot interaction, methods of classifying corridors and rooms using a predefined ontology based on shape and image features have been proposed (Zender et al., 2008; Pronobis and Jensfelt, 2012).”
MethodFrom Pixels to Buildings: End-to-end Probabilistic Deep Networks for Large-scale Semantic Mapping
Zheng, K., & Pronobis, A. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967568 · Cited by 21
“Pronobis and Jensfelt [2] applied Markov Random Fields to model pairwise dependencies between semantic categories of rooms according to a topological map.”
BackgroundMethodInfluentialSelf-citationBuilding a grid-semantic map for the navigation of service robots through human–robot interaction
Zhao, C., Mei, W., & Pan, W. (2015) · Digital Communications and Networks · doi:10.1016/j.dcan.2015.09.002 · Cited by 22
“Pronobis and Jensfelt presented a probabilistic framework combining heterogeneous, uncertain information such as object observations, shape, size, appearance of rooms and human input for semantic mapping [13].”
BackgroundLuperto, M., Kucner, T. P., Tassi, A., Magnusson, M., & Francesco Amigoni, A. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3186495 · Cited by 21
Inferring Compact Representations for Efficient Natural Language Understanding of Robot Instructions
Patki, S., Daniele, A. F., Walter, M. R., & Howard, T. M. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793667 · Cited by 28
“…measurements into a single, flat representation of the environment (i.e., the “world model”) that expresses all metric [6–12] as well as semantic [4, 5, 13–16] knowledge Siddharth Patki and Thomas M. Howard are with the University of Rochester, Rochester, NY USA, spatki@ur.rochester.edu,…”
Multiple Hypothesis Semantic Mapping for Robust Data Association
Bernreiter, L., Gawel, A., Sommer, H., Nieto, J. I., Siegwart, R., & Cadena, C. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2925756 · Cited by 24
Liu, Z., & Wichert, G. V. (2013) · Future Generation Computer Systems · doi:10.1016/j.future.2013.06.026 · Cited by 22
“Pronobis and Jensfelt [20] integrated multi-modal sensory information, human intervention and some common-sense knowledge to classify places with semantic types.”
BackgroundSemantic Map Augmentation for Robot Navigation: A Learning Approach Based on Visual and Depth Data
Bersan, D., Martins, R., Campos, M. F. M., & Nascimento, E. R. (2018) · Latin American Robotics Symposium, Brazilian Symposium on Robotics and Workshop on Robotics in Education (LARS/SBR/WRE) · doi:10.1109/lars/sbr/wre.2018.00018 · Cited by 20
“Typical examples are self-driving cars, which need to recognize other vehicles, pedestrians and the free road space for safe navigation [1], [2], or personal assistant robots that need to recognize the environment and humans for safe interaction [3], [4].”
BackgroundPredicting the Layout of Partially Observed Rooms from Grid Maps
Luperto, M., Arcerito, V., & Francesco Amigoni, A. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793489 · Cited by 19
“The system proposed in [15] predicts the existence of a room (and its label) in the unexplored space, but it does not provide any information on the geometry of the room.”
BackgroundHierarchical Semantic Mapping Using Convolutional Neural Networks for Intelligent Service Robotics
Luo, R. C., & Chiou, M. (2018) · IEEE Access · doi:10.1109/access.2018.2873597 · Cited by 19
“[14] H. Zender, O. M. Mozos, P. Jensfelt, G.-J. M. Kruijff, and W. Burgard, ‘‘Conceptual spatial representations for indoor mobile robots,’’ Robot.”
MethodInfluentialComparison between Bayesian network classifiers and SVMs for semantic localization
Rubio, F., Martínez-Gómez, J., Flores, M. J., & Puerta, J. M. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.08.029 · Cited by 20
“…there are different types of semantic information that can be xtracted from the environment ( Kostavelis & Gasteratos, 2015; Liu, i, Hu, & Gu, 2016 ), but recent works are focused in the acquiition of semantic maps ( Kostavelis, Charalampous, Gasteratos, & sotsos, 2016; Pronobis & Jensfelt, 2012 ).”
BackgroundA survey of Semantic Reasoning frameworks for robotic systems
Liu, W., Daruna, A., Patel, M., Ramachandruni, K., & Chernova, S. (2022) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2022.104294 · Cited by 29
“This is particularly true for models of large spaces composed of many rooms or locations, such as an entire kitchen [19] or a whole office floor [44].”
BackgroundInfluentialA proposal for semantic map representation and evaluation
Capobianco, R., Serafin, J., Dichtl, J., Grisetti, G., Iocchi, L., & Nardi, D. (2015) · European Conference on Mobile Robots · doi:10.1109/ecmr.2015.7324198 · Cited by 18
“Pronobis and Jensfelt [7], instead, represent a conceptual map as a probabilistic chain graph model and evaluate their method by comparing the robot belief to be in a certain location against the ground truth.”
BackgroundContext-aware robot navigation using interactively built semantic maps
Cosgun, A., & Christensen, H. I. (2018) · Paladyn Journal of Behavioral Robotics · doi:10.1515/pjbr-2018-0020 · Cited by 25
“[62] proposed a joint spatial-semantic environment model by fusing multi-modal data including natural language and object classifiers.”
MethodA scene-based dependable indoor navigation system
Ko, D. W., Kim, Y. N., Lee, J. H., & Suh, I. H. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759248 · Cited by 17
Enhancing semantic segmentation with detection priors and iterated graph cuts for robotics
Antonello, M., Chiesurin, S., & Ghidoni, S. (2020) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2019.103467 · Cited by 18
Multitask Learning for Scalable and Dense Multilayer Bayesian Map Inference
Gan, L., Kim, Y., Grizzle, J. W., Walls, J. M., Kim, A., Eustice, R. M., & Ghaffari, M. (2022) · IEEE Transactions on Robotics · doi:10.1109/tro.2022.3197106 · Cited by 24
“Pronobis and Jensfelt [34] develop a probabilistic framework based on chain graphs to build a four-layer”
BackgroundTopological Semantic Mapping by Consolidation of Deep Visual Features
Sousa, Y. C. N., & Bassani, H. F. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3149572 · Cited by 17
Mozos, Ó. M., Mizutani, H., Jung, H., Kurazume, R., & Hasegawa, T. (2013) · Advanced Robotics · doi:10.1080/01691864.2013.839091 · Cited by 16
“[4,5] A final application of place categorization constitutes the high-level representation of indoor environments by autonomous robots.[6,7] In this paper, we present an approach to categorize places”
Information-theoretic dialog to improve spatial-semantic representations
Hemachandra, S. M., & Walter, M. R. (2015) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7354097 · Cited by 22
“Several approaches exist that construct semantic environment models using traditional robot sensors [7, 1, 2, 8], while others have looked at additionally integrating natural language descriptions to improve the semantic representations [3, 9, 4].”
BackgroundRobot Navigation in Unseen Spaces Using an Abstract Map
Talbot, B., Dayoub, F., Corke, P., & Wyeth, G. F. (2020) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2020.2993855 · Cited by 25
“Significant progress has been demonstrated in the area of semantic mapping [40], with approaches for attaching symbols to maps including using discrete areas of segmented robot maps [9], [41], linking perceptions with ontological representations [42], [43], and probabilistically inferring symbolic labels from object d…”
BackgroundMulti-modal panoramic 3D outdoor datasets for place categorization
Jung, H., Oto, Y., Mozos, O. M., Iwashita, Y., & Kurazume, R. (2016) · 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759669 · Cited by 16
“This information greatly improves communication between robots and humans [3], [4] and it allows autonomous robots to make decisions with context-based understanding when completing high-level tasks [5].”
BackgroundMaking Sense of Indoor Spaces Using Semantic Web Mining and Situated Robot Perception
Young, J., Basile, V., Suchi, M., Kunze, L., Hawes, N., Vincze, M., & Barbara Caputo, B. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-70407-4_39 · Cited by 16
“It is typical in robotics that object recognition is facilitated by a training step prior to deployment [12,15] (though unsupervised approaches do exist [1]) whereby selected objects from the robot’s environment are learned and later re-recognised and used for space categorisation.”
BackgroundStereo Visual Odometry and Semantics based Localization of Aerial Robots in Indoor Environments
Bavle, H., Manthe, S., de la Puente, P., Rodríguez-Ramos, A., Sampedro, C., & Campoy, P. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593426 · Cited by 16
“Pronobis et al. [6] proposed a probabilistic framework for inference of room categories considering common sense knowledge in domestic applications.”
BackgroundDesign of Robot Monitoring System for Aviation
Mohan, R., Suraj, A. K., Agarawal, S., Majumdar, S., & Nath, V. (2018) · Lecture notes in electrical engineering · doi:10.1007/978-981-13-0776-8_50 · Cited by 15
COSMO: Contextualized scene modeling with Boltzmann Machines
Bozcan, İ., & Kalkan, S. (2019) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2018.12.009 · Cited by 15
Hypermap Mapping Framework and its Application to Autonomous Semantic Exploration
Zaenker, T., Verdoja, F., & Kyrki, V. (2020) · IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI) · doi:10.1109/mfi49285.2020.9235231 · Cited by 19
“, “chair”, “plant” and “desk” [10]–[12], or more abstract descriptions of places, like “office”, “floor” and “kitchen” [13], [14].”
SEMAP - a semantic environment mapping framework
Deeken, H., Wiemann, T., Lingemann, K., & Hertzberg, J. (2015) · European Conference on Mobile Robots · doi:10.1109/ecmr.2015.7324176 · Cited by 14
“Pronobis and Jensfelt analysed features on position-tagged images and geometric attributes, like area and shape, on 2D grid maps to identify different types of rooms [2].”
BackgroundTemporal persistence modeling for object search
Toris, R., & Chernova, S. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989365 · Cited by 14
“Many approaches have been developed for tackling this broad research problem within the robotics and perception communities [9], [3], [8], [10], [11].”
BackgroundRobot navigation in unseen spaces using an abstract map
Talbot, B., Dayoub, F., Corke, P., & Wyeth, G. F. (2021) · QUT ePrints (Queensland University of Technology) · Cited by 14
Human Aware Robot Navigation in Semantically Annotated Domestic Environments
Kostavelis, I., Giakoumis, D., Malassiotis, S., & Tzovaras, D. K. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-40244-4_40 · Cited by 17
“Moreover, it is an evidence that contemporary robots use to navigate in their environments by computing their pose within metric maps and, therefore, the vast amount of the semantic mapping methods reported in the literature use these metric maps to add semantic information on top of it [12], [11].”
Background
Metadata
- Authors
- Pronobis, A., & Jensfelt, P.
- Affiliations
- KTH Royal Institute of Technology (SE)
- ORCID
- Jensfelt, Patric
- Venue
- 2012 IEEE International Conference on Robotics and Automation
- Publisher
- IEEE
- Published
- 2012-05
- Pages
- 3515-3522
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2000321478
- Cited by
- 248
- References
- 27
- Citations by year
- 2026: 5 · 2025: 9 · 2024: 6 · 2023: 6 · 2022: 20 · 2021: 17 · 2020: 23 · 2019: 16 · 2018: 34 · 2017: 27 · 2016: 22 · 2015: 23 · 2014: 25 · 2013: 14 · 2012: 1
- FWCI
- 37.56
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Physical Sciences › Computer Science › Artificial Intelligence › Logic, Reasoning, and Knowledge
- Keywords
- semantic mapping, heterogeneous modalities, multimodal sensor fusion, probabilistic graphical models, chain graphs, spatial reasoning, commonsense knowledge, object prediction, spatial characteristics, human-robot interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.