Semantic structure from motion with points, regions, and objects
The DOI matches its Crossref record; 34 of the 35 references it cites were found in a registry or an index.
Abstract
Structure from motion (SFM) aims at jointly recovering the structure of a scene as a collection of 3D points and estimating the camera poses from a number of input images. In this paper we generalize this concept: not only do we want to recover 3D points, but also recognize and estimate the location of high level semantic scene components such as regions and objects in 3D. As a key ingredient for this joint inference problem, we seek to model various types of interactions between scene components. Such interactions help regularize our solution and obtain more accurate results than solving these problems in isolation. Experiments on public datasets demonstrate that: 1) our framework estimates camera poses more robustly than SFM algorithms that use points only; 2) our framework is capable of accurately estimating pose and location of objects, regions, and points in the 3D scene; 3) our framework recognizes objects and regions more accurately than state-of-the-art single image recognition methods.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recently, the limitations of purely geometric maps have been recognized and this has spawned a significant and ongoing body of work in semantic mapping of environments”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [9]
In the citing paper
“in order to enhance robot's autonomy and robustness, facilitate more complex tasks (e.g. avoid muddy-road while driving), move from path-planning to task-planning, and enable advanced human-robot interaction [9, 26, 217]”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [9]
In the citing paper
“In their work, the authors demonstrate the improved object recognition performance and robustness”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [9]
In the citing paper
“at the cost of a run-time of 20 minutes per image-pair”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [9]
In the citing paper
“Bao et al. [9] propose one of the first approaches to jointly estimate camera parameters, scene points and object labels using both geometric and semantic attributes in the scene.”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [9]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · cited 2×
“For instance, points do not carry semantic information, but can be robustly matched across views (e.g. by [22]).”
BackgroundOptimization by Simulated Annealing
Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983) · Science · doi:10.1126/science.220.4598.671 · Cited by 45,134 · cited 1×
“We use simulated annealing [16] to search for the solution for Eq.”
MethodObject Detection with Discriminatively Trained Part-Based Models
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.167 · Cited by 9,853 · cited 2×
“However, [3] has a few limitations: 1) it focuses on modeling " structured " objects (e.g. cars, cups, and bottles), and it ignores " amorphous " scene components (e.g. road, sky, and table surfaces which can be hardly identified by standard object detectors such as [8] [21]).”
BackgroundBundle Adjustment — A Modern Synthesis
Triggs, B., McLauchlan, P. F., Hartley, R. I., & Fitzgibbon, A. W. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-44480-7_21 · Cited by 3,988 · cited 1×
“Most SFM algorithms [29] [28] represent the structure of a scene as a set of 3D points.”
BackgroundScale & Affine Invariant Interest Point Detectors
Mikolajczyk, K., & Schmid, C. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000027790.02288.f2 · Cited by 3,436
Modeling the World from Internet Photo Collections
Snavely, N., Seitz, S. M., & Szeliski, R. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0107-3 · Cited by 2,155 · cited 2×
“[28] [3] Table 3: The error for camera pose estimation.”
BackgroundAn efficient solution to the five-point relative pose problem
Nister, D. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.17 · Cited by 2,009 · cited 1×
“We follow the evaluation criteria in [24].”
Learning a classification model for segmentation
Ren, & Malik (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238308 · Cited by 1,765 · cited 1×
“The region measurements are segmented regions (e.g. by segmentation algorithms such as [26] [33]).”
Segmentation and Recognition Using Structure from Motion Point Clouds
Brostow, G. J., Shotton, J., Fauqueur, J., & Cipolla, R. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88682-2_5 · Cited by 1,050 · cited 1×
“Among these works (e.g. [12] [15] [11] [25] [10] [20] [6] [13]), most assume that only a single image is available, whereas a handful of them model this problem with multiple images.”
BackgroundCombined Object Categorization and Segmentation With An Implicit Shape Model
Leibe, B., Leonardis, A., & Schiele, B. (2004) · Max Planck Digital Library · Cited by 890
Cited by
Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
“muddy-road while driving), move from path-planning to taskplanning, and enable advanced human-robot interaction [10],”
Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments
Yu, C., Liu, Z., Liu, X.-J., Xie, F., Yang, Y., Wei, Q., & Fei, Q. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593691 · Cited by 1,154
“[14] first try to estimate camera pose, points, and objects jointly utilizing both geometry and semantic attributes in the scene, which significantly improve the object recognition accuracy.”
BackgroundSLAM++: Simultaneous Localisation and Mapping at the Level of Objects
Salas-Moreno, R. F., Newcombe, R. A., Strasdat, H., Kelly, P. H. J., & Davison, A. J. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.178 · Cited by 945
“[3] represented a scene as a set of points, objects and regions in two-view SfM, solving ex-pensively and jointly in a graph optimisation for a labelling and reconstruction solution taking account of interactions between all scene entities.”
BackgroundPL-SLAM: Real-time monocular visual SLAM with points and lines
Pumarola, A., Vakhitov, A., Agudo, A., Sanfeliu, A., & Moreno-Noguer, F. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989522 · Cited by 482
“Scene prior information have been also exploited to provide a significant boost to SLAM systems [3], [4].”
MethodCubeSLAM: Monocular 3-D Object SLAM
Yang, S., & Scherer, S. (2019) · IEEE Transactions on Robotics · doi:10.1109/tro.2019.2909168 · Cited by 454
“Bao et al. proposed the first Semantic SfM to jointly optimize camera poses, objects, points and planes [19].”
BackgroundMini-Unmanned Aerial Vehicle-Based Remote Sensing: Techniques, applications, and prospects
Xiang, T.-Z., Xia, G.-S., & Zhang, L. (2019) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2019.2918840 · Cited by 189
Salas-Moreno, R. F., Glocken, B., Kelly, P. H. J., & Davison, A. J. (2014) · 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) · doi:10.1109/ismar.2014.6948422 · Cited by 175
“[3], is far from being feasible in real-time operation, unlike [21].”
BackgroundStereo Vision-Based Semantic 3D Object and Ego-Motion Tracking for Autonomous Driving
Li, P., Qin, T., & Shen, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01216-8_40 · Cited by 158
Keyframe-based monocular SLAM: design, survey, and future directions
Younes, G., Asmar, D., Shammas, E., & Zelek, J. (2017) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2017.09.010 · Cited by 191
Occupancy Anticipation for Efficient Exploration and Navigation
Ramakrishnan, S. K., Al-Halah, Z., & Grauman, K. L. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58558-7_24 · Cited by 218
Real-time monocular object SLAM
Gálvez‐López, D., Salas, M., Tardós, J. D., & Montiel, J. M. M. (2015) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2015.08.009 · Cited by 153
Choudhary, S., Carlone, L., Nieto-Granda, C., Rogers, J. G., Christensen, H. I., & Dellaert, F. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364917732640 · Cited by 158
“Bao et al. (2012) model semantic structure from motion as a joint inference problem where they jointly recognize and estimate the location of high-level semantic scene components such as regions and objects in 3D. SLAM++, proposed by Salas-Moreno et al. (2013), train domain-specific object detectors…”
BackgroundMonocular SLAM Supported Object Recognition
Pillai, S., & Leonard, J. (2015) · Robotics: Science and Systems XI · doi:10.15607/rss.2015.xi.034 · Cited by 118
“Bao et al. [2, 3] proposed one of the first approaches to jointly estimate camera parameters, scene points and object labels using both geometric and semantic attributes in the scene.”
BackgroundChen, K., Lai, Y., Wu, Y., Martin, R. R., & Hu, S. (2014) · ACM Transactions on Graphics · doi:10.1145/2661229.2661239 · Cited by 118
Scene Graph Generation: A comprehensive survey
Li, H., Zhu, G., Zhang, L., Jiang, Y., Dang, Y., Hou, H., Shen, P., Zhao, X., Shah, S. A. A., & Bennamoun, M. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.127052 · Cited by 236
Monocular Object and Plane SLAM in Structured Environments
Yang, S., & Scherer, S. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2924848 · Cited by 113
“Semantic Structure from Motion [20] jointly optimizes various geometry components.”
BackgroundLarge-Scale Structure from Motion with Semantic Constraints of Aerial Images
Chen, Y., Wang, Y., Lu, P., Chen, Y., & Wang, G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-03398-9_30 · Cited by 148
Joint Attention in Autonomous Driving (JAAD)Preprint
Kotseruba, I., Rasouli, A., & Tsotsos, J. K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.04741 · Cited by 166
“• Semantic Structure From Motion (SSFM) [47]: As the name implies, this dataset is collected for scene understanding.”
BackgroundReal-Time Monocular Object-Model Aware Sparse SLAM
Hosseinzadeh, M., Li, K., Latif, Y., & Reid, I. D. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793728 · Cited by 93
“Other work [8] has admitted more general objects (and constraints) but only in a slow, offline structure-from-motion context.”
BackgroundSimultaneous Localization And Mapping: Present, Future, and the Robust-Perception Age.Preprint
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · arXiv (Cornell University) · Cited by 90
Semantic alignment of LiDAR data at city scale
Yu, F., Xiao, J., & Funkhouser, T. A. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298781 · Cited by 80
“Our experimental results indicate that the proposed methods can achieve significantly better alignments than those provided by alternative methods for four different cities.”
MethodStructure Aware SLAM Using Quadrics and Planes
Hosseinzadeh, M., Latif, Y., Pham, T. T., Sünderhauf, N., & Reid, I. D. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20893-6_26 · Cited by 75
“n of the SLAM problem. In this work, we propose a map representation that consists of points, and higher level geometric entities such as planes and objects as landmarks. Unlike previous work such as [8] we explicitly target real-time performance, and integrate within an online SLAM framework. Such performance would b…”
MethodOA-SLAM: Leveraging Objects for Camera Relocalization in Visual SLAM
Zins, M., Simon, G., & Berger, M. (2022) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar55827.2022.00090 · Cited by 80
“A pioneering work, introducing objects in localization and mapping has been developed in [1], by Bao et al.”
MethodObject-aware bundle adjustment for correcting monocular scale drift
Frost, D., Kähler, O., & Murray, D. W. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487680 · Cited by 67
“In contrast, more complicated object models e.g. in [33], [34], can only be interpreted if the pose of the object is known and a particular 3D object model is given.”
BackgroundNYC3DCars: A Dataset of 3D Vehicles in Geographic Context
Matzen, K., & Snavely, N. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.99 · Cited by 55
“Finally, other work combines multi-view reasoning with object recognition [3, 2], generally for images taken at the same time.”
BackgroundROLS : Robust Object-Level SLAM for Grape Counting
Nellithimaru, A., & Kantor, G. A. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00321 · Cited by 64
“However, recent advances in object-detectors and semantic mapping networks have made it possible for researchers to use classification, detection and segmentation in SLAM [2, 12, 15, 19].”
MethodJoint Detection, Tracking and Mapping by Semantic Bundle Adjustment
Fioraio, N., & Stefano, L. D. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.202 · Cited by 58
“Initial proposals were mainly based on feature matching and incremental optimization [14, 13], but soon Newcombe et al. developed Kinect Fusion [23], a new dense algorithm providing outstanding performance both in terms of extremely low drift as well as smooth surface reconstruction.”
BackgroundMethodSemantic Mapping for View-Invariant Relocalization
Li, J., Meger, D., & Dudek, G. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793624 · Cited by 59
“which also infers 3D cuboids from 2D bounding boxes under the structure from motion setting [30].”
BackgroundContinuous Scene Representations for Embodied AI
Gadre, S. Y., Ehsani, K., Song, S., & Mottaghi, R. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01443 · Cited by 68
“There are object-based maps that encode object-level semantics and the relationships [3, 7, 42].”
BackgroundApplications of structure from motion: a survey
Wei, Y., Kang, L., Yang, B., & Wu, L. (2013) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/jzus.cide1302 · Cited by 50
“Recent studies have also integrated SfM into segmentation and recognition for more sophisticated algorithms (Brostow et al., 2008; Sturgess et al., 2009; Bao et al., 2012).”
BackgroundSemantic Visual Simultaneous Localization and Mapping: A Survey
Chen, K., Xiao, J., Liu, J., Tong, Q., Zhang, H., Liu, R., Zhang, J., Ajoudani, A., & Chen, S. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3556928 · Cited by 61
“Bao et al. [35], [36], [37] take a different approach by integrating camera parameters, geometric and semantic information, and object shapes into the structure-from-motion problem.”
Planar Surface Reconstruction from Sparse Views
Jin, L., Qian, S., Owens, A., & Fouhey, D. F. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01275 · Cited by 59
“Traditional tools from multi-view geometry [18, 1] largely rely on correspondence for reconstruction and are fundamentally limited to the small part of the scene that directly overlaps, even when the camera pose is known.”
BackgroundBao, S. Y., Yu, X., & Savarese, S. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33718-5_7 · Cited by 39
“Given multiple views of a scene, SSFM methods such as [4,20,21] use high level semantic information to help estimate the camera viewpoint changes.”
MethodSelf-citationProbabilistic Structure from Motion with Objects (PSfMO)
Gay, P., Bansal, V., Rubino, C., & Bue, A. D. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.334 · Cited by 47
“The usefulness of this problem is also stressed by the attempts of previous works injecting higher semantic in classical 3D reconstruction problems [4, 45, 55, 20, 14, 17].”
BackgroundSemantic Mapping with Simultaneous Object Detection and Localization
Zeng, Z., Zhou, Y., Jenkins, O. C., & Desingh, K. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594205 · Cited by 39
Visual Graphs from Motion (VGfM): Scene Understanding with Object Geometry Reasoning
Gay, P., James, S., & Bue, A. D. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20893-6_21 · Cited by 50
Understanding the 3D layout of a cluttered room from multiple images
Bao, S. Y., Furlan, A., Fei-Fei, L., & Savarese, S. (2014) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2014.6836035 · Cited by 31
“Various experiments demonstrate that our framework can achieve better estimation accuracy and higher reconstruction completeness than alternative state-of-the-art approaches.”
BackgroundSelf-citationUnsupervised extrinsic calibration of depth sensors in dynamic scenes
Miller, S., Teichman, A., & Thrun, S. (2013) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2013.6696737 · Cited by 28
“[13], and numerous others have shown that sensor pose and scene geometry can be inferred from multiple, often uncalibrated sensors.”
BackgroundView-Invariant Loop Closure with Oriented Semantic Landmarks
Li, J., Koreitem, K., Meger, D., & Dudek, G. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196886 · Cited by 31
“Inspired by the work of Bao et al. [13] and Meger et al. [14], we represent objects as bounding cuboids that tightly fit target objects.”
MethodFrom Points to Planes - Adding Planar Constraints to Monocular SLAM Factor Graphs
Arndt, C., Sabzevari, R., & Civera, J. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9340805 · Cited by 29
“Over the years there have been significant efforts to use semantic information in Structure from Motion [4], [5] and mapping [6] or to incorporate elements like lines [7], [8] or objects [9]–[11] into monocular SLAM systems.”
BackgroundSemantically-Aware Aerial Reconstruction from Multi-modal Data
Cabezas, R., Straub, J., & Fisher, J. S. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.249 · Cited by 28
Associative3D: Volumetric Reconstruction from Sparse Views
Qian, S., Jin, L., & Fouhey, D. F. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58555-6_9 · Cited by 25
An Occupancy Information Grid Model for Path Planning of Intelligent Robots
Zhang, J., Wang, X., Xu, L., & Zhang, X. (2022) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi11040231 · Cited by 22
“The semantic map describes the spatial environment according to semantic concepts and processes each element in a categorical hierarchy, presenting a map to the robot in a similar manner to that in which people use maps [8].”
BackgroundMethodSpace Subdivision in Indoor Mobile Laser Scanning Point Clouds Based on Scanline Analysis
Zheng, Y., Peter, M., Zhong, R., Elberink, S. O., & Zhou, Q. (2018) · Sensors · doi:10.3390/s18061838 · Cited by 22
“To date, several algorithms have been proposed for the detection of indoor objects based on trajectories [4,12,13].”
MethodRobot In a Room: Toward Perfect Object Recognition in Closed EnvironmentsPreprint
Song, S., Zhang, L., & Xiao, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.02703 · Cited by 32
“There are also several seminal works in image domain as well [3, 2, 6, 15].”
BackgroundSISE: Self-Updating of Indoor Semantic Floorplans for General Entities
Teng, X., Guo, D., Guo, Y., Zhao, X. H., & Liu, Z. (2018) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2018.2812752 · Cited by 19
“Recently, many approaches have been proposed to label or infer semantics for some places and entities in indoor space [7], [8], [9], [10], [11], [12], [13].”
BackgroundMethodInfluentialIT-HYFAO-VO: Interpretation Tree-Based VO With Hybrid Feature Association and Optimization
Sun, Q., Yuan, J., & Zhang, X. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3107058 · Cited by 17
3D Detection and Pose Estimation of Vehicle in Cooperative Vehicle Infrastructure System
Guo, E., Chen, Z., Rahardja, S., & Yang, J. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3101497 · Cited by 19
Marchesi, G., Eichhorn, C., Plecher, D. A., Itoh, Y., & Klinker, G. (2021) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi10110772 · Cited by 16
CubeSLAM: Monocular 3D Object Detection and SLAM without Prior ModelsPreprint
Yang, S., & Scherer, S. (2018) · arXiv (Cornell University) · Cited by 15
Context-coherent scenes of objects for camera pose estimation
Li, J., Meger, D., & Dudek, G. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202221 · Cited by 14
“that perform pose estimation using objects as a feature representation [12], [13], [14], [15], [16], [17].”
MethodProbabilistic Projective Association and Semantic Guided Relocalization for Dense Reconstruction
Yang, S. Y., Kuang, Z., Cao, Y., Lai, Y., & Hu, S. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794299 · Cited by 14
“Starting from primitive-level geometric entities such as lines [15], [16] and planes [17], [18], coarse [19], [20] and fine [21] representations for instances extracted by recent object detection [22] and segmentation [7] approaches are also introduced nowadays for establishing a richer representation of the involved…”
BackgroundDSOD: DSO in Dynamic Environments
Ma, P., Bai, Y. L., Zhu, J., Wang, C., & Peng, C. (2019) · IEEE Access · doi:10.1109/access.2019.2958374 · Cited by 14
“[23] estimate camera pose by exploiting geometric and semantic information about the frame simultaneously to improve the accuracy of object recognition.”
BackgroundOnline Object Detection and Localization on Stereo Visual SLAM System
Pire, T., Corti, J., & Grinblat, G. L. (2019) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-019-01074-2 · Cited by 18
“In [1] the authors raise the problem of Semantic Structure from Motion (SSfM), where the theoretical framework for the simultaneous adjustment of points, surfaces and objects is defined from two views of the same scene.”
BackgroundLp-slam: language-perceptive RGB-D SLAM framework exploiting large language model
Zhang, W., Guo, Y., Niu, L., Li, P., Wan, Z., Shao, F., Nian, C., Farrukh, F. U. D., Zhang, D., Zhang, C., et al. (2024) · Complex & Intelligent Systems · doi:10.1007/s40747-024-01408-0 · Cited by 20
“In addition, semantic information was used to estimate the camera pose in work [20].”
Robust Visual Teach and Repeat for UGVs Using 3D Semantic Maps
Mahdavian, M., Yin, K., & Chen, M. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3189165 · Cited by 10
Simultaneous dense scene reconstruction and object labeling
Asif, U., Bennamoun, M., & Sohel, F. A. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487374 · Cited by 11
“, [15], [16]) proposes a SLAM-based object recognition approach, where SLAM and object recognition are jointly addressed in a unified architecture.”
BackgroundBao, Y., Pan, Y., Yang, Z., & Huan, R. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3091396 · Cited by 13
“Bao et al. [35] jointly utilize geometry and semantic attributes in the scene to model various types of interactions between scene components, which significantly improves the object recognition accuracy.”
BackgroundSemantic Scene Models for Visual Localization under Large Viewpoint Changes
Li, J., Xu, Z., Meger, D., & Dudek, G. (2018) · Canadian Conference on Computer and Robot Vision · doi:10.1109/crv.2018.00033 · Cited by 10
“We have been inspired by several of the geometric constructions, especially Bao’s relation between image bounding boxes to 3D cuboids [12].”
BackgroundMethodLifting GIS maps into strong geometric context for scene understanding
Diaz, R., Lee, M., Schubert, J. E., & Fowlkes, C. C. (2016) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2016.7477647 · Cited by 11
Tian, Y., Zhang, J., Zhang, Z., & Wu, J. (2024) · IEEE Access · doi:10.1109/access.2024.3396148 · Cited by 10
“Bao et al. [23] introduced a semantic motion recovery structure algorithm that enhances algorithm robustness through the recognition and estimation of high-level semantic information such as regions and objects in the 3D scene.”
BackgroundCoarse Semantic-Based Motion Removal for Robust Mapping in Dynamic Environments
Wang, S., Lv, X., Li, J., & Ye, D. (2020) · IEEE Access · doi:10.1109/access.2020.2989317 · Cited by 9
“Bao applied the geometric properties and semantic properties of the surrounding scene to estimate the camera pose, map points, and objects jointly [32].”
MethodXiang, G., Dian, S., Zhao, N., & Wang, G. (2023) · Sensors · doi:10.3390/s23031125 · Cited by 7
Jose Oramas, M., Raedt, L. D., & Tuytelaars, T. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.43 · Cited by 7
“Recently, methods related to structure from motion such as [1, 2, 12, 37] aim at understanding the full scene layout.”
MethodStereo Vision-based Semantic 3D Object and Ego-motion Tracking for Autonomous DrivingPreprint
Li, P., Qin, T., & Shen, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.02062 · Cited by 11
“In [16, 17], the authors propose a semantic structure from motion (SfM) approach to jointly estimate camera, object with considering scene components interaction.”
BackgroundSalas-Moreno, R. F., Glocker, B., Kelly, P. H. J., & Davison, A. J. (2014) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2014.6948492 · Cited by 6
Homography‐based traffic sign localisation and pose estimation from image sequence
Cui, Z., Liu, Y., & Ren, F. (2019) · IET Image Processing · doi:10.1049/iet-ipr.2019.0023 · Cited by 6
“[20] proposed a semantic SfM method for both recognition and localisation of points, regions and objects.”
MethodIncremental multi-view object detection from a moving camera
Konno, T., Amma, A., & Kanezaki, A. (2021) · ACM International Conference on Multimedia in Asia (MMAsia) · doi:10.1145/3444685.3446257 · Cited by 6
“In the context of semantic mapping, object recognition results are often utilized to improve the performance of structure from motion (SfM) [3, 4, 19], bundle adjustment [16, 17], and simultaneous localization and mapping (SLAM) [18, 31, 37, 42].”
MethodRobust Improvement in 3D Object Landmark Inference for Semantic Mapping
Lin, X., Yang, Y., He, L., Chen, W., Guan, Y., & Zhang, H. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561596 · Cited by 5
“…dense points and texture, 3D enveloping surface inferred from 2D object bounding boxes in multiple views is widely adopted to represent object [4, 5], due to its potential to tremendously reduce the amount of data while maintaining essential information such as semantic label, dimensions and…”
BackgroundA comparative study of video-based object recognition from an egocentric viewpoint
Shao, M., Tang, D., Liu, Y., & Kim, T. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.07.023 · Cited by 7
Object-Oriented 3D Semantic Mapping Based on Instance Segmentation
Chi, J., Wu, H., & Tian, G. (2019) · Journal of Advanced Computational Intelligence and Intelligent Informatics · doi:10.20965/jaciii.2019.p0695 · Cited by 5
Object detection and depth estimation for 3D trajectory extraction
Boukhers, Z., Shirahama, K., Li, F., & Grzegorzek, M. (2015) · International Workshop on Content-Based Multimedia Indexing (CBMI) · doi:10.1109/cbmi.2015.7153632 · Cited by 4
“In addition, in [16], [17], SfM and SLAM are combined with object detection to obtain 3D positions of objects.”
An overview of recent progress in volumetric semantic 3D reconstruction
Häne, C., & Pollefeys, M. (2016) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2016.7900143 · Cited by 5
Wang, T., Zhang, Y., Zhang, Y., Ying, Y., Li, L., Liu, S., Zhao, X., Zhang, Z., & Wang, L. (2022) · Remote Sensing · doi:10.3390/rs14153521 · Cited by 4
Robust Shape Registration using Fuzzy CorrespondencesPreprint
Kolagunda, A., Sorensen, S., Saponaro, P. J., Treible, W., & Kambhamettu, C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.05664 · Cited by 4
Structure Aware SLAM using Quadrics and PlanesPreprint
Hosseinzadeh, M., Latif, Y., Pham, T. T., Suenderhauf, N., & Reid, I. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.09111 · Cited by 14
“Unlike previous work such as [8] we explicitly target real-time performance, and integrate within an online SLAM framework.”
BackgroundMethodInfluentialFirst-person indoor navigation via vision-inertial data fusion
Farnoosh, A., Nabian, M., Closas, P., & Ostadabbas, S. (2018) · IEEE/ION Position Location and Navigation Symposium · doi:10.1109/plans.2018.8373507 · Cited by 3
“In another work [22], authors proposed an approach to jointly estimate camera poses, recover scene points and recognize object labels by modeling various types of interactions between scene components and using both geometric and semantic attributes in the scene.”
BackgroundRe-localization for Self-Driving Cars using Semantic Maps
Kenye, L., Palugulla, R., Arora, M., Bhat, B., Kala, R., & Nayak, A. (2020) · IEEE International Conference on Robotic Computing (IRC) · doi:10.1109/irc.2020.00018 · Cited by 3
“Bao et al. [9] presented an approach of performing structure from motion along with the localization of the detected objects.”
BackgroundYige, 赵. Z., Kai, 王. W., Xiaoke, 张. Z., Chen, 杨. Y., & Hui, 陈. C. (2025) · Laser & Optoelectronics Progress · doi:10.3788/lop241888 · Cited by 2
Plane Object-Based High-Level Map Representation for SLAM
Gritsenko, P., Gritsenko, I., Seidakhmet, A., & Kwolek, B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00692-1_9 · Cited by 2
Single sensor-based 3D feature point location for a small flying robot application using one camera
Shah, S. I. A., Johnson, E. N., Wu, A. D., & Watanabe, Y. (2013) · Proceedings of the Institution of Mechanical Engineers Part G Journal of Aerospace Engineering · doi:10.1177/0954410013500614 · Cited by 2
Factorization based structure from motion with object priors
Gay, P., Rubino, C., Crocco, M., & Bue, A. D. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.11.005 · Cited by 2
Visual SLAM Location Methods Based on Complex Scenes: A Review
Zhang, H., & Peng, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-57881-7_43 · Cited by 2
Interactive user experience - Effects of augmented reality on consumer psychology and behavior
Stoyanova, J. (2015) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) · doi:10.34626/9qm5-3z15 · Cited by 5
Planar Reconstruction of Indoor Scenes from Sparse Views and Relative Camera Poses
Guan, F., Liu, J., Zhang, J., Yan, L., & Jiang, L. (2024) · Remote Sensing · doi:10.3390/rs16091616 · Cited by 1
Scene-driven Cues for Viewpoint Classification for Elongated Object Classes
Jose Oramas, M., & Tuytelaars, T. (2014) · British Machine Vision Conference (BMVC) · doi:10.5244/c.28.75 · Cited by 1
Learning Visual Landmarks for Localization with Minimal Supervision
Haris, M., Franzius, M., & Bauer-Wersing, U. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-06427-2_64 · Cited by 1
GIS for Disaster Management: A System to Aid Users with Relevant Information
Aghav, S., Solanki, P., & Palwe, S. (2022) · Algorithms for intelligent systems · doi:10.1007/978-981-19-2065-3_19 · Cited by 1
Research on Active Firefighting Robot Navigation Based on the Improved AUKF Algorithm
Du, H., Li, Q., Tanglong, C., Liu, Y., Zhang, H., & Guan, Z. (2023) · Atlantis Highlights in Intelligent Systems/Atlantis highlights in intelligent systems · doi:10.2991/978-94-6463-222-4_9 · Cited by 1
Argyriou, V., del Rincón, J. M., Villarini, B., & Roche, A. (2015) · Wiley eBooks · doi:10.1002/9781118702451.ch6 · Cited by 1
Object Oriented Structure from Motion: Can a Scribble Help?
Rahal, R., Asmar, D., Shammas, E., & Ghanem, B. (2018) · International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP) · doi:10.5220/0006596005410548 · Cited by 1
“(Bao et al., 2012) introduced what is known as semantic SfM.”
BackgroundNakajima, Y., & Saitô, H. (2018) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2017mvp0024 · Cited by 1
“Bao et al. [16], [17] proposed other multi-view-based recognition methods that jointly estimate camera pose, 3D points, and object regions by expanding the Structure-from-Motion (SfM) framework.”
BackgroundSeMap: automatic semantic mapping for large-scale indoor spaces
Teng, X., Guo, D., Guo, Y., Xu, P., Meng, Y., Hu, R., Chai, H., & Liu, Z. (2019) · ACM SIGSPATIAL Workshop Recommendations for Location-based Services and Social Networks · doi:10.1145/3356994.3365500 · Cited by 1
“Although great progress has been achieved in recent years, semantic mapping remains a challenging problem due to scene appearance variations caused by semantic diversity and complexity of large-scale indoor spaces [2, 3, 7].”
BackgroundVisual Localization and Mapping with Hybrid SFA
Haris, M., Franzius, M., Bauer-Wersing, U., & Karanam, S. K. K. (2020) · Conference on Robot Learning · Cited by 1
Lifting GIS Maps into Strong Geometric Context.Preprint
Díaz, R. A., Lee, M., Schubert, J. E., & Fowlkes, C. C. (2015) · arXiv (Cornell University) · Cited by 1
Direction-Aware Semi-Dense SLAMPreprint
Straub, J., Cabezas, R., Leonard, J. J., & Fisher, J. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.05774 · Cited by 3
“Bao et al. [3, 2] jointly estimate object and region segmentation of a sparse point cloud in the batch structure from motion framework.”
Structure recovery from uncalibrated images using iterative perspective factorization
Qiu, S., & Wen, G. (2016) · Optik · doi:10.1016/j.ijleo.2016.04.007
Context-based object viewpoint estimation: A 2D relational approach
Jose Oramas, M., Raedt, L. D., & Tuytelaars, T. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.04.005
“still images and does not require image sequences as in some SfM-based approaches [3, 4].”
MethodFuzzy Correspondences for Robust Shape Registration
Kolagunda, A., Sorensen, S., Saponaro, P. J., Treible, W., Kambhamettu, C., & Sherbondy, K. D. (2017) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00083
“To test our method we have manually identified vehicle models in the Ford Campus Dataset by looking through the photographs that are provided with the LiDAR scans.”
BackgroundMethodInfluential
Metadata
- Authors
- Bao, S. Y., Bagra, M., Chao, Y.-W., & Savarese, S.
- Affiliations
- University of Michigan (US)
- ORCID
- Yu-Wei Chao
- Venue
- 2012 IEEE Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2012-06
- Pages
- 2703-2710
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W1997452076
- Cited by
- 107
- References
- 35
- Citations by year
- 2025: 3 · 2024: 3 · 2023: 3 · 2022: 9 · 2021: 7 · 2020: 9 · 2019: 17 · 2018: 14 · 2017: 10 · 2016: 9 · 2015: 10 · 2014: 5 · 2013: 7 · 2012: 1
- FWCI
- 5.22
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Funders
- National Science Foundation
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Keywords
- structure from motion, 3D object recognition, camera pose estimation, multi-view images
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.