Semantic structure from motion with points, regions, and objects
The DOI matches its Crossref record; 34 of the 35 references it cites were found in a registry or an index.
Abstract
Structure from motion (SFM) aims at jointly recovering the structure of a scene as a collection of 3D points and estimating the camera poses from a number of input images. In this paper we generalize this concept: not only do we want to recover 3D points, but also recognize and estimate the location of high level semantic scene components such as regions and objects in 3D. As a key ingredient for this joint inference problem, we seek to model various types of interactions between scene components. Such interactions help regularize our solution and obtain more accurate results than solving these problems in isolation. Experiments on public datasets demonstrate that: 1) our framework estimates camera poses more robustly than SFM algorithms that use points only; 2) our framework is capable of accurately estimating pose and location of objects, regions, and points in the 3D scene; 3) our framework recognizes objects and regions more accurately than state-of-the-art single image recognition methods.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recently, the limitations of purely geometric maps have been recognized and this has spawned a significant and ongoing body of work in semantic mapping of environments”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [9]
In the citing paper
“in order to enhance robot's autonomy and robustness, facilitate more complex tasks (e.g. avoid muddy-road while driving), move from path-planning to task-planning, and enable advanced human-robot interaction [9, 26, 217]”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [9]
In the citing paper
“In their work, the authors demonstrate the improved object recognition performance and robustness”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [9]
In the citing paper
“at the cost of a run-time of 20 minutes per image-pair”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [9]
In the citing paper
“Bao et al. [9] propose one of the first approaches to jointly estimate camera parameters, scene points and object labels using both geometric and semantic attributes in the scene.”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [9]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · cited 2×
“For instance, points do not carry semantic information, but can be robustly matched across views (e.g. by [22]).”
BackgroundOptimization by Simulated Annealing
Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983) · Science · doi:10.1126/science.220.4598.671 · Cited by 45,134 · cited 1×
“We use simulated annealing [16] to search for the solution for Eq.”
MethodObject Detection with Discriminatively Trained Part-Based Models
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.167 · Cited by 9,853 · cited 2×
“However, [3] has a few limitations: 1) it focuses on modeling " structured " objects (e.g. cars, cups, and bottles), and it ignores " amorphous " scene components (e.g. road, sky, and table surfaces which can be hardly identified by standard object detectors such as [8] [21]).”
BackgroundBundle Adjustment — A Modern Synthesis
Triggs, B., McLauchlan, P. F., Hartley, R. I., & Fitzgibbon, A. W. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-44480-7_21 · Cited by 3,988 · cited 1×
“Most SFM algorithms [29] [28] represent the structure of a scene as a set of 3D points.”
BackgroundScale & Affine Invariant Interest Point Detectors
Mikolajczyk, K., & Schmid, C. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000027790.02288.f2 · Cited by 3,436
Modeling the World from Internet Photo Collections
Snavely, N., Seitz, S. M., & Szeliski, R. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0107-3 · Cited by 2,155 · cited 2×
“[28] [3] Table 3: The error for camera pose estimation.”
BackgroundAn efficient solution to the five-point relative pose problem
Nister, D. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.17 · Cited by 2,009 · cited 1×
“We follow the evaluation criteria in [24].”
Learning a classification model for segmentation
Ren, & Malik (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238308 · Cited by 1,765 · cited 1×
“The region measurements are segmented regions (e.g. by segmentation algorithms such as [26] [33]).”
Segmentation and Recognition Using Structure from Motion Point Clouds
Brostow, G. J., Shotton, J., Fauqueur, J., & Cipolla, R. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88682-2_5 · Cited by 1,050 · cited 1×
“Among these works (e.g. [12] [15] [11] [25] [10] [20] [6] [13]), most assume that only a single image is available, whereas a handful of them model this problem with multiple images.”
BackgroundCombined Object Categorization and Segmentation With An Implicit Shape Model
Leibe, B., Leonardis, A., & Schiele, B. (2004) · Max Planck Digital Library · Cited by 890
Cited by
Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
“muddy-road while driving), move from path-planning to taskplanning, and enable advanced human-robot interaction [10],”
Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments
Yu, C., Liu, Z., Liu, X.-J., Xie, F., Yang, Y., Wei, Q., & Fei, Q. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593691 · Cited by 1,154
“[14] first try to estimate camera pose, points, and objects jointly utilizing both geometry and semantic attributes in the scene, which significantly improve the object recognition accuracy.”
BackgroundSLAM++: Simultaneous Localisation and Mapping at the Level of Objects
Salas-Moreno, R. F., Newcombe, R. A., Strasdat, H., Kelly, P. H. J., & Davison, A. J. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.178 · Cited by 945
“[3] represented a scene as a set of points, objects and regions in two-view SfM, solving ex-pensively and jointly in a graph optimisation for a labelling and reconstruction solution taking account of interactions between all scene entities.”
BackgroundPL-SLAM: Real-time monocular visual SLAM with points and lines
Pumarola, A., Vakhitov, A., Agudo, A., Sanfeliu, A., & Moreno-Noguer, F. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989522 · Cited by 482
“Scene prior information have been also exploited to provide a significant boost to SLAM systems [3], [4].”
Method
Metadata
- Authors
- Bao, S. Y., Bagra, M., Chao, Y.-W., & Savarese, S.
- Affiliations
- University of Michigan (US)
- ORCID
- Yu-Wei Chao
- Venue
- 2012 IEEE Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2012-06
- Pages
- 2703-2710
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W1997452076
- Cited by
- 107
- References
- 35
- Citations by year
- 2025: 3 · 2024: 3 · 2023: 3 · 2022: 9 · 2021: 7 · 2020: 9 · 2019: 17 · 2018: 14 · 2017: 10 · 2016: 9 · 2015: 10 · 2014: 5 · 2013: 7 · 2012: 1
- FWCI
- 5.22
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Funders
- National Science Foundation
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Keywords
- structure from motion, 3D object recognition, camera pose estimation, multi-view images
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Semantic structure from motion with points, regions, and objects. https://app.cochat.ai/papers/10.1109/cvpr.2012.6247992
- The paper
- Bao, S. Y., Bagra, M., Chao, Y.-W., & Savarese, S. (2012). Semantic structure from motion with points, regions, and objects. 2012 IEEE Conference on Computer Vision and Pattern Recognition. https://doi.org/10.1109/cvpr.2012.6247992
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.