Semantic octree: Unifying recognition, reconstruction and representation via an octree constrained higher order MRF
The DOI matches its Crossref record; 27 of the 30 references it cites were found in a registry or an index.
Abstract
On the one hand, mainly within the computer vision community, multi-resolution image labelling problems with pixel, super-pixel and object levels, have made great progress towards the modelling of holistic scene understanding. On the other hand, mainly within the robotics and graphics communities, multi-resolution 3Drepresentations of the world have matured to be efficient and accurate. In this paper we bring together the two hands and move towards the new direction of unified recognition, reconstruction and representation. We tackle the problem by embedding an octree into a hierarchical robust PNMarkov Random Field. This allows us to jointly infer the multi-resolution 3Dvolume along with the object-class labels, all within the constraints of an octree data-structure. The octree representation is chosen as this data-structure is efficient for further processing such as dynamic updates, data compression, and surface reconstruction. We perform experiments in inferring our semantic octree on the The kitti Vision Benchmark Suite in order to demonstrate its efficacy.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“a similar idea was proposed earlier by Sengupta et al. [219] using stereo cameras.”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [219]
In the citing paper
“It should be noted that [147] and [219] focus only on the mapping part and they do not refine the early computed poses in this late stage.”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [219]
In the citing paper
“Although some progress has been done in terms of temporal fusion of, for instance, per frame semantic evidence [219, 251], the problem of consistently fusing several sources of semantic information with metric information coming at different points in time is still open.”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [219]
Citations
Cites
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897
Fast approximate energy minimization via graph cuts
Boykov, Y., Veksler, O., & Zabih, R. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.969114 · Cited by 6,988
Stereo Processing by Semiglobal Matching and Mutual Information
Hirschmuller, H. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1166 · Cited by 4,195
OctoMap: an efficient probabilistic 3D mapping framework based on octrees
Hornung, A., Wurm, K. M., Bennewitz, M., Stachniss, C., & Burgard, W. (2013) · Autonomous Robots · doi:10.1007/s10514-012-9321-0 · Cited by 2,866 · cited 7×
“These values are observed to be sufficient for outdoor scene experiments [6].”
BackgroundMethodInfluentialGeometric modeling using octree encoding
Meagher, D. (1982) · Computer Graphics and Image Processing · doi:10.1016/0146-664x(82)90104-6 · Cited by 1,266 · cited 3×
“Here we choose to exploit a D data-structure called the octree [5], which was introduced into graphics over 30 years ago, and is still of major interest till date.”
MethodStereoScan: Dense 3d reconstruction in real-time
Geiger, A., Ziegler, J., & Stiller, C. (2011) · 2011 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2011.5940405 · Cited by 1,089 · cited 1×
“To estimate the camera matrix, we follow the approach presented in [24].”
MethodShotton, J., Winn, J., Rother, C., & Criminisi, A. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-007-0109-1 · Cited by 1,026 · cited 4×
“The need for higher order information (information obtained from more than 2 elementary units) was identified early on in this area by TextonBoost [9].”
BackgroundMethodInfluentialRobust Higher Order Potentials for Enforcing Label Consistency
Kohli, P., Ladický, L., & Torr, P. H. S. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-008-0202-0 · Cited by 853 · cited 6×
“In order to perform recognition we follow the state-of-the-art line of work in structured prediction [2], [1], [7], which was propelled by the advent of efficient maximum a posteriori (MAP) estimation”
BackgroundMethodInfluentialAssociative hierarchical CRFs for object class image segmentation
Ladicky, L., Russell, C., Kohli, P., & Torr, P. H. S. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459248 · Cited by 596 · cited 10×
“In order to perform recognition we follow the state-of-the-art line of work in structured prediction [2], [1], [7], which was propelled by the advent of efficient maximum a posteriori (MAP) estimation”
BackgroundMethodInfluentialOctrees for faster isosurface generation
Wilhelms, J., & Van Gelder, A. (1992) · ACM Transactions on Graphics · doi:10.1145/130881.130882 · Cited by 569 · cited 2×
“Moreover the octree based representation is flexible and can be extended to generate a mesh of the scene [20] if desired.”
BackgroundWhat, Where and How Many? Combining Object Detectors and CRFs
Ladický, Ľ., Sturgess, P., Alahari, K., Russell, C., & Torr, P. H. S. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_31 · Cited by 310 · cited 1×
“The AHCRF framework is very general and has been used to incorporate many other types of higher order information, such as bounding box detectors [11] and co-occurrence statistics [7].”
MethodSelf-citationJoint Semantic Segmentation and 3D Reconstruction from Monocular Video
Kundu, A., Li, Y., Dellaert, F., Li, F., & Rehg, J. M. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10599-4_45 · Cited by 220 · cited 3×
“octrees’ shear diversity that drives us to investigate its use as a semantic representation, but also its proven practical use in our particular application of D modelling for both indoor [14] and outdoor [12], [15], [13] scenes.”
BackgroundMethodOctree-based fusion for realtime 3D reconstruction
Zeng, M., Zhao, F., Zheng, J., & Liu, X. (2012) · Graphical Models · doi:10.1016/j.gmod.2012.09.002 · Cited by 143 · cited 1×
“data compression [19], generating isosurfaces [20], and D data fusion [21], plus many more 1.”
BackgroundUrban 3D semantic modelling using stereo vision
Sengupta, S., Greveson, E., Shahrokni, A., & Torr, P. H. S. (2013) · 2013 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2013.6630632 · Cited by 142 · cited 6×
“We evaluate our method quantitatively as shown in table I where we compare the results from [18] and [17].”
BackgroundMethodInfluentialSelf-citationMesh Based Semantic Modelling for Indoor and Outdoor Scenes
Valentin, J. P. C., Sengupta, S., Warrell, J., Shahrokni, A., & Torr, P. H. S. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.269 · Cited by 116 · cited 4×
“We evaluate our method quantitatively as shown in table I where we compare the results from [18] and [17].”
BackgroundMethodInfluentialSelf-citation3D Scene Understanding by Voxel-CRF
Kim, B.-S., Kohli, P., & Savarese, S. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.180 · Cited by 115 · cited 4×
“D scene, and also against other popular representations, including fixed size voxel grids [3], point clouds, elevation maps [16].”
BackgroundMethodInfluentialP³ & Beyond: Move Making Algorithms for Solving Higher Order Functions
Kohli, P., Kumar, M. P., & Torr, P. H. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.217 · Cited by 102 · cited 2×
“Setting γ l c to zero has the effect of penalising inconsistent grouping of voxels and thus enforcing label consistency as in the P model [10].”
MethodHigh efficiency video coding: the next frontier in video compression [Standards in a Nutshell]
Ohm, J., & Sullivan, G. J. (2013) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2219672 · Cited by 91 · cited 1×
“data compression [19], generating isosurfaces [20], and D data fusion [21], plus many more 1.”
BackgroundImageSpirit: Verbal Guided Image Parsing
Cheng, M., Zheng, S., Lin, W.-Y., Vineet, V., Sturgess, P., Crook, N. T., Mitra, N. J., & Torr, P. H. S. (2014) · ACM Transactions on Graphics · doi:10.1145/2682628 · Cited by 72
Self-citationInference Methods for CRFs with Co-occurrence Statistics
Ladický, Ľ., Russell, C., Kohli, P., & Torr, P. H. S. (2012) · International Journal of Computer Vision · doi:10.1007/s11263-012-0583-y · Cited by 52
Nonparametric semantic segmentation for 3D street scenes
He, H., & Upcroft, B. (2013) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2013.6696884 · Cited by 45 · cited 3×
“octrees’ shear diversity that drives us to investigate its use as a semantic representation, but also its proven practical use in our particular application of D modelling for both indoor [14] and outdoor [12], [15], [13] scenes.”
BackgroundMethodImageSpirit: Verbal Guided Image Parsing
Cheng, M., Zheng, S., Lin, W.-Y., et al. (2013) · Radar (Oxford Brookes University) · doi:10.48550/arxiv.1310.4389 · Cited by 32
Hierarchies of octrees for efficient 3D mapping
Wurm, K. M., Hennes, D., Holz, D., Rusu, R. B., Stachniss, C., Konolige, K., & Burgard, W. (2011) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2011.6094571 · Cited by 31 · cited 2×
“octrees’ shear diversity that drives us to investigate its use as a semantic representation, but also its proven practical use in our particular application of D modelling for both indoor [14] and outdoor [12], [15], [13] scenes.”
BackgroundStixmentation - Probabilistic Stixel based Traffic Scene Labeling
Erbs, F., Schwarz, B., & Franke, U. (2012) · British Machine Vision Conference (BMVC) · doi:10.5244/c.26.71 · Cited by 28 · cited 1×
“D scene, and also against other popular representations, including fixed size voxel grids [3], point clouds, elevation maps [16].”
BackgroundHierarchies of octrees for efficient 3D mapping
Wurm, K. M., Hennes, D., Holz, D., Rusu, R. B., Stachniss, C., Konolige, K., & Burgard, W. (2011) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2011.6048189 · Cited by 23
Scalable Cascade Inference for Semantic Image Segmentation
Sturgess, P., Ladický, Ľ., Crook, N., & Torr, P. H. S. (2012) · British Machine Vision Conference (BMVC) · doi:10.5244/c.26.62 · Cited by 12 · cited 1×
“would be to leverage the flexibility of the AHCRF model by exploring other higher order information, such as label counts, co-occurrence, whole object detectors, symmetry, topological constraints and scalability [25] for our semantic octree.”
MethodSelf-citationSemantic 3D Octree Maps based on Conditional Random Fields
Lang, D., Friedmann, S., & Paulus, D. W. R. (2013) · Machine Vision and Applications · Cited by 12
Cited by
Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
“As mentioned in Section II data association matches each measurement to the portion of the state the measurement refers to.”
Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments
Yu, C., Liu, Z., Liu, X.-J., Xie, F., Yang, Y., Wei, Q., & Fei, Q. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593691 · Cited by 1,154
“[12], which embedded an octo-tree into a hierarchical robust Markov Random Filed to provide each voxel in the map with the semantic label.”
MethodRecent trends in social aware robot navigation: A survey
Charalampous, K., Kostavelis, I., & Gasteratos, A. (2017) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2017.03.002 · Cited by 168
Building and optimization of 3D semantic map based on Lidar and camera fusion
Li, J., Zhang, X., Li, J., Liu, Y., & Wang, J. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.06.004 · Cited by 114
Metadata
- Authors
- Sengupta, S., & Sturgess, P.
- Affiliations
- Oxford Brookes University (GB)
- Venue
- 2015 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2015-05
- Pages
- 1874-1879
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W1595654559
- Cited by
- 54
- References
- 30
- Citations by year
- 2026: 1 · 2024: 4 · 2023: 3 · 2022: 8 · 2021: 6 · 2020: 3 · 2019: 5 · 2018: 10 · 2017: 6 · 2016: 5 · 2015: 3
- FWCI
- 14.71
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- Markov random fields, object recognition, surface reconstruction, data compression, KITTI dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.