Manhattan scene understanding using monocular, stereo, and 3D features
The DOI matches its Crossref record; 16 of the 17 references it cites were found in a registry or an index.
Abstract
This paper addresses scene understanding in the context of a moving camera, integrating semantic reasoning ideas from monocular vision with 3D information available through structure-from-motion. We combine geometric and photometric cues in a Bayesian framework, building on recent successes leveraging the indoor Manhattan assumption in monocular vision. We focus on indoor environments and show how to extract key boundaries while ignoring clutter and decorations. To achieve this we present a graphical model that relates photometric cues learned from labeled data, stereo photo-consistency across multiple views, and depth cues derived from structure-from-motion point clouds. We show how to solve MAP inference using dynamic programming, allowing exact, global inference in ~100 ms (in addition to feature computation of under one second) without using specialized hardware. Experiments show our system out-performing the state-of-the-art.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The online system of Flint et al. [79] presents a model that leverages the Manhattan world assumption to segment the map in the main planes in indoor scenes.”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [79]
In the citing paper
“Prior information about a scene has increasingly been shown to provide a significant boost to SLAM systems.”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [79]
Citations
Cites
A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms
Scharstein, D., & Szeliski, R. (2002) · International Journal of Computer Vision · doi:10.1023/a:1014573219977 · Cited by 6,740
Parallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346
Self-citationSegmentation and Recognition Using Structure from Motion Point Clouds
Brostow, G. J., Shotton, J., Fauqueur, J., & Cipolla, R. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88682-2_5 · Cited by 1,050
Scene Text Recognition using Higher Order Language Priors
Mishra, A., Alahari, K., & Jawahar, C. (2012) · Procedings of the British Machine Vision Conference 2012 · doi:10.5244/c.26.127 · Cited by 831
Manhattan World: compass direction from a single image by Bayesian inference
Coughlan, J. M., & Yuille, A. L. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.790349 · Cited by 475
Geometric reasoning for single image structure recovery
Lee, D. C., Hebert, M., & Kanade, T. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206872 · Cited by 407
Furukawa, Y., Curless, B., Seitz, S. M., & Szeliski, R. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206867 · Cited by 338
Blocks World Revisited: Image Understanding Using Qualitative Geometry and Mechanics
Gupta, A., Efros, A. A., & Hebert, M. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_35 · Cited by 308
Košecká, J., & Zhang, W. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-47979-1_32 · Cited by 293
Combining Appearance and Structure from Motion Features for Road Scene Understanding
Sturgess, P., Alahari, K., Ladicky, L., & Torr, P. H. S. (2009) · Procedings of the British Machine Vision Conference 2009 · doi:10.5244/c.23.62 · Cited by 189
Research works printed but not linked: 2
- roberts, 1965, Machine Perception of 3-D Solids
- flint, 2009, Learning textons for realtime scene context, Proc IEEE Conf Computer Vision and Pattern Recognition
Cited by
Items 101–131 of 131
Help Me Through: Imitation Learning Based Active View Planning to Avoid SLAM Tracking Failures
Naveed, K., Hussain, W., Hussain, I., Lee, D., & Anjum, M. L. (2025) · IEEE Transactions on Robotics · doi:10.1109/tro.2025.3582817 · Cited by 6
“Salas et al. [28] suggested a high-level scene layout understanding [31], [32] to account for such failures.”
First-person indoor navigation via vision-inertial data fusion
Farnoosh, A., Nabian, M., Closas, P., & Ostadabbas, S. (2018) · IEEE/ION Position Location and Navigation Symposium · doi:10.1109/plans.2018.8373507 · Cited by 3
“Authors in [21] presented an online system for recovering the structure of indoor scenes in the context of a moving camera by leveraging the Manhattan world assumption.”
BackgroundSemantic Visual Understanding of Indoor Environments: From Structures to Opportunities for Action
Tsai, G., Johnson, C., & Kuipers, B. J. (2014) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2014.61 · Cited by 3
“Other methods [8, 6, 7] use the Manhattan-world assumption to reconstruct a planar model of an indoor environment from a collection of images.”
MethodRoom layout estimation on mobile devices
Angladon, V. (2018) · theses.fr (ABES) · doi:10.70675/16e4b911z377bz47d8zac58z5a0fea7f57d8 · Cited by 4
Surface Layout Estimation Using Multiple Segmentation Methods and 3D Reasoning
Hödlmoser, M., & Mičušík, B. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-38628-2_5 · Cited by 2
“Surface labeling is also obtained by using a point cloud, as proposed in [9, 8, 10].”
Method3D Spatial Layout Propagation in a Video Sequence
Rituerto, A., Manduchi, R., Murillo, A. C., & Guerrero, J. J. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-11755-3_42 · Cited by 2
“[5] combined geometric and photometric cues to obtain their scene model from a moving camera.”
BackgroundMethodPlane Object-Based High-Level Map Representation for SLAM
Gritsenko, P., Gritsenko, I., Seidakhmet, A., & Kwolek, B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00692-1_9 · Cited by 2
3D Layout Propagation to Improve Object Recognition in Egocentric Videos
Rituerto, A., Murillo, A. C., & Guerrero, J. J. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16199-0_58 · Cited by 2
“[18] combined geometric and photometric cues to obtain their scene model from a moving camera.”
BackgroundScaled layout recovery with wide field of view RGB-D
Pérez-Yus, A., Nicolás, G. L., & Guerrero, J. J. (2019) · Image and Vision Computing · doi:10.1016/j.imavis.2019.04.008 · Cited by 2
Yang, G., & Liu, P. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5123187 · Cited by 2
Handling perceptual clutter for robot vision with partial model-based interpretations
Tsai, G., & Kuipers, B. J. (2014) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2014.6942802 · Cited by 1
“A more concise, large-granularity model that would be useful to a robot in planning must then be constructed from the point cloud.”
BackgroundPlane Extraction for Indoor Place Recognition
Potena, C., Pretto, A., Bloisi, D. D., & Nardi, D. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-25903-1_46 · Cited by 1
“Indeed, with this kind of geometric model, it is possible to reconstruct the indoor world, as done in [13], or make a decomposition of the scene, as described in [6,7].”
Efficient Pose Tracking based on Line Segment Matching
Wang, J., Cui, J., Gao, Y., Ma, Y., Gan, Z., & Yang, C. (2019) · American Journal of Engineering and Applied Sciences · doi:10.3844/ajeassp.2019.1.9 · Cited by 1
“This bears certain similarities to a few works such as (Taylor and Kriegman, 1994; Drummond and Cipolla, 2002; Rosten and Drummond, 2005; Flint et al. , 2011).”
BackgroundPredictive and Semantic Layout Estimation for Robotic Applications in Manhattan WorldsPreprint
Shariati, A., Pfrommer, B., & Taylor, C. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.07442 · Cited by 1
“Among the first of these approaches are [4] and [18] which use a Bayesian filtering framework to update a model of the scene based on observations provided by some state-estimation subsystem.”
BackgroundDeveloping a slam-based backpack mobile mapping system for indoor mapping
Karam, S. (2021) · doi:10.3990/1.9789036552561 · Cited by 1
Creative Transformations of Personal Photographs
Wu, Y., Seshadrinathan, K., Sun, W. M., Choubassi, M. E., Ratcliff, J., & Kozintsev, I. V. (2012) · IEEE International Conference on Multimedia and Expo workshops · doi:10.1109/icmew.2012.87
“We assume urban and indoor environments that typically consist of building facades, ground and ceiling can be approximated using a set of planar structures [8].”
BackgroundFast Binocular Depth Inference via Bidirectional Motion Based Interpolation
Geng, W., Yang, Y., Ju, R., Ren, T., & Wu, G. (2014) · International Conference on Internet Multimedia Computing and Service · doi:10.1145/2632856.2632872
Combining recognition and geometry for data-driven 3D reconstruction
Owens, A. (2013) · DSpace@MIT (Massachusetts Institute of Technology)
Indoor Frame Recovery from Refined Line SegmentsPreprint
Guo, L., & Chu, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.00279
On-line, Incremental Visual Scene Understanding for an Indoor Navigating Robot.
Tsai, G. (2014) · Deep Blue (University of Michigan)
A New Method for Atlanta World Frame Estimation.Preprint
Liu, Y., Knoll, A., & Chen, G. (2019) · arXiv (Cornell University)
Parsing Indoor Scenes from RGB-D Image Using Superpixel and Region Merging
He, K., Zhang, Z., Pan, X., Yang, C., Yang, S. C., Lü, X., & Chen, R. (2019) · IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA) · doi:10.1109/ispa-bdcloud-sustaincom-socialcom48970.2019.00164
“The dynamic programming algorithm is used to find the optimal labeling problem, which is similar to Flint et al. [6, 5] method.”
MethodAn unsupervised monocular image depth prediction algorithm using Fourier domain analysis
Chen, L., & Tang, X. (2022) · IET Signal Processing · doi:10.1049/sil2.12135
Robust Tracking and Localization using Stable Features of Static Planes in Dynamic Scenes
Zhang, Y., Zhang, X., Feng, S., Shen, Q., Dang, S., & Xu, K. (2023) · IEEE International Conference on Real-time Computing and Robotics (RCAR) · doi:10.1109/rcar58764.2023.10249352
“Inspired by the world of Manhattan [4], [5], we can observe that planes are ubiquitous geometric structures in natural scenes.”
BackgroundRecurrence-Based Vanishing Point Detection
Bharadwaj, S. S., Collins, R. T., & Liu, Y. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00865
“Vanishing Point Detection (VPD) plays a vital role in computer vision applications such as structure from motion [16, 23], 3D reconstruction [17, 41, 47], camera calibration [6, 9, 11, 17], scene understanding [14], SLAM [21,22,45], facade detection [27] and autonomous driving [20].”
Optimization Methods in Machine Drawings
Kulkarni, A. J. (2025) · Springer eBooks · doi:10.1007/978-981-96-5257-0_9
Yang, G., Liu, P., & Ma, W. (2025) · PLoS ONE · doi:10.1371/journal.pone.0330839
“This assumption is based on the orthogonal structural patterns of buildings and indoor environments, where 3 dominant orthogonal directions exist [14].”
Indoor 3D geometry estimation using single panorama image
Sharma, S. K., Jain, K., & Shukla, A. K. (2026) · Applied Geomatics · doi:10.1007/s12518-025-00684-w
Gohard, P.-A. (2018) · theses.fr (ABES) · doi:10.70675/d90fc3e3zd301z43d6zac80z5a16f803431f · Cited by 1
Indoor Scene Understanding using Non-Conventional Cameras
Labrador, C. F. (2020) · theses.fr (ABES) · doi:10.70675/7af47b94z536cz49cdza539zc99752cca57b · Cited by 2
PolyLayout: Multi-room Manhattan Layout Estimation
Hanning, G., Liu, S., Pautrat, R., Pollefeys, M., Åström, K., & Larsson, V. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37023-5_18
“[ 10] reconstruct indoor scenes from video sequences by combining geometric and photometric cues.”
Metadata
- Authors
- Flint, A., Murray, D., & Reid, I.
- Affiliations
- University of Oxford (GB)
- Venue
- 2011 International Conference on Computer Vision
- Publisher
- IEEE
- Published
- 2011-11
- Pages
- 2228-2235
- OpenAlex
- W2083347703
- Cited by
- 132
- References
- 17
- Citations by year
- 2026: 2 · 2025: 5 · 2023: 5 · 2022: 5 · 2021: 6 · 2020: 15 · 2019: 13 · 2018: 15 · 2017: 6 · 2016: 11 · 2015: 12 · 2014: 16 · 2013: 13 · 2012: 8
- FWCI
- 8.81
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
- Funders
- Engineering and Physical Sciences Research Council (EP/H050795/1)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Keywords
- Manhattan world assumption, scene understanding, indoor scene understanding, monocular vision, stereo vision, structure from motion, point clouds, photo-consistency, Bayesian graphical models, MAP inference, dynamic programming, semantic reasoning, boundary extraction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.