Manhattan scene understanding using monocular, stereo, and 3D features
The DOI matches its Crossref record; 16 of the 17 references it cites were found in a registry or an index.
Abstract
This paper addresses scene understanding in the context of a moving camera, integrating semantic reasoning ideas from monocular vision with 3D information available through structure-from-motion. We combine geometric and photometric cues in a Bayesian framework, building on recent successes leveraging the indoor Manhattan assumption in monocular vision. We focus on indoor environments and show how to extract key boundaries while ignoring clutter and decorations. To achieve this we present a graphical model that relates photometric cues learned from labeled data, stereo photo-consistency across multiple views, and depth cues derived from structure-from-motion point clouds. We show how to solve MAP inference using dynamic programming, allowing exact, global inference in ~100 ms (in addition to feature computation of under one second) without using specialized hardware. Experiments show our system out-performing the state-of-the-art.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The online system of Flint et al. [79] presents a model that leverages the Manhattan world assumption to segment the map in the main planes in indoor scenes.”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [79]
In the citing paper
“Prior information about a scene has increasingly been shown to provide a significant boost to SLAM systems.”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [79]
Citations
Cites
A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms
Scharstein, D., & Szeliski, R. (2002) · International Journal of Computer Vision · doi:10.1023/a:1014573219977 · Cited by 6,740
Parallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346
Self-citationSegmentation and Recognition Using Structure from Motion Point Clouds
Brostow, G. J., Shotton, J., Fauqueur, J., & Cipolla, R. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88682-2_5 · Cited by 1,050
Scene Text Recognition using Higher Order Language Priors
Mishra, A., Alahari, K., & Jawahar, C. (2012) · Procedings of the British Machine Vision Conference 2012 · doi:10.5244/c.26.127 · Cited by 831
Manhattan World: compass direction from a single image by Bayesian inference
Coughlan, J. M., & Yuille, A. L. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.790349 · Cited by 475
Geometric reasoning for single image structure recovery
Lee, D. C., Hebert, M., & Kanade, T. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206872 · Cited by 407
Furukawa, Y., Curless, B., Seitz, S. M., & Szeliski, R. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206867 · Cited by 338
Blocks World Revisited: Image Understanding Using Qualitative Geometry and Mechanics
Gupta, A., Efros, A. A., & Hebert, M. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_35 · Cited by 308
Košecká, J., & Zhang, W. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-47979-1_32 · Cited by 293
Combining Appearance and Structure from Motion Features for Road Scene Understanding
Sturgess, P., Alahari, K., Ladicky, L., & Torr, P. H. S. (2009) · Procedings of the British Machine Vision Conference 2009 · doi:10.5244/c.23.62 · Cited by 189
Research works printed but not linked: 2
- roberts, 1965, Machine Perception of 3-D Solids
- flint, 2009, Learning textons for realtime scene context, Proc IEEE Conf Computer Vision and Pattern Recognition
Cited by
Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
“jects [57], [217] or prior knowledge about the expected structure in the scene, like smoothness as in DTAM [184], Manhattan constraints as in [80], or even the expected relationships between objects [10].”
Self-citationPast, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
Self-citationTowards unified depth and semantic prediction from a single image
Wang, P., Shen, X., Lin, Z., Cohen, S., Price, B., & Yuille, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298897 · Cited by 447
“Since the two tasks are mutually beneficial, extensive investigations have been done towards jointly solving them in videos [2] [8] [19] [34], in which 3D information can be easily obtained through structure from motion.”
BackgroundNavigation Assistance for the Visually Impaired Using RGB-D Sensor With Range Expansion
Aladren, A., Lopez-Nicolas, G., Puig, L., & Guerrero, J. J. (2016) · IEEE Systems Journal · doi:10.1109/jsyst.2014.2320639 · Cited by 231
“Scene understanding has been also considered by combining geometric and photometric cues [25] or from a single omnidirectional image [26], [27].”
BackgroundLearning to Parse Wireframes in Images of Man-Made Environments
Huang, K., Wang, Y., Zhou, Z., Ding, T., Gao, S., & Ma, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00072 · Cited by 221
“In the literature, several methods have been proposed to detect line segments [45] or junctions [35, 47] in the image, and to further reason about the 3D scene geometry using the detected features [21, 11, 10, 34, 51].”
BackgroundSingle-Image Depth Estimation Based on Fourier Domain Analysis
Lee, J.-H., Heo, M., Kim, K.-R., & Kim, C.-S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00042 · Cited by 163
“Depths can be estimated from stereo images [31] or motion sequences [2, 7, 17, 31, 40], which provide relatively rich information for understanding 3D structures.”
BackgroundPop-up SLAM: Semantic monocular plane SLAM for low-texture environments
Yang, S., Song, Y., Kaess, M., & Scherer, S. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759204 · Cited by 170
“[12] formulate it as Bayesian framework using monocular and 3D features.”
MethodDPPTAM: Dense piecewise planar tracking and mapping from a monocular sequence
Concha, A., & Civera, J. (2015) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7354184 · Cited by 157
“[20], [21] use a multiview sparse feature map to estimate a more robust layout.”
MethodNavigating to objects in the real world
Gervet, T., Chintala, S., Batra, D., Malik, J., & Chaplot, D. S. (2023) · Science Robotics · doi:10.1126/scirobotics.adf6991 · Cited by 125
“Semantic SLAM meth-ods [19, 20, 21, 22, 23, 24, 25] naturally extend SLAM to detect objects and keep them in memory but offer no solution for efficient semantic exploration.”
BackgroundMethodPiecewise Planar and Compact Floorplan Reconstruction from Images
Cabral, R., & Furukawa, Y. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.546 · Cited by 151
Efficient structured prediction for 3D indoor scene understanding
Schwing, A. G., Hazan, T., Pollefeys, M., & Urtasun, R. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248006 · Cited by 130
State‐of‐the‐art in Automatic 3D Reconstruction of Structured Indoor Environments
Pintore, G., Mura, C., Ganovelli, F., Perez, L. J. F., Pajarola, R., & Gobbetti, E. (2020) · Computer Graphics Forum · doi:10.1111/cgf.14021 · Cited by 123
Corners for Layout: End-to-End Layout Recovery From 360 Images
Fernandez-Labrador, C., Fácil, J. M., Pérez-Yus, A., Demonceaux, C., Civera, J., & Guerrero, J. J. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2967274 · Cited by 137
Semantics for Robotic Mapping, Perception and Interaction: A Survey
Garg, S., Sünderhauf, N., Dayoub, F., Morrison, D., Cosgun, A., Carneiro, G., Wu, Q., Chin, T.-J., Reid, I., Gould, S., et al. (2020) · Foundations and Trends® in Robotics · doi:10.1561/2300000059 · Cited by 105
Self-citationEfficient Exact Inference for 3D Indoor Scene Understanding
Schwing, A. G., & Urtasun, R. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33783-3_22 · Cited by 104
“More recently, [10, 11] have tackled the problem of inferring the 3D layout of corridors.”
BackgroundFloor-SP: Inverse CAD for Floorplans by Sequential Room-Wise Shortest Path
Chen, J., Liu, C., Wu, J., & Furukawa, Y. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00275 · Cited by 104
“For a room beyond a box shape, Dynamic Programming (DP) was applied to search for an optimal room structure [8, 9].”
MethodPPGNet: Learning Point-Pair Graph for Line Segment Detection
Zhang, Z., Li, Z., Bi, N., Zheng, J., Wang, J., Huang, K., Luo, W., Xu, Y., & Gao, S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00727 · Cited by 116
“The prior assumption about a spacial structure like Man-hattan World [7,11,38] or room topology [24,57,60] could significantly benefit the 3D reconstruction, but they are often violated in real man-made environments.”
3D-Based Reasoning with Blocks, Support, and Stability
Jia, Z., Gallagher, A., Saxena, A., & Chen, T. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.8 · Cited by 96
Shu, F., Wang, J., Pagani, A., & Stricker, D. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160452 · Cited by 119
“tion, or constraints such as Manhattan World (MW) [5], [8], [51].”
Multi-modal Auto-Encoders as Joint Estimators for Robotics Scene Understanding
Cadena, C., Dick, A., & D. Reid, I. (2016) · Robotics: Science and Systems XII · doi:10.15607/rss.2016.xii.041 · Cited by 93
“Subsequent to this, various works considered how 3D “pop-ups” or more complex scenes could be reconstructed automatically by combining machine learning of scene labels with perspective inversion [9, 12, 13].”
BackgroundSelf-citationPeople Watching: Human Actions as a Cue for Single View Geometry
Fouhey, D. F., Delaitre, V., Gupta, A., Efros, A. A., Laptev, I., & Šivic, J. (2014) · International Journal of Computer Vision · doi:10.1007/s11263-014-0710-z · Cited by 88
“[15]: our functional surfaces are not tied to the box-prediction paradigm.”
Estimating the 3D Layout of Indoor Scenes and Its Clutter from Depth Sensors
Zhang, J., Kan, C., Schwing, A. G., & Urtasun, R. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.161 · Cited by 84
“More general layouts than 3D cuboids were predicted in [3].”
BackgroundSimultaneous Localization And Mapping: Present, Future, and the Robust-Perception Age.Preprint
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · arXiv (Cornell University) · Cited by 90
Self-citationThe Manhattan Frame Model—Manhattan World Inference in the Space of Surface Normals
Straub, J., Freifeld, O., Rosman, G., Leonard, J. J., & Fisher, J. S. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2662686 · Cited by 72
“Many VP extraction algorithms rely on the MW assumption [1], [20], [22], [25], [26], [32], [34], [35] which is manifested as three OVPs.”
BackgroundManhattan Junction Catalogue for Spatial Reasoning of Indoor Scenes
Ramalingam, S., Pillai, J. K., Jain, A., & Taguchi, Y. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.394 · Cited by 67
“In this paper, we detect these junctions auto- ∗now at A9, a subsidiary of Amazon. matically from a given image and use them to improve the spatial understanding of indoor scenes.”
BackgroundUsing superpixels in monocular SLAM
Concha, A., & Civera, J. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6906883 · Cited by 78
“In these works object categories –car, screen– [7]; object instances –a particular chair or table model– [8]; or the room layout [9] are the semantic features that are explicitly modelled in the estimation.”
BackgroundMethodLifting 3D Manhattan Lines from a Single Image
Ramalingam, S., & Brand, M. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.67 · Cited by 61
“Our work is related to the geometrical constraints used in [19, 7].”
MethodParsing Indoor Scenes Using RGB-D Imagery
Taylor, C. J., & Cowley, A. (2012) · Robotics: Science and Systems · doi:10.15607/rss.2012.viii.051 · Cited by 61
“The principal differences between this work and that of Flint et al.is that it takes as input an RGB-D frame and begins by explicitly extracting planar surfaces, further the subsequent dynamic programming optimization procedure is phrased in terms of the RGB-D measurements as opposed to monocular and stereo cues.”
MethodResultInfluentialLaidlow, T., Czarnowski, J., & Leutenegger, S. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793527 · Cited by 74
“To combat this, dense systems often use a regulariser based on planar ([6]–[10]) or smoothness assumptions ([4], [5]).”
BackgroundCapturing indoor scenes with smartphones
Sankar, A., & Seitz, S. M. (2012) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2380116.2380168 · Cited by 57
“The visual SLAM and computer vision communities have developed automatic approaches [6, 7, 19, 4, 20] to re-construct indoor scenes from images.”
BackgroundPintore, G., Agus, M., & Gobbetti, E. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58598-3_26 · Cited by 54
“A notable example is the work of Flint et al. [8,7], who, exploiting the homography between floor and ceiling, reduce the structure classification problem to the estimation of the y-coordinate of the ceiling-wall boundary in each image column.”
MethodEvaluation of non-geometric methods for visual odometry
Ciarfuglia, T. A., Costante, G., Valigi, P., & Ricci, E. (2014) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2014.08.001 · Cited by 61
“Until now the features used have all been quantizations of the optical flow input, but we hypothesize that other kinds of features, for example extracted from an image segmentation process [11], could be used as well.”
MethodVisual Navigation Using Projection of Spatial Right-Angle In Indoor Environment
Wei, H., & Wang, L. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2818931 · Cited by 44
“If spatial right-angles satisfy the Manhattan World assumption in a 3D scene, their projection would still acquire special appearance in the 2D image.”
InfluentialPredicting Complete 3D Models of Indoor ScenesPreprint
Guo, R., Zou, C., & Hoiem, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.02437 · Cited by 59
“A box provides a good approximation to many rooms, and it has few parameters so that accurate estimation is possible from single RGB images [17, 8, 33] or panoramas [42].”
BackgroundOmnidirectional image capture on mobile devices for fast automatic generation of 2.5D indoor maps
Pintore, G., Garro, V., Ganovelli, F., Gobbetti, E., & Agus, M. (2016) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2016.7477631 · Cited by 50
“Applying the transform for an unknown wall height through a specialized voting scheme we individuate a points set Sm with a high likelihood to belong to the real room boundaries, coupled with an estimation of the wall height.”
BackgroundMethodReal-time manhattan world rotation estimation in 3D
Straub, J., Bhandari, N., Leonard, J. J., & Fisher, J. S. (2015) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7353628 · Cited by 38
“[12] integrate information across a stream of RGB images to obtain a semantic segmentation of the scene which relies on the MW structure of the observed environment.”
BackgroundDRM-SLAM: Towards dense reconstruction of monocular SLAM with scene depth fusion
Ye, X., Ji, X., Sun, B., Chen, S., Wang, Z., & Li, H. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.02.044 · Cited by 41
Nauata, N., & Furukawa, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58598-3_42 · Cited by 52
“red cost functions [15,25,17,6]. For a room beyond a box shape, Markov Random Field (MRF) infers detailed architectural structures [12] and Dynamic Programming (DP) searches for an optimal room shape [10,11], again via hand-engineered cost functions. High-level reconstruction (knowledge): Given a prior knowledge about…”
BackgroundDeep vanishing point detection: Geometric priors make dataset variations vanish
Lin, Y., Wiersma, R., Pintea, S. L., Hildebrandt, K., Eisemann, E., & van Gemert, J. C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00601 · Cited by 48
“Vanishing point detection in images has non-vanishing real-world returns: camera calibration [1, 9, 21], scene understanding [18], visual SLAM [13, 33], or even autonomous driving [28].”
BackgroundEfficient 3D Scene Labeling Using Fields of Trees
Kähler, O., & Reid, I. D. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.380 · Cited by 42
“A separate stream of work has explicitly considered inferring the coarse scene layout, usually by restricting it to some simple form such as a cuboid [16] or an indoor Manhattan world [5].”
BackgroundSelf-citationDense piecewise planar RGB-D SLAM for indoor environments
Le, P.-H., & Košecká, J. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206375 · Cited by 34
“This is in contrast to pixel level labeling schemes of [5] where each pixel in the RGB frame is assigned a label.”
BackgroundResultQuasi-Globally Optimal and Efficient Vanishing Point Estimation in Manhattan World
Li, H., Zhao, J. Y., Bazin, J., Chen, W., Liu, Z., & Liu, Y. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00173 · Cited by 33
“It has been successfully applied to various fields such as structure from motion [15,23], scene understanding [14] and SLAM [21,22].”
MethodHong Kong World: Leveraging Structural Regularity for Line-based SLAM
Li, H., Zhao, J. Y., Bazin, J., Kim, P., Joo, K., Zhao, Z., & Liu, Y. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3276204 · Cited by 31
Free your Camera: 3D Indoor Scene Understanding from Arbitrary Camera Motion
Furlan, A., Miller, S. D., Sorrenti, D. G., Fei-Fei, L., & Savarese, S. (2013) · British Machine Vision Conference (BMVC) · doi:10.5244/c.27.24 · Cited by 32
“[7, 8, 9] present a model that leverages the Manhattan world assumption to estimate dominant vanish ing points and relates them with learned photometric cues.”
BackgroundYaw aided Radar Inertial Odometry using Manhattan World Assumptions
Doer, C., & Trommer, G. F. (2021) · Saint Petersburg International Conference on Integrated Navigation Systems · doi:10.23919/icins43216.2021.9470842 · Cited by 32
Complete 3D Scene Parsing from an RGBD Image
Zou, C., Guo, R., Li, Z., & Hoiem, D. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-018-1133-z · Cited by 31
“w; and we make use of depth information, which eases the difficulty of interpreting geometry of visible surfaces. Other related topics are as follows. Room layout is often modeled as a 3D box (cuboid) [21, 11, 38, 53, 54]. A box provides a good approximation to many rooms, and it has few parameters so that accurate est…”
MethodRobotics and Artificial Intelligence
Ghallab, M., & Ingrand, F. (2020) · Springer eBooks · doi:10.1007/978-3-030-06170-8_12 · Cited by 33
Understanding the 3D layout of a cluttered room from multiple images
Bao, S. Y., Furlan, A., Fei-Fei, L., & Savarese, S. (2014) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2014.6836035 · Cited by 31
“Using a small set of images, [1] shows better 3D geometry estimation and object recognition results than the geometry estimation methods or the semantic reasoning methods that work in isolation.”
3D Reasoning from Blocks to Stability
Jia, Z., Gallagher, A., Saxena, A., & Chen, T. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2359435 · Cited by 30
A fast, modular scene understanding system using context-aware object detection
Cadena, C., Dick, A., & Reid, I. D. (2015) · 2015 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2015.7139874 · Cited by 29
“On the other hand, the proposed systems of [2,10,15,25] segment the scene into coarser classes (e.”
BackgroundSelf-citationUnderstanding of indoor scenes based on projection of spatial rectangles
Wei, H., & Wang, L. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.04.017 · Cited by 29
Manhattan and Piecewise-Planar Constraints for Dense Monocular Mapping
Concha, A., Hussain, W., Montano, L., & Civera, J. (2014) · Robotics: Science and Systems · doi:10.15607/rss.2014.x.016 · Cited by 28
“Multiview sparse 3D reconstructions have made use of layout estimation and scene understanding in [7, 21]; and superpixels in [14, 4].”
Background3D floor plan recovery from overlapping spherical images
Pintore, G., Ganovelli, F., Pintus, R., Scopigno, R., & Gobbetti, E. (2018) · Computational Visual Media · doi:10.1007/s41095-018-0125-9 · Cited by 27
Quasi-Globally Optimal and Near/True Real-Time Vanishing Point Estimation in Manhattan World
Li, H., Zhao, J. Y., Bazin, J., & Liu, Y. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3023183 · Cited by 26
Dynamic visual understanding of the local environment for an indoor navigating robot
Tsai, G., & Kuipers, B. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6385735 · Cited by 28
Depth and motion cues with phosphene patterns for prosthetic vision
Perez-Yus, A., Bermudez-Cameo, J., Guerrero, J. J., & Lopez-Nicolas, G. (2017) · 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw.2017.179 · Cited by 24
“Taking this assumption into account, denoted as Manhattan world assumption, some works have been proposed for recovering the scene layout [26, 20, 16], or just the orientation of the scene [10] as we do in this work.”
MethodGlobally Optimal Inlier Set Maximization for Atlanta Frame Estimation
Joo, K., Oh, T.-H., Kweon, I. S., & Bazin, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00600 · Cited by 24
“Since information about the scene structure can increase the accuracy, most VP estimation methods rely on the structure assumption, especially the MW assumption [7, 20, 10, 34].”
MethodDense monocular reconstruction using surface normals
Weerasekera, C. S., Latif, Y., Garg, R., & Reid, I. D. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989293 · Cited by 24
“Some of them include smoothness in disparity/depth priors [4], [11], [17] or Manhatton/piewise-planar priors [6], [18], [7], [19], [8].”
BackgroundSelf-citationPSMNet: Position-aware Stereo Merging Network for Room Layout Estimation
Wang, H., Hutchcroft, W., Li, Y., Wan, Z., Boyadzhiev, I., Tian, Y., & Kang, S. B. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00842 · Cited by 23
Yi, D.-H., Lee, T.-J., & Cho, D. (2018) · Sensors · doi:10.3390/s18010171 · Cited by 21
Automatic modeling of cluttered multi‐room floor plans from panoramic images
Pintore, G., Ganovelli, F., Villanueva, A. J., & Gobbetti, E. (2019) · Computer Graphics Forum · doi:10.1111/cgf.13842 · Cited by 21
“For example, several authors exploit the heavily constraining Manhattan World assumption to reconstruct the 3D structure of moderately cluttered interiors e.g., [FCSS09, FMR11, TXLK11]) or the 3D footprint of interior objects (e.g., [LGHK10,HHF12]).”
BackgroundVedaldi, A., & Zisserman, A. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33709-3_7 · Cited by 20
“The estimation of the image vanishing points [20–26] is a standard step in several geometry-based applications.”
BackgroundPlanar Structures from Line Correspondences in a Manhattan World
Kim, C., & Manduchi, R. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16865-4_33 · Cited by 20
“[24] addressed the spatial layout estimation problem by integrating information from image features, stereo features, and 3-D point clouds in a MAP optimization problem, which is solved using dynamic programming.”
BackgroundIncorporating scene priors to dense monocular mapping
Concha, A., Hussain, W., Montano, L., & Civera, J. (2015) · Autonomous Robots · doi:10.1007/s10514-015-9465-9 · Cited by 20
Understanding of Curved Corridor Scenes Based on Projection of Spatial Right-Angles
Wang, L., & Wei, H. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3026628 · Cited by 20
“In restricted domains, such as those satisfying the Manhattan world assumption [23], [24], there has been renewed interest in 3D structures.”
BackgroundThinking Outside the Box: Generation of Unconstrained 3D Room Layouts
Howard-Jenkins, H., Li, S., & Prisacariu, V. A. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20887-5_27 · Cited by 19
“approaches have been used, such as that in [2]. Flint et al. used a combination of monocular features, with multiple-view and 3D features to infer a Manhattan World representation of the environment [9]. In [30], a RGB-D panorama is split and half of the scene is taken as input with the aim of producing reasonable, cl…”
Line-based Absolute and Relative Camera Pose Estimation in Structured Environments
Li, H., Zhao, J. Y., Bazin, J., Chen, W., Chen, K., & Liu, Y. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968444 · Cited by 19
“These models have been applied to various fields to improve the accuracy and/or efficiency, such as SLAM [4] [5] and scene understanding [6].”
MethodLayout aware visual tracking and mapping
Salas, M., Hussain, W., Concha, A., Montano, L., Civera, J., & Montiel, J. M. M. (2015) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7353367 · Cited by 19
“, connecting corridors [14] and non-cuboid scenes [11], [12].”
BackgroundResultFlexibly Connectable Light Field System For Free View Exploration
Jung, H., Lee, H.-J., & Rhee, C. E. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2934819 · Cited by 18
Shape Anchors for Data-Driven Multi-view Reconstruction
Owens, A., Xiao, J., Torralba, A. B., & Freeman, W. T. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.461 · Cited by 17
“For example, [20] [8] build piecewise-planar or highly”
BackgroundUnderstanding of wheelchair ramp scenes for disabled people with visual impairments
Wang, L., & Wei, H. (2020) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2020.103569 · Cited by 18
Layered Interpretation of Street View Images
Liu, M.-Y., Lin, S., Ramalingam, S., & Tuzel, O. (2015) · Robotics: Science and Systems · doi:10.15607/rss.2015.xi.025 · Cited by 16
Building facade detection, segmentation, and parameter estimation for mobile robot stereo vision
Delmerico, J., David, P., & Corso, J. J. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2013.08.006 · Cited by 14
“The high-level representation is a Potts model, where each hidden variable represents the labeling of the associated pixel with one of the candidate planes, or with no plane if it is not part of a building.”
BackgroundVistas and Wall-Floor Intersection Features: Enabling Autonomous Flight in Man-made Environments
Ok, K., Ta, D.-N., & Dellaert, F. (2012) · Cited by 14
Peripheral Expansion of Depth Information via Layout Estimation with Fisheye Camera
Pérez-Yus, A., Nicolás, G. L., & Guerrero, J. J. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46484-8_24 · Cited by 14
“Other approaches make use of video sequences instead of single images [18,19].”
Background360-DFPE: Leveraging Monocular 360-Layouts for Direct Floor Plan Estimation
Solarte, B., Liu, Y.-C., Wu, C.-H., Tsai, Y.-H., & Sun, M. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3173730 · Cited by 14
TT-SLAM: Dense Monocular SLAM for Planar Environments
Wang, X., Christie, M., & Marchand, É. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561164 · Cited by 13
“Such an assumption is well suited in standard indoor and urban scenarios and simplifies the model, improving performance and precision in specific use-cases [10], [11], [12].”
BackgroundSpatial Geometric Reasoning for Room Layout Estimation via Deep Reinforcement Learning
Ren, L., Song, Y., Lu, J., & Zhou, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58583-9_33 · Cited by 12
“Room Layout Estimation: For simplicity, most work consider the room as a 3D cuboid as in [11], instead of general Manhattan world models [7,15,39].”
BackgroundNicolás, G. L., Omedes, J., & Guerrero, J. J. (2014) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2014.03.018 · Cited by 11
“Keywords: Omnidirectional vision, Vanishing points, Homography, Manhattan-world, Spatial layout”
BackgroundMonocular Camera Based Real-Time Dense Mapping Using Generative Adversarial Network
Yang, X. Y., Chen, J., Wang, Z., Zhang, Q., Liu, W., Liao, C., & Cheng, K.-T. T. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240564 · Cited by 10
“Rather than a piecewise planar assumption, several methods proposed to integrate shape priors based on object recognition [4, 18, 28] and/or predefined structure/layout [9, 35] into monocular SLAM for dense mapping.”
MethodTrajectory bundle estimation For perception-driven planning
Bachrach, A. (2013) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 16
Linear Four-Point LiDAR SLAM for Manhattan World Environments
Jeong, E., Lee, J., Kang, S., & Kim, P. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3315205 · Cited by 9
“this challenging problem by effectively utilizing the Manhattan world (MW) structural regularities [9] of the indoor environments, and line RANSAC to represent detected surrounding walls as orthogonal lines directly modeled as landmarks with insufficient range measurements.”
I Want to Change My Floor: Dominant Plane Recognition from a Single Image to Augment the Scene
Osuna-Coutiño, J. A. D. J., Cruz-Martínez, C., Martínez-Carranza, J., Arias-Estrada, M., & Mayol-Cuevas, W. W. (2016) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar-adjunct.2016.0060 · Cited by 8
“To increase plane recognition other works process both, 3D map and images [11] or use Manhattan assumptions [12].”
MethodLi, H., Chen, K., Kim, P., Yoon, K., Liu, Z., Joo, K., & Liu, Y. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00561 · Cited by 8
“VP has various applications such as scene understanding [12], camera orientation estimation [14] and 3D reconstruction [13].”
BackgroundSingle-image depth estimation using relative depths
Lee, J.-H., & Kim, C. (2022) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2022.103459 · Cited by 8
Line Primitives and Their Applications in Geometric Computer Vision
Zhang, L. (2013) · MACAU: Open Access Repository of Kiel University (University Library Kiel) · Cited by 8
Real-time Monocular Dense Mapping for Augmented Reality
Xue, T., Luo, H., Cheng, D., Yuan, Z., & Yang, X. Y. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3123348 · Cited by 7
“In recent years, some efforts have been made to integrate semantic information based on object recognition [3, 18, 22] or structure/layout of a scene [10] with monocular SLAM for dense mapping.”
BackgroundA sketch is worth a thousand navigational instructions
Ahmad, H., Usama, S. M., Hussain, W., & Anjum, M. L. (2021) · Autonomous Robots · doi:10.1007/s10514-020-09965-2 · Cited by 7
“On the other hand, extracting 3D layout from sensor readings has been shown for limited (Manhattan) environments (Hedau et al. 2009; Flint et al. 2011; Furlan et al. 2013; Salas et al. 2015; Liu et al. 2018).”
BackgroundDominant plane recognition in interior scenes from a single image
Osuna-Coutiño, J. A. D. J., Martínez-Carranza, J., Arias-Estrada, M., & Mayol-Cuevas, W. W. (2016) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2016.7899917 · Cited by 7
3D Aware Correction and Completion of Depth Maps in Piecewise Planar Scenes
Thabet, A., Lahoud, J., Asmar, D., & Ghanem, B. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16808-1_16 · Cited by 6
“This scene prior has been used in previous work for the purpose of stereo vision or 3D reconstruction [9, 10], semantic labelling [22], and scene understanding from a single RGB-D image [19, 8] or a single RGB image [16, 14, 15].”
MethodAn Interactive Indoor 3D Reconstruction Method Based on Conformal Geometry Algebra
Liu, N., Lin, B., Yuan, L., Lv, G., Yu, Z., & Zhou, L. (2018) · Advances in Applied Clifford Algebras · doi:10.1007/s00006-018-0880-9 · Cited by 6
“The first approach reconstructs indoor scene using a single image [5, 13,15–17, 20, 21 , 25 The second approach uses stereo matching algorithm to recover the depth of the scene from a bunch of images [4,9–12 , 30 ].”
MethodRelative Pose Estimation and Planar Reconstruction via Superpixel-Driven Multiple Homographies
Wang, X., Christie, M., & Marchand, É. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341707 · Cited by 5
“Typically, the Manhattan assumption is widely exploited in planar vision tasks [11], [12], [13].”
BackgroundTa, D.-N., Ok, K., & Dellaert, F. (2014) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2014.03.010 · Cited by 6
“Moreover, vistas are also derived from scale-space features and inherit the properties such that they are easily and reliably detected and tracked in many types of environments.”
BackgroundModelado Semántico 3D de Ambientes Interiores basado en Nubes de Puntos y Relaciones Contextuales
Quijano, A., & Prieto, F. (2016) · Ingeniería · doi:10.14483/udistrital.jour.reving.2016.3.a04 · Cited by 8
Wireframe Parsing With Guidance of Distance Map
Huang, K., & Gao, S. (2019) · IEEE Access · doi:10.1109/access.2019.2943885 · Cited by 5
“The detection results are further used to recover the geometric of the scenes in [2], [4], [9]–[11].”
MethodSparse Point Cloud Densification by Combining Multiple Segmentation Methods
Hödlmoser, M., Mičušík, B., & Kampel, M. (2013) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2013.64 · Cited by 5
Indoor Manhattan spatial layout recovery from monocular videos via line matching
Kim, C., & Manduchi, R. (2016) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2016.09.012 · Cited by 4
Single-image depth estimation by refined segmentation and consistency reconstruction
Liu, H., Lei, D., Zhu, Q., Sui, H., Zhang, H., & Wang, Z. (2020) · Signal Processing Image Communication · doi:10.1016/j.image.2020.116048 · Cited by 4
Globally Optimal and Efficient Manhattan Frame Estimation by Delimiting Rotation Search Space
Ge, W., Song, Y., Zhang, B., & Dong, Z. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01493 · Cited by 4
“understanding [7, 8, 9] and SLAM [10, 11, 12].”
Higher-order clique based image segmentation using evolutionary game theory
Li, J., Zeng, G., Gan, R., Zha, H., & Wang, L. (2014) · Artificial Intelligence Research · doi:10.5430/air.v3n2p1 · Cited by 4
Metadata
- Authors
- Flint, A., Murray, D., & Reid, I.
- Affiliations
- University of Oxford (GB)
- Venue
- 2011 International Conference on Computer Vision
- Publisher
- IEEE
- Published
- 2011-11
- Pages
- 2228-2235
- OpenAlex
- W2083347703
- Cited by
- 132
- References
- 17
- Citations by year
- 2026: 2 · 2025: 5 · 2023: 5 · 2022: 5 · 2021: 6 · 2020: 15 · 2019: 13 · 2018: 15 · 2017: 6 · 2016: 11 · 2015: 12 · 2014: 16 · 2013: 13 · 2012: 8
- FWCI
- 8.81
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
- Funders
- Engineering and Physical Sciences Research Council (EP/H050795/1)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Keywords
- Manhattan world assumption, scene understanding, indoor scene understanding, monocular vision, stereo vision, structure from motion, point clouds, photo-consistency, Bayesian graphical models, MAP inference, dynamic programming, semantic reasoning, boundary extraction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.