3D Scene Reconstruction With Multi-Layer Depth and Epipolar Transformers
Open accessAbstract
We tackle the problem of automatically reconstructing a complete 3D model of a scene from a single RGB image. This challenging task requires inferring the shape of both visible and occluded surfaces. Our approach utilizes viewer-centered, multi-layer representation of scene geometry adapted from recent methods for single object shape completion. To improve the accuracy of view-centered representations for complex scenes, we introduce a novel "Epipolar Feature Transformer" that transfers convolutional network features from an input view to other virtual camera viewpoints, and thus better covers the 3D scene geometry. Unlike existing approaches that first detect and localize objects in 3D, and then infer object shape using category-specific models, our approach is fully convolutional, end-to-end differentiable, and avoids the resolution and memory limitations of voxel representations. We demonstrate the advantages of multi-layer depth representations and epipolar feature transformers on the reconstruction of a large database of indoor scenes.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Several surveys dedicated to Transformers have been recently introduced, with a range of emphases.
“Several surveys dedicated to Transformers have been recently introduced, with a range of emphases including general Transformers [48], efficient designs [49], visualization [50], computer vision tasks [51], [52], [53], [54], medical imaging [55], video tasks [56], and vision language pretraining [57].”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [50]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Charles, R., Su, H., Mo, K., & Guibas, L. J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.16 · Cited by 9,767 · see citing works
Indoor segmentation and support inference from rgbd images
Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_54 · Cited by 5,760 · see citing works
3D ShapeNets: A deep representation for volumetric shapes
Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., & Xiao, J. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298801 · Cited by 5,413 · see citing works
Jaderberg, M., Simonyan, K., Zisserman, A., & Kavukcuoglu, K. (2015) · arXiv · doi:10.48550/arxiv.1506.02025 · Cited by 5,137 · see citing works
ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes
Dai, A., Chang, A. X., Savva, M., Halber, M., Funkhouser, T., & Niessner, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.261 · Cited by 4,368 · see citing works
VoxNet: A 3D Convolutional Neural Network for real-time object recognition
Maturana, D., & Scherer, S. (2015) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7353481 · Cited by 3,674 · see citing works
Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
Eigen, D., Puhrsch, C., & Fergus, R. (2014) · arXiv · doi:10.48550/arxiv.1406.2283 · Cited by 3,442 · see citing works
Unsupervised Learning of Depth and Ego-Motion from Video
Zhou, T., Brown, M., Snavely, N., & Lowe, D. G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.700 · Cited by 2,910 · see citing works
KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera
Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R. A., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al. (2011) · ACM Symposium on User Interface Software and Technology · doi:10.1145/2047196.2047270 · Cited by 2,601 · see citing works
A Point Set Generation Network for 3D Object Reconstruction from a Single Image
Fan, H., Su, H., & Guibas, L. J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.264 · Cited by 2,410 · see citing works
ShapeNet: An Information-Rich 3D Model Repository
Chang, A. X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al. (2015) · arXiv · doi:10.48550/arxiv.1512.03012 · Cited by 2,330 · see citing works
Modeling the World from Internet Photo Collections
Snavely, N., Seitz, S. M., & Szeliski, R. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-007-0107-3 · Cited by 2,147 · see citing works
3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction
Choy, C., Xu, D., Gwak, J., Chen, K., & Savarese, S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46484-8_38 · Cited by 1,741 · see citing works
OctNet: Learning Deep 3D Representations at High Resolutions
Riegler, G., Ulusoy, A. O., & Geiger, A. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.701 · Cited by 1,662 · see citing works
ShapeNet: An Information-Rich 3D Model Repository
Chang, A. X., Funkhouser, T. A., Guibas, L. J., Hanrahan, P., Qi-xing Huang, A., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al. (2023) · doi:10.5281/zenodo.10089705 · Cited by 1,624 · see citing works
MACHINE PERCEPTION OF THREE-DIMENSIONAL SOLIDS,
Roberts, L. G. (1963) · Defense Technical Information Center (DTIC) · Cited by 1,456 · see citing works
Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images
Wang, N., Zhang, Y., Li, Z., Fu, Y., Liu, W., & Jiang, Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_4 · Cited by 1,362 · see citing works
Semantic Scene Completion from a Single Depth Image
Song, S., Yu, F., Zeng, A., Chang, A. X., Savva, M., & Funkhouser, T. A. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.28 · Cited by 1,244 · see citing works
Representing moving images with layers
Wang, J. Y. A., & Adelson, E. H. (1994) · IEEE Transactions on Image Processing · doi:10.1109/83.334981 · Cited by 1,102 · see citing works
Kato, H., Ushiku, Y., & Harada, T. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00411 · Cited by 954 · see citing works
Zhou, T., Tucker, R. P., Flynn, J., Fyffe, G., & Snavely, N. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201323 · Cited by 918 · see citing works
SPLATNet: Sparse Lattice Networks for Point Cloud Processing
Su, H., Jampani, V., Sun, D., Maji, S., Kalogerakis, E., Yang, M., & Kautz, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00268 · Cited by 804 · see citing works
Learning a Predictable and Generative Vector Representation for Objects
Girdhar, R., Fouhey, D. F., Rodriguez, M. D., & Gupta, A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46466-4_29 · Cited by 673 · see citing works
Dai, A., Nießner, M., Zollhöfer, M., Izadi, S., & Theobalt, C. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3054739 · Cited by 665 · see citing works
Shape Completion Using 3D-Encoder-Predictor CNNs and Shape Synthesis
Dai, A., Qi, C. R., & NieBner, M. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.693 · Cited by 633 · see citing works
Multi-View Supervision for Single-View Reconstruction via Differentiable Ray Consistency
Tulsiani, S., Zhou, T., Efros, A. A., & Malik, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.30 · Cited by 528 · see citing works
Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D SupervisionPreprint
Yan, X., Yang, J., Yumer, E., Guo, Y., & Lee, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00814 · Cited by 425 · see citing works
Single-Image Depth Perception in the WildPreprint
Chen, W., Zhao, F., Yang, D., & Deng, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.03901 · Cited by 418 · see citing works
Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction
Lin, C.-H., Kong, C., & Lucey, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12278 · Cited by 385 · see citing works
In-place Activated BatchNorm for Memory-Optimized Training of DNNs
Bulò, S. R., Porzi, L., & Kontschieder, P. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00591 · Cited by 324 · see citing works
Hierarchical Surface Prediction for 3D Object Reconstruction
Häne, C., Tulsiani, S., & Malik, J. (2017) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00054 · Cited by 308 · see citing works
Transformation-Grounded Image Generation Network for Novel 3D View Synthesis
Park, E., Yang, J., Yumer, E., Ceylan, D., & Berg, A. C. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.82 · Cited by 276 · see citing works
Category-specific object reconstruction from a single image
Kar, A., Tulsiani, S., Carreira, J., & Malik, J. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298807 · Cited by 263 · see citing works
Physically-Based Rendering for Indoor Scene Understanding Using Convolutional Neural Networks
Zhang, Y., Song, S., Yumer, E., Savva, M., Lee, J., Jin, H., & Funkhouser, T. A. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.537 · Cited by 251 · see citing works
Multiresolution Tree Networks for 3D Point Cloud Processing
Gadelha, M., Wang, R., & Maji, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01234-2_7 · Cited by 241 · see citing works
Dai, A., Nießner, M., Zollhöfer, M., Izadi, S., & Theobalt, C. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3126814 · Cited by 211 · see citing works
Aligning 3D models to RGB-D images of cluttered scenes
Gupta, S., Arbeláez, P., Girshick, R., & Malik, J. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299105 · Cited by 199 · see citing works
3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks
Zou, C., Yumer, E., Yang, J., Ceylan, D., & Hoiem, D. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.103 · Cited by 195 · see citing works
Learning Efficient Point Cloud Generation for Dense 3D Object ReconstructionPreprint
Lin, C.-H., Kong, C., & Lucey, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.07036 · Cited by 164 · see citing works
Izadinia, H., Shan, Q., & Seitz, S. M. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.260 · Cited by 156 · see citing works
Completing 3D object shape from one depth image
Rock, J., Gupta, T., Thorsen, J., Gwak, J., Shin, D., & Hoiem, D. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298863 · Cited by 154 · see citing works
Self-citationLayer-Structured 3D Scene Inference via View Synthesis
Tulsiani, S., Tucker, R. P., & Snavely, N. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01234-2_19 · Cited by 152 · see citing works
Moulding Humans: Non-Parametric 3D Human Shape Estimation From Single Images
Gabeur, V., Franco, J.-S., Martin, X., Schmid, C., & Rogez, G. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00232 · Cited by 150 · see citing works
Stereo Magnification: Learning View Synthesis using Multiplane Images
Zhou, T., Tucker, R., Flynn, J., Fyffe, G., & Snavely, N. (2018) · arXiv · doi:10.48550/arxiv.1805.09817 · Cited by 150 · see citing works
Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scene
Tulsiani, S., Gupta, S., Fouhey, D. F., Efros, A. A., & Malik, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00039 · Cited by 113 · see citing works
Shin, D., Fowlkes, C. C., & Hoiem, D. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00323 · Cited by 110 · see citing works
Self-citationLearning to Reconstruct Shapes from Unseen ClassesPreprint
Zhang, X., Zhang, Z., Zhang, C., Tenenbaum, J. B., Freeman, W. T., & Wu, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.11166 · Cited by 107 · see citing works
Layered motion segmentation and depth ordering by tracking edges
Smith, P., Drummond, T. W., & Cipolla, R. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.1265863 · Cited by 105 · see citing works
Matryoshka Networks: Predicting 3D Geometry via Nested Shape Layers
Roth, S., & Richter, S. R. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00207 · Cited by 98 · see citing works
Layered segmentation and optical flow estimation over time
Sun, D., Sudderth, E. B., & Black, M. J. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247873 · Cited by 90 · see citing works
Self-citationSelf-Supervised Relative Depth Learning for Urban Scene Understanding
Jiang, H., Larsson, G., Maire, M., Shakhnarovich, G., & Miller, E. G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_2 · Cited by 81 · see citing works
Peeking behind objects: Layered depth prediction from a single image
Dhamo, H., Tateno, K., Laina, I., Navab, N., & Tombari, F. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2019.05.007 · Cited by 60 · see citing works
Ji, D., Kwon, J., McFarland, M., & Savarese, S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.750 · Cited by 58 · see citing works
Scene Collaging: Analysis and Synthesis of Natural Images with Semantic Layers
Isola, P., & Liu, C. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.457 · Cited by 57 · see citing works
3D-Assisted Feature Synthesis for Novel Views of an Object
Su, H., Wang, F., Yi, E., & Guibas, L. J. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.307 · Cited by 37 · see citing works
Geometric Pose Affordance: 3D Human Pose with Scene ConstraintsPreprint
Wang, Z., Chen, L., Rathore, S., Shin, D., & Fowlkes, C. C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.07718 · Cited by 31 · see citing works
Self-citation3D-Assisted Image Feature Synthesis for Novel Views of an ObjectPreprint
Su, H., Wang, F., Li, Y., et al. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.0003 · Cited by 18 · see citing works
3D Object Dense Reconstruction from a Single Depth View.Preprint
Yang, B., Rosa, S., Markham, A., Trigoni, N., & Wen, H. (2018) · arXiv (Cornell University) · Cited by 14 · see citing works
Nonparametric learning for layered segmentation of natural images
Ghosh, S., & Sudderth, E. B. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247937 · Cited by 13 · see citing works
Self-citationX-Section: Cross-Section Prediction for Enhanced RGB-D Fusion
Nicastro, A., Clark, R., & Leutenegger, S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00160 · Cited by 12 · see citing works
Complete 3D Scene Parsing from Single RGBD Image.Preprint
Zou, C., Li, Z., & Hoiem, D. (2017) · arXiv (Cornell University) · Cited by 7 · see citing works
Self-Supervised Depth Learning for Urban Scene Understanding
Jiang, H., Miller, E. G., Larsson, G., Maire, M., & Shakhnarovich, G. (2017) · Cited by 3 · see citing works
Flagged first, then by citations
13 of the 75 references couldn’t be matched to a known paper.
Research works printed but not linked: 6
- Hamid Izadinia, Qi Shan, and Steven M Seitz. Im2cad. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. 3
- Andrea Nicastro, Ronald Clark, and Stefan Leutenegger. X- section: Cross-section prediction for enhanced rgbd fusion. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2019. 3
- Edward Smith, Scott Fujimoto, and David Meger. Multi- view silhouette and depth decomposition for high resolution 3d object representation. In Advances in Neural Information Processing Systems (NeurIPS), pages 6479-6489, 2018. 3
- Edward J. Smith and David Meger. Improved adversarial systems for 3d object generation and reconstruction. In 1st Annual Conference on Robot Learning, CoRL 2017, Moun- tain View, California, USA, November 13-15, 2017, Proceed- ings, pages 87-96, 2017. 3
- Jiajun Wu, Yifan Wang, Tianfan Xue, Xingyuan Sun, Bill Freeman, and Josh Tenenbaum. Marrnet: 3d shape recon- struction via 2.5 d sketches. In Advances in Neural Informa- tion Processing Systems (NeurIPS), pages 540-550, 2017. 3
- Jiajun Wu, Chengkai Zhang, Xiuming Zhang, Zhoutong Zhang, William T Freeman, and Joshua B Tenenbaum. Learning 3D Shape Priors for Shape Completion and Recon- struction. In Proceedings of the European Conference on Computer Vision (ECCV), 2018. 3
Cited by
Sorted by citations
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072 · see citing works
End-to-End Human Pose and Mesh Reconstruction with Transformers
Lin, K., Wang, L., & Liu, Z. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00199 · Cited by 740 · see citing works
SynSin: End-to-End View Synthesis From a Single Image
Wiles, O., Gkioxari, G., Szeliski, R., & Johnson, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00749 · Cited by 418 · see citing works
Deep Stereo Using Adaptive Thin Volume Representation With Uncertainty Awareness
Cheng, S., Xu, Z., Zhu, S., Li, Z., Li, L. E., Ramamoorthi, R., & Su, H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00260 · Cited by 385 · see citing works
Computer vision applications in construction: Current state, opportunities & challenges
Paneru, S., & Jeelani, I. (2021) · Automation in Construction · doi:10.1016/j.autcon.2021.103940 · Cited by 309 · see citing works
Nie, Y., Han, X., Guo, S., Zheng, Y., Chang, J., & Zhang, J. J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00013 · Cited by 238 · see citing works
SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation
Di, Y., Manhardt, F., Wang, G., Ji, X., Navab, N., & Tombari, F. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01217 · Cited by 159 · see citing works
Deep learning-based 3D reconstruction: a survey
Samavati, T., & Soryani, M. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10399-2 · Cited by 105 · see citing works
Human-Aware Object Placement for Visual Environment Reconstruction
Yi, H., Huang, C.-H. P., Tzionas, D., Kocabas, M., Hassan, M. S., Tang, S., Thies, J., & Black, M. J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00393 · Cited by 67 · see citing works
Holistic 3D Human and Scene Mesh Estimation from Single View Images
Weng, Z., & Yeung, S. Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00040 · Cited by 50 · see citing works
MatrixVT: Efficient Multi-Camera to BEV Transformation for 3D Perception
Zhou, H., Ge, Z., Li, Z., & Zhang, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00785 · Cited by 51 · see citing works
UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Rendering
Banani, M. E., Gao, L., & Johnson, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00705 · Cited by 50 · see citing works
Building displacement measurement and analysis based on UAV images
Sun, J., Peng, B., Wang, C., Chen, K., Zhong, B., & Wu, J. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104367 · Cited by 42 · see citing works
Pseudo-LiDAR Point Cloud Interpolation Based on 3D Motion Representation and Spatial Supervision
Liu, H., Liao, K., Lin, C., Zhao, Y., & Guo, Y. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3056048 · Cited by 40 · see citing works
3-D Semantic Terrain Reconstruction of Monocular Close-Up Images of Martian Terrains
Tian, P., Yao, M., Xiao, X., Zheng, B., Cao, T., Xi, Y., Liu, H., & Cui, H. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3378192 · Cited by 39 · see citing works
Exploring Efficiency of Vision Transformers for Self-Supervised Monocular Depth Estimation
Karpov, A., & Makarov, I. (2022) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar55827.2022.00089 · Cited by 36 · see citing works
Panoptic 3D Scene Reconstruction From a Single RGB Image
Dahnert, M., Hou, J., Nießner, M., & Dai, A. (2021) · arXiv · doi:10.48550/arxiv.2111.02444 · Cited by 36 · see citing works
3D Reconstruction of a Complex Grid Structure Combining UAS Images and Deep Learning
Knyaz, V. A., Kniaz, V. V., Fabio Remondino, A., Zheltov, S. Y., & Gruen, A. (2020) · Remote Sensing · doi:10.3390/rs12193128 · Cited by 32 · see citing works
Han, Z., Qiao, G., Liu, Y.-S., & Zwicker, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58586-0_36 · Cited by 31 · see citing works
Cai, S., Chan, E. R., Peng, S., Shahbazi, M., Obukhov, A., Gool, L. V., & Wetzstein, G. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00204 · Cited by 30 · see citing works
Camera Pose Matters: Improving Depth Prediction by Mitigating Pose Distribution Bias
Zhao, Y., Kong, S., & Fowlkes, C. C. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01550 · Cited by 27 · see citing works
Self-citationDepth Inpainting via Vision Transformer
Makarov, I., & Borisenko, G. V. (2021) · IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) · doi:10.1109/ismar-adjunct54149.2021.00065 · Cited by 27 · see citing works
Distribution-Aligned Diffusion for Human Mesh Recovery
Foo, L. G., Gong, J. L., Rahmani, H., & Liu, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00846 · Cited by 28 · see citing works
Shi, J., Yong, A., Jin, Y., Li, D., Niu, H., Jin, Z., & Wang, H. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611152 · Cited by 26 · see citing works
Image-to-Voxel Model Translation for 3D Scene Reconstruction and Segmentation
Kniaz, V. V., Knyaz, V. A., Fabio Remondino, A., Bordodymov, A. N., & Moshkantsev, P. V. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58571-6_7 · Cited by 23 · see citing works
OutCast: Outdoor Single‐image Relighting with Cast Shadows
Griffiths, D., Ritschel, T., & Philip, J. (2022) · Computer Graphics Forum · doi:10.1111/cgf.14467 · Cited by 23 · see citing works
RTRBench: A Benchmark Suite for Real-Time Robotics
Bakhshalipour, M., Likhachev, M. A., & Gibbons, P. B. (2022) · IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) · doi:10.1109/ispass55109.2022.00024 · Cited by 14 · see citing works
Geometric Pose Affordance: Monocular 3D Human Pose Estimation with Scene Constraints
Wang, Z., Chen, L., Rathore, S., Shin, D., & Fowlkes, C. C. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25075-0_1 · Cited by 14 · see citing works
Self-citationUni-3D: A Universal Model for Panoptic 3D Scene Reconstruction
Zhang, X., Chen, Z., Wei, F., & Zhuowen Tu, A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00849 · Cited by 13 · see citing works
Saroha, A., Gladkova, M., Curreli, C., Muhle, D., Yenamandra, T., & Cremers, D. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-85187-2_15 · Cited by 13 · see citing works
SAR: Spatial-Aware Regression for 3D Hand Pose and Mesh Reconstruction from a Monocular RGB Image
Zheng, X., Ren, P., Sun, H., Wang, J., Qi, Q., & Liao, J. (2021) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar52148.2021.00024 · Cited by 10 · see citing works
WIRE STRUCTURE IMAGE-BASED 3D RECONSTRUCTION AIDED BY DEEP LEARNING
Kniaz, V. V., Zheltov, S. Y., Fabio Remondino, A., Knyaz, V. A., Bordodymov, A. N., & Gruen, A. (2020) · The international archives of the photogrammetry, remote sensing and spatial information sciences/International archives of the photogrammetry, remote sensing and spatial information sciences · doi:10.5194/isprs-archives-xliii-b2-2020-435-2020 · Cited by 9 · see citing works
UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable RenderingPreprint
Banani, M. E., Gao, L., & Johnson, J. (2021) · arXiv (Cornell University) · Cited by 9 · see citing works
Creating a Forensic Database of Shoeprints from Online Shoe-Tread Photos
Shafique, S., Kong, B., Kong, S., & Fowlkes, C. C. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00092 · Cited by 8 · see citing works
Self-citationCD 2 : Fine-grained 3D Mesh Reconstruction with Twice Chamfer Distance
Zeng, R., Su, M., Yu, R., & Wang, X. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3582694 · Cited by 7 · see citing works
Equine Pain Behavior Classification via Self-Supervised Disentangled Pose Representation
Rashid, M., Broomé, S., Ask, K., Hernlund, E., Andersen, P. H., Kjellström, H., & Lee, Y. J. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00023 · Cited by 7 · see citing works
Recognizing Scenes from Novel Viewpoints
Qian, S., Kirillov, A., Ravi, N., Chaplot, D. S., Johnson, J., Fouhey, D. F., & Gkioxari, G. (2021) · CaltechAUTHORS (California Institute of Technology) · doi:10.48550/arxiv.2112.01520 · Cited by 5 · see citing works
Heightfields for Efficient Scene Reconstruction for AR
Watson, J., Vicente, S., Aodha, O. M., Godard, C., Brostow, G., & Firman, M. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00580 · Cited by 5 · see citing works
Improved Modeling of 3D Shapes with Multi-view Depth Maps
Gupta, K., Jabbireddy, S., Shah, K., Shrivastava, A., & Zwicker, M. (2020) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv50981.2020.00017 · Cited by 4 · see citing works
SIMstack: A Generative Shape and Instance Model for Unordered Object Stacks
Landgraf, Z., Scona, R., Laidlow, T., James, S., Leutenegger, S., & Davison, A. J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01277 · Cited by 4 · see citing works
Monocular Depth Prediction through Continuous 3D Loss
Zhu, M., Ghaffari, M., Zhong, Y., Lu, P., Cao, Z. L., Eustice, R. M., & Peng, H. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341767 · Cited by 4 · see citing works
Learning to Infer 3D Object Models from Images.Preprint
Chen, C., Deng, F., & Ahn, S. (2020) · arXiv (Cornell University) · Cited by 4 · see citing works
Novel View Synthesis Of Transparent Object From a Single Image
Zhou, S., Wang, Z., & Ye, D. (2022) · Computer Graphics Forum · doi:10.1111/cgf.14714 · Cited by 3 · see citing works
Peng, B., & Li, Y. (2024) · Journal of Mechanical Science and Technology · doi:10.1007/s12206-024-0922-2 · Cited by 3 · see citing works
What's on the Other Side? A Single-View 3D Scene Reconstruction
Staszak, R., Kulecki, B., Sempruch, W., & Belter, D. (2022) · International Conference on Control, Automation, Robotics and Vision (ICARCV) · doi:10.1109/icarcv57592.2022.10004340 · Cited by 3 · see citing works
Attentive Partial Convolution for RGBD Image Inpainting
Dash, A., Wang, G., & Han, T. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589335.3651906 · Cited by 3 · see citing works
Object-Centric Representation and Rendering of 3D ScenesPreprint
Chen, C., Deng, F., & Ahn, S. (2020) · arXiv (Cornell University) · Cited by 3 · see citing works
Unsupervised Moving Object Segmentation with Atmospheric Turbulence
Qin, D., Saha, R. K., Chung, W., Jayasuriya, S., Ye, J., & Li, N. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72658-3_2 · Cited by 2 · see citing works
Depth Self-Supervision for Single Image Novel View Synthesis
Minelli, G., Poggi, M., & Salti, S. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10342058 · Cited by 2 · see citing works
Explicit3D: Graph network with spatial inference for single image 3D object detection
Liu, Y., & Yang, W. (2024) · Signal Processing Image Communication · doi:10.1016/j.image.2024.117120 · Cited by 1 · see citing works
Learning Nonparametric Human Mesh Reconstruction from a Single Image without Ground Truth MeshesPreprint
Lin, K., Wang, L., Jin, Y., Liu, Z., & Sun, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.00052
3D Reconstruction of Grabbed Objects Using a Single Image Based on Palletizing Task
Lü, L., Yang, J., Liang, J., & Zhang, A. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-63403-2_46
3D Shape Completion Using Multilayer Spherical Depth Parameters
Ağca, A., & Atalay, V. F. B. (2021) · Signal Processing and Communications Applications Conference (SIU) · doi:10.1109/siu53274.2021.9477782
Semantic 3D Reconstruction of a Scene and Its Effective Visualisation
Kniaz, V. V., Moshkantsev, P. V., Bordodymov, A. N., Mizginov, V. A., & Novikov, D. I. (2022) · International Conference on Computer Graphics and Vision (GraphiCon) · doi:10.20948/graphicon-2022-179-193
3D Semantic Scene Reconstruction from a Single Viewport
Denninger, M., & Triebel, R. (2023) · International Conference on Image Processing and Vision Engineering (IMPROVE) · doi:10.5220/0011747700003497
Direct Object Reconstruction on RGB-D Images in Cluttered Environment
Zieliński, M., Kulecki, B., & Belter, D. (2023) · European Conference on Mobile Robots · doi:10.1109/ecmr59166.2023.10256391
Explicit3d: Graph Network with Spatial Inference for Single Image 3d Object DetectionPreprint
Liu, Y., & Yang, W. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4604919
Ma, K., Cheng, Y., Ge, W., Zhang, Z., & Li, S. (2026) · Open Geosciences · doi:10.1515/geo-2025-0991
- Authors
- Shin, D., Ren, Z., Sudderth, E., & Fowlkes, C.
- Affiliations
- University of California, Irvine (US)
- Georgia Institute of Technology (US)
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 2172-2182
- OpenAlex
- W2970295759
- Cited by
- 57
- References
- 75
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 6 · 2023: 14 · 2022: 9 · 2021: 13 · 2020: 13
- FWCI
- 10.16
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1730158, 1253538, 1618806)
- Topics
- Physical Sciences › Engineering › Computational Mechanics › 3D Shape Modeling and Analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- 3D scene reconstruction, single-image 3D reconstruction, indoor scenes, fully convolutional network
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
