The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Cost Volume Pyramid Based Depth Inference for Multi-View Stereo
Abstract
We propose a cost volume-based neural network for depth inference from multi-view images. We demonstrate that building a cost volume pyramid in a coarse-to-fine manner instead of constructing a cost volume at a fixed resolution leads to a compact, lightweight network and allows us inferring high resolution depth maps to achieve better reconstruction results. To this end, we first build a cost volume based on uniform sampling of fronto-parallel planes across the entire depth range at the coarsest resolution of an image. Then, given current depth estimate, we construct new cost volumes iteratively on the pixelwise depth residual to perform depth map refinement. While sharing similar insight with Point-MVSNet as predicting and refining depth iteratively, we show that working on cost volume pyramid can lead to a more compact, yet efficient network structure compared with the Point-MVSNet on 3D points. We further provide detailed analyses of the relation between (residual) depth sampling and image resolution, which serves as a principle for building compact cost volume pyramid. Experimental results on benchmark datasets show that our model can perform 6× faster and has similar performance as state-of-the-art methods. Code is available at https://github.com/JiayuYANG/CVP-MVSNet.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
(2017) · Cited by 11,625
Structure-from-Motion Revisited
Schonberger, J. L., & Frahm, J.-M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.445 · Cited by 6,433
Accurate, Dense, and Robust Multiview Stereopsis
Furukawa, Y., & Ponce, J. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.161 · Cited by 3,019
PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume
Sun, D., Yang, X., Liu, M.-Y., & Kautz, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00931 · Cited by 2,648
Self-citationA Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms
Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.19 · Cited by 2,457
Pixelwise View Selection for Unstructured Multi-View Stereo
Schönberger, J. L., Zheng, E., Frahm, J.-M., & Pollefeys, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_31 · Cited by 2,315
Tanks and temples: benchmarking large-scale scene reconstruction
Knapitsch, A., Park, J., Zhou, Q.-Y., & Koltun, V. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073599 · Cited by 1,707
MVSNet: Depth Inference for Unstructured Multi-view Stereo
Yao, Y., Luo, Z., Li, S., Fang, T., & Quan, L. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01237-3_47 · Cited by 1,455
A Theory of Shape by Space Carving
Kutulakos, K. N., & Seitz, S. M. (2000) · International Journal of Computer Vision · doi:10.1023/a:1008191222954 · Cited by 1,254
Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches
Žbontar, J., & LeCun, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.05970 · Cited by 931
Efficient Deep Learning for Stereo Matching
Luo, W., Schwing, A. G., & Urtasun, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.614 · Cited by 741
Multi-camera Scene Reconstruction via Graph Cuts
Kolmogorov, V., & Zabih, R. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-47977-5_6 · Cited by 739
Large-Scale Data for Multiple-View Stereopsis
Aanæs, H., Jensen, R. R., Vogiatzis, G., Tola, E., & Dahl, A. B. (2016) · International Journal of Computer Vision · doi:10.1007/s11263-016-0902-9 · Cited by 672
Recurrent MVSNet for High-Resolution Multi-View Stereo Depth Inference
Yao, Y., Luo, Z., Li, S., Shen, T., Fang, T., & Quan, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00567 · Cited by 666
A space-sweep approach to true multi-image matching
Collins, R. T. (1996) · Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1996.517097 · Cited by 610
Silhouette and stereo fusion for 3D object modeling
Hernández Esteban, C., & Schmitt, F. (2004) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2004.03.016 · Cited by 435
Using Multiple Hypotheses to Improve Depth-Maps for Multi-View Stereo
Campbell, N. D. F., Vogiatzis, G., Hernández, C., & Cipolla, R. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88682-2_58 · Cited by 390
Point-Based Multi-View Stereo Network
Chen, R., Han, S., Xu, J., & Su, H. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00162 · Cited by 369
SurfaceNet: An End-to-End 3D Neural Network for Multiview Stereopsis
Ji, M., Gall, J., Zheng, H., Liu, Y., & Fang, L. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.253 · Cited by 361
Real-Time Visibility-Based Fusion of Depth Maps
Merrell, P., Akbarzadeh, A., Wang, L., Mordohai, P., Frahm, J.-M., Yang, R., Nister, D., & Pollefeys, M. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408984 · Cited by 358
Handling occlusions in dense multi-view stereo
Kang, S. B., Szeliski, R., & Chai, J. (2005) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 · doi:10.1109/cvpr.2001.990462 · Cited by 353
Efficient large-scale multi-view stereo for ultra high-resolution image sets
Tola, E., Strecha, C., & Fua, P. (2012) · Machine Vision and Applications · doi:10.1007/s00138-011-0346-8 · Cited by 328
DPSNet: End-to-end Deep Plane Sweep StereoPreprint
Im, S., Jeon, H., Lin, S. C.-F., & Kweon, I. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.00538 · Cited by 267
P-MVSNet: Learning Patch-Wise Matching Confidence Aggregation for Multi-View Stereo
Luo, K., Guan, T., Ju, L., Huang, H., & Luo, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01055 · Cited by 247
Object-centered surface reconstruction: Combining multi-image stereo and shading
Fua, P., & Leclerc, Y. G. (1995) · International Journal of Computer Vision · doi:10.1007/bf01428192 · Cited by 246
Variational principles, surface evolution, PDE's, level set methods and the stereo problem
Faugeras, O., & Keriven, R. (2005) · 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002. · doi:10.1109/ssbi.2002.1233990 · Cited by 244
Learning a Multi-View Stereo Machine
Kar, A., Häne, C., & Malik, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.05375 · Cited by 209
BA-Net: Dense Bundle Adjustment NetworksPreprint
Tang, C., & Tan, P. (2018) · arXiv (Cornell University) · Cited by 137
MVSCRF: Learning Multi-View Stereo With Conditional Random Fields
Xue, Y., Chen, J., Wan, W., Huang, Y., Cheng, Y., Li, T., & Bao, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00441 · Cited by 130
Learned Multi-patch Similarity
Hartmann, W., Galliani, S., Havlena, M., Gool, L. V., & Konrad Schindler, B. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.176 · Cited by 119
RayNet: Learning Volumetric 3D Reconstruction with Ray Potentials
Paschalidou, D., Ulusoy, A. O., Schmitt, C., Gool, L. V., & Geiger, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00410 · Cited by 102
CBMV: A Coalesced Bidirectional Matching Volume for Disparity Estimation
Batsos, K., Cai, C., & Mordohai, P. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00220 · Cited by 72
Shape Reconstruction Using Volume Sweeping and Learned Photoconsistency
Leroy, V., Franco, J.-S., & Boyer, E. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01240-3_48 · Cited by 63
Poxels: Probabilistic Voxelized Volume Reconstruction
Bonet, J. S. D. (1999) · Cited by 60
Variational stereovision and 3D scene flow estimation with statistical similarity measures
Pons, Keriven, Faugeras, & Hermosillo (2003) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2003.1238402 · Cited by 36
Variational Stereovision and 3D Scene Flow Estimation with Statistical Similarity Measures
Pons, J.-P., Keriven, R., Faugeras, O. D., & Hermosillo, G. (2003) · Cited by 28
Cited by
PatchmatchNet: Learned Multi-View Patchmatch Stereo
Wang, F., Galliani, S., Vogel, C., Speciale, P., & Pollefeys, M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01397 · Cited by 420
Track to Detect and Segment: An Online Multi-Object Tracker
Wu, J., Cao, J., Song, L., Wang, Y., Yang, M., & Yuan, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01217 · Cited by 406
CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching
Shen, Z., Dai, Y., & Rao, Z. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01369 · Cited by 351
Iterative Geometry Encoding Volume for Stereo Matching
Xu, G., Wang, X., Ding, X., & Yang, X. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.02099 · Cited by 349
Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers
Li, Z., Liu, X., Drenkow, N., Ding, A., Creighton, F. X., Taylor, R. H., & Unberath, M. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00614 · Cited by 335
Metadata
- Authors
- Yang, J., Mao, W., Alvarez, J. M., & Liu, M.
- Venue
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2020-06
- Pages
- 4876-4885
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3034600477
- Cited by
- 346
- References
- 44
- FWCI
- 14.36
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.