The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Consistent Depth Maps Recovery from a Video Sequence
Abstract
This paper presents a novel method for recovering consistent depth maps from a video sequence. We propose a bundle optimization framework to address the major difficulties in stereo reconstruction, such as dealing with image noise, occlusions, and outliers. Different from the typical multi-view stereo methods, our approach not only imposes the photo-consistency constraint, but also explicitly associates the geometric coherence with multiple frames in a statistical way. It thus can naturally maintain the temporal coherence of the recovered dense depth maps without over-smoothing. To make the inference tractable, we introduce an iterative optimization scheme by first initializing the disparity maps using a segmentation prior and then refining the disparities by means of bundle optimization. Instead of defining the visibility parameters, our method implicitly models the reconstruction noise as well as the probabilistic visibility. After bundle optimization, we introduce an efficient space-time fusion algorithm to further reduce the reconstruction noise. Our automatic depth recovery is evaluated using a variety of challenging video examples.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070
Mean shift: a robust approach toward feature space analysis
Comaniciu, D., & Meer, P. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.1000236 · Cited by 11,305
Fast approximate energy minimization via graph cuts
Boykov, Y., Veksler, O., & Zabih, R. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.969114 · Cited by 6,988
A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms
Scharstein, D., & Szeliski, R. (2002) · International Journal of Computer Vision · doi:10.1023/a:1014573219977 · Cited by 6,740
What energy functions can be minimized via graph cuts?
Kolmogorov, V., & Zabih, R. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.1262177 · Cited by 3,127
A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms
Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.19 · Cited by 2,457
The visual hull concept for silhouette-based image understanding
Laurentini, A. (1994) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.273735 · Cited by 1,450
Efficient Belief Propagation for Early Vision
Felzenszwalb, P. F., & Huttenlocher, D. P. (2006) · International Journal of Computer Vision · doi:10.1007/s11263-006-7899-4 · Cited by 1,345
Stereo matching using belief propagation
Sun, J., Zheng, N.-N., & Shum, H.-Y. (2003) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2003.1206509 · Cited by 1,271
Computing visual correspondence with occlusions using graph cuts
Kolmogorov, V., & Zabih, R. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937668 · Cited by 1,119
Okutomi, M., & Kanade, T. (1993) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.206955 · Cited by 939
Visual Modeling with a Hand-Held Camera
Pollefeys, M., Van Gool, L., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., & Koch, R. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000025798.50602.3a · Cited by 874
Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure
Klaus, A., Sormann, M., & Karner, K. (2006) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2006.1033 · Cited by 837
On benchmarking camera calibration and multi-view stereo for high resolution imagery
Strecha, C., von Hansen, W., Van Gool, L., Fua, P., & Thoennessen, U. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587706 · Cited by 814
Multi-View Stereo for Community Photo Collections
Goesele, M., Snavely, N., Curless, B., Hoppe, H., & Seitz, S. M. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408933 · Cited by 717
A space-sweep approach to true multi-image matching
Collins, R. T. (1996) · Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1996.517097 · Cited by 610
Photorealistic Scene Reconstruction by Voxel Coloring
Seitz, S. M., & Dyer, C. R. (1999) · International Journal of Computer Vision · doi:10.1023/a:1008176507526 · Cited by 492
Bobick, A. F., & Intille, S. S. (1999) · International Journal of Computer Vision · doi:10.1023/a:1008150329890 · Cited by 460
Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters
Tappen, & Freeman (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238444 · Cited by 451
A parallel stereo algorithm that produces dense depth maps and preserves image features
Fua, P. (1993) · Machine Vision and Applications · doi:10.1007/bf01212430 · Cited by 441
Variational principles, surface evolution, PDEs, level set methods, and the stereo problem
Faugeras, O., & Keriven, R. (1998) · IEEE Transactions on Image Processing · doi:10.1109/83.661183 · Cited by 424
Symmetric Stereo Matching for Occlusion Handling
Sun, J., Li, Y., Kang, S. B., & Shum, H.-Y. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.337 · Cited by 415
Real-Time Visibility-Based Fusion of Depth Maps
Merrell, P., Akbarzadeh, A., Wang, L., Mordohai, P., Frahm, J.-M., Yang, R., Nister, D., & Pollefeys, M. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408984 · Cited by 358
Handling occlusions in dense multi-view stereo
Kang, S. B., Szeliski, R., & Chai, J. (2005) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 · doi:10.1109/cvpr.2001.990462 · Cited by 353
Real-Time Plane-Sweeping Stereo with Multiple Sweeping Directions
Gallup, D., Frahm, J.-M., Mordohai, P., Yang, Q., & Pollefeys, M. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383245 · Cited by 322
Multilevel computational processes for visual surface reconstruction
Terzopoulos, D. (1983) · Computer Vision, Graphics, and Image Processing · doi:10.1016/0734-189x(83)90020-8 · Cited by 317
Stereo Matching Using Belief Propagation
Sun, J., Shum, H.-Y., & Zheng, N.-N. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-47967-8_34 · Cited by 296
Stereo for Image-Based Rendering using Image Over-Segmentation
Zitnick, C. L., & Kang, S. B. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-006-0018-8 · Cited by 282
Multi-View Stereo via Volumetric Graph-Cuts
Vogiatzis, G., Torr, P. H. S., & Cipolla, R. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.238 · Cited by 275
A global matching framework for stereo computation
Tao, H., Sawhney, H. S., & Kumar, R. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937562 · Cited by 268
Variable baseline/resolution stereo
Gallup, D., Frahm, J., Mordohai, P., & Pollefeys, M. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587671 · Cited by 245
Variational principles, surface evolution, PDE's, level set methods and the stereo problem
Faugeras, O., & Keriven, R. (2005) · 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002. · doi:10.1109/ssbi.2002.1233990 · Cited by 244
A Globally Optimal Algorithm for Robust TV-L1 Range Image Integration
Zach, C., Pock, T., & Bischof, H. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408983 · Cited by 220
Yang, Q., Wang, L., Yang, R., Stewenius, H., & Nister, D. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.292 · Cited by 181
Combined Depth and Outlier Estimation in Multi-View Stereo
Strecha, C., Fransens, R., & Van Gool, L. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.78 · Cited by 180
Accurate multi-view reconstruction using robust binocular stereo and surface meshing
Bradley, D., Boubekeur, T., & Heidrich, W. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587792 · Cited by 169
Determination of optical flow and its discontinuities using non-linear diffusion
Proesmans, M., Van Gool, L., Pauwels, E., & Oosterlinck, A. (1994) · Lecture Notes in Computer Science · doi:10.1007/bfb0028362 · Cited by 164
Wide-baseline stereo from multiple views: a probabilistic account
Strecha, C., Fransens, R., & Gool, L. V. (2004) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2004.1315080 · Cited by 131
Global stereo reconstruction under second order smoothness priors
Woodford, O. J., Torr, P. H. S., Reid, I. D., & Fitzgibbon, A. W. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587672 · Cited by 124
Symmetrical Dense Optical Flow Estimation with Occlusions Detection
Álvarez, L., Rachid Deriche, B., Papadopoulo, T., & Sánchez, J. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-007-0041-4 · Cited by 122
TransforMesh : A Topology-Adaptive Mesh-Based Approach to Surface Evolution
Zaharescu, A., Boyer, E., & Horaud, R. (2007) · Springer eBooks · doi:10.1007/978-3-540-76390-1_17 · Cited by 113
Using Photographs to Enhance Videos of a Static Scene
Bhat, P., Zitnick, C. L., Snavely, N., Agarwala, A., Agrawala, M., Cohen, M., Curless, B., & Kang, S. B. (2007) · Eurographics · doi:10.5555/2383847.2383889 · Cited by 105
Bayesian 3D Modeling from Images Using Multiple Depth Maps
Gargallo, P., & Sturm, P. F. (2005) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2005.84 · Cited by 101
Probabilistic visibility for multi-view stereo
Hernandez, C., Vogiatzis, G., & Cipolla, R. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383193 · Cited by 98
Stereo reconstruction with mixed pixels using adaptive over-segmentation
Taguchi, Y., Wilburn, B. S., & Zitnick, C. L. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587691 · Cited by 96
Dense 3D motion capture from synchronized video streams
Furukawa, Y., & Ponce, J. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587495 · Cited by 94
Temporally Consistent Reconstruction from Multiple Video Streams Using Enhanced Belief Propagation
Larsen, E. S., Mordohai, P., Pollefeys, M., & Fuchs, H. (2007) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2007.4409013 · Cited by 84
Extracting View-Dependent Depth Maps from a Collection of Images
Kang, S. B., & Szeliski, R. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000015917.35451.df · Cited by 79
A symmetric patch-based correspondence model for occlusion handling
Deng, Y. P., Yang, Q., Lin, X., & Tang, X. (2005) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2005.23 · Cited by 70
Robust Metric Reconstruction from Challenging Video Sequences
Zhang, G., Qin, X., Hua, W., Wong, T., Heng, P., & Bao, H. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383118 · Cited by 56
Self-citationDynamic depth recovery from multiple synchronized video streams
Hai, T., Sawhney, H., & Kumar, R. (2005) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2001.990464 · Cited by 53
PDE-based multi-view depth estimation
Strecha, C., & Gool, L. V. (2003) · International Symposium on 3D Data Processing, Visualization and Transmission · doi:10.1109/tdpvt.2002.1024097 · Cited by 25
Cited by
Fattal, R. (2014) · ACM Transactions on Graphics · doi:10.1145/2651362 · Cited by 1,167
Depth Transfer: Depth Extraction from Video Using Non-Parametric Sampling
Karsch, K., Liu, C., & Kang, S. B. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2316835 · Cited by 453
Accurate Multiple View 3D Reconstruction Using Patch-Based Stereo for Large-Scale Scenes
Shen, S. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2013.2237921 · Cited by 312
Dehazing Evaluation: Real-World Benchmark Datasets, Criteria, and Baselines
Zhao, S., Zhang, L., Huang, S., Shen, Y., & Zhao, S. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2995264 · Cited by 169
3D Object Detection from Images for Autonomous Driving: A Survey
Ma, X., Ouyang, W., Simonelli, A., & Ricci, E. (2022) · arXiv · doi:10.48550/arxiv.2202.02980 · Cited by 147
Metadata
- Authors
- Zhang, G., Jia, J., Wong, T.-T., & Bao, H.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2009-06
- Volume
- 31
- Issue
- 6
- Pages
- 974-988
- ISSN
- 0162-8828 (ISSN-L)
- OpenAlex
- W2156380531
- Cited by
- 270
- References
- 52
- FWCI
- 30.42
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.