The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Scene reconstruction from high spatio-angular resolution light fields
Abstract
This paper describes a method for scene reconstruction of complex, detailed environments from 3D light fields. Densely sampled light fields in the order of 10 9 light rays allow us to capture the real world in unparalleled detail, but efficiently processing this amount of data to generate an equally detailed reconstruction represents a significant challenge to existing algorithms. We propose an algorithm that leverages coherence in massive light fields by breaking with a number of established practices in image-based reconstruction. Our algorithm first computes reliable depth estimates specifically around object boundaries instead of interior regions, by operating on individual light rays instead of image patches. More homogeneous interior regions are then processed in a fine-to-coarse procedure rather than the standard coarse-to-fine approaches. At no point in our method is any form of global optimization performed. This allows our algorithm to retain precise object contours while still ensuring smooth reconstructions in less detailed areas. While the core reconstruction method handles general unstructured input, we also introduce a sparse representation and a propagation scheme for reliable depth estimates which make our algorithm particularly effective for 3D input, enabling fast and memory efficient processing of "Gigaray light fields" on a standard GPU. We show dense 3D reconstructions of highly detailed scenes, enabling applications such as automatic segmentation and image-based rendering, and provide an extensive evaluation and comparison to existing image-based reconstruction techniques.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mean shift: a robust approach toward feature space analysis
Comaniciu, D., & Meer, P. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.1000236 · Cited by 11,305
A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms
Scharstein, D., & Szeliski, R. (2002) · International Journal of Computer Vision · doi:10.1023/a:1014573219977 · Cited by 6,740
Levoy, M., & Hanrahan, P. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237199 · Cited by 3,755
Pattern Classification and Scene Analysis.
Clarke, M. R. B., Duda, R. O., & Hart, P. E. (1974) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2344977 · Cited by 3,222
Accurate, Dense, and Robust Multiview Stereopsis
Furukawa, Y., & Ponce, J. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.161 · Cited by 3,019
Pattern Classification (2nd ed.)
Duda, R. O., Hart, P. E., & Stork, D. G. (1999) · Cited by 2,520
Gortler, S. J., Grzeszczuk, R., Szeliski, R., & Cohen, M. F. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237200 · Cited by 2,458
A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms
Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.19 · Cited by 2,457
Modeling the World from Internet Photo Collections
Snavely, N., Seitz, S. M., & Szeliski, R. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0107-3 · Cited by 2,155
Light Field Photography with a Hand-held Plenoptic Camera
Ng, R., Levoy, M., Brédif, M., Duval, G., Horowitz, M., & Hanrahan, P. (2005) · HAL · Cited by 1,749
Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information
Hirschmuller, H. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.56 · Cited by 1,698
High-quality video view interpolation using a layered representation
Zitnick, C. L., Kang, S. B., Uyttendaele, M., Winder, S., & Szeliski, R. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015766 · Cited by 1,385
Computing visual correspondence with occlusions using graph cuts
Kolmogorov, V., & Zabih, R. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937668 · Cited by 1,119
Single lens stereo with a plenoptic camera
Adelson, E. H., & Wang, J. Y. A. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.121783 · Cited by 1,101
High performance imaging using large camera arrays
Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Horowitz, M., & Levoy, M. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073259 · Cited by 1,042
Epipolar-plane image analysis: An approach to determining structure from motion
Bolles, R. C., Baker, H. H., & Marimont, D. H. (1987) · International Journal of Computer Vision · doi:10.1007/bf00128525 · Cited by 892
Unstructured lumigraph rendering
Buehler, C., Bosse, M., McMillan, L., Gortler, S., & Cohen, M. (2001) · Proceedings of the 28th annual conference on Computer graphics and interactive techniques · doi:10.1145/383259.383309 · Cited by 881
Efficient Large-Scale Stereo Matching
Geiger, A., Roser, M., & Urtasun, R. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-19315-6_3 · Cited by 761
Towards Internet-scale multi-view stereo
Furukawa, Y., Curless, B., Seitz, S. M., & Szeliski, R. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539802 · Cited by 751
Dynamically reparameterized light fields
Isaksen, A., McMillan, L., & Gortler, S. J. (2000) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/344779.344929 · Cited by 507
Depth from Combining Defocus and Correspondence Using Light-Field Cameras
Tao, M. W., Hadap, S., Malik, J., & Ramamoorthi, R. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.89 · Cited by 506
Photorealistic Scene Reconstruction by Voxel Coloring
Seitz, S. M., & Dyer, C. R. (1999) · International Journal of Computer Vision · doi:10.1023/a:1008176507526 · Cited by 492
Fast cost-volume filtering for visual correspondence and beyond
Rhemann, C., Hosni, A., Bleyer, M., Rother, C., & Gelautz, M. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995372 · Cited by 485
Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A., & Tumblin, J. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276463 · Cited by 469
Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A., & Tumblin, J. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276463 · Cited by 444
Surface light fields for 3D photography
Wood, D. N., Azuma, D. I., Aldinger, K., Curless, B., Duchamp, T., Salesin, D. H., & Stuetzle, W. (2000) · Proceedings of the 27th annual conference on Computer graphics and interactive techniques - SIGGRAPH '00 · doi:10.1145/344779.344925 · Cited by 430
Globally consistent depth labeling of 4D light fields
Wanner, S., & Goldlüecke, B. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247656 · Cited by 383
High-quality single-shot capture of facial geometry
Beeler, T., Bickel, B., Beardsley, P., Sumner, B., & Gross, M. (2010) · ACM Transactions on Graphics · doi:10.1145/1778765.1778777 · Cited by 313
Self-citationStereo for Image-Based Rendering using Image Over-Segmentation
Zitnick, C. L., & Kang, S. B. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-006-0018-8 · Cited by 282
Programmable aperture photography: multiplexed light field acquisition
Liang, C.-K., Lin, T.-H., Wong, B.-Y., Liu, C., & Chen, H. H. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360654 · Cited by 261
Towards high-resolution large-scale multi-view stereo
Hiep, V. H., Keriven, R., Labatut, P., & Pons, J.-P. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvprw.2009.5206617 · Cited by 228
Reconstructing Occluded Surfaces Using Synthetic Apertures: Stereo, Focus and Robust Measures
Vaish, V., Levoy, M., Szeliski, R., Zitnick, C. L., & Kang, S. B. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.244 · Cited by 225
Davis, A., Levoy, M., & Durand, F. (2012) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2012.03009.x · Cited by 224
Bishop, T. E., Zanetti, S., & Favaro, P. (2009) · IEEE International Conference on Computational Photography (ICCP) · doi:10.1109/iccphot.2009.5559010 · Cited by 216
Light field mapping: efficient representation and hardware rendering of surface light fields
Chen, W.-C., Bouguet, J.-Y., Chu, M. H., & Grzeszczuk, R. (2002) · Proceedings of the 29th annual conference on Computer graphics and interactive techniques · doi:10.1145/566570.566601 · Cited by 172
Object stereo — Joint stereo matching and object segmentation
Bleyer, M., Rother, C., Kohli, P., Scharstein, D., & Sinha, S. N. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995581 · Cited by 168
A self-reconfigurable camera array
Zhang, C., & Chen, T. (2004) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1186223.1186412 · Cited by 125
Sparse Occlusion Detection with Optical Flow
Ayvacı, A., Raptis, M., & Soatto, S. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-011-0490-7 · Cited by 121
Image-Based Rendering Using Image-Based Priors
Fitzgibbon, A. W., Wexler, Y., & Zisserman, A. (2005) · International Journal of Computer Vision · doi:10.1007/s11263-005-6643-9 · Cited by 104
Stereo Matching with Reliable Disparity Propagation
Sun, X., Mei, X., Jiao, S., Zhou, M., & Wang, H. (2011) · International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission · doi:10.1109/3dimpvt.2011.24 · Cited by 94
Efficient Sampling of Disparity Space for Fast And Accurate Matching
Čech, J., & Šára, R. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383355 · Cited by 89
Reducing Plenoptic Camera Artifacts
Georgiev, T., & Lumsdaine, A. (2010) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2010.01662.x · Cited by 88
Extracting View-Dependent Depth Maps from a Collection of Images
Kang, S. B., & Szeliski, R. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000015917.35451.df · Cited by 86
Symmetric Sub-pixel Stereo Matching
Szeliski, R., & Scharstein, D. (2002) · Lecture notes in computer science · doi:10.1007/3-540-47967-8_35 · Cited by 85
Extracting objects from range and radiance images
Yu, Y., Ferencz, A., & Malik, J. (2001) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/2945.965349 · Cited by 64
Learning to find occlusion regions
Humayun, A., Aodha, O. M., & Brostow, G. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995517 · Cited by 62
Generating EPI Representations of 4D Light Fields with a Single Lens Focused Plenoptic Camera
Wanner, S., Keuper, J., & Jähne, B. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24028-7_9 · Cited by 56
Structure and motion from scene registration
Basha, T., Avidan, S., Hornung, A., & Matusik, W. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247830 · Cited by 31
Panoramic EPI generation and analysis of video from a moving platform with vibration
Zhu, Z., Xu, G., & Lin, X. (2003) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.1999.784732 · Cited by 23
Das, S., & Ahuja, N. (1996) · Artificial Intelligence · doi:10.1016/0004-3702(95)00036-4 · Cited by 5
Reconstruction of coarse depth estimates using hierarchical image partitions
Dórea, C. C., & de Queiroz, R. L. (2012) · Multidimensional Systems and Signal Processing · doi:10.1007/s11045-012-0191-6 · Cited by 1
Cited by
Fattal, R. (2014) · ACM Transactions on Graphics · doi:10.1145/2651362 · Cited by 1,167
A Dataset and Evaluation Methodology for Depth Estimation on 4D Light Fields
Honauer, K., Johannsen, O., Kondermann, D., & Goldluecke, B. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-54187-7_2 · Cited by 642
Towards real-time photorealistic 3D holography with deep neural networks
Shi, L., Li, B., Kim, C., Kellnhofer, P., & Matusik, W. (2021) · Nature · doi:10.1038/s41586-020-03152-0 · Cited by 637
Self-citationFurukawa, Y., & Hernández, C. (2015) · Foundations and Trends in Computer Graphics and Vision · doi:10.1561/0600000052 · Cited by 598
Light Field Image Processing: An Overview
Wu, G., Masia, B., Jarabo, A., Zhang, Y., Wang, L., Dai, Q., Chai, T., & Liu, Y. (2017) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2017.2747126 · Cited by 586
Metadata
- Authors
- Kim, C., Zimmer, H., Pritch, Y., Sorkine-Hornung, A., & Gross, M.
- Venue
- ACM Transactions on Graphics
- Published
- 2013-07-21
- Volume
- 32
- Issue
- 4
- Pages
- 1-12
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W2007165614
- Cited by
- 439
- References
- 51
- FWCI
- 47.21
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.