The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Learning-based view synthesis for light field cameras
Open accessAbstract
With the introduction of consumer light field cameras, light field imaging has recently become widespread. However, there is an inherent trade-off between the angular and spatial resolution, and thus, these cameras often sparsely sample in either spatial or angular domain. In this paper, we use machine learning to mitigate this trade-off. Specifically, we propose a novel learning-based approach to synthesize new views from a sparse set of input views. We build upon existing view synthesis techniques and break down the process into disparity and color estimation components. We use two sequential convolutional neural networks to model these two components and train both networks simultaneously by minimizing the error between the synthesized and ground truth images. We show the performance of our approach using only four corner sub-aperture views from the light fields captured by the Lytro Illum camera. Experimental results show that our approach synthesizes high-quality images that are superior to the state-of-the-art techniques on a variety of challenging real-world scenes. We believe our method could potentially decrease the required angular resolution of consumer light field cameras, which allows their spatial resolution to increase.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
(2010) · Cited by 14,134
Learning a Deep Convolutional Network for Image Super-Resolution
Dong, C., Loy, C. C., He, K., & Tang, X. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10593-2_13 · Cited by 5,494
Levoy, M., & Hanrahan, P. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237199 · Cited by 3,755
Accurate, Dense, and Robust Multiview Stereopsis
Furukawa, Y., & Ponce, J. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.161 · Cited by 3,019
MatConvNet: Convolutional Neural Networks for MATLAB
Vedaldi, A., & Lenc, K. (2015) · Proceedings of the 23rd ACM international conference on Multimedia · doi:10.1145/2733373.2807412 · Cited by 2,444
Light Field Photography with a Hand-held Plenoptic Camera
Ng, R., Levoy, M., Brédif, M., Duval, G., Horowitz, M., & Hanrahan, P. (2005) · HAL · Cited by 1,749
Image denoising: Can plain neural networks compete with BM3D?
Burger, H. C., Schuler, C. J., & Harmeling, S. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247952 · Cited by 1,325
Single lens stereo with a plenoptic camera
Adelson, E. H., & Wang, J. Y. A. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.121783 · Cited by 1,101
High performance imaging using large camera arrays
Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Horowitz, M., & Levoy, M. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073259 · Cited by 1,042
Learning a convolutional neural network for non-uniform motion blur removal
Sun, J., Cao, W., Xu, Z., & Ponce, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298677 · Cited by 906
Learning to generate chairs with convolutional neural networks
Dosovitskiy, A., Springenberg, J. T., & Brox, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298761 · Cited by 640
Variational Light Field Analysis for Disparity Estimation and Super-Resolution
Wanner, S., & Goldluecke, B. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.147 · Cited by 615
View Synthesis by Appearance Flow
Zhou, T., Tulsiani, S., Sun, W., Malik, J., & Efros, A. A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_18 · Cited by 579
MatConvNet - Convolutional Neural Networks for MATLAB
Vedaldi, A., & Lenc, K. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.4564 · Cited by 578
Deep Stereo: Learning to Predict New Views from the World's Imagery
Flynn, J., Neulander, I., Philbin, J., & Snavely, N. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.595 · Cited by 574
Accurate depth map estimation from a lenslet light field camera
Jeon, H., Park, J., Choe, G., Park, J., Bok, Y., Tai, Y., & Kweon, I. S. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298762 · Cited by 542
Depth from Combining Defocus and Correspondence Using Light-Field Cameras
Tao, M. W., Hadap, S., Malik, J., & Ramamoorthi, R. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.89 · Cited by 506
Self-citationLearning a Deep Convolutional Network for Light-Field Image Super-Resolution
Yoon, Y., Jeon, H.-G., Yoo, D., Lee, J.-Y., & Kweon, I. S. (2015) · 2015 IEEE International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2015.17 · Cited by 403
Compressive light field photography using overcomplete dictionaries and optimized projections
Marwah, K., Wetzstein, G., Bando, Y., & Raskar, R. (2013) · ACM Transactions on Graphics · doi:10.1145/2461912.2461914 · Cited by 398
Globally consistent depth labeling of 4D light fields
Wanner, S., & Goldlüecke, B. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247656 · Cited by 383
Depth synthesis and local warps for plausible image-based navigation
Chaurasia, G., Duchene, S., Sorkine-Hornung, O., & Drettakis, G. (2013) · ACM Transactions on Graphics · doi:10.1145/2487228.2487238 · Cited by 357
Occlusion-Aware Depth Estimation Using Light-Field Cameras
Wang, T.-C., Efros, A. A., & Ramamoorthi, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.398 · Cited by 326
Self-citationHigh performance imaging using large camera arrays
Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Horowitz, M., & Levoy, M. (2005) · ACM SIGGRAPH 2005 Papers · doi:10.1145/1186822.1073259 · Cited by 258
Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis
Yang, J., Reed, S., Yang, M., & Lee, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.00706 · Cited by 246
Bishop, T. E., Zanetti, S., & Favaro, P. (2009) · IEEE International Conference on Computational Photography (ICCP) · doi:10.1109/iccphot.2009.5559010 · Cited by 216
Linear view synthesis using a dimensionality gap light field prior
Levin, A., & Durand, F. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5539854 · Cited by 214
Mitra, K., & Veeraraghavan, A. (2012) · 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2012.6239346 · Cited by 198
3D object manipulation in a single photograph using stock 3D models
Kholgade, N., Simon, T., Efros, A. A., & Sheikh, Y. A. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601209 · Cited by 175
Moving gradients: a path-based method for plausible image interpolation
Mahajan, D., Huang, F.-C., Matusik, W., Ramamoorthi, R., & Belhumeur, P. (2009) · ACM Transactions on Graphics · doi:10.1145/1531326.1531348 · Cited by 172
Self-citationCho, D., Lee, M., Kim, S., & Tai, Y. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.407 · Cited by 163
Image-based rendering using image-based priors
Fitzgibbon, Wexler, & Zisserman (2003) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2003.1238625 · Cited by 161
Convolutional Networks for Shape from Light Field
Heber, S., & Pock, T. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.407 · Cited by 149
PlenoPatch: Patch-Based Plenoptic Image Manipulation
Zhang, F., Wang, J., Shechtman, E., Zhou, Z., Shi, J., & Hu, S. (2016) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2016.2532329 · Cited by 94
Silhouette‐Aware Warping for Image‐Based Rendering
Chaurasia, G., Sorkine‐Hornung, O., & George Drettakis, B. (2011) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2011.01981.x · Cited by 90
Shechtman, E., Rav-Acha, A., Irani, M., & Seitz, S. M. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540159 · Cited by 76
Single-view to Multi-view: Reconstructing Unseen Views with a Convolutional NetworkPreprint
Tatarchenko, M., Dosovitskiy, A., & Brox, T. (2015) · arXiv (Cornell University) · Cited by 68
3D-Assisted Feature Synthesis for Novel Views of an Object
Su, H., Wang, F., Yi, E., & Guibas, L. J. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.307 · Cited by 39
3D-Assisted Image Feature Synthesis for Novel Views of an ObjectPreprint
Su, H., Wang, F., Yi, L., & Guibas, L. J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.0003 · Cited by 18
Directional Super-Resolution by Means of Coded Sampling and Guided Upsampling
Schedl, D. C., Birklbauer, C., & Bimber, O. (2015) · IEEE International Conference on Computational Photography (ICCP) · doi:10.1109/iccphot.2015.7168365 · Cited by 13
Cited by
Deferred neural rendering: image synthesis using neural textures
Thies, J., Zollhöfer, M., & Nießner, M. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323035 · Cited by 1,356
Local light field fusion: practical view synthesis with prescriptive sampling guidelines
Mildenhall, B., Srinivasan, P. P., Ortiz-Cayon, R., Kalantari, N. K., Ramamoorthi, R., Ng, R., & Kar, A. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3322980 · Cited by 1,175
Self-citationMVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo
Chen, A., Xu, Z., Zhao, F., Zhang, X., Xiang, F., Yu, J., & Su, H. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01386 · Cited by 797
Stereo magnification: learning view synthesis using multiplane images
Zhou, T., Tucker, R., Flynn, J., Fyffe, G., & Snavely, N. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201323 · Cited by 776
Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation
Caballero, J., Ledig, C., Aitken, A., Acosta, A., Totz, J., Wang, Z., & Shi, W. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.304 · Cited by 763
Metadata
- Authors
- Kalantari, N. K., Wang, T.-C., & Ramamoorthi, R.
- Venue
- ACM Transactions on Graphics
- Published
- 2016-11-11
- Volume
- 35
- Issue
- 6
- Pages
- 1-10
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W2551052086
- Cited by
- 726
- References
- 42
- FWCI
- 38.34
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.