Stereo Magnification: Learning View Synthesis using Multiplane Images
Open accessAbstract
The view synthesis problem--generating novel views of a scene from known imagery--has garnered recent attention due in part to compelling applications in virtual and augmented reality. In this paper, we explore an intriguing scenario for view synthesis: extrapolating views from imagery captured by narrow-baseline stereo cameras, including VR cameras and now-widespread dual-lens camera phones. We call this problem stereo magnification, and propose a learning framework that leverages a new layered representation that we call multiplane images (MPIs). Our method also uses a massive new data source for learning view extrapolation: online videos on YouTube. Using data mined from such videos, we train a deep network that predicts an MPI from an input stereo image pair. This inferred MPI can then be used to synthesize a range of novel views of the scene, including views that extrapolate significantly beyond the input baseline. We show that our method compares favorably with several recent view synthesis methods, and demonstrate applications in magnifying narrow-baseline stereo images.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892 · see citing works
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392 · see citing works
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070 · see citing works
TensorFlow: A system for large-scale machine learningPreprint
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.5555/3026877.3026899 · Cited by 16,369 · see citing works
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623 · see citing works
Layer NormalizationPreprint
Ba, J., Kiros, J., & Hinton, G. E. (2016) · arXiv.org · doi:10.48550/arxiv.1607.06450 · Cited by 13,745 · see citing works
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283 · see citing works
ORB-SLAM: A Versatile and Accurate Monocular SLAM System
Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2463671 · Cited by 6,778 · see citing works
Structure-from-Motion Revisited
Schonberger, J. L., & Frahm, J.-M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.445 · Cited by 6,433 · see citing works
Levoy, M., & Hanrahan, P. (1996) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/237170.237199 · Cited by 3,789 · see citing works
Unsupervised Monocular Depth Estimation with Left-Right Consistency
Godard, C., Aodha, O. M., & Brostow, G. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.699 · Cited by 3,270 · see citing works
Engel, J. J., Koltun, V., & Cremers, D. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2658577 · Cited by 2,995 · see citing works
Unsupervised Learning of Depth and Ego-Motion from Video
Zhou, T., Brown, M., Snavely, N., & Lowe, D. G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.700 · Cited by 2,910 · see citing works
Self-citationGortler, S. J., Grzeszczuk, R., Szeliski, R., & Cohen, M. F. (1996) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/237170.237200 · Cited by 2,459 · see citing works
SVO: Fast semi-direct monocular visual odometry
Forster, C., Pizzoli, M., & Scaramuzza, D. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6906584 · Cited by 2,205 · see citing works
Modeling and rendering architecture from photographs
Debevec, P. E., Taylor, C. J., & Malik, J. (1996) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/237170.237191 · Cited by 1,928 · see citing works
Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
Garg, R., B.G., V. K., Carneiro, G., & Reid, I. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_45 · Cited by 1,632 · see citing works
Shade, J., Gortler, S., He, L.-W., & Szeliski, R. (1998) · Proceedings of the 25th annual conference on Computer graphics and interactive techniques - SIGGRAPH '98 · doi:10.1145/280814.280882 · Cited by 1,115 · see citing works
View interpolation for image synthesis
Chen, S. E., & Williams, L. R. (1993) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/166117.166153 · Cited by 1,108 · see citing works
Representing moving images with layers
Wang, J. Y. A., & Adelson, E. H. (1994) · IEEE Transactions on Image Processing · doi:10.1109/83.334981 · Cited by 1,102 · see citing works
Photographic Image Synthesis with Cascaded Refinement Networks
Chen, Q., & Koltun, V. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.168 · Cited by 887 · see citing works
Video Frame Synthesis Using Deep Voxel Flow
Liu, Z., Yeh, R. A., Tang, X., Liu, Y., & Agarwala, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.478 · Cited by 766 · see citing works
Porter, T., & Duff, T. (1984) · ACM SIGGRAPH Computer Graphics · doi:10.1145/964965.808606 · Cited by 728 · see citing works
View Synthesis by Appearance Flow
Zhou, T., Tulsiani, S., Sun, W., Malik, J., & Efros, A. A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46493-0_18 · Cited by 614 · see citing works
Self-citationMulti-View Supervision for Single-View Reconstruction via Differentiable Ray Consistency
Tulsiani, S., Zhou, T., Efros, A. A., & Malik, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.30 · Cited by 528 · see citing works
Self-citationBurst photography for high dynamic range and low-light imaging on mobile cameras
Hasinoff, S. W., Sharlet, D., Geiss, R., et al. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2980254 · Cited by 483 · see citing works
(2016) · doi:10.1007/978-3-319-46493-0_51 · Cited by 411 · see citing works
High-quality video view interpolation using a layered representation
Zitnick, C. L., Kang, S. B., Uyttendaele, M., Winder, S., & Szeliski, R. (2004) · ACM SIGGRAPH 2004 Papers · doi:10.1145/1186562.1015766 · Cited by 367 · see citing works
Soft 3D reconstruction for view synthesis
Penner, E., & Zhang, L. (2017) · ACM Transactions on Graphics · doi:10.1145/3130800.3130855 · Cited by 358 · see citing works
SfM-Net: Learning of Structure and Motion from VideoPreprint
Vijayanarasimhan, S., Ricco, S., Schmid, C., Sukthankar, R., & Fragkiadaki, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.07804 · Cited by 327 · see citing works
Learning to Synthesize a 4D RGBD Light Field from a Single Image
Srinivasan, P. P., Wang, T., Sreelal, A., Ramamoorthi, R., & Ng, R. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.246 · Cited by 243 · see citing works
Wetzstein, G., Lanman, D. R., Heidrich, W., et al. (2011) · ACM SIGGRAPH Conference Papers (SIGGRAPH) · doi:10.1145/1964921.1964990 · Cited by 148 · see citing works
Hedman, P., Alsisan, S., Szeliski, R., & Kopf, J. (2017) · ACM Transactions on Graphics · doi:10.1145/3130800.3130828 · Cited by 119 · see citing works
Joint view expansion and filtering for automultiscopic 3D displays
Didyk, P., Sitthi‐Amorn, P., Freeman, W. T., et al. (2013) · ACM Transactions on Graphics · doi:10.1145/2508363.2508376 · Cited by 47 · see citing works
Kellnhofer, P., Didyk, P., Wang, S.-P., et al. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073617 · Cited by 20 · see citing works
Optimizing stereo‐to‐multiview conversion for autostereoscopic displays
Chapiro, A., Heinzle, S., Aydın, T. O., Poulakos, S., Zwicker, M., Smolić, A., & Groß, M. H. (2014) · Computer Graphics Forum · doi:10.1111/cgf.12291 · Cited by 19 · see citing works
Computing and fabricating multilayer models
Holroyd, M., Baran, I., Lawrence, J., & Matusik, W. (2011) · Proceedings of the 2011 SIGGRAPH Asia Conference · doi:10.1145/2024156.2024221
Light field from micro-baseline image pair
Zhang, Z., Liu, Y., & Dai, Q. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299004
Flagged first, then by citations
Research works printed but not linked: 6
- Alexey Dosovitskiy and Thomas Brox. 2016. Generating images with perceptual similarity metrics based on deep networks. In NIPS .
- John Flynn, Ivan Neulander, James Philbin, and Noah Snavely. 2016. DeepStereo: Learning to Predict New Views From the World's Imagery. In CVPR .
- Hyowon Ha, Sunghoon Im, Jaesik Park, Hae-Gon Jeon, and In So Kweon. 2016. Highquality Depth from Uncalibrated Small Motion Clip. In CVPR .
- Max Jaderberg, Karen Simonyan, Andrew Zisserman, et al. 2015. Spatial transformer networks. In NIPS .
- Nima Khademi Kalantari, Ting-Chun Wang, and Ravi Ramamoorthi. 2016. LearningBased View Synthesis for Light Field Cameras. In Proc. SIGGRAPH Asia .
- Fisher Yu and David Gallup. 2014. 3D Reconstruction from Accidental Motion. In CVPR . Fisher Yu and Vladlen Koltun. 2016. Multi-Scale Context Aggregation by Dilated Convolutions. In ICLR .
Cited by
Sorted by citations
Plenoxels: Radiance Fields without Neural Networks
Fridovich-Keil, S., Yu, A., Tancik, M., Chen, Q., Recht, B., & Kanazawa, A. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00542 · Cited by 1,418 · see citing works
NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections
Martin-Brualla, R., Radwan, N., Sajjadi, M. S. M., Barron, J. T., Dosovitskiy, A., & Duckworth, D. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00713 · Cited by 1,414 · see citing works
PlenOctrees for Real-time Rendering of Neural Radiance Fields
Yu, A., Li, R., Tancik, M., Li, H., Ng, R., & Kanazawa, A. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00570 · Cited by 876 · see citing works
Block-NeRF: Scalable Large Scene Neural View Synthesis
Tancik, M., Casser, V., Yan, X., Pradhan, S., Mildenhall, B., Srinivasan, P. P., Barron, J. T., & Kretzschmar, H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00807 · Cited by 808 · see citing works
Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes
Li, Z., Niklaus, S., Snavely, N., & Wang, O. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00643 · Cited by 713 · see citing works
Self-citationLombardi, S., Simon, T., Saragih, J., Schwartz, G., Lehrmann, A., & Sheikh, Y. A. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323020 · Cited by 729 · see citing works
BARF: Bundle-Adjusting Neural Radiance Fields
Lin, C.-H., Ma, W.-C., Torralba, A. B., & Lucey, S. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00569 · Cited by 649 · see citing works
Point-NeRF: Point-based Neural Radiance Fields
Xu, Q., Xu, Z., Philip, J., Bi, S., Shu, Z., Sunkavalli, K., & Neumann, U. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00536 · Cited by 601 · see citing works
DeepView: View Synthesis With Learned Gradient Descent
Flynn, J., Broxton, M. J., Debevec, P. E., DuVall, M., Fyffe, G., Overbeck, R., Snavely, N., & Tucker, R. P. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00247 · Cited by 432 · see citing works
Self-citationNeural 3D Video Synthesis from Multi-view Video
Li, T., Slavcheva, M., Zollhöfer, M., Green, S., Lassner, C., Kim, C., Schmidt, T., Lovegrove, S. J., Goesele, M., Newcombe, R., et al. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00544 · Cited by 406 · see citing works
NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images
Mildenhall, B., Hedman, P., Martin-Brualla, R., Srinivasan, P. P., & Barron, J. T. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01571 · Cited by 369 · see citing works
Is Pseudo-Lidar needed for Monocular 3D Object detection?
Park, D., Ambrus, R., Guizilini, V., Li, J., & Gaidon, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00313 · Cited by 334 · see citing works
Rematas, K., Liu, A., Srinivasan, P., Barron, J. T., Tagliasacchi, A., Funkhouser, T. A., & Ferrari, V. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01259 · Cited by 298 · see citing works
DONeRF: Towards Real‐Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks
Neff, T., Stadlbauer, P., Parger, M., Kurz, A., Mueller, J. H., Chaitanya, C. R. A., Kaplanyan, A., & Steinberger, M. (2021) · Computer Graphics Forum · doi:10.1111/cgf.14340 · Cited by 256 · see citing works
NoPe-NeRF: Optimising Neural Radiance Field with No Pose Prior
Bian, W., Wang, Z., Li, K., & Bian, J.-W. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00405 · Cited by 247 · see citing works
GRAM: Generative Radiance Manifolds for 3D-Aware Image Generation
Deng, Y., Yang, J., Xiang, J., & Tong, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01041 · Cited by 207 · see citing works
Riegler, G., & Koltun, V. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01204 · Cited by 181 · see citing works
Pulsar: Efficient Sphere-based Neural Rendering
Lassner, C., & Zollhöfer, M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00149 · Cited by 162 · see citing works
Learning Light Field Angular Super-Resolution via a Geometry-Aware Network
Jin, J., Hou, J., Yuan, H., & Kwong, S. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6771 · Cited by 142 · see citing works
HDR-NeRF: High Dynamic Range Neural Radiance Fields
Huang, X., Zhang, Q., Feng, Y., Li, H., Wang, X., & Wang, Q. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01785 · Cited by 130 · see citing works
Hallucinated Neural Radiance Fields in the Wild
Chen, X., Zhang, Q., Li, X., Chen, Y., Feng, Y., Wang, X., & Wang, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01260 · Cited by 130 · see citing works
Rückert, D., Franke, L., & Stamminger, M. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530122 · Cited by 168 · see citing works
HyperReel: High-Fidelity 6-DoF Video with Ray-Conditioned Sampling
Attal, B., Huang, J., Richardt, C., Zollhöfer, M., Kopf, J., O’Toole, M., & Kim, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01594 · Cited by 127 · see citing works
Kellnhofer, P., Jebe, L., Jones, A., Spicer, R. P., Pulli, K., & Wetzstein, G. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00427 · Cited by 126 · see citing works
Bacon: Band-limited Coordinate Networks for Multiscale Scene Representation
Lindell, D. B., Veen, D. V., Park, J. J., & Wetzstein, G. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01577 · Cited by 121 · see citing works
NeRF-SR: High Quality Neural Radiance Fields using Supersampling
Wang, C., Wu, X., Guo, Y.-C., Zhang, S., Tai, Y., & Hu, S.-M. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3547808 · Cited by 121 · see citing works
Novel View Synthesis of Dynamic Scenes With Globally Coherent Depths From a Monocular Camera
Yoon, J. S., Kim, K., Gallo, O., Park, H. S., & Kautz, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00538 · Cited by 109 · see citing works
Unconstrained Scene Generation with Locally Conditioned Radiance Fields
DeVries, T., Bautista, M. Á., Srivastava, N., Taylor, G. W., & Susskind, J. M. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01404 · Cited by 107 · see citing works
Web Stereo Video Supervision for Depth Prediction from Dynamic Scenes
Wang, C., Lucey, S., Perazzi, F., & Wang, O. (2019) · 2019 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2019.00046 · Cited by 103 · see citing works
NeuTex: Neural Texture Mapping for Volumetric Neural Rendering
Xiang, F., Xu, Z., Hašan, M., Hold-Geoffroy, Y., Sunkavalli, K., & Su, H. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00704 · Cited by 102 · see citing works
Nguyen-Phuoc, T., Liu, F., & Xiao, L. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530107 · Cited by 100 · see citing works
Learning to Stylize Novel Views
Huang, H., Tseng, H.-Y., Saini, S., Singh, M. K., & Yang, M.-H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01361 · Cited by 84 · see citing works
Spatial-Angular Attention Network for Light Field Reconstruction
Wu, G., Wang, Y., Liu, Y., Fang, L., & Chai, T. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3122089 · Cited by 78 · see citing works
Light Field Super-Resolution with Zero-Shot Learning
Cheng, Z., Xiong, Z., Chen, C., Liu, D., & Zha, Z.-J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00988 · Cited by 76 · see citing works
$\text{NPBG++}$: Accelerating Neural Point-Based Graphics
Rakhimov, R., Ardelean, A.-T., Lempitsky, V., & Burnaev, E. V. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01550 · Cited by 75 · see citing works
Revisiting Light Field Rendering with Deep Anti-Aliasing Neural Network
Wu, G., Liu, Y., Fang, L., & Chai, T. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3073739 · Cited by 66 · see citing works
Physics informed neural fields for smoke reconstruction with sparse data
Chu, M., Liu, L., Zheng, Q., Franz, A., Seidel, H., Theobalt, C., & Zayer, R. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530169 · Cited by 63 · see citing works
Depth Completion using Plane-Residual Representation
Lee, B.-U., Lee, K., & Kweon, I. S. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01370 · Cited by 59 · see citing works
3D Scene Reconstruction With Multi-Layer Depth and Epipolar Transformers
Shin, D., Ren, Z., Sudderth, E., & Fowlkes, C. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00226 · Cited by 57 · see citing works
Monocular Neural Image Based Rendering With Continuous View Control
Song, J., Chen, X., & Hilliges, O. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00419 · Cited by 55 · see citing works
SLIDE: Single Image 3D Photography with Soft Layering and Depth-aware Inpainting
Jampani, V., Chang, H., Sargent, K., Kar, A., Tucker, R. P., Krainin, M., Käser, D. P., Freeman, W. T., Salesin, D., Curless, B., et al. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01229 · Cited by 49 · see citing works
Self-citationFWD: Real-time Novel View Synthesis with Forward Warping and Depth
Cao, A., Rockwell, C., & Johnson, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01526 · Cited by 49 · see citing works
Intelligent layout generation based on deep generative models: A comprehensive survey
Shi, Y., Shang, M., & Qi, Z. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101940 · Cited by 52 · see citing works
Ost, J., Laradji, I., Newell, A., Bahat, Y., & Heide, F. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01787 · Cited by 47 · see citing works
A Neural Rendering Framework for Free-Viewpoint Relighting
Chen, Z., Chen, A., Zhang, G., Wang, C., Ji, Y., Kutulakos, K. N., & Jingyi Yu, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00564 · Cited by 43 · see citing works
ReLight My NeRF: A Dataset for Novel View Synthesis and Relighting of Real World Objects
Toschi, M., Matteo, R. D., Spezialetti, R., Gregorio, D. D., Stefano, L. D., & Salti, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01989 · Cited by 43 · see citing works
Robust Semi-Supervised Monocular Depth Estimation with Reprojected DistancesPreprint
Guizilini, V., Li, J., Ambruş, R., Pillai, S., & Gaidon, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.01765 · Cited by 43 · see citing works
ViP-NeRF: Visibility Prior for Sparse Input Neural Radiance Fields
Somraj, N., & Soundararajan, R. (2023) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3588432.3591539 · Cited by 42 · see citing works
Towards efficient and photorealistic 3D human reconstruction: A brief survey
Chen, L., Peng, S., & Zhou, X. (2021) · Visual Informatics · doi:10.1016/j.visinf.2021.10.003 · Cited by 41 · see citing works
Neural Canvas: Supporting Scenic Design Prototyping by Integrating 3D Sketching and Generative AI
Shen, Y., Shen, Y., Cheng, J., Jiang, C., Fan, M., & Wang, Z. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642096 · Cited by 46 · see citing works
Semantic Implicit Neural Scene Representations With Semi-Supervised Training
Kohli, A., Sitzmann, V., & Wetzstein, G. (2020) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv50981.2020.00052 · Cited by 39 · see citing works
Geometric Unsupervised Domain Adaptation for Semantic Segmentation
Guizilini, V., Li, J., Ambruş, R., & Gaidon, A. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00842 · Cited by 37 · see citing works
Real-Time Depth Video-Based Rendering for 6-DoF HMD Navigation and Light Field Displays
Bonatto, D., Fachada, S., Rogge, S., Munteanu, A., & Lafruit, G. (2021) · IEEE Access · doi:10.1109/access.2021.3123529 · Cited by 36 · see citing works
Seeing the World in a Bag of Chips
Park, J. J., Holynski, A., & Seitz, S. M. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00149 · Cited by 34 · see citing works
DoF-NeRF: Depth-of-Field Meets Neural Radiance Fields
Wu, Z., Li, X., Peng, J., Lü, H., Cao, Z., & Zhong, W. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548088 · Cited by 34 · see citing works
Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objects
Noguchi, A., Iqbal, U., Tremblay, J., Harada, T., & Gallo, O. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00366 · Cited by 34 · see citing works
Chen, T., Wang, P., Fan, Z., & Wang, Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01476 · Cited by 32 · see citing works
SimpleNeRF: Regularizing Sparse Input Neural Radiance Fields with Simpler Solutions
Somraj, N., Karanayil, A., & Soundararajan, R. (2023) · SIGGRAPH Asia Conference Papers · doi:10.1145/3610548.3618188 · Cited by 31 · see citing works
Benchmarking neural radiance fields for autonomous robots: An overview
Ming, Y., Yang, X., Wang, W., Chen, Z., Feng, J., Xing, Y., & Zhang, G. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109685 · Cited by 26 · see citing works
HVH: Learning a Hybrid Neural Volumetric Representation for Dynamic Hair Performance Capture
Wang, Z., Nam, G., Stuyck, T., Lombardi, S., Zollhöfer, M., Hodgins, J. K., & Lassner, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00605 · Cited by 24 · see citing works
Remote Sensing Novel View Synthesis With Implicit Multiplane Representations
Wu, Y., Zou, Z., & Shi, Z. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3197409 · Cited by 23 · see citing works
Simple and Effective Synthesis of Indoor 3D Scenes
Koh, J. Y., Agrawal, H., Batra, D., Tucker, R. P., Waters, A., Lee, H., Yang, Y., Baldridge, J., & Anderson, P. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25199 · Cited by 22 · see citing works
Self-citationGrad-CAM++ is Equivalent to Grad-CAM With Positive Gradients
Lerma, M., & Lucas, M. (2022) · International Machine Vision and Image Processing Conference (IMVIP) · doi:10.56541/awjv6348 · Cited by 23 · see citing works
Learning Dynamic Interpolation for Extremely Sparse Light Fields with Wide Baselines
Guo, M., Jin, J., Liu, H., & Hou, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00245 · Cited by 20 · see citing works
NeLF: Neural Light-transport Field for Portrait View Synthesis and RelightingPreprint
Sun, T., Lin, K.-E., Bi, S., Xu, Z., & Ramamoorthi, R. (2021) · arXiv (Cornell University) · doi:10.2312/sr.20211299 · Cited by 19 · see citing works
3D Moments from Near-Duplicate Photos
Wang, Q., Li, Z., Salesin, D., Snavely, N., Curless, B., & Kontkanen, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00388 · Cited by 18 · see citing works
Self-citationDifferentiable Diffusion for Dense Depth Estimation from Multi-view Images
Khan, N., Kim, M. H., & Tompkin, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00880 · Cited by 17 · see citing works
NeLF: Neural Light-transport Field for Portrait View Synthesis and RelightingPreprint
CHEN, T. (2021) · doi:10.31219/osf.io/gsfbd · Cited by 16 · see citing works
Fang, S., Xu, W., Wang, H., Yang, Y., Wang, Y., & Zhou, S. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25135 · Cited by 15 · see citing works
Visually Imbalanced Stereo Matching
Liu, Y., Ren, J., Zhang, J., Liu, J., & Lin, M. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00210 · Cited by 15 · see citing works
TRANSPR: Transparency Ray-Accumulating Neural 3D Scene Point Renderer
Kolos, M., Sevastopolsky, A., & Lempitsky, V. (2020) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv50981.2020.00127 · Cited by 14 · see citing works
Monocular Depth Estimation for Soft Visuotactile Sensors
Ambruş, R., Guizilini, V., Kuppuswamy, N. S., Beaulieu, A., Gaidon, A., & Alspach, A. N. (2021) · IEEE International Conference on Soft Robotics (RoboSoft) · doi:10.1109/robosoft51838.2021.9479234 · Cited by 14 · see citing works
Yang, Z., Sang, X., Yan, B., Chen, D., Wang, P., Wan, H., Chen, S., & Li, J. (2022) · Optics Express · doi:10.1364/oe.478853 · Cited by 13 · see citing works
Wang, Y., Curless, B., & Seitz, S. M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58607-2_26 · Cited by 13 · see citing works
Acquiring a Dynamic Light Field through a Single-Shot Coded Image
Mizuno, R., Takahashi, K., Yoshida, M., Tsutake, C., Fujii, T., & Nagahara, H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01921 · Cited by 13 · see citing works
Implicit neural refinement based multi-view stereo network with adaptive correlation
Song, B., Hu, X., Xiao, J., Zhang, G., & Chen, T. (2022) · Image and Vision Computing · doi:10.1016/j.imavis.2022.104511 · Cited by 12 · see citing works
Lee, M.-J., Um, G., Yun, J., Cheong, W., & Park, S.-Y. (2021) · Sensors · doi:10.3390/s21196680 · Cited by 12 · see citing works
GeoFill: Reference-Based Image Inpainting with Better Geometric Understanding
Zhao, Y., Barnes, C., Zhou, Y., Shechtman, E., Amirghodsi, S., & Fowlkes, C. C. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00182 · Cited by 12 · see citing works
DARF: Depth-Aware Generalizable Neural Radiance Field
Shi, Y., Rong, D., Chen, C., Ma, C., Ni, B., & Zhang, W. (2025) · Displays · doi:10.1016/j.displa.2025.102996 · Cited by 12 · see citing works
Self-improving Multiplane-to-layer Images for Novel View Synthesis
Solovev, P., Khakhulin, T., & Korzhenkov, D. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00429 · Cited by 12 · see citing works
Occupancy Planes for Single-View RGB-D Human Reconstruction
Zhao, X., Hu, Y.-T., Ren, Z., & Schwing, A. G. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25474 · Cited by 11 · see citing works
IE-NeRF: Exploring transient mask inpainting to enhance neural radiance fields in the wild
Wang, S., Xu, H., Li, Y., Chen, J., & Tan, G. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.129112 · Cited by 10 · see citing works
Light Field Rendering for non-Lambertian Objects
Fachada, S., Bonatto, D., Teratani, M., & Lafruit, G. (2021) · Electronic Imaging · doi:10.2352/issn.2470-1173.2021.2.sda-054 · Cited by 10 · see citing works
DirectVoxGO++: Grid-based fast object reconstruction using radiance fields
Perazzo, D., Lima, J. P., Velho, L. C. P. R., & Teichrieb, V. (2023) · Computers & Graphics · doi:10.1016/j.cag.2023.05.027 · Cited by 10 · see citing works
VET: Visual Error Tomography for Point Cloud Completion and High-Quality Neural Rendering
Franke, L., Rückert, D., Fink, L., Innmann, M., & Stamminger, M. (2023) · SIGGRAPH Asia Conference Papers · doi:10.1145/3610548.3618212 · Cited by 9 · see citing works
Deshpande, R. R., Bhatt, M. R., & Madhavi, C. H. R. (2021) · IEEE Access · doi:10.1109/access.2021.3109865 · Cited by 9 · see citing works
Martel, J., Lindell, D. B., Lin, C. Z., Chan, E. R., Monteiro, M., & Wetzstein, G. (2021) · ACM Transactions on Graphics · doi:10.1145/3476576.3476607 · Cited by 9 · see citing works
Generating High-Quality Panorama by View Synthesis Based on Optical Flow Estimation
Zhang, W., Wang, Y., & Liu, Y. (2022) · Sensors · doi:10.3390/s22020470 · Cited by 9 · see citing works
Compact And Adaptive Multiplane Images For View Synthesis
Navarro, J., & Sabater, N. (2021) · Proceedings - International Conference on Image Processing · doi:10.1109/icip42928.2021.9506403 · Cited by 8 · see citing works
Geometric Warping Error Aware CNN for DIBR Oriented View Synthesis
Li, S., Wang, K., Gao, Y., Cai, X., & Ye, M. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3547946 · Cited by 8 · see citing works
View-dependent Scene Appearance Synthesis using Inverse Rendering from Light Fields
Kang, D., Jeon, D. S., Kim, H., Jang, H., & Kim, M. H. (2021) · IEEE International Conference on Computational Photography (ICCP) · doi:10.1109/iccp51581.2021.9466274 · Cited by 8 · see citing works
Box Supervised Video Segmentation Proposal Network
Hannan, T., Koner, R., Kobold, J., & Schubert, M. (2022) · International Machine Vision and Image Processing Conference (IMVIP) · doi:10.56541/azwk8552 · Cited by 7 · see citing works
Gildea, K., Mercadal-Baudart, C., Blythman, R., Smolić, A., & Simms, C. K. (2022) · International Machine Vision and Image Processing Conference (IMVIP) · doi:10.56541/qtuv2945 · Cited by 6 · see citing works
Zhang, R., & Chen, J. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680821 · Cited by 6 · see citing works
Novel view synthesis with wide-baseline stereo pairs based on local–global information
Song, K., & Zhang, L. (2024) · Computers & Graphics · doi:10.1016/j.cag.2024.104139 · Cited by 6 · see citing works
An unsupervised video stabilization algorithm based on gyroscope image fusion
Rena, Z., Zoub, M., Bib, L., & Fanga, M. (2024) · Computers & Graphics · doi:10.1016/j.cag.2024.104154 · Cited by 6 · see citing works
Qi, S., Sang, X., Yan, B., Wang, P., Chen, D., Wang, H., Ye, X., & Wan, H. (2022) · Optics Communications · doi:10.1016/j.optcom.2022.128679 · Cited by 6 · see citing works
Depth-Guided Robust and Fast Point Cloud Fusion NeRF for Sparse Input Views
Guo, S., Wang, Q., Gao, Y., Xie, R., & Song, L. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i3.27968 · Cited by 6 · see citing works
Virtual view synthesis for 3D light-field display based on feature reprojection and fusion
Qi, S., Sang, X., Yan, B., Chen, D., Wang, P., Wang, H., & Ye, X. (2022) · Optics Communications · doi:10.1016/j.optcom.2022.128383 · Cited by 6 · see citing works
- Authors
- Zhou, T., Tucker, R., Flynn, J., Fyffe, G., & Snavely, N.
- Affiliations
- University of California System (US)
- Google (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1805.09817
- OpenAlex
- W2804091141
- Cited by
- 150
- References
- 5
- Citations by year
- 2025: 3 · 2024: 9 · 2023: 26 · 2022: 56 · 2021: 38 · 2020: 11 · 2019: 7
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- Figshare (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Engineering › Computational Mechanics › 3D Shape Modeling and Analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- view synthesis, layered representation, stereo camera, virtual reality, augmented reality
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
