Neural point cloud rendering via multi-plane projection
Open accessAbstract
We present a new deep point cloud rendering pipeline through multi-plane projections. The input to the network is the raw point cloud of a scene and the output are image or image sequences from a novel view or along a novel camera trajectory. Unlike previous approaches that directly project features from 3D points onto 2D image domain, we propose to project these features into a layered volume of camera frustum. In this way, the visibility of 3D points can be automatically learnt by the network, such that ghosting effects due to false visibility check as well as occlusions caused by noise interferences are both avoided successfully. Next, the 3D feature volume is fed into a 3D CNN to produce multiple planes of images w.r.t. the space division in the depth directions. The multi-plane images are then blended based on learned weights to produce the final rendering results. Experiments show that our network produces more stable renderings compared to previous methods, especially near the object boundaries. Moreover, our pipeline is robust to noisy and relatively sparse point cloud for a variety of challenging scenes.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The warped images usually contain isolated pixels, and the foreground and background are entangled, which jeopardizes video quality.
“In addition, the warped images usually contain isolated pixels, and the foreground and background are entangled, which jeopardizes video quality [5].”
From SVG: 3D Stereoscopic Video Generation via Denoising Frame Matrix (2024) · cites this paper as [5]
Section 4.4 reports that 2D-projection methods can produce a mixture of foreground and background under noisy depth, and that changing projections can cause jittering artifacts in videos. The Figure 2 caption describes holes in re-projected point-cloud images, but our copy does not say that warped images usually contain isolated pixels. The claim also generalizes these problems beyond the conditions the paper discusses. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602 · see citing works
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930 · see citing works
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070 · see citing works
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928 · see citing works
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623 · see citing works
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283 · see citing works
ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes
Dai, A., Chang, A. X., Savva, M., Halber, M., Funkhouser, T., & Niessner, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.261 · Cited by 4,368 · see citing works
Levoy, M., & Hanrahan, P. (1996) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/237170.237199 · Cited by 3,789 · see citing works
Conditional Image Generation with PixelCNN DecodersPreprint
van den Oord, A., Kalchbrenner, N., Vinyals, O., Espeholt, L., Graves, A., & Kavukcuoglu, K. (2016) · Neural Information Processing Systems · doi:10.48550/arxiv.1606.05328 · Cited by 3,066 · see citing works
Snavely, N., Seitz, S. M., & Szeliski, R. (2006) · ACM Transactions on Graphics · doi:10.1145/1141911.1141964 · Cited by 2,975 · see citing works
Semantic Image Synthesis With Spatially-Adaptive Normalization
Park, T., Liu, M.-Y., Wang, T.-C., & Zhu, J.-Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00244 · Cited by 2,974 · see citing works
Gortler, S. J., Grzeszczuk, R., Szeliski, R., & Cohen, M. F. (1996) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/237170.237200 · Cited by 2,459 · see citing works
Image Inpainting for Irregular Holes Using Partial Convolutions
Liu, G., Reda, F. A., Shih, K. J., Wang, T.-C., Tao, A., & Catanzaro, B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_6 · Cited by 2,419 · cited 1× · see citing works
“On the other hand, many deep learning approaches show strong capability in inpainting [31], refining [36], and even constructing images from only a few indications [34, 2].”
BackgroundGlobally and locally consistent image completion
Iizuka, S., Simo-Serra, E., & Ishikawa, H. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073659 · Cited by 2,203 · see citing works
Free-Form Image Inpainting With Gated Convolution
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., & Huang, T. S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00457 · Cited by 2,082 · see citing works
Matterport3D: Learning from RGB-D Data in Indoor Environments
Chang, A., Dai, A., Funkhouser, T., Halber, M., Niebner, M., Savva, M., Song, S., Zeng, A., & Zhang, Y. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00081 · Cited by 1,858 · see citing works
Self-citationThies, J., Zollhöfer, M., & Nießner, M. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323035 · Cited by 1,410 · see citing works
Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images
Wang, N., Zhang, Y., Li, Z., Fu, Y., Liu, W., & Jiang, Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_4 · Cited by 1,362 · see citing works
Self-citationMcMillan, L., & Bishop, G. (1995) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/218380.218398 · Cited by 1,266 · see citing works
The RADIANCE lighting simulation and rendering system
Ward, G. J. (1994) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/192161.192286 · Cited by 1,047 · see citing works
(1994) · Elsevier eBooks · doi:10.1016/b978-0-12-336156-1.50041-0 · Cited by 973 · see citing works
Pfister, H., Zwicker, M., van Baar, J., & Groß, M. H. (2000) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/344779.344936 · Cited by 840 · see citing works
Learning-based view synthesis for light field cameras
Kalantari, N. K., Wang, T.-C., & Ramamoorthi, R. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2980251 · Cited by 727 · see citing works
Seitz, S. M., & Dyer, C. R. (1996) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/237170.237196 · Cited by 723 · see citing works
Simulation of wrinkled surfaces
Blinn, J. F. (1978) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/800248.507101 · Cited by 712 · see citing works
Dai, A., Nießner, M., Zollhöfer, M., Izadi, S., & Theobalt, C. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3054739 · Cited by 665 · see citing works
Deep blending for free-viewpoint image-based rendering
Hedman, P., Philip, J., Price, T., Frahm, J., George Drettakis, B., & Brostow, G. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275084 · Cited by 622 · see citing works
Deep Stereo: Learning to Predict New Views from the World's Imagery
Flynn, J., Neulander, I., Philbin, J. F., & Snavely, N. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.595 · Cited by 599 · see citing works
DeepVoxels: Learning Persistent 3D Feature Embeddings
Sitzmann, V., Thies, J., Heide, F., Nießner, M., Wetzstein, G., & Zollhöfer, M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00254 · Cited by 551 · see citing works
Efficient View-Dependent Image-Based Rendering with Projective Texture-Mapping
Debevec, P. E., Yu, Y., & Borshukov, G. (1998) · Eurographics · doi:10.1007/978-3-7091-6453-2_10 · Cited by 490 · see citing works
An introduction to ray tracing
Anderson, D. C., & Cychosz, J. M. (1990) · Image and Vision Computing · doi:10.1016/0262-8856(90)90033-2 · Cited by 477 · see citing works
Content-preserving warps for 3D video stabilization
Liu, F., Gleicher, M. L., Jin, H., & Agarwala, A. (2009) · ACM Transactions on Graphics · doi:10.1145/1531326.1531350 · Cited by 468 · see citing works
Surface light fields for 3D photography
Wood, D. N., Azuma, D. I., Aldinger, K., Curless, B., Duchamp, T., Salesin, D., & Stuetzle, W. (2000) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/344779.344925 · Cited by 430 · see citing works
Qiu, J., Cui, Z., Zhang, Y., Zhang, X., Liu, S., Zeng, B., & Pollefeys, M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00343 · Cited by 428 · see citing works
Self-citationAliev, K.-A., Sevastopolsky, A., Kolos, M., Ulyanov, D., & Lempitsky, V. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58542-6_42 · Cited by 378 · see citing works
Simulation of wrinkled surfaces
Blinn, J. F. (1978) · ACM SIGGRAPH Computer Graphics · doi:10.1145/965139.507101 · Cited by 320 · see citing works
A survey of point-based techniques in computer graphics
Kobbelt, L. P., & Botsch, M. (2004) · Computers & Graphics · doi:10.1016/j.cag.2004.08.009 · Cited by 308 · see citing works
Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder
Chaitanya, C. R. A., Kaplanyan, A., Schied, C., Salvi, M., Lefohn, A. E., Nowrouzezahrai, D., & Aila, T. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073601 · Cited by 301 · see citing works
Grossman, J. P., & Dally, W. J. (1998) · Eurographics · doi:10.1007/978-3-7091-6453-2_17 · Cited by 300 · see citing works
Semantic photo manipulation with a generative image priorPreprint
Bau, D., Strobelt, H., Peebles, W., Wulff, J., Zhou, B., Zhu, J.-Y., & Torralba, A. B. (2019) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 280 · see citing works
Shape and Spatially-Varying BRDFs from Photometric Stereo
Goldman, D. B., Curless, B., Hertzmann, A., & Seitz, S. M. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.102 · Cited by 270 · see citing works
Dai, A., Nießner, M., Zollhöfer, M., Izadi, S., & Theobalt, C. (2017) · ACM Transactions on Graphics · doi:10.1145/3054739 · Cited by 259 · see citing works
Deep outdoor illumination estimation
Hold-Geoffroy, Y., Sunkavalli, K., Hadap, S., Gambaretto, E., & Lalonde, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.255 · Cited by 216 · see citing works
Semantic photo manipulation with a generative image prior
Bau, D., Strobelt, H., Peebles, W., Wulff, J., Zhou, B., Zhu, J.-Y., & Torralba, A. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323023 · Cited by 207 · see citing works
Video stabilization using epipolar geometry
Goldstein, A., & Fattal, R. (2012) · TOGS · doi:10.1145/2231816.2231824 · Cited by 180 · see citing works
Deep Shading: Convolutional Neural Networks for Screen Space Shading
Nalbach, O., Arabadzhiyska, E., Mehta, D., Seidel, H., & Ritschel, T. (2017) · Computer Graphics Forum · doi:10.1111/cgf.13225 · Cited by 131 · see citing works
Point Cloud Noise and Outlier Removal for Image-Based 3D Reconstruction
Wolff, K., Kim, C., Zimmer, H., Schroers, C., Botsch, M., Sorkine‐Hornung, O., & Sorkine‐Hornung, A. (2016) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.20 · Cited by 103 · see citing works
Deferred Neural Rendering: Image Synthesis using Neural TexturesPreprint
Thies, J., Zollhöfer, M., & Nießner, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.12356 · Cited by 90 · see citing works
Material classification using BRDF slices
Wang, O., Gunawardane, P., Scher, S., & Davis, J. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206558 · Cited by 74 · see citing works
Chen, A., Wu, M., Zhang, Y., Li, N., Lu, J., Gao, S., & Jingyi Yu, A. (2018) · Proceedings of the ACM on Computer Graphics and Interactive Techniques · doi:10.1145/3203192 · Cited by 64 · see citing works
Self-citationCalibration Between Depth and Color Sensors for Commodity Depth Cameras
Zhang, C., & Zhang, Z. (2014) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-319-08651-4_3 · Cited by 53 · see citing works
Point-based rendering enhancement via deep learning
Bui, G. T. T., Le, T., Morago, B., & Duan, Y. (2018) · The Visual Computer · doi:10.1007/s00371-018-1550-6 · Cited by 32 · see citing works
Flagged first, then by citations
2 of the 57 references couldn’t be matched to a known paper.
Research works printed but not linked: 9
- Markus Gross and Hanspeter Pfister. Point-based graphics. Elsevier, 2011. 1
- Leonard McMillan and Gary Bishop. Plenoptic mod- eling: An image-based rendering system. In Proc. of conference on Computer graphics and interactive tech- niques, pages 39-46, 1995. 1, 2
- Hanspeter Pfister, Matthias Zwicker, Jeroen Van Baar, and Markus Gross. Surfels: Surface elements as ren- dering primitives. In Proceedings of the 27th annual conference on Computer graphics and interactive tech- niques, pages 335-342, 2000. 1
- Kara-Ali Aliev, Dmitry Ulyanov, and Victor Lempitsky. Neu- ral point-based graphics. arXiv preprint arXiv:1906.08240, 2019. 1, 2 · arXiv
- Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. arXiv preprint arXiv:1709.06158, 2017. 1 · arXiv
- Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly- annotated 3d reconstructions of indoor scenes. In Proc. CVPR, pages 5828-5839, 2017. 1
- Peter Hedman, Tobias Ritschel, George Drettakis, and Gabriel Brostow. Scalable inside-out image-based rendering. ACM Trans. Graphics, 35(6):1-11, 2016. 1
- Jonas Uhrig, Nick Schneider, Lukas Schneider, Uwe Franke, Thomas Brox, and Andreas Geiger. Sparsity invariant cnns. In 2017 International Conference on 3D Vision (3DV), pages 11-20. IEEE, 2017. 1
- Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. Free-form image inpainting with gated con- volution. In Proc. CVPR, pages 4471-4480, 2019.
Cited by
Sorted by citations
pixelNeRF: Neural Radiance Fields from One or Few Images
Yu, A., Ye, V., Tancik, M., & Kanazawa, A. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00455 · Cited by 1,603 · see citing works
NeRF-Editing: Geometry Editing of Neural Radiance Fields
Yuan, Y.-J., Sun, Y.-T., Lai, Y., Ma, Y., Jia, R., & Gao, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01781 · Cited by 237 · see citing works
Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering
Yang, B., Zhang, Y., Xu, Y., Li, Y., Zhou, H., Bao, H., Zhang, G., & Cui, Z. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01352 · Cited by 217 · see citing works
Self-citationSC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes
Huang, Y., Sun, Y.-T., Yang, Z., Lyu, X., Cao, Y., & Qi, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00404 · Cited by 202 · see citing works
Riegler, G., & Koltun, V. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01204 · Cited by 181 · see citing works
Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splatting
Yan, Y., Lin, H., Zhou, C., Wang, W., Sun, H., Zhan, K., Lang, X., Zhou, X., & Peng, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73464-9_10 · Cited by 172 · see citing works
Hallucinated Neural Radiance Fields in the Wild
Chen, X., Zhang, Q., Li, X., Chen, Y., Feng, Y., Wang, X., & Wang, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01260 · Cited by 130 · see citing works
Rückert, D., Franke, L., & Stamminger, M. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530122 · Cited by 168 · see citing works
PLA: Language-Driven Open-Vocabulary 3D Scene Understanding
Ding, R., Yang, J., Xue, C., Zhang, W., Bai, S., & Qi, X. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00677 · Cited by 131 · see citing works
Transfer Learning from Synthetic to Real LiDAR Point Cloud for Semantic Segmentation
Xiao, A., Huang, J., Guan, D., Zhan, F., & Lu, S. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20183 · Cited by 134 · see citing works
Unconstrained Scene Generation with Locally Conditioned Radiance Fields
DeVries, T., Bautista, M. Á., Srivastava, N., Taylor, G. W., & Susskind, J. M. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01404 · Cited by 107 · see citing works
Grad-PU: Arbitrary-Scale Point Cloud Upsampling via Gradient Descent with Learned Distance Functions
He, Y., Tang, D., Zhang, Y., Xue, X., & Fu, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00518 · Cited by 103 · see citing works
Self-citationStyle-based Point Generator with Adversarial Rendering for Point Cloud Completion
Xie, C., Wang, C., Zhang, B., Yang, H., Chen, D., & Wen, F. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00459 · Cited by 93 · see citing works
Direct-PoseNet: Absolute Pose Regression with Photometric Consistency
Chen, S., Wang, Z., & Prisacariu, V. A. (2021) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00125 · Cited by 72 · see citing works
Single-View View Synthesis in the Wild with Learned Adaptive Multiplane Images
Han, Y., Wang, R., & Yang, J. (2022) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3528233.3530755 · Cited by 69 · see citing works
Fu, C., Li, Q. L., Xu, K., & Wu, J. (2023) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3576915.3616631 · Cited by 45 · see citing works
ADOP: Approximate Differentiable One-Pixel Point RenderingPreprint
Rückert, D., Franke, L., & Stamminger, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.06635 · Cited by 38 · see citing works
Maier, A., Köstler, H., Heisig, M., Krauß, P., & Yang, S. H. (2022) · Progress in Biomedical Engineering · doi:10.1088/2516-1091/ac5b13 · Cited by 35 · see citing works
Lowis3D: Language-Driven Open-World Instance-Level 3D Scene Understanding
Ding, R., Yang, J., Xue, C., Zhang, W., Bai, S., & Qi, X. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3410324 · Cited by 34 · see citing works
Li, S., Zhou, J., Ma, B., Liu, Y.-S., & Han, Z. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i4.28102 · Cited by 28 · see citing works
Fast and Explicit Neural View Synthesis
Guo, P., Bautista, M. Á., Colburn, A., Yang, L., Ulbricht, D., Susskind, J. M., & Shan, Q. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00009 · Cited by 25 · see citing works
Sharing Ambient Objects Using Real-time Point Cloud Streaming in Web-based XR Remote Collaboration
Lee, Y., Yoo, B., & Lee, S.-H. (2021) · international conference on 3D web technology · doi:10.1145/3485444.3487642 · Cited by 24 · see citing works
Pointersect: Neural Rendering with Cloud-Ray Intersection
Chang, J.-H. R., Chen, W.-Y., Ranjan, A., Yi, K. M., & Tuzel, O. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00808 · Cited by 24 · see citing works
MultiDiff: Consistent Novel View Synthesis from a Single Image
Müller, N., Schwarz, K., Roessle, B., Porzi, L., Bulò, S. R., Nießner, M., & Kontschieder, P. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00977 · Cited by 23 · see citing works
Wang, Y., Wang, J., Qu, Y., & Qi, Y. (2023) · International Conference on Multimedia Retrieval · doi:10.1145/3591106.3592276 · Cited by 22 · see citing works
Real-Time Rendering of Point Clouds With Photorealistic Effects: A Survey
Kivi, P. E. J., Makitalo, M. J., Žádník, J., Ikkala, J., Vadakital, V. K. M., & Jääskeläinen, P. (2022) · IEEE Access · doi:10.1109/access.2022.3146768 · Cited by 20 · see citing works
Hybrid Neural Rendering for Large-Scale Scenes with Motion Blur
Dai, P., Zhang, Y., Yu, X., Lyu, X., & Qi, X. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00023 · Cited by 20 · see citing works
Self-citationPoint2Pix: Photo-Realistic Point Cloud Rendering via Neural Radiance Fields
Hu, T., Xu, X., Liu, S., & Jia, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00807 · Cited by 19 · see citing works
Self-citationYang, B., Zhang, Y., Li, Y., Cui, Z., Fanello, S., Bao, H., & Zhang, G. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530163 · Cited by 18 · see citing works
Self-citationBoosting Point Clouds Rendering via Radiance Mapping
Huang, X., Zhang, Y., Ni, B., Li, T., Chen, K., & Zhang, W. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25175 · Cited by 18 · see citing works
Self-citationTriVol: Point Cloud Rendering via Triple Volumes
Hu, T., Xu, X., Chu, R., & Jia, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01986 · Cited by 17 · see citing works
Seal-3D: Interactive Pixel-Level Editing for Neural Radiance Fields
Wang, X., Zhu, J., Ye, Q., Huo, Y., Ran, Y., Zhong, Z., & Chen, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01621 · Cited by 17 · see citing works
Differentiable Diffusion for Dense Depth Estimation from Multi-view Images
Khan, N., Kim, M. H., & Tompkin, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00880 · Cited by 17 · see citing works
Frequency-Modulated Point Cloud Rendering with Easy Editing
Zhang, Y., Huang, X., Ni, B., Zhang, W., & Li, T. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00020 · Cited by 16 · see citing works
Self-citationNDF: Neural Deformable Fields for Dynamic Human Modelling
Zhang, R., & Chen, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19824-3_3 · Cited by 14 · see citing works
SparseFormer: Sparse transformer network for point cloud classification
Wang, Y., Liu, Y., Zhou, P., Geng, G., & Zhang, Q. (2023) · Computers & Graphics · doi:10.1016/j.cag.2023.07.040 · Cited by 12 · see citing works
Precise indoor localization with 3D facility scan data
Xia, J., & Gong, J. (2021) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12795 · Cited by 12 · see citing works
RDNeRF: relative depth guided NeRF for dense free view synthesis
Qiu, J., Zhu, Y., Jiang, P.-T., Cheng, M., & Ren, B. (2023) · The Visual Computer · doi:10.1007/s00371-023-02863-5 · Cited by 12 · see citing works
Interactive NeRF Geometry Editing With Shape Priors
Yuan, Y.-J., Sun, Y.-T., Lai, Y., Ma, Y., Jia, R., Kobbelt, L. P., & Gao, L. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3315068 · Cited by 12 · see citing works
IE-NeRF: Exploring transient mask inpainting to enhance neural radiance fields in the wild
Wang, S., Xu, H., Li, Y., Chen, J., & Tan, G. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.129112 · Cited by 10 · see citing works
Multi-Layer Scene Representation from Composed Focal Stacks
Ishikawa, R., Saitô, H., Kalkofen, D., & Mori, S. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3320248 · Cited by 9 · see citing works
Learning a Locally Unified 3D Point Cloud for View Synthesis
You, M., Guo, M., Lyu, X., Liu, H., & Hou, J. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3321458 · Cited by 9 · see citing works
Bidirectional Optical Flow NeRF: High Accuracy and High Quality under Fewer Views
Chen, S., Yan, B., Sang, X., Chen, D., Wang, P., Guo, X., Zhong, C., & Wan, H. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25109 · Cited by 8 · see citing works
Tian, A., Li, W., Yang, M., Ding, J., Pei, L., & Weng, Y. (2024) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.13369 · Cited by 7 · see citing works
MPR-GAN: A Novel Neural Rendering Framework for MLS Point Cloud With Deep Generative Learning
Xu, Q., Guan, X., Cao, J., Ma, Y., & Wu, H. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3212389 · Cited by 7 · see citing works
SynLiDAR: Learning From Synthetic LiDAR Sequential Point Cloud for Semantic Segmentation.Preprint
Xiao, A., Huang, J., Guan, D., Zhan, F., & Lu, S. (2021) · arXiv (Cornell University) · Cited by 7 · see citing works
Zhang, R., & Chen, J. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680821 · Cited by 6 · see citing works
Point-Based Neural Scene Rendering for Street Views
Li, Z., & Zhu, J. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3304347 · Cited by 6 · see citing works
Self-citationDynamic View Synthesis with Spatio-Temporal Feature Warping from Sparse Views
Li, D., Huang, S.-S., Shen, T., & Huang, H. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612419 · Cited by 6 · see citing works
Style-based Point Generator with Adversarial Rendering for Point Cloud CompletionPreprint
Xie, C., Wang, C., Zhang, B., Yang, H., Chen, D., & Wen, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.02535 · Cited by 6 · see citing works
PFGS: High Fidelity Point Cloud Rendering via Feature Splatting
Wang, J., Zhang, Z., He, J., & Xu, R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73010-8_12 · Cited by 5 · see citing works
Z2P: Instant Visualization of Point Clouds
Metzer, G., Hanocka, R., Giryes, R., Mitra, N. J., & Cohen‐Or, D. (2022) · Computer Graphics Forum · doi:10.1111/cgf.14487 · Cited by 5 · see citing works
Robust Local Light Field Synthesis via Occlusion-aware Sampling and Deep Visual Feature Fusion
Xing, W., Chen, J., & Guo, Y. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1381-9 · Cited by 5 · see citing works
Zhang, R., Chen, J., & Wang, Q. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611707 · Cited by 5 · see citing works
HashPoint: Accelerated Point Searching and Sampling for Neural Rendering
Ma, J., Liu, M., Ahmedt‐Aristizabal, D., & Nguyen, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00427 · Cited by 4 · see citing works
Scale-Consistent Fusion: From Heterogeneous Local Sampling to Global Immersive Rendering
Xing, W., Chen, J., Yang, Z., Wang, Q., & Guo, Y. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3205745 · Cited by 4 · see citing works
State of the Art on Deep Learning-enhanced Rendering Methods
Wang, Q., Zhong, Z., Huo, Y., Bao, H., & Wang, R. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1400-x · Cited by 4 · see citing works
NeRF-IS: Explicit Neural Radiance Fields in Semantic Space
Sha, J., Zhang, H., Pan, Y., Kou, G., & Yi, X. (2023) · ACM International Conference on Multimedia in Asia (MMAsia) · doi:10.1145/3595916.3626379 · Cited by 4 · see citing works
RPBG: Towards Robust Neural Point-Based Graphics in the Wild
Zhu, Q., Wei, Z., Zheng, Z., Zhan, Y., Yao, Z., Zhang, J., Wu, K., & Zheng, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72633-0_22 · Cited by 4 · see citing works
Gao, G., Yang, L., Zhang, Q., Wang, C., Bao, H., & Rao, C. (2023) · Remote Sensing · doi:10.3390/rs15235449 · Cited by 4 · see citing works
SplatMesh: Interactive 3D Segmentation and Editing Using Mesh-Based Gaussian Splatting
Zhou, K., Hong, L., Chang, X., Zhong, Y., Xie, E., Dong, H., Li, Z., Yang, Y., Li, Z., & Zhang, W. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00035 · Cited by 3 · see citing works
Self-citationNVSS: High-quality Novel View Selfie Synthesis
Bian, J.-W., Zhan, H., & Reid, I. D. (2021) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00116 · Cited by 3 · see citing works
Optimal Transport Regularization for Simulation-Informed Room Impulse Response Estimation
Björkman, A., Sundström, D., Jakobsson, A., & Elvander, F. (2025) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2025.3643595 · Cited by 2 · see citing works
Hashed, binned A-buffer for real-time outlier removal and rendering of noisy point clouds
Sommerhoff, H., & Kolb, A. (2023) · The Visual Computer · doi:10.1007/s00371-023-02888-w · Cited by 2 · see citing works
Enhancing Scene Simulation for Autonomous Driving with Neural Point Rendering
Yang, J., Yan, Y., Chen, S., & Zheng, N. (2023) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc57777.2023.10422354 · Cited by 2 · see citing works
Oriented-grid Encoder for 3D Implicit Representations
Gaur, A., Pais, G. D., & Miraldo, P. (2024) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv62453.2024.00101 · Cited by 2 · see citing works
Neural radiance fields assisted by image features for UAV scene reconstruction
Chen, Z. H., Chen, X., Ye, C., Wu, S., & Wu, X. (2025) · Scientific Reports · doi:10.1038/s41598-025-16386-7 · Cited by 2 · see citing works
SVG: 3D Stereoscopic Video Generation via Denoising Frame Matrix
Dai, P., Tan, F., Xu, Q., Futschik, D., Du, R., Fanello, S., Qi, X., & Zhang, Y. (2024) · arXiv · doi:10.48550/arxiv.2407.00367
Self-citationZ2P: Instant Rendering of Point Clouds.Preprint
Metzer, G., Hanocka, R., Giryes, R., Mitra, N. J., & Cohen‐Or, D. (2021) · arXiv (Cornell University)
Neural-based Rendering and Application
Dai, P. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3481031
Self-citationCoarse Aggregate Particle Size Classification Based on 3d Multi-View MethodPreprint
Tian, A., Li, W., Yang, M., Ding, J., Weng, Y., & Pei, L. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4511549
Point Cloud Rendering via Multi-plane NeRF
Dong-mei, M., Cao, J., & Chen, Z. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-50072-5_16
ProbIBR: Fast Image-Based Rendering With Learned Probability-Guided Sampling
Zhou, Y., Yu, T., Zheng, Z., Wu, G., Zhao, G., Jiang, W., Fu, Y., & Liu, Y. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3372152
Sparse Point Cloud Upsampling Based on Neural Implicit Functions
Wang, W., Kong, X., Wang, D., & Zhao, X. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-5600-1_32
Multi-Frequency-Aware Patch Adversarial Learning for Neural Point Cloud Rendering
Karhade, J., Zhu, H., Chung, K.-S., Tripathy, R. K., Lin, W., & Ang, M. H. (2024) · IEEE International Conference on Advanced Intelligent Mechatronics (AIM) · doi:10.1109/aim55361.2024.10637094
Ray-Transformer Neural Radiance Field for Novel 3D View Synthesis from Single Image
Yu, T.-M., & Wu, Y.-L. (2024) · IIAI International Congress on Advanced Applied Informatics (IIAI-AAI) · doi:10.1109/iiai-aai63651.2024.00056
Street Gaussians: Modeling Dynamic Urban Scenes With Gaussian Primitives
Peng, S., Long, Y., Yan, Y., Lin, H., Zhou, C., Sun, H., Zhan, K., Lang, X., Bao, H., & Zhou, X. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3634385
TransLiDAR: A Dataset and Benchmark for Cross-Sensor Point Cloud Translation
Chen, Y.-F., Xu, Z., Huang, X., Zhao, S., Lang, Z., & Gao, X. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3664621
Implementations, Streaming, and Rendering for 3D Point Cloud Coding
Gao, W. (2026) · Springer eBooks · doi:10.1007/978-981-95-0660-6_10
Sato, S., Murasaki, K., & Tanida, R. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31583-0_40
A structure-preserving diffusion model for point cloud upsampling
Wu, H., Li, Y., Liu, Y., Wang, Z., & Guo, H. (2026) · Machine Vision and Applications · doi:10.1007/s00138-026-01931-9
- Authors
- Dai, P., Zhang, Y., Li, Z., Liu, S., & Zeng, B.
- Affiliations
- University of Electronic Science and Technology of China (CN)
- Google (United States) (US)
- Venue
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2020-06
- Pages
- 7827-7836
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3035318263
- Cited by
- 80
- References
- 57
- Citations by year
- 2026: 4 · 2025: 4 · 2024: 16 · 2023: 28 · 2022: 13 · 2021: 15
- FWCI
- 12.45
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (61872067, 61720106004)
- Topics
- Physical Sciences › Engineering › Computational Mechanics › 3D Shape Modeling and Analysis
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Computer Graphics and Visualization Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- point cloud rendering, neural rendering, 3D convolutional neural networks, occlusion, ghosting, sparse point clouds, novel view synthesis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
