The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
LangSplat: 3D Language Gaussian Splatting
Open accessAbstract
Humans live in a 3D world and commonly use natural language to interact with a 3D scene. Modeling a 3D language field to support open-ended language queries in 3D has gained increasing attention recently. This paper introduces LangSplat, which constructs a 3D language field that enables precise and efficient open-vocabulary querying within 3D spaces. Unlike existing methods that ground CLIP language embeddings in a NeRF model, LangSplat advances the field by utilizing a collection of 3D Gaussians, each encoding language features distilled from CLIP, to represent the language field. By employing a tile-based splatting technique for rendering language features, we circumvent the costly rendering process inherent in NeRF. Instead of directly learning CLIP embeddings, LangSplat first trains a scene-wise language autoencoder and then learns language features on the scene-specific latent space, thereby alleviating substantial memory demands imposed by explicit modeling. Existing methods struggle with imprecise and vague 3D language fields, which fail to discern clear boundaries between objects. We delve into this issue and propose to learn hierarchical semantics using SAM, thereby eliminating the need for extensively querying the language field across various scales and the regularization of DINO features. Extensive experimental results show that LangSplat significantly outperforms the previous state-of-the-art method LERF by a large margin. Notably, LangSplat is extremely efficient, achieving a 199 x speedup compared to LERF at the resolution of 1440 x 1080. We strongly recommend readers to check out our video results at https://langsplat.github.io/
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., et al. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00371 · Cited by 11,953
3D Gaussian Splatting for Real-Time Radiance Field Rendering
Kerbl, B., Kopanas, G., Leimkuehler, T., & Drettakis, G. (2023) · ACM Transactions on Graphics · doi:10.1145/3592433 · Cited by 6,025
Emerging Properties in Self-Supervised Vision Transformers
Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., & Joulin, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00951 · Cited by 5,638 · cited 1×
“They learned 3D-consistent features by distilling LSeg [21] or DINO [4] features across multiple views into a NeRF.”
BackgroundInstant neural graphics primitives with a multiresolution hash encoding
Müller, T., Evans, A., Schied, C., & Keller, A. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530127 · Cited by 4,114
Learning to Prompt for Vision-Language Models
Zhou, K., Yang, J., Loy, C. C., & Liu, Z. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01653-1 · Cited by 3,304 · cited 1×
“Most existing methods [18, 23, 33] employ the CLIP image encoder V to extract image features and utilize the extracted CLIP embeddings to supervise the 3D language field Φ , leveraging the well-aligned text-image latent space provided by CLIP, thus facilitating open-vocabulary queries.”
MethodSegment anything in medical images
Ma, J., He, Y., Li, F., Han, L., You, C., & Wang, B. (2024) · Nature Communications · doi:10.1038/s41467-024-44824-z · Cited by 2,913
pixelNeRF: Neural Radiance Fields from One or Few Images
Yu, A., Ye, V., Tancik, M., & Kanazawa, A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00455 · Cited by 1,560
D-NeRF: Neural Radiance Fields for Dynamic Scenes
Pumarola, A., Corona, E., Pons-Moll, G., & Moreno-Noguer, F. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01018 · Cited by 1,355
TensoRF: Tensorial Radiance Fields
Chen, A., Xu, Z., Geiger, A., Yu, J., & Su, H. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19824-3_20 · Cited by 1,184
Nerfies: Deformable Neural Radiance Fields
Park, K., Sinha, U., Barron, J. T., Bouaziz, S., Goldman, D. B., Seitz, S. M., & Martin-Brualla, R. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00581 · Cited by 1,059
Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction
Sun, C., Sun, M., & Chen, H.-T. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00538 · Cited by 939
4D Gaussian Splatting for Real-Time Dynamic Scene Rendering
Wu, G., Yi, T., Fang, J., Xie, L., Zhang, X., Wei, W., Liu, W., Tian, Q., & Wang, X. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01920 · Cited by 806
Segment anything model for medical image analysis: An experimental study
Mazurowski, M. A., Dong, H., Gu, H., Yang, J., Konz, N., & Zhang, Y. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102918 · Cited by 802 · cited 1×
“[38] introduced [48], super-resolution [25], image matting [45], object tracking [11, 42], medical image segmentation [29], image editing [12], and so on.”
BackgroundDreamGaussian: Generative Gaussian Splatting for Efficient 3D Content CreationPreprint
Tang, J., Ren, J., Zhou, H., Liu, Z., & Zeng, G. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.16653 · Cited by 695
pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis
Chan, E. R., Monteiro, M., Kellnhofer, P., Wu, J., & Wetzstein, G. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00574 · Cited by 689
Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis
Luiten, J., Kopanas, G., Leibe, B., & Ramanan, D. (2024) · 2024 International Conference on 3D Vision (3DV) · doi:10.1109/3dv62453.2024.00044 · Cited by 469
Open-Vocabulary Semantic Segmentation with Mask-adapted CLIP
Liang, F., Wu, B., Dai, X., Li, K., Zhao, Y., Zhang, H., Zhang, P., Vajda, P., & Marculescu, D. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00682 · Cited by 464
Decomposing NeRF for Editing Via Feature Field Distillation
Kobayashi, S., Matsumoto, E., & Sitzmann, V. (2022) · neural information processing systems · doi:10.52202/068431-1694 · Cited by 462
Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction
Yang, Z., Gao, X., Zhou, W., Jiao, S., Zhang, Y., & Jin, X. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01922 · Cited by 452
Segment and Track AnythingPreprint
Cheng, Y., Li, L., Xu, Y., Li, X., Yang, Z., Wang, W., & Yang, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.06558 · Cited by 408
In-Place Scene Labelling and Understanding with Implicit Scene Representation
Zhi, S., Laidlow, T., Leutenegger, S., & Davison, A. J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01554 · Cited by 396
Visual Language Maps for Robot Navigation
Huang, C., Mees, O., Zeng, A., & Burgard, W. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10160969 · Cited by 376
LERF: Language Embedded Radiance Fields
Kerr, J., Kim, C. M., Goldberg, K., Kanazawa, A., & Tancik, M. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01807 · Cited by 370
Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models
Xu, J., Liu, S., Vahdat, A., Byeon, W., Wang, X., & De Mello, S. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00289 · Cited by 359
IQA: Visual Question Answering in Interactive Environments
Gordon, D., Kembhavi, A., Rastegari, M., Redmon, J., Fox, D., & Farhadi, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00430 · Cited by 352
Language-driven Semantic Segmentation
Li, B., Weinberger, K. Q., Belongie, S., Koltun, V., & Ranftl, R. (2022) · arXiv · doi:10.48550/arxiv.2201.03546 · Cited by 316 · cited 2×
“They learned 3D-consistent features by distilling LSeg [21] or DINO [4] features across multiple views into a NeRF.”
BackgroundInfluentialOpen-vocabulary Queryable Scene Representations for Real World Planning
Chen, B., Xia, F., Ichter, B., Rao, K., Gopalakrishnan, K., Ryoo, M. S., Stone, A., & Kappler, D. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161534 · Cited by 283
Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representations
Tschernezki, V., Laina, I., Larlus, D., & Vedaldi, A. (2022) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv57658.2022.00056 · Cited by 242
ConceptFusion: Open-set multimodal 3D mapping
Jatavallabhula, K., Kuwajerwala, A., Gu, Q., Omama, M., Iyer, G., Saryazdi, S., Chen, T., Maalouf, A., Li, S., Keetha, N., et al. (2023) · Robotics: Science and Systems XIX · doi:10.15607/rss.2023.xix.066 · Cited by 228
Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language ModelPreprint
Huang, S., Jiang, Z., Dong, H., Qiao, Y., Gao, P., & Li, H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.11176 · Cited by 228
ConceptFusion: Open-set Multimodal 3D Mapping
Jatavallabhula, K. M., Kuwajerwala, A., Gu, Q., Omama, M., Chen, T., Maalouf, A., Li, S., Iyer, G., Saryazdi, S., Keetha, N., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.07241 · Cited by 228
CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memory
Shafiullah, N. M. M., Paxton, C., Pinto, L., Chintala, S., & Szlam, A. D. (2023) · Robotics: Science and Systems · doi:10.15607/rss.2023.xix.074 · Cited by 224
ScanQA: 3D Question Answering for Spatial Scene Understanding
Azuma, D., Miyanishi, T., Kurita, S., & Kawanabe, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01854 · Cited by 197
Weakly Supervised 3D Open-vocabulary Segmentation
Liu, K., Zhan, F., Zhang, J., Xu, M., Yu, Y., Saddik, A. E., Theobalt, C., Xing, E., & Lu, S. (2023) · neural information processing systems · doi:10.52202/075280-2325 · Cited by 192
Text-to-3D using Gaussian Splatting
Chen, Z., Wang, F., Wang, Y., & Liu, H. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02022 · Cited by 185
Distilled Feature Fields Enable Few-Shot Language-Guided ManipulationPreprint
Shen, W., Yang, G., Yu, A., Wong, J., Kaelbling, L. P., & Isola, P. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.07931 · Cited by 183
Segment Any Point Cloud Sequences by Distilling Vision Foundation Models
Liu, Y., Kong, L., Cen, J., Chen, R., Zhang, W., Pan, L., Chen, K., & Liu, Z. (2023) · neural information processing systems · doi:10.52202/075280-1617 · Cited by 177
Panoptic Lifting for 3D Scene Understanding with Neural Fields
Siddiqui, Y., Porzi, L., Bulò, S. R., Muller, N. E., Nießner, M., Dai, A., & Kontschieder, P. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00873 · Cited by 163
Learning Dynamic Facial Radiance Fields for Few-Shot Talking Head Synthesis
Shen, S., Li, W., Zhu, Z. D., Duan, Y., Zhou, J., & Lu, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19775-8_39 · Cited by 162
Self-citationSegment Anything in 3D with Radiance Fields
Cen, J., Fang, J., Zhou, Z., Yang, C., Xie, L., Zhang, X., Shen, W., & Tian, Q. C. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02421-7 · Cited by 116
Track Anything: Segment Anything Meets VideosPreprint
Yang, J., Gao, M., Li, Z., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.11968 · Cited by 93
LMM Modality Transfer: A Pre-Requisite for Autonomous GIS AgentsPreprint
Mai, G., Huang, W., Sun, J., et al. (2023) · arXiv (Cornell University) · doi:10.4230/lipics.cosit.2026.7 · Cited by 65
SD-NeRF: Towards Lifelike Talking Head Animation via Spatially-Adaptive Dual-Driven NeRFs
Shen, S., Li, W., Huang, X., Zhu, Z. D., Zhou, J., & Lu, J. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3308441 · Cited by 59
Self-citationSim VQA: Exploring Simulated Environments for Visual Question Answering
Cascante-Bonilla, P., Wu, H., Wang, L., Feris, R., & Ordóñez, V. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00500 · Cited by 55
Anything-3D: Towards Single-view Anything Reconstruction in the WildPreprint
Shen, Q., Yang, X., & Wang, X. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.10261 · Cited by 33
EditAnything: Empowering Unparalleled Flexibility in Image Editing and Generation
Gao, S., Lin, Z., Xie, X., Zhou, P., Cheng, M., & Yan, S. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612680 · Cited by 26
TriSAM: Tri-Plane SAM for Zero-Shot Cortical Blood Vessel Segmentation in VEM Images
Wan, J., Li, W., Adhinarta, J. K., Banerjee, A., Sjöstedt, E., Wu, J., Lichtman, J. W., Pfister, H., & Wei, D. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3577625 · Cited by 17
Self-citation
Metadata
- Authors
- Qin, M., Li, W., Zhou, J., Wang, H., & Pfister, H.
- Venue
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2024-06-16
- Pages
- 20051-20060
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4402727529
- Cited by
- 268
- References
- 48
- FWCI
- 76.96
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.