CoChat

CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memory

Open access
(Mahi)Shafiullah, Nur, Paxton, Chris, Pinto, Lerrel, Chintala, Soumith, Szlam, Arthur · 2023 · Robotics: Science and Systems XIX
Cited by 82

The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.

Abstract

We propose CLIP-Fields, an implicit scene model that can be used for a variety of tasks, such as segmentation, instance identification, semantic search over space, and view localization.CLIP-Fields learns a mapping from spatial locations to semantic embedding vectors.Importantly, we show that this mapping can be trained with supervision coming only from webimage and web-text trained models such as CLIP, Detic, and Sentence-BERT; and thus uses no direct human supervision.When compared to baselines like Mask-RCNN, our method outperforms on few-shot instance identification or semantic segmentation on the HM3D dataset with only a fraction of the examples.Finally, we show that using CLIP-Fields as a scene memory, robots can perform semantic navigation in real-world environments.Our code and demonstration videos are available here: https://mahis.life/clip-fields

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.15607/rss.2023.xix.074

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 12 references

Cited by

See all 82

Metadata

Authors
(Mahi)Shafiullah, N., Paxton, C., Pinto, L., Chintala, S., & Szlam, A.
Venue
Robotics: Science and Systems XIX
Published
2023-07-10
OpenAlex
W4385430678
Cited by
82
References
12
FWCI
15.50
Type
conference paper
Language
en
Open access
gold

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memory. https://app.cochat.ai/papers/10.15607/rss.2023.xix.074
The paper
(Mahi)Shafiullah, N., Paxton, C., Pinto, L., Chintala, S., & Szlam, A. (2023). CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memory. Robotics: Science and Systems XIX. https://doi.org/10.15607/rss.2023.xix.074

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.