Cited by 59Topic ModelingDomain Adaptation and Few-Shot LearningNatural Language Processing Techniques

Can We Edit Factual Knowledge by In-Context Learning?

Open access
Zheng, Ce, Li, Lei, Dong, Qingxiu, Fan, Yuxuan, Wu, Zhiyong, Xu, Jingjing, Chang, Baobao·2023·Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

Abstract

Previous studies have shown that large language models (LLMs) like GPTs store massive factual knowledge in their parameters.However, the stored knowledge could be false or outdated.Traditional knowledge editing methods refine LLMs via fine-tuning on texts containing specific knowledge.However, with the increasing scales of LLMs, these gradient-based approaches bring large computation costs.The trend of model-as-a-service also makes it impossible to modify knowledge in black-box LLMs.Inspired by in-context learning (ICL), a new paradigm based on demonstration contexts without parameter updating, we explore whether ICL can edit factual knowledge.To answer this question, we give a comprehensive empirical study of ICL strategies.Experiments show that in-context knowledge editing (IKE), without any gradient and parameter updating, achieves a competitive success rate compared to gradient-based methods on GPT-J (6B) but with much fewer side effects, including less over-editing on similar but unrelated facts and less knowledge forgetting on previously stored knowledge.We also apply the method to larger LMs with tens or hundreds of parameters like OPT-175B, which shows the scalability of our method.The code is available at https://github.com/pkunlp-icler/IKE.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2023.emnlp-main.296

Cites

37 of 37 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Memory-based approaches require training to update memory representations or routing, and memory consumption grows linearly with the number of edits, limiting practicality in lifelong model editing.

    “However, these methods still require training to update memory representations or routing, and inference must be rerouted through the memory. In addition, they maintain one entry per edit, causing memory consumption to grow linearly with the number of edits, which limits their practicality in lifelong model editing with high-frequency or large-scale updates.”

    From UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Models (2025) · cites this paper as Zheng et al., 2023

    Section E.1 (page unavailable) describes a memory-based method that stores multiple factual edits externally and uses a scope classifier to retrieve a relevant edit for a prompt; Section 2 also describes an edit-memory retriever. The copy does not say that memory representations or routing must be trained, that storage grows linearly with edits, or that this limits lifelong editing. (LLM, conf=0.83)

    Experimental — accuracy not yet measured
  • In-context learning ability enhances models' understanding of updated knowledge.

    “Although in-context learning ability enhances models' understanding of updated knowledge (Zheng et al., 2023)”

    From Reason-KE++: Aligning the Process, Not Just the Outcome, for Faithful LLM Knowledge Editing (2025) · cites this paper as Zheng et al., 2023

    The Main Results and Conclusion (sections 5.2 and 7; page numbers unavailable) report that in-context knowledge editing improves models’ use and generalization of updated facts. They do not measure models’ understanding or establish that stronger in-context learning ability itself causes that improvement. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. PMLR.
  2. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
  3. Yunzhi Yao, Peng Wang, Bo Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. 2023. Editing large language mod- els: Problems, methods, and opportunities. ArXiv, abs/2305.13172. · arXiv

Cited by

Sorted by citations

Authors
Zheng, C., Li, L., Dong, Q., Fan, Y., Wu, Z., Xu, J., & Chang, B.
Affiliations
  • Peking University (CN)
  • Shanghai Artificial Intelligence Laboratory (CN)
Venue
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Published
2023
Pages
4862-4876
OpenAlex
W4389518797
Cited by
54
References
36
Citations by year
2026: 17 · 2025: 17 · 2024: 16 · 2023: 4
FWCI
9.57
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
  • arXiv (Cornell University)
Funders
National Natural Science Foundation of China (61936012)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
in-context learning, large language models, knowledge forgetting, GPT-J

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review