CoChat
Sign Up
PreprintCited by 1Topic ModelingDomain Adaptation and Few-Shot LearningNatural Language Processing Techniques

The DOI matches its DataCite record; 4 of the 4 references it cites were found in a registry or an index.

WikiBigEdit: Understanding the Limits of Lifelong Knowledge Editing in LLMs

Open access
Thede, Lukas, Roth, Karsten, Bethge, Matthias, Akata, Zeynep, Hartvigsen, Tom·2025·arXiv
DOI 10.48550/arxiv.2503.05683arXiv 2503.05683
Open this paper in CoChat — free accountOpen PDF

Abstract

Keeping large language models factually up-to-date is crucial for deployment, yet costly retraining remains a challenge. Knowledge editing offers a promising alternative, but methods are only tested on small-scale or synthetic edit benchmarks. In this work, we aim to bridge research into lifelong knowledge editing to real-world edits at a practically relevant scale. We first introduce WikiBigEdit; a large-scale benchmark of real-world Wikidata edits, built to automatically extend lifelong for future-proof benchmarking. In its first instance, it includes over 500K question-answer pairs for knowledge editing alongside a comprehensive evaluation pipeline. Finally, we use WikiBigEdit to study existing knowledge editing techniques' ability to incorporate large volumes of real-world facts and contrast their capabilities to generic modification techniques such as retrieval augmentation and continual finetuning to acquire a complete picture of the practical extent of current lifelong knowledge editing.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2503.05683

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • WikiBigEdit's Update, Rephrase, and Locality metrics correspond closely to the standard editing criteria of Efficacy, Generalization, and Specificity, and are evaluated using the same methodology as in ZsRE.

    “Among them, Update , Rephrase , and Locality correspond closely to the standard editing criteria of Efficacy , Generalization , and Specificity , and are evaluated using the same methodology as in ZsRE.”

    From UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Models (2025) · cites this paper as Thede et al., 2025

    Section 4.1 (p. 6) identifies Update, Rephrase, and Locality as evaluation axes, and Appendix A.2 (p. 16) explicitly describes Rephrase as testing generalization. Appendix A.1 (p. 15) describes Locality as checking that unrelated facts remain unaffected. These passages support the claimed correspondence, but the paper's discussion of ZsRE in Related Works (p. 2) does not establish that WikiBigEdit uses the same evaluation methodology as ZsRE. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • A total of 490,519 examples in WikiBigEdit support the first four metrics, while 17,541 are specifically annotated for multi-hop reasoning.

    “A total of 490,519 examples support the first four metrics, while 17,541 are specifically annotated for multi-hop reasoning, enabling focused evaluation of a model's ability to handle compositional queries and long-range dependencies.”

    From UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Models (2025) · cites this paper as Thede et al., 2025

    Section 4.1 (p. 6) describes five evaluation axes, including multi-hop reasoning, and Section 3.1 (p. 4) explains how multi-hop question-answer pairs are generated. But the cited passages do not establish either the 490,519-example count for four metrics or the 17,541-example count for multi-hop reasoning; Section 3.2 (p. 4) instead gives an overall total of 502,382 question-answer pairs. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • WikiBigEdit contains 506,035 editing pairs, all derived from real-world Wikidata revisions collected across eight time steps over a six-month period.

    “The dataset contains a total of 506,035 editing pairs, all derived from real-world Wikidata revisions collected across eight time steps over a six-month period.”

    From UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Models (2025) · cites this paper as Thede et al., 2025

    Table 2 and Section 3.2 (p. 4) report 502,382 question-answer pairs, not 506,035. The Introduction (p. 1) supports the description of eight timesteps over six months, with edits derived from real-world Wikidata changes. (LLM, conf=0.90)

    Experimental — accuracy not yet measured

  • WikiBigEdit is a large-scale benchmark designed for lifelong knowledge editing.

    “WikiBigEdit (Thede et al., 2025) is a large-scale benchmark designed for lifelong knowledge editing.”

    From UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Models (2025) · cites this paper as Thede et al., 2025

  • WikiBigEdit supports iterative assessment of a model's capacity to incorporate factual updates over time.

    “Reflecting realistic and evolving knowledge dynamics, WikiBigEdit supports iterative assessment of a model's capacity to incorporate factual updates over time.”

    From UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Models (2025) · cites this paper as Thede et al., 2025

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 76

  1. Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., Rosenberg, M., Song, X., Stoica, A., Tiwary, S., and Wang, T. Ms marco: A human generated machine reading comprehension dataset, 2018. URL https://arxiv. org/abs/1611.09268 .
  2. Bao, Z., Chen, W., Xiao, S., Ren, K., Wu, J., Zhong, C., Peng, J., Huang, X., and Wei, Z. Disc-medllm: Bridging general large language models and real-world medical consultation. arXiv preprint arXiv:2308.14346 , 2023. · arXiv
  3. Borgeaud, S., Mensch, A., Hoffmann, J., Cai, T., Rutherford, E., Millican, K., van den Driessche, G., Lespiau, J.-B., Damoc, B., Clark, A., de Las Casas, D., Guy, A., Menick, J., Ring, R., Hennigan, T., Huang, S., Maggiore, L., Jones, C., Cassirer, A., Brock, A., Paganini, M., Irving, G., Vinyals, O., Osindero, S., Simonyan, K., Rae, J. W., Elsen, E., and Sifre, L. Improving language models by retrieving from trillions of tokens, 2022. URL https://arxiv. org/abs/2112.04426 .
  4. Chen, Q., Zhang, T., He, X., Li, D., Wang, C., Huang, L., and Xue, H. Lifelong knowledge editing for llms with retrieval-augmented continuous prompt learning, 2024. URL https://arxiv.org/abs/2405.03279 . · arXiv
  5. Cheng, J., Marone, M., Weller, O., Lawrie, D., Khashabi, D., and Durme, B. V. Dated data: Tracing knowledge cutoffs in large language models. In First Conference on Language Modeling , 2024. URL https: //openreview.net/forum?id=wS7PxDjy6m .
  6. Colombo, P., Pires, T. P., Boudiaf, M., Culver, D., Melo, R., Corro, C., Martins, A. F., Esposito, F., Raposo, V . L., Morgado, S., et al. Saullm-7b: A pioneering large language model for law. arXiv preprint arXiv:2403.03883 , 2024. · arXiv
  7. Dziadzio, S., Udandarao, V., Roth, K., Prabhu, A., Akata, Z., Albanie, S., and Bethge, M. How to merge your multimodal models over time? arXiv preprint arXiv:2412.06712 , 2024. et al., A. G. The llama 3 herd of models, 2024a. URL https://arxiv.org/abs/2407.21783 . et al., D.-A. Deepseek llm: Scaling open-source language models with longtermism, 2024b. URL https: //arxiv.org/abs/2401.02954 . et al., D.-A. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model, 2024c. URL https://arxiv.org/abs/2405.04434 . et al., D.-A. Deepseek-v3 technical report, 2024d. URL https://arxiv.org/abs/2412.19437 . et al., H. T. Llama 2: Open foundation and fine-tuned chat models, 2023. URL https://arxiv.org/abs/ 2307.09288 . · arXiv
  8. Fan, A., Jernite, Y ., Perez, E., Grangier, D., Weston, J., and Auli, M. Eli5: Long form question answering, 2019. URL https://arxiv.org/abs/1907.09190 . · arXiv
  9. Garg, S., Farajtabar, M., Pouransari, H., Vemulapalli, R., Mehta, S., Tuzel, O., Shankar, V., and Faghri, F. Tic-clip: Continual training of clip models, 2024. URL https: //arxiv.org/abs/2310.16226 .
  10. Ge, X., Mousavi, A., Grave, E., Joulin, A., Qian, K., Han, B., Arefiyan, M., and Li, Y. Time sensitive knowledge editing through efficient finetuning. arXiv [cs.CL] , June 2024.
  11. Ghosh, A., Dziadzio, S., Prabhu, A., Udandarao, V., Albanie, S., and Bethge, M. Onebench to test them all: Sample-level benchmarking over open-ended capabilities. arXiv preprint arXiv:2412.06745 , 2024. · arXiv
  12. Gui, Z., Sun, S., Li, R., Yuan, J., An, Z., Roth, K., Prabhu, A., and Torr, P. knn-clip: Retrieval enables training-free segmentation on continually expanding large vocabularies. arXiv preprint arXiv:2404.09447 , 2024. · arXiv
  13. Gupta, A., Baskaran, S., and Anumanchipalli, G. Rebuilding ROME : Resolving model collapse during sequential model editing. arXiv [cs.CL] , March 2024a.
  14. Gupta, A., Rao, A., and Anumanchipalli, G. Model editing at scale leads to gradual and catastrophic forgetting, 2024b. URL https://arxiv.org/abs/ 2401.07453 .
  15. Guti´ errez, B. J., Shu, Y ., Gu, Y ., Yasunaga, M., and Su, Y . Hipporag: Neurobiologically inspired long-term memory for large language models, 2025. URL https: //arxiv.org/abs/2405.14831 .
  16. Guu, K., Lee, K., Tung, Z., Pasupat, P., and Chang, M.W. Realm: Retrieval-augmented language model pretraining, 2020. URL https://arxiv.org/abs/ 2002.08909 .
  17. Hartvigsen, T., Sankaranarayanan, S., Palangi, H., Kim, Y., and Ghassemi, M. Aging with GRACE: Lifelong model editing with discrete key-value adaptors. Advances in Neural Information Processing Systems , November 2023.
  18. Honovich, O., Aharoni, R., Herzig, J., Taitelbaum, H., Kukliansy, D., Cohen, V., Scialom, T., Szpektor, I., Hassidim, A., and Matias, Y. True: Re-evaluating factual consistency evaluation, 2022. URL https://arxiv.org/ abs/2204.04991 . · arXiv
  19. Hsueh, C.-H., Huang, P. K.-M., Lin, T.-H., Liao, C.-W., Fang, H.-C., Huang, C.-W., and Chen, Y.-N. Editing the mind of giants: An in-depth exploration of pitfalls of knowledge editing in large language models. arXiv [cs.CL] , June 2024.
  20. Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models, 2021. URL https://arxiv. org/abs/2106.09685 .
  21. Ibrahim, A., Th´ erien, B., Gupta, K., Richter, M. L., Anthony, Q., Lesort, T., Belilovsky, E., and Rish, I. Simple and scalable strategies to continually pre-train large language models, 2024. URL https://arxiv.org/ abs/2403.08763 . · arXiv
  22. Ilharco, G., Wortsman, M., Gadre, S. Y., Song, S., Hajishirzi, H., Kornblith, S., Farhadi, A., and Schmidt, L. Patching open-vocabulary models by interpolating weights, 2022. URL https://arxiv.org/abs/2208.05592 . · arXiv
  23. Izacard, G., Lewis, P., Lomeli, M., Hosseini, L., Petroni, F., Schick, T., Dwivedi-Yu, J., Joulin, A., Riedel, S., and Grave, E. Atlas: Few-shot learning with retrieval augmented language models, 2022. URL https:// arxiv.org/abs/2208.03299 . · arXiv
  24. Jang, J., Ye, S., Lee, C., Yang, S., Shin, J., Han, J., Kim, G., and Seo, M. TemporalWiki: A lifelong benchmark for training and evaluating Ever-Evolving language models. In Goldberg, Y., Kozareva, Z., and Zhang, Y. (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pp. 6237-6250, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics.
  25. Jeong, D. P., Garg, S., Lipton, Z. C., and Oberst, M. Medical adaptation of large language and vision-language models: Are we making progress? arXiv preprint arXiv:2411.04118 , 2024. · arXiv
  26. Joshi, M., Choi, E., Weld, D. S., and Zettlemoyer, L. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension, 2017. URL https: //arxiv.org/abs/1705.03551 .
  27. Karpukhin, V., O˘ guz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and tau Yih, W. Dense passage retrieval for open-domain question answering, 2020. URL https: //arxiv.org/abs/2004.04906 .
  28. Khodja, H. A., B´ echet, F., Brabant, Q., Nasr, A., and Lecorv´ e, G. Wikifactdiff: A large, realistic, and temporally adaptable dataset for atomic factual knowledge update in causal language models, 2024. URL https: //arxiv.org/abs/2403.14364 .
  29. Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., Hassabis, D., Clopath, C., Kumaran, D., and Hadsell, R. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences , 114 (13):3521-3526, March 2017. ISSN 1091-6490. doi: 10.1073/pnas.1611835114. URL http://dx.doi. org/10.1073/pnas.1611835114 . · DOI · Paper page
  30. Kozal, J., Wasilewski, J., Krawczyk, B., and Wo´ zniak, M. Continual learning with weight interpolation, 2024. URL https://arxiv.org/abs/2404.04002 . · arXiv
  31. Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Kelcey, M., Devlin, J., Lee, K., Toutanova, K. N., Jones, L., Chang, M.-W., Dai, A., Uszkoreit, J., Le, Q., and Petrov, S. Natural questions: a benchmark for question answering research. Transactions of the Association of Computational Linguistics , 2019.
  32. Langley, P. Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning (ICML 2000) , pp. 1207-1216, Stanford, CA, 2000. Morgan Kaufmann.
  33. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., K¨ uttler, H., Lewis, M., Yih, W.-t., Rockt¨ aschel, T., Riedel, S., and Kiela, D. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems , volume 33, pp. 9459-9474. Curran Associates, Inc., 2020. URL https://proceedings.neurips. cc/paper\_files/paper/2020/file/ 6b493230205f780e1bc26945df7481e5-Paper. pdf .
  34. Li, B. Z., Liu, E., Ross, A., Zeitoun, A., Neubig, G., and Andreas, J. Language modeling with editable external knowledge, 2024. URL https://arxiv.org/abs/ 2406.11830 .
  35. Li, J., Armandpour, M., Mirzadeh, I., Mehta, S., Shankar, V., Vemulapalli, R., Bengio, S., Tuzel, O., Farajtabar, M., Pouransari, H., and Faghri, F. Tic-lm: A web-scale benchmark for time-continual llm pretraining, 2025. URL https://arxiv.org/abs/2504.02107 . · arXiv
  36. Liu, J., Min, S., Zettlemoyer, L., Choi, Y ., and Hajishirzi, H. Infini-gram: Scaling unbounded n-gram language models to a trillion tokens, 2024a. URL https://arxiv. org/abs/2401.17377 .
  37. Ma, H., Xu, W., Wei, Y., Chen, L., Wang, L., Liu, Q., Wu, S., and Wang, L. Ex-fever: A dataset for multihop explainable fact verification, 2024. URL https: //arxiv.org/abs/2310.09754 .
  38. Mallen, A., Asai, A., Zhong, V., Das, R., Khashabi, D., and Hajishirzi, H. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories, 2023. URL https://arxiv.org/abs/ 2212.10511 .
  39. Marczak, D., Twardowski, B., Trzci´ nski, T., and Cygert, S. Magmax: Leveraging model merging for seamless continual learning, 2024. URL https://arxiv.org/ abs/2407.06322 . · arXiv
  40. Marouf, I. E., Roy, S., Tartaglione, E., and Lathuili` ere, S. Weighted ensemble models are strong continual learners, 2024. URL https://arxiv.org/abs/2312. 08977 .
  41. Meng, K., Sharma, A. S., Andonian, A., Belinkov, Y., and Bau, D. Mass-editing memory in a transformer. arXiv [cs.CL] , October 2022.
  42. Meng, K., Bau, D., Andonian, A., and Belinkov, Y. Locating and editing factual associations in gpt, 2023. URL https://arxiv.org/abs/2202.05262 . · arXiv
  43. Min, T. D., Mancini, M., Alahari, K., Alameda-Pineda, X., and Ricci, E. On the effectiveness of layernorm tuning for continual learning in vision transformers, 2023. URL https://arxiv.org/abs/2308.09610 . · arXiv
  44. Mitchell, E., Lin, C., Bosselut, A., Finn, C., and Manning, C. D. Fast model editing at scale. arXiv [cs.LG] , October 2021.
  45. Mitchell, E., Lin, C., Bosselut, A., Manning, C. D., and Finn, C. Memory-based model editing at scale, 2022. URL https://arxiv.org/abs/2206.06520 . · arXiv
  46. Nijkamp, E., Xie, T., Hayashi, H., Pang, B., Xia, C., Xing, C., Vig, J., Yavuz, S., Laban, P., Krause, B., et al. Xgen7b technical report. arXiv preprint arXiv:2309.03450 , 2023. · arXiv
  47. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32 , pp. 80248035. Curran Associates, Inc., 2019.
  48. Prabhu, A., Cai, Z., Dokania, P., Torr, P., Koltun, V., and Sener, O. Online continual learning without the storage constraint. arXiv preprint arXiv:2305.09253 , 2023a. · arXiv
  49. Prabhu, A., Hammoud, H. A. A. K., Dokania, P., Torr, P. H. S., Lim, S.-N., Ghanem, B., and Bibi, A. Computationally budgeted continual learning: What does matter?, 2023b. URL https://arxiv.org/abs/ 2303.11165 .
  50. Prabhu, A., Udandarao, V., Torr, P., Bethge, M., Bibi, A., and Albanie, S. Lifelong benchmarks: Efficient model evaluation in an era of rapid progress. arXiv preprint arXiv:2402.19472 , 2024. · arXiv
  51. Ram´ e, A., Vieillard, N., Hussenot, L., Dadashi, R., Cideron, G., Bachem, O., and Ferret, J. Warm: On the benefits of weight averaged reward models, 2024. URL https: //arxiv.org/abs/2401.12187 .
  52. Rashkin, H., Nikolaev, V., Lamm, M., Aroyo, L., Collins, M., Das, D., Petrov, S., Tomar, G. S., Turc, I., and Reitter, D. Measuring attribution in natural language generation models, 2022. URL https://arxiv.org/ abs/2112.12870 . · arXiv
  53. Reimers, N. and Gurevych, I. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing . Association for Computational Linguistics, 11 2019. URL https://arxiv.org/ abs/1908.10084 . · arXiv
  54. Roth, K., Udandarao, V., Dziadzio, S., Prabhu, A., Cherti, M., Vinyals, O., H´ enaff, O., Albanie, S., Bethge, M., and Akata, Z. A practitioner's guide to continual multimodal pretraining. arXiv preprint arXiv:2408.14471 , 2024. · arXiv
  55. Song, K., Tan, X., Qin, T., Lu, J., and Liu, T.-Y. Mpnet: Masked and permuted pre-training for language understanding. Advances in neural information processing systems , 33:16857-16867, 2020.
  56. Stojanovski, Z., Roth, K., and Akata, Z. Momentum-based weight interpolation of strong zero-shot models for continual learning, 2022. URL https://arxiv.org/ abs/2211.03186 . · arXiv
  57. Team, G., Mesnard, T., Hardin, C., Dadashi, R., Bhupatiraju, S., Pathak, S., Sifre, L., Rivi` ere, M., Kale, M. S., Love, J., et al. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295 , 2024a. · arXiv
  58. Team, G., Riviere, M., Pathak, S., Sessa, P. G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahriari, B., Ram´ e, A., et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118 , 2024b. · arXiv
  59. Team, L., Modi, A., Veerubhotla, A. S., Rysbek, A., Huber, A., Wiltshire, B., Veprek, B., Gillick, D., Kasenberg, D., Ahmed, D., et al. Learnlm: Improving gemini for learning. arXiv preprint arXiv:2412.16429 , 2024c. · arXiv
  60. Thede, L., Roth, K., H´ enaff, O. J., Bethge, M., and Akata, Z. Reflecting on the state of rehearsal-free continual learning with pretrained models, 2024. URL https: //arxiv.org/abs/2406.09384 .
  61. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozi` ere, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., and Lample, G. Llama: Open and efficient foundation language models, 2023. URL https://arxiv.org/ abs/2302.13971 . · arXiv
  62. Trivedi, H., Balasubramanian, N., Khot, T., and Sabharwal, A. Is multihop qa in dire condition? measuring and reducing disconnected reasoning, 2020. URL https: //arxiv.org/abs/2005.00789 .
  63. Udandarao, V., Prabhu, A., Ghosh, A., Sharma, Y., Torr, P., Bibi, A., Albanie, S., and Bethge, M. No 'zeroshot' without exponential data: Pretraining concept frequency determines multimodal model performance. In The Thirty-eighth Annual Conference on Neural Information Processing Systems , 2024. URL https: //openreview.net/forum?id=9VbGjXLzig .
  64. Vu, T., Iyyer, M., Wang, X., Constant, N., Wei, J., Wei, J., Tar, C., Sung, Y.-H., Zhou, D., Le, Q., et al. Freshllms: Refreshing large language models with search engine augmentation. arXiv preprint arXiv:2310.03214 , 2023. · arXiv
  65. Wang, P., Li, Z., Zhang, N., Xu, Z., Yao, Y., Jiang, Y., Xie, P., Huang, F., and Chen, H. WISE: Rethinking the knowledge memory for lifelong model editing of large language models. arXiv [cs.CL] , May 2024a.
  66. Wang, S., Xu, T., Li, H., Zhang, C., Liang, J., Tang, J., Yu, P. S., and Wen, Q. Large language models for education: A survey and outlook. arXiv preprint arXiv:2403.18105 , 2024b. · arXiv
  67. Wortsman, M., Ilharco, G., Kim, J. W., Li, M., Kornblith, S., Roelofs, R., Gontijo-Lopes, R., Hajishirzi, H., Farhadi, A., Namkoong, H., and Schmidt, L. Robust fine-tuning of zero-shot models, 2022. URL https://arxiv.org/ abs/2109.01903 . · arXiv
  68. Yang, E., Shen, L., Guo, G., Wang, X., Cao, X., Zhang, J., and Tao, D. Model merging in llms, mllms, and beyond: Methods, theories, applications and opportunities. arXiv preprint arXiv:2408.07666 , 2024a. · arXiv
  69. Yang, W., Sun, F., Ma, X., Liu, X., Yin, D., and Cheng, X. The butterfly effect of model editing: Few edits can trigger large language models collapse. arXiv [cs.AI] , February 2024b.
  70. Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W. W., Salakhutdinov, R., and Manning, C. D. Hotpotqa: A dataset for diverse, explainable multi-hop question answering, 2018. URL https://arxiv.org/abs/ 1809.09600 .
  71. Yin, X., Jiang, J., Yang, L., and Wan, X. History matters: Temporal knowledge editing in large language model. AAAI , 38(17):19413-19421, March 2024.
  72. Zaken, E. B., Ravfogel, S., and Goldberg, Y. Bitfit: Simple parameter-efficient fine-tuning for transformerbased masked language-models, 2022. URL https: //arxiv.org/abs/2106.10199 .
  73. Zenke, F., Poole, B., and Ganguli, S. Continual learning through synaptic intelligence, 2017. URL https:// arxiv.org/abs/1703.04200 . · arXiv
  74. Zhang, N., Yao, Y., Tian, B., Wang, P., Deng, S., Wang, M., Xi, Z., Mao, S., Zhang, J., Ni, Y ., et al. A comprehensive study of knowledge editing for large language models. arXiv preprint arXiv:2401.01286 , 2024. · arXiv
  75. Zhao, J., Zhang, Z., Chen, B., Wang, Z., Anandkumar, A., and Tian, Y. Galore: Memory-efficient llm training by gradient low-rank projection, 2024. URL https: //arxiv.org/abs/2403.03507 .
  76. Zheng, D., Lapata, M., and Pan, J. Z. Large language models as reliable knowledge bases? arXiv preprint arXiv:2407.13578 , 2024. · arXiv

Cited by

Sorted by citations

Metadata

Authors
Thede, L., Roth, K., Bethge, M., Akata, Z., & Hartvigsen, T.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2503.05683
arXiv
2503.05683
OpenAlex
W4415249762
Cited by
0
References
4
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
large language models, Wikidata, retrieval augmentation, continual fine-tuning, factual accuracy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review