Cited by 21Topic ModelingMultimodal Machine Learning ApplicationsInformation Retrieval and Search Behavior

The DOI matches its Crossref record, with a small difference in title or year; 1 of the 1 references it cites were found in a registry or an index.

Towards Verifiable Text Generation with Evolving Memory and Self-Reflection

Open access
Sun, Hao, Cai, Hengyi, Wang, Bo, Hou, Yingyan, Wei, Xiaochi, Wang, Shuaiqiang, Zhang, Yan, Yin, Dawei·2024·Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing

Abstract

Despite the remarkable ability of large language models (LLMs) in language comprehension and generation, they often suffer from producing factually incorrect information, also known as hallucination.A promising solution to this issue is verifiable text generation, which prompts LLMs to generate content with citations for accuracy verification.However, verifiable text generation is non-trivial due to the focus-shifting phenomenon, the intricate reasoning needed to align the claim with correct citations, and the dilemma between the precision and breadth of retrieved documents.In this paper, we present VTG, an innovative framework for Verifiable Text Generation with evolving memory and self-reflection.VTG introduces evolving long short-term memory to retain both valuable documents and recent documents.A two-tier verifier equipped with an evidence finder is proposed to rethink and reflect on the relationship between the claim and citations.Furthermore, active retrieval and diverse query generation are utilized to enhance both the precision and breadth of the retrieved documents.We conduct extensive experiments on five datasets across three knowledge-intensive tasks and the results reveal that VTG significantly outperforms baselines.

Overview

Integrity signals

Registry

Crossref · 10.18653/v1/2024.emnlp-main.469

Cites

0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Existing long-term memory methods typically follow two directions: continuously recording, summarizing, and updating dialogue histories as memory, or transforming conversations into structured graph-based memory using OpenIE.

    “These approaches typically follow two directions: (i) continuously recording, summarizing, and updating dialogue histories as memory [2, 23, 39, 44], or (ii) transforming conversations into structured graphbased memory using techniques such as OpenIE [19, 40, 57, 58].”

Citations

Cites

The publisher hasn’t deposited this paper’s reference list.

Cited by

Sorted by citations

Metadata

Authors
Sun, H., Cai, H., Wang, B., Hou, Y., Wei, X., Wang, S., Zhang, Y., & Yin, D.
Affiliations
  • Peking University (CN)
  • Chinese Academy of Sciences (CN)
  • Beijing Institute of Technology (CN)
  • Baidu (China) (CN)
Venue
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Published
2024
Pages
8211-8227
OpenAlex
W4404783066
Cited by
8
References
0
Citations by year
2026: 3 · 2025: 4 · 2024: 1
FWCI
2.31
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
Keywords
hallucination mitigation, large language models, long short-term memory, self-reflection, query generation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review