Controlled Hallucinations: Learning to Generate Faithfully from Noisy Data
Open accessAbstract
Neural text generation (data-or text-to-text) demonstrates remarkable performance when training data is abundant which for many applications is not the case.To collect a large corpus of parallel data, heuristic rules are often used but they inevitably let noise into the data, such as phrases in the output which cannot be explained by the input.Consequently, models pick up on the noise and may hallucinategenerate fluent but unsupported text.Our contribution is a simple but powerful technique to treat such hallucinations as a controllable aspect of the generated text, without dismissing any input and without modifying the model architecture.On the WikiBio corpus (Lebret et al., 2016), a particularly noisy dataset, we demonstrate the efficacy of the technique both in an automatic and in a human evaluation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Under a regularity assumption, for any algorithm, with probability at least 99% over training sets, the hallucination rate is at least the monofact estimate minus miscalibration minus 300 times the ratio of facts to possible hallucinations minus 7 divided by the square root of n.
“In particular, under a “regularity” assumption on the factoid distribution, our simplest bound (Corollary 1) implies that, for any algorithm, with probability ≥ 99% over training sets, the Hallucination rate is at least 𝑀𝐹 -Miscalibration - 300|Facts| |Possible hallucinations| - 7 √ 𝑛 ,”
From Calibrated Language Models Must Hallucinate (2024) · cites this paper as [14]
Section 3 describes per-example hallucination scores, and Section 6 reports an empirical reduction in hallucinations; neither gives a lower bound for every algorithm. Searches for the regularity assumption, monofact estimate, miscalibration, Corollary 1, and the stated constants found no matching result in our copy. Page numbers were not available. (LLM, conf=0.77)
Experimental — accuracy not yet measuredIf the maximum probability of any fact is bounded, the chance of generating a hallucination is close to the fraction of facts that occur exactly once in training data, even with ideal error-free training data.
“Specifically, if the maximum probability of any fact is bounded, we show that the probability of generating a hallucination is close to the fraction of facts that occur exactly once in the training data (a “Good-Turing” estimate), even assuming ideal training data without errors.”
From Calibrated Language Models Must Hallucinate (2024) · cites this paper as [14]
The Introduction and Section 3 describe detecting and controlling hallucinations associated with noisy training data. Searches for Good–Turing, facts occurring exactly once, bounded fact probabilities, and error-free training data found no passage stating the claimed probability relationship. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 21 · Cited by 79
Research works printed but not linked: 20
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Ben- gio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the 3rd International Conference on Learning Rep- resentations.
- Ondřej Dušek, David M. Howcroft, and Verena Rieser. 2019. Semantic noise matters for neural natural lan- guage generation. In Proceedings of the 12th Inter- national Conference on Natural Language Genera- tion, pages 421-426, Tokyo, Japan. Association for Computational Linguistics. · DOI · Paper page
- Jessica Ficler and Yoav Goldberg. 2017. Controlling linguistic style aspects in neural language genera- tion. In Proceedings of the Workshop on Stylistic Variation.
- Katja Filippova, Enrique Alfonseca, Carlos A. Col- menares, Lukasz Kaiser, and Oriol Vinyals. 2015. Sentence compression by deletion with LSTMs. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 360-368, Lisbon, Portugal. Association for Compu- tational Linguistics. · DOI
- Karl Moritz Hermann, Tomas Kocisky, Edward Grefen- stette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Advances in Neural Informa- tion Processing Systems 28.
- Divyansh Kaushik and Zachary C. Lipton. 2018. How much reading does reading comprehension require? a critical investigation of popular benchmarks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 5010-5015, Brussels, Belgium. Association for Computational Linguistics. · DOI
- Yuta Kikuchi, Graham Neubig, Ryohei Sasano, Hiroya Takamura, and Manabu Okumura. 2016. Control- ling output length in neural encoder-decoders. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.
- Philipp Koehn. 2005. Europarl: A Parallel Corpus for Statistical Machine Translation. In Conference Proceedings: the tenth Machine Translation Summit, pages 79-86, Phuket, Thailand. AAMT, AAMT.
- Taku Kudo and John Richardson. 2018. SentencePiece: A simple and language independent subword tok- enizer and detokenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 66-71, Brussels, Belgium. Association for Computational Linguistics. · DOI
- Rémi Lebret, David Grangier, and Michael Auli. 2016. Neural text generation from structured data with application to the biography domain. In Proceed- ings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1203-1213, Austin, Texas. Association for Computational Lin- guistics. · DOI
- Tianyu Liu, Kexiang Wang, Lei Sha, Baobao Chang, and Zhifang Sui. 2018. Table-to-text generation by structure-aware seq2seq learning. In Proceedings of the 32 AAAI Conference on Artificial Intelligence.
- Feng Nie, Jin-Ge Yao, Jinpeng Wang, Rong Pan, and Chin-Yew Lin. 2019. A simple recipe towards re- ducing hallucination in neural surface realisation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2673-2679, Florence, Italy. Association for Compu- tational Linguistics. · DOI · Paper page
- Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. pages 311-318.
- Laura Perez-Beltrachini and Claire Gardent. 2017. Analysing data-to-text generation benchmarks. In Proceedings of the 10th International Conference on Natural Language Generation, pages 238-242, San- tiago de Compostela, Spain. Association for Compu- tational Linguistics. · DOI
- Ratish Puduppully, Li Dong, and Mirella Lapata. 2019. Data-to-text generation with entity modeling. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2023-2035, Florence, Italy. Association for Compu- tational Linguistics.
- Ehud Reiter. 2018. A structured review of the validity of BLEU. Computational Linguistics, 44(3):393- 401. · DOI · Paper page
- Sascha Rothe, Shashi Narayan, and Aliaksei Severyn. 2019. Leveraging pre-trained checkpoints for se- quence generation tasks. TACL. To appear.
- Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sen- tence summarization. In Proceedings of the 2015 Conference on Empirical Methods in Natural Lan- guage Processing, pages 379-389, Lisbon, Portugal. Association for Computational Linguistics. · DOI
- Sam Wiseman, Stuart Shieber, and Alexander Rush. 2017. Challenges in data-to-document generation. In Proceedings of the 2017 Conference on Empiri- cal Methods in Natural Language Processing, pages 2253-2263, Copenhagen, Denmark. Association for Computational Linguistics. · DOI · Paper page
- Ye Zhang, Nan Ding, and Radu Soricut. 2018. SHAPED: Shared-private encoder-decoder for text style adaptation. In Proceedings of the 2018 Con- ference of the North American Chapter of the Asso- ciation for Computational Linguistics: Human Lan- guage Technologies, pages 1528-1538.
- Authors
- Filippova, K.
- Affiliations
- Google (United States) (US)
- Corresponding author
- Filippova, Katja
- ORCID
- Filippova, Katja
- Venue
- Findings of the Association for Computational Linguistics: EMNLP 2020
- Published
- 2020
- Pages
- 864-870
- OpenAlex
- W3093852412
- Cited by
- 79
- References
- 21
- Citations by year
- 2026: 11 · 2025: 22 · 2024: 9 · 2023: 14 · 2022: 10 · 2021: 13
- FWCI
- 3.53
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- hallucination mitigation, noisy training data, heuristic rules, text-to-text generation, human evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
