The DOI matches its DataCite record; 12 of the 21 references it cites were found in a registry or an index.
Efficient Guided Generation for Large Language Models
Open accessAbstract
In this article we show how the problem of neural text generation can be constructively reformulated in terms of transitions between the states of a finite-state machine. This framework leads to an efficient approach to guiding text generation with regular expressions and context-free grammars by allowing the construction of an index over a language model's vocabulary. The approach is model agnostic, allows one to enforce domain-specific knowledge and constraints, and enables the construction of reliable interfaces by guaranteeing the structure of the generated text. It adds little overhead to the token sequence generation process and significantly outperforms existing solutions. An implementation is provided in the open source Python library Outlines
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“We are concerned with the problem of generating sequences of tokens from a large language model (LLM) [Vaswani et al., 2017, Radford et al., 2019] that conform to regular expressions or context-free grammars (CFGs).”
BackgroundNeural Machine Translation of Rare Words with Subword Units
Sennrich, R., Haddow, B., & Birch, A. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1162 · Cited by 974 · cited 2×
“The vocabularies, V, are composed of strings from a fixed alphabet [Sennrich et al., 2015] and N is often on the order of 104 or larger.”
BackgroundDille, G. E. (1930) · doi:10.1177/002205743011202012 · Cited by 802
PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models
Scholak, T., Schucher, N., & Bahdanau, D. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.779 · Cited by 407 · cited 1×
“…output usable under rigid formatting requirements that are either hard or costly to capture through fine-tuning alone [Beurer-Kellner et al., 2023, Scholak et al., 2021, Poesia et al., 2022a, Rabinovich et al., 2017, Weng, 2021, Dong et al., 2023, Poesia et al., 2022b, Geng et al., 2023, Wang et…”
BackgroundAbstract Syntax Networks for Code Generation and Semantic Parsing
Rabinovich, M., Stern, M., & Klein, D. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1105 · Cited by 368 · cited 1×
“…under rigid formatting requirements that are either hard or costly to capture through fine-tuning alone [Beurer-Kellner et al., 2023, Scholak et al., 2021, Poesia et al., 2022a, Rabinovich et al., 2017, Weng, 2021, Dong et al., 2023, Poesia et al., 2022b, Geng et al., 2023, Wang et al., 2023].”
BackgroundGrammar Prompting for Domain-Specific Language Generation with Large Language Models
Wang, B.-L., Wang, Z., Wang, X.-Z., Cao, Y., Saurous, R. A., & Kim, Y. (2023) · Neural Information Processing Systems · doi:10.48550/arxiv.2305.19234 · Cited by 144 · cited 1×
“…under rigid formatting requirements that are either hard or costly to capture through fine-tuning alone [Beurer-Kellner et al., 2023, Scholak et al., 2021, Poesia et al., 2022a, Rabinovich et al., 2017, Weng, 2021, Dong et al., 2023, Poesia et al., 2022b, Geng et al., 2023, Wang et al., 2023].”
BackgroundPrompting Is Programming: A Query Language for Large Language Models
Beurer-Kellner, L., Fischer, M., & Vechev, M. (2023) · Proceedings of the ACM on Programming Languages · doi:10.1145/3591300 · Cited by 117
Synchromesh: Reliable code generation from pre-trained language models
Poesia, G., Polozov, O., Le, V., Tiwari, A., Soares, G., Meek, C., & Gulwani, S. (2022) · arXiv · doi:10.48550/arxiv.2201.11227 · Cited by 72 · cited 1×
“…under rigid formatting requirements that are either hard or costly to capture through fine-tuning alone [Beurer-Kellner et al., 2023, Scholak et al., 2021, Poesia et al., 2022a, Rabinovich et al., 2017, Weng, 2021, Dong et al., 2023, Poesia et al., 2022b, Geng et al., 2023, Wang et al., 2023].”
BackgroundSequential Monte Carlo Steering of Large Language Models using Probabilistic ProgramsPreprint
Lew, A. K., Zhi‐Xuan, T., Grand, G., & Mansinghka, V. K. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.03081 · Cited by 71
Gasser, U., & Mayer-Schönberger, V. (2024) · doi:10.2307/jj.6605394 · Cited by 29
CODEP: Grammatical Seq2Seq Model for General-Purpose Code Generation
Dong, Y., Li, G., & Jin, Z. G. (2023) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3597926.3598048 · Cited by 27 · cited 1×
“…under rigid formatting requirements that are either hard or costly to capture through fine-tuning alone [Beurer-Kellner et al., 2023, Scholak et al., 2021, Poesia et al., 2022a, Rabinovich et al., 2017, Weng, 2021, Dong et al., 2023, Poesia et al., 2022b, Geng et al., 2023, Wang et al., 2023].”
BackgroundValidating Large Language Models with ReLM
Kuchnik, M., Smith, V., & Amvrosiadis, G. (2022) · arXiv · doi:10.48550/arxiv.2211.15458 · Cited by 12
Research works printed but not linked: 1
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
Cited by
Sorted by citations
Large language models for generative information extraction: a survey
Xu, D., Chen, W., Peng, W., Zhang, C., Xu, T., Zhao, X., Wu, X., Zheng, Y., Wang, Y., & Chen, E. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40555-y · Cited by 326
SGLang: Efficient Execution of Structured Language Model Programs
Zheng, L., Yin, L., Xie, Z., Sun, C., Huang, J., Yu, C. H., Cao, S., Kozyrakis, C., Stoica, I., Gonzalez, J. E., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.07104 · Cited by 145
“To achieve this, we convert the regex into a Finite State Machine (FSM) to guide the generation process during decoding [54].”
RCAgent: Cloud Root Cause Analysis by Autonomous Agents with Tool-Augmented Large Language Models
Wang, Z., Liu, Z., Zhang, Y., Zhong, A., Wang, J., Yin, F., Fan, L., Wu, L., & Wen, Q. (2024) · Proceedings of the 33rd ACM International Conference on Information and Knowledge Management · doi:10.1145/3627673.3680016 · Cited by 99
Reducing hallucination in structured outputs via Retrieval-Augmented Generation
Ayala, O., & Bechard, P. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 6: Industry Track) · doi:10.18653/v1/2024.naacl-industry.19 · Cited by 97
Type-Constrained Code Generation with Language Models
Mündler-Sasahara, N., He, J.-X., Wang, H., Sen, K., Song, D., & Vechev, M. T. (2025) · Proc. ACM Program. Lang. · doi:10.1145/3729274 · Cited by 77
Metadata
- Authors
- Willard, B. T., & Louf, R.
- ORCID
- Louf, Rémi
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2307.09702
- OpenAlex
- W4385014449
- Cited by
- 27
- References
- 21
- Citations by year
- 2026: 7 · 2025: 12 · 2024: 7 · 2023: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- large language models, finite automata, regular expressions, context-free grammars, domain-specific constraints
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
