The DOI matches its DataCite record.
PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models
Open accessAbstract
Large pre-trained language models for textual data have an unconstrained output space; at each decoding step, they can produce any of 10,000s of sub-word tokens. When fine-tuned to target constrained formal languages like SQL, these models often generate invalid code, rendering it unusable. We propose PICARD (code and trained models available at https://github.com/ElementAI/picard), a method for constraining auto-regressive decoders of language models through incremental parsing. PICARD helps to find valid output sequences by rejecting inadmissible tokens at each decoding step. On the challenging Spider and CoSQL text-to-SQL translation tasks, we show that PICARD transforms fine-tuned T5 models with passable performance into state-of-the-art solutions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
No references could be resolved yet.
Research works printed but not linked: 18
- Ruisheng Cao, Lu Chen, Zhi Chen, Yanbin Zhao, Su Zhu, and Kai Yu. 2021. LGESQL: Line graph en- hanced text-to-SQL model with mixed local and non- local relations. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2541-2555, Online. Association for Computational Linguistics. · DOI
- DongHyun Choi, Myeong Cheol Shin, EungGyun Kim, and Dong Ryeol Shin. 2021. RYANSQL: Re- cursively Applying Sketch-based Slot Fillings for Complex Text-to-SQL in Cross-Domain Databases. Computational Linguistics, 47(2):309-332. · DOI
- Daan Leijen and Erik Meijer. 2001. Parsec: Direct style monadic parser combinators for the real world. Technical Report UU-CS-2001-27. User Model- ing 2007, 11th International Conference, UM 2007, Corfu, Greece, June 25-29, 2007.
- Xi Victoria Lin, Richard Socher, and Caiming Xiong. 2020. Bridging textual and tabular data for cross- domain text-to-sql semantic parsing. Findings of the Association for Computational Linguistics: EMNLP 2020. · DOI
- Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the lim- its of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21:1-67.
- Ohad Rubin and Jonathan Berant. 2021. SmBoP: Semi-autoregressive bottom-up semantic parsing. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 311-324, Online. Association for Computa- tional Linguistics. · DOI
- Peter Shaw, Ming-Wei Chang, Panupong Pasupat, and Kristina Toutanova. 2021. Compositional general- ization and natural language variation: Can a se- mantic parsing approach handle both? In Proceed- ings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Interna- tional Joint Conference on Natural Language Pro- cessing (Volume 1: Long Papers), pages 922-938, Online. Association for Computational Linguistics. · DOI
- Noam Shazeer and Mitchell Stern. 2018. Adafactor: Adaptive learning rates with sublinear memory cost. In International Conference on Machine Learning, pages 4596-4604.
- PMLR.
- Peng Shi, Patrick Ng, Zhiguo Wang, Henghui Zhu, Alexander Hanbo Li, Jun Wang, Cicero Nogueira dos Santos, and Bing Xiang. 2021. Learning con- textual representations for semantic parsing with generation-augmented pre-training. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 13806-13814.
- Alane Suhr, Ming-Wei Chang, Peter Shaw, and Ken- ton Lee. 2020. Exploring unexplored generalization challenges for cross-database semantic parsing. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 8372- 8388, Online. Association for Computational Lin- guistics. · DOI
- Bailin Wang, Richard Shin, Xiaodong Liu, Oleksandr Polozov, and Matthew Richardson. 2020. Rat-sql: Relation-aware schema encoding and linking for text-to-sql parsers. Proceedings of the 58th Annual Meeting of the Association for Computational Lin- guistics. · DOI
- Peng Xu, Dhruv Kumar, Wei Yang, Wenjie Zi, Keyi Tang, Chenyang Huang, Jackie Chi Kit Cheung, Si- mon J.D. Prince, and Yanshuai Cao. 2021. Optimiz- ing deeper transformers on small datasets. In Pro- ceedings of the 59th Annual Meeting of the Associa- tion for Computational Linguistics and the 11th In- ternational Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2089- 2102, Online. Association for Computational Lin- guistics. · DOI
- Pengcheng Yin and Graham Neubig. 2018. Tranx: A transition-based neural abstract syntax parser for se- mantic parsing and code generation. Proceedings of the 2018 Conference on Empirical Methods in Natu- ral Language Processing: System Demonstrations. · DOI · Paper page
- Tao Yu, Rui Zhang, Heyang Er, Suyi Li, Eric Xue, Bo Pang, Xi Victoria Lin, Yi Chern Tan, Tianze Shi, Zihan Li, Youxuan Jiang, Michihiro Yasunaga, Sungrok Shim, Tao Chen, Alexander Fabbri, Zifan Li, Luyao Chen, Yuwen Zhang, Shreya Dixit, Vin- cent Zhang, Caiming Xiong, Richard Socher, Wal- ter Lasecki, and Dragomir Radev. 2019. CoSQL: A conversational text-to-SQL challenge towards cross- domain natural language interfaces to databases. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan- guage Processing (EMNLP-IJCNLP), pages 1962- 1979, Hong Kong, China. Association for Computa- tional Linguistics. · DOI
- Tao Yu, Rui Zhang, Alex Polozov, Christopher Meek, and Ahmed Hassan Awadallah. 2021. Score: Pre- training for context representation in conversational semantic parsing. In International Conference on Learning Representations.
- Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingn- ing Yao, Shanelle Roman, and et al. 2018. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. · DOI · Paper page
- Ruiqi Zhong, Tao Yu, and Dan Klein. 2020. Seman- tic evaluation for text-to-sql with distilled test suites. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). · DOI · Paper page
Cited by
Sorted by citations
Synchromesh: Reliable code generation from pre-trained language models
Poesia, G., Polozov, O., Le, V., Tiwari, A., Soares, G., Meek, C., & Gulwani, S. (2022) · arXiv · doi:10.48550/arxiv.2201.11227 · Cited by 72
SAFE-SQL: Self-Augmented In-Context Learning with Fine-grained Example Selection for Text-to-SQL
Lee, J., Baek, I., Kim, B., Bae, H., & Lee, H. (2025) · Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2025.emnlp-main.962 · Cited by 3
Metadata
- Authors
- Scholak, T., Schucher, N., & Bahdanau, D.
- Affiliations
- University of Toronto (CA)
- McGill University (CA)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- Pages
- 9895-9901
- arXiv
- 2109.05093
- OpenAlex
- W3200285169
- Cited by
- 346
- References
- 0
- Citations by year
- 2026: 98 · 2025: 76 · 2024: 50 · 2023: 91 · 2022: 31
- FWCI
- 19.86
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- constrained decoding, incremental parsing, language model decoding, text-to-SQL, T5, formal language generation, Spider benchmark
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
