CoChat
Sign Up
PreprintCited by 100Topic ModelingNatural Language Processing TechniquesAdvanced Database Systems and Queries

The DOI matches its DataCite record.

Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL

Open access
Hong, Zijin, Yuan, Zheng, Zhang, Qinggang, Chen, Hao, Dong, Junnan, Huang, Feiran, Huang, Xiao·2024·arXiv
DOI 10.48550/arxiv.2406.08426arXiv 2406.08426
Open this paper in CoChat — free accountOpen PDF

Abstract

Generating accurate SQL from users' natural language questions (text-to-SQL) remains a long-standing challenge due to the complexities involved in user question understanding, database schema comprehension, and SQL generation. Traditional text-to-SQL systems, which combine human engineering and deep neural networks, have made significant progress. Subsequently, pre-trained language models (PLMs) have been developed for text-to-SQL tasks, achieving promising results. However, as modern databases and user questions grow more complex, PLMs with a limited parameter size often produce incorrect SQL. This necessitates more sophisticated and tailored optimization methods, which restricts the application of PLM-based systems. Recently, large language models (LLMs) have shown significant capabilities in natural language understanding as model scale increases. Thus, integrating LLM-based solutions can bring unique opportunities, improvements, and solutions to text-to-SQL research. In this survey, we provide a comprehensive review of existing LLM-based text-to-SQL studies. Specifically, we offer a brief overview of the technical challenges and evolutionary process of text-to-SQL. Next, we introduce the datasets and metrics designed to evaluate text-to-SQL systems. Subsequently, we present a systematic analysis of recent advances in LLM-based text-to-SQL. Finally, we make a summarization and discuss the remaining challenges in this field and suggest expectations for future research directions. All the related resources of LLM-based, including research papers, benchmarks, and open-source projects, are collected for the community in our repository: https://github.com/DEEP-PolyU/Awesome-LLM-based-Text2SQL.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2406.08426

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 148

  1. J. Li, B. Hui, G. QU, J. Yang, B. Li, B. Li, B. Wang, B. Qin, R. Geng, N. Huo, X. Zhou, C. Ma, G. Li, K. Chang, F. Huang, R. Cheng, and Y. Li, "Can LLM already serve as a database interface? a BIg bench for large-scale database grounded text-to-SQLs," in Advances in Neural Information Processing Systems (NeurIPS), 2023.
  2. L. Wang, B. Qin, B. Hui, B. Li, M. Yang, B. Wang, B. Li, J. Sun, F. Huang, L. Si, and Y. Li, "Proton: Probing schema linking information from pre-trained language models for text-to-sql parsing," in Conference on Knowledge Discovery and Data Mining (KDD), 2022.
  3. B. Qin, B. Hui, L. Wang, M. Yang, J. Li, B. Li, R. Geng, R. Cao, J. Sun, L. Si et al., "A survey on text-to-sql parsing: Concepts, methods, and future directions," arXiv preprint arXiv:2208.13629, 2022. · arXiv
  4. S. Xu, S. Semnani, G. Campagna, and M. Lam, "Autoqa: From databases to qa semantic parsers with only syn- thetic training data," in Empirical Methods in Natural Language Processing (EMNLP), 2020.
  5. S. Lin, J. Hilton, and O. Evans, "Truthfulqa: Measuring how models mimic human falsehoods," in Association for Computational Linguistics (ACL), 2021.
  6. N. Rajkumar, R. Li, and D. Bahdanau, "Evaluating the text-to-sql capabilities of large language models," arXiv preprint arXiv:2204.00498, 2022. · arXiv
  7. M. Pourreza and D. Rafiei, "DIN-SQL: Decomposed in-context learning of text-to-SQL with self-correction," in Advances in Neural Information Processing Systems (NeurIPS), 2023.
  8. D. Gao, H. Wang, Y. Li, X. Sun, Y. Qian, B. Ding, and J. Zhou, "Text-to-sql empowered by large language models: A benchmark evaluation," in International Conference on Very Large Data Bases (VLDB), 2024.
  9. F. Li and H. V. Jagadish, "Constructing an interactive natural language interface for relational databases," in International Conference on Very Large Data Bases (VLDB), 2014.
  10. T. Mahmud, K. A. Hasan, M. Ahmed, and T. H. C. Chak, "A rule based approach for nlp based query processing," in International Conference on Electrical Information and Communication Technologies (EICT), 2015.
  11. T. Yu, R. Zhang, K. Yang, M. Yasunaga, D. Wang, Z. Li, J. Ma, I. Li, Q. Yao, S. Roman, Z. Zhang, and D. Radev, "Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-SQL task," in Empirical Methods in Natural Language Processing (EMNLP), 2018.
  12. V. Zhong, C. Xiong, and R. Socher, "Seq2sql: Generating structured queries from natural language using reinforce- ment learning," arXiv preprint arXiv:1709.00103, 2017. · arXiv
  13. M. Pourreza and D. Rafiei, "Evaluating cross-domain text-to-SQL models and benchmarks," in Empirical Methods in Natural Language Processing (EMNLP), 2023.
  14. I. Sutskever, O. Vinyals, and Q. V. Le, "Sequence to sequence learning with neural networks," in Advances in Neural Information Processing Systems (NeurIPS), 2014.
  15. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, "Attention is all you need," in Advances in Neural Information Processing Systems (NeurIPS), 2017.
  16. B. Hui, X. Shi, R. Geng, B. Li, Y. Li, J. Sun, and X. Zhu, "Improving text-to-sql with schema dependency learning," arXiv preprint arXiv:2103.04399, 2021. · arXiv
  17. P. Yin, G. Neubig, W.-t. Yih, and S. Riedel, "Tabert: Pretraining for joint understanding of textual and tabular data," arXiv preprint arXiv:2005.08314, 2020. · arXiv
  18. H. Li, J. Zhang, C. Li, and H. Chen, "Resdsql: Decou- pling schema linking and skeleton parsing for text-to-sql," in Conference on Artificial Intelligence (AAAI), 2023.
  19. J. Li, B. Hui, R. Cheng, B. Qin, C. Ma, N. Huo, F. Huang, W. Du, L. Si, and Y. Li, "Graphix-t5: Mixing pre-trained transformers with graph-aware layers for text-to-sql parsing," in Conference on Artificial Intelligence (AAAI), 2023.
  20. D. Rai, B. Wang, Y. Zhou, and Z. Yao, "Improv- ing generalization in language model-based text-to- sql semantic parsing: Two simple semantic boundary- based techniques," in Association for Computational Linguistics (ACL), 2023.
  21. Q. Lyu, K. Chakrabarti, S. Hathi, S. Kundu, J. Zhang, and Z. Chen, "Hybrid ranking network for text-to-sql," arXiv preprint arXiv:2008.04759, 2020. · arXiv
  22. T. Yu, C.-S. Wu, X. V. Lin, bailin wang, Y. C. Tan, X. Yang, D. Radev, richard socher, and C. Xiong, "Grappa: Grammar-augmented pre-training for table se- mantic parsing," in International Conference on Learning Representations (ICLR), 2021.
  23. H. Li, J. Zhang, H. Liu, J. Fan, X. Zhang, J. Zhu, R. Wei, H. Pan, C. Li, and H. Chen, "Codes: Towards building open-source language models for text-to-sql," in Conference on Management of Data (SIGMOD), 2024.
  24. A. Liu, X. Hu, L. Wen, and P. S. Yu, "A comprehensive evaluation of chatgpt's zero-shot text-to-sql capability," arXiv preprint arXiv:2303.13547, 2023. · arXiv
  25. J. Yang, H. Jin, R. Tang, X. Han, Q. Feng, H. Jiang, S. Zhong, B. Yin, and X. Hu, "Harnessing the power of llms in practice: A survey on chatgpt and beyond," ACM Transactions on Knowledge Discovery from Data (TKDD), 2024.
  26. W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y. Hou, Y. Min, B. Zhang, J. Zhang, Z. Dong et al., "A survey of large language models," arXiv preprint arXiv:2303.18223, 2023. · arXiv
  27. X. Dong, C. Zhang, Y. Ge, Y. Mao, Y. Gao, J. Lin, D. Lou et al., "C3: Zero-shot text-to-sql with chatgpt," arXiv preprint arXiv:2307.07306, 2023. · arXiv
  28. Z. Yuan, H. Chen, Z. Hong, Q. Zhang, F. Huang, Q. Li, and X. Huang, "Knapsack optimization-based schema linking for llm-based text-to-sql generation," arXiv preprint arXiv:2502.12911, 2025. · arXiv
  29. Q. Zhang, J. Dong, H. Chen, W. Li, F. Huang, and X. Huang, "Structure guided large language model for sql generation," in International Conference on Machine Learning (ICML), 2024.
  30. L. Wang, A. Zhang, K. Wu, K. Sun, Z. Li, H. Wu, M. Zhang, and H. Wang, "DuSQL: A large-scale and pragmatic Chinese text-to-SQL dataset," in Empirical Methods in Natural Language Processing (EMNLP), 2020.
  31. T. Yu, R. Zhang, H. Er, S. Li, E. Xue, B. Pang, X. V. Lin, Y. C. Tan, T. Shi, Z. Li et al., "CoSQL: A conversational text-to-SQL challenge towards cross-domain natural language interfaces to databases," in Empirical Methods in Natural Language Processing and International Joint Conference on Natural Language Processing (EMNLP- IJCNLP), 2019.
  32. C.-H. Lee, O. Polozov, and M. Richardson, "KaggleD- BQA: Realistic evaluation of text-to-SQL parsers," in Association for Computational Linguistics and Interna- tional Joint Conference on Natural Language Processing (ACL-IJCNLP), 2021.
  33. X. Pi, B. Wang, Y. Gao, J. Guo, Z. Li, and J.-G. Lou, "Towards robustness of text-to-SQL models against natural and realistic adversarial table perturbation," in Association for Computational Linguistics (ACL), 2022.
  34. Y. Gan, X. Chen, Q. Huang, and M. Purver, "Measuring and improving compositional generalization in text-to- SQL via component alignment," in Findings of North American Chapter of the Association for Computational Linguistics (NAACL), 2022.
  35. Y. Gan, X. Chen, and M. Purver, "Exploring underex- plored limitations of cross-domain text-to-SQL gener- alization," in Empirical Methods in Natural Language Processing (EMNLP), 2021.
  36. Y. Gan, X. Chen, Q. Huang, M. Purver, J. R. Woodward, J. Xie, and P. Huang, "Towards robustness of text-to-SQL models against synonym substitution," in Association for Computational Linguistics and International Joint Conference on Natural Language Processing (ACL- IJCNLP), 2021.
  37. X. Deng, A. H. Awadallah, C. Meek, O. Polozov, H. Sun, and M. Richardson, "Structure-grounded pretraining for text-to-SQL," in North American Chapter of the Associ- ation for Computational Linguistics: Human Language Technologies (NAACL-HLT), 2021.
  38. Q. Min, Y. Shi, and Y. Zhang, "A pilot study for Chinese SQL semantic parsing," in Empirical Methods in Natural Language Processing and International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019.
  39. T. Yu, R. Zhang, M. Yasunaga, Y. C. Tan, X. V. Lin, S. Li, H. Er, I. Li, B. Pang, T. Chen, E. Ji, S. Dixit, D. Proctor, S. Shim, J. Kraft, V. Zhang, C. Xiong, R. Socher, and D. Radev, "SParC: Cross-domain semantic parsing in context," in Association for Computational Linguistics (ACL), 2019.
  40. T. Shi, C. Zhao, J. Boyd-Graber, H. Daumé III, and L. Lee, "On the potential of lexico-logical alignments for semantic parsing to SQL queries," in Findings of Empirical Methods in Natural Language Processing (EMNLP), 2020.
  41. F. Lei, J. Chen, Y. Ye, R. Cao, D. Shin, H. SU, Z. SUO, H. Gao, W. Hu, P. Yin, V. Zhong, C. Xiong, R. Sun, Q. Liu, S. Wang, and T. Yu, "Spider 2.0: Evaluating language models on real-world enterprise text-to-SQL workflows," in International Conference on Learning Representations (ICLR), 2025.
  42. A. Tuan Nguyen, M. H. Dao, and D. Q. Nguyen, "A pilot study of text-to-SQL semantic parsing for Vietnamese," in Findings of Empirical Methods in Natural Language Processing (EMNLP), 2020.
  43. S. Chang, J. Wang, M. Dong, L. Pan, H. Zhu, A. H. Li, W. Lan, S. Zhang, J. Jiang, J. Lilien et al., "Dr. spider: A diagnostic evaluation benchmark towards text-to-sql robustness," in International Conference on Learning Representations (ICLR), 2023.
  44. C. Zhang, Y. Mao, Y. Fan, Y. Mi, Y. Gao, L. Chen, D. Lou, and J. Lin, "Finsql: model-agnostic llms- based text-to-sql framework for financial analysis," in Conference on Management of Data (SIGMOD), 2024.
  45. T. Zhang, T. Yu, T. B. Hashimoto, M. Lewis, W. tau Yih, D. Fried, and S. I. Wang, "Coder reviewer reranking for code generation," in International Conference on Machine Learning (ICML), 2023.
  46. C.-Y. Tai, Z. Chen, T. Zhang, X. Deng, and H. Sun, "Exploring chain of thought style prompting for text- to-SQL," in Empirical Methods in Natural Language Processing (EMNLP), 2023.
  47. B. Wang, C. Ren, J. Yang, X. Liang, J. Bai, L. Chai, Z. Yan, Q.-W. Zhang, D. Yin, X. Sun, and Z. Li, "Mac- sql: A multi-agent collaborative framework for text- to-sql," in International Conference on Computational Linguistics (COLING), 2024.
  48. Y. Xie, X. Jin, T. Xie, M. Lin, L. Chen, C. Yu, L. Cheng, C. Zhuo, B. Hu, and Z. Li, "Decomposition for enhancing attention: Improving llm-based text-to-sql through workflow paradigm," in Findings of Association for Computational Linguistics (ACL), 2024.
  49. Y. Fan, Z. He, T. Ren, C. Huang, Y. Jing, K. Zhang, and X. S. Wang, "Metasql: A generate-then-rank framework for natural language to sql translation," in International Conference on Data Engineering (ICDE), 2024.
  50. H. Xia, F. Jiang, N. Deng, C. Wang, G. Zhao, R. Mi- halcea, and Y. Zhang, ""this is my sql, are you with me?" a consensus-based multi-agent system for text-to- sql tasks," in ACL 2025 Fourth Table Representation Learning Workshop (ACL), 2024.
  51. Z. Li, X. Wang, J. Zhao, S. Yang, G. Du, X. Hu, B. Zhang, Y. Ye, Z. Li, R. Zhao, and H. Mao, "Pet-sql: A prompt-enhanced two-stage text-to-sql framework with cross-consistency," arXiv preprint arXiv:2403.09732, 2024. · arXiv
  52. T. Ren, Y. Fan, Z. He, R. Huang, J. Dai, C. Huang, Y. Jing, K. Zhang, Y. Yang, and X. S. Wang, "Purple: Making a large language model a better sql writer," in International Conference on Data Engineering (ICDE), 2024.
  53. G. Qu, J. Li, B. Li, B. Qin, N. Huo, C. Ma, and R. Cheng, "Before generation, align it! a novel and effective strategy for mitigating hallucinations in text-to-SQL generation," in Findings of Association for Computational Linguistics (ACL), 2024.
  54. D. Lee, C. Park, J. Kim, and H. Park, "MCS-SQL: Lever- aging multiple prompts and multiple-choice selection for text-to-SQL generation," in International Conference on Computational Linguistics (COLING), 2025.
  55. S. Talaei, M. Pourreza, Y.-C. Chang, A. Mirhoseini, and A. Saberi, "Chess: Contextual harnessing for efficient sql synthesis," arXiv preprint arXiv:2405.16755, 2024. · arXiv
  56. B. Li, Y. Luo, C. Chai, G. Li, and N. Tang, "The dawn of natural language to sql: Are we fully ready?" in International Conference on Very Large Data Bases (VLDB), 2024.
  57. K. Maamari, F. Abubaker, D. Jaroslawicz, and A. Mhed- hbi, "The death of schema linking? text-to-SQL in the age of well-reasoned language models," in NeurIPS 2024 Third Table Representation Learning Workshop (NeurIPS), 2024.
  58. Z. Cao, Y. Zheng, Z. Fan, X. Zhang, and W. Chen, "Rsl- sql: Robust schema linking in text-to-sql generation," arXiv preprint arXiv:2411.00073, 2024. · arXiv
  59. J. Shi, B. Xu, J. Liang, Y. Xiao, J. Chen, C. Xie, P. Wang, and W. Wang, "Gen-SQL: Efficient text-to-SQL by bridging natural language question and database schema with pseudo-schema," in International Conference on Computational Linguistics (COLING), 2025.
  60. M. Deng, A. Ramachandran, C. Xu, L. Hu, Z. Yao, A. Datta, and H. Zhang, "Reforce: A text-to-sql agent with self-refinement, format restriction, and column exploration," in ICLR 2025 The VerifAI Workshop (ICLR), 2025.
  61. Y. Wang and P. Liu, "Linkalign: Scalable schema linking for real-world large-scale multi-database text-to-sql," arXiv preprint arXiv:2503.18596, 2025. · arXiv
  62. C. Guo, Z. Tian, J. Tang, P. Wang, Z. Wen, K. Yang, and T. Wang, "Prompting gpt-3.5 for text-to-sql with de-semanticization and skeleton retrieval," in Pacific Rim International Conference on Artificial Intelligence (PRICAI), 2024.
  63. J. Jiang, K. Zhou, Z. Dong, K. Ye, X. Zhao, and J.- R. Wen, "StructGPT: A general framework for large language model to reason over structured data," in Empirical Methods in Natural Language Processing (EMNLP), 2023.
  64. L. Nan, Y. Zhao, W. Zou, N. Ri, J. Tae, E. Zhang, A. Cohan, and D. Radev, "Enhancing text-to-SQL capabilities of large language models: A study on prompt design strategies," in Findings of Empirical Methods in Natural Language Processing (EMNLP), 2023.
  65. C. Guo, Z. Tian, J. Tang, S. Li, Z. Wen, K. Wang, and T. Wang, "Retrieval-augmented gpt-3.5-based text-to-sql framework with sample-aware prompting and dynamic revision chain," in International Conference on Neural Information Processing (ICONIP), 2024.
  66. S. Chang and E. Fosler-Lussier, "Selective demonstra- tions for cross-domain text-to-SQL," in Findings of Empirical Methods in Natural Language Processing (EMNLP), 2023.
  67. H. Zhang, R. Cao, L. Chen, H. Xu, and K. Yu, "ACT-SQL: In-context learning for text-to-SQL with automatically-generated chain-of-thought," in Findings of Empirical Methods in Natural Language Processing (EMNLP), 2023.
  68. D. Wang, L. Dou, X. Zhang, Q. Zhu, and W. Che, "Improving demonstration diversity by human-free fusing for text-to-sql," in Findings of Empirical Methods in Natural Language Processing (EMNLP), 2024.
  69. Z. Hong, Z. Yuan, H. Chen, Q. Zhang, F. Huang, and X. Huang, "Knowledge-to-sql: Enhancing sql generation with data expert llm," in Findings of Association for Computational Linguistics (ACL), 2024.
  70. D. G. Thorpe, A. J. Duberstein, and I. A. Kinsey, "Dubo- sql: Diverse retrieval-augmented generation and fine tuning for text-to-sql," arXiv preprint arXiv:2404.12560, 2024. · arXiv
  71. R. Toteja, A. Sarkar, and P. M. Comar, "In-context reinforcement learning with retrieval-augmented gener- ation for text-to-SQL," in International Conference on Computational Linguistics (COLING), 2025.
  72. J. Lee, I. Baek, B. Kim, and H. Lee, "Safe-sql: Self-augmented in-context learning with fine-grained example selection for text-to-sql," arXiv preprint arXiv:2502.11438, 2025. · arXiv
  73. R. Sun, S. O. Arik, H. Nakhost, H. Dai, R. Sinha, P. Yin, and T. Pfister, "Sql-palm: Improved large language model adaptation for text-to-sql," Transactions on Machine Learning Research (TMLR), 2023.
  74. H. Xia, F. Jiang, N. Deng, C. Wang, G. Zhao, R. Mi- halcea, and Y. Zhang, "Sql-craft: Text-to-sql through interactive refinement and enhanced reasoning," arXiv preprint arXiv:2402.14851, 2024. · arXiv
  75. Y. Gu, Y. Shu, H. Yu, X. Liu, Y. Dong, J. Tang, J. Srinivasa, H. Latapie, and Y. Su, "Middleware for llms: Tools are instrumental for language agents in complex environments," in Empirical Methods in Natural Language Processing (EMNLP), 2024.
  76. M. Pourreza, H. Li, R. Sun, Y. Chung, S. Talaei, G. T. Kakkar, Y. Gan, A. Saberi, F. Ozcan, and S. O. Arik, "CHASE-SQL: Multi-path reasoning and preference optimized candidate selection in text-to-SQL," in Interna- tional Conference on Learning Representations (ICLR), 2025.
  77. F. Shi, D. Fried, M. Ghazvininejad, L. Zettlemoyer, and S. I. Wang, "Natural language to code translation with execution," in Empirical Methods in Natural Language Processing (EMNLP), 2022.
  78. A. Ni, S. Iyer, D. Radev, V. Stoyanov, W.-t. Yih, S. I. Wang, and X. V. Lin, "Lever: Learning to verify language-to-code generation with execution," in Interna- tional Conference on Machine Learning (ICML), 2023.
  79. X. Chen, M. Lin, N. Schärli, and D. Zhou, "Teaching large language models to self-debug," in International Conference on Learning Representations (ICLR), 2024.
  80. H. A. Caferoglu and Ö. Ulusoy, "E-sql: Direct schema linking via question enrichment in text-to-sql," arXiv preprint arXiv:2409.16751, 2024. · arXiv
  81. S. Kou, L. Hu, Z. He, Z. Deng, and H. Zhang, "Cllms: Consistency large language models," in International Conference on Machine Learning (ICML), 2024.
  82. F. Xu, Z. Wu, Q. Sun, S. Ren, F. Yuan, S. Yuan, Q. Lin, Y. Qiao, and J. Liu, "Symbol-llm: Towards foundational symbol-centric interface for large language models," in Association for Computational Linguistics (ACL), 2024.
  83. A. Zhuang, G. Zhang, T. Zheng, X. Du, J. Wang, W. Ren, S. W. Huang, J. Fu, X. Yue, and W. Chen, "Structlm: Towards building generalist models for structured knowl- edge grounding," in Conference on Language Modeling (COLM), 2024.
  84. Y. Gao, Y. Liu, X. Li, X. Shi, Y. Zhu, Y. Wang, S. Li, W. Li, Y. Hong, Z. Luo et al., "Xiyan-sql: A multi- generator ensemble framework for text-to-sql," arXiv preprint arXiv:2411.08599, 2024. · arXiv
  85. G. Qu, J. Li, B. Qin, X. Li, N. Huo, C. Ma, and R. Cheng, "SHARE: An SLM-based hierarchical action CorREction assistant for text-to-SQL," in Association for Computational Linguistics (ACL), 2025.
  86. B. Roziere, J. Gehring, F. Gloeckle, S. Sootla, I. Gat, X. E. Tan, Y. Adi, J. Liu, T. Remez, J. Rapin et al., "Code llama: Open foundation models for code," arXiv preprint arXiv:2308.12950, 2023. · arXiv
  87. M. Pourreza and D. Rafiei, "Dts-sql: Decomposed text- to-sql with small large language models," in Findings of Empirical Methods in Natural Language Processing (EMNLP), 2024.
  88. S. K. Gorti, I. Gofman, Z. Liu, J. Wu, N. Vouitsis, G. Yu, J. C. Cresswell, and R. Hosseinzadeh, "MSc- SQL: Multi-sample critiquing small language models for text-to-SQL translation," in North American Chapter of the Association for Computational Linguistics (NAACL), 2024.
  89. Y. Qin, C. Chen, Z. Fu, Z. Chen, D. Peng, P. Hu, and J. Ye, "ROUTE: Robust multitask tuning and collabo- ration for text-to-SQL," in International Conference on Learning Representations (ICLR), 2025.
  90. D. Xu, W. Chen, W. Peng, C. Zhang, T. Xu, X. Zhao, X. Wu, Y. Zheng, and E. Chen, "Large language models for generative information extraction: A survey," arXiv preprint arXiv:2312.17617, 2023. · arXiv
  91. G. Katsogiannis-Meimarakis and G. Koutrika, "A survey on deep learning approaches for text-to-sql," Journal on Very Large Data Bases (VLDBJ), 2023.
  92. N. Deng, Y. Chen, and Y. Zhang, "Recent advances in text-to-SQL: A survey of what we have and what we expect," in International Conference on Computational Linguistics (COLING), 2022.
  93. W. Zhang, Y. Wang, Y. Song, V. J. Wei, Y. Tian, Y. Qi, J. H. Chan, R. C.-W. Wong, and H. Yang, "Natural language interfaces for tabular data querying and visualization: A survey," IEEE Transactions on Knowledge and Data Engineering (TKDE), 2024.
  94. P. Ma and S. Wang, "Mt-teql: evaluating and augment- ing neural nlidb on real-world linguistic and schema variations," in International Conference on Very Large Data Bases (VLDB), 2021.
  95. P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, "SQuAD: 100,000+ questions for machine comprehen- sion of text," in Empirical Methods in Natural Language Processing (EMNLP), 2016.
  96. P. Rajpurkar, R. Jia, and P. Liang, "Know what you don't know: Unanswerable questions for SQuAD," in Association for Computational Linguistics (ACL), 2018.
  97. H. Yang, Y. Zhang, J. Xu, H. Lu, P.-A. Heng, and W. Lam, "Unveiling the generalization power of fine- tuned large language models," in North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 2024.
  98. J. Guo, Z. Zhan, Y. Gao, Y. Xiao, J.-G. Lou, T. Liu, and D. Zhang, "Towards complex text-to-sql in cross- domain database with intermediate representation," arXiv preprint arXiv:1905.08205, 2019. · arXiv
  99. X. Xu, C. Liu, and D. Song, "Sqlnet: Generating structured queries from natural language without re- inforcement learning," arXiv preprint arXiv:1711.04436, 2017. · arXiv
  100. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of deep bidirectional transformers for language understanding," in North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 2019.
  101. Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, "Roberta: A robustly optimized bert pretraining approach," arXiv preprint arXiv:1907.11692, 2019. · arXiv
  102. L. Dou, Y. Gao, X. Liu, M. Pan, D. Wang, W. Che, D. Zhan, M.-Y. Kan, and J.-G. Lou, "Towards knowledge- intensive text-to-SQL semantic parsing with formulaic knowledge," in Empirical Methods in Natural Language Processing (EMNLP), 2022.
  103. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., "Language models are few-shot learners," in Advances in Neural Information Processing Systems (NeurIPS), 2020.
  104. J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al., "Gpt-4 technical report," arXiv preprint arXiv:2303.08774, 2023. · arXiv
  105. S. Chang and E. Fosler-Lussier, "How to prompt LLMs for text-to-SQL: A study in zero-shot, single-domain, and cross-domain settings," in NeurIPS 2023 Second Table Representation Learning Workshop (NeurIPS), 2023.
  106. P. Sahoo, A. K. Singh, S. Saha, V. Jain, S. Mondal, and A. Chadha, "A systematic survey of prompt engineering in large language models: Techniques and applications," arXiv preprint arXiv:2402.07927, 2024. · arXiv
  107. J. Wang, E. Shi, S. Yu, Z. Wu, C. Ma, H. Dai, Q. Yang, Y. Kang, J. Wu, H. Hu et al., "Prompt engineering for healthcare: Methodologies and applications," arXiv preprint arXiv:2304.14670, 2023. · arXiv
  108. B. Chen, Z. Zhang, N. Langrené, and S. Zhu, "Unleash- ing the potential of prompt engineering in large lan- guage models: a comprehensive review," arXiv preprint arXiv:2310.14735, 2023. · arXiv
  109. J. Wei, M. Bosma, V. Y. Zhao, K. Guu, A. W. Yu, B. Lester, N. Du, A. M. Dai, and Q. V. Le, "Finetuned language models are zero-shot learners," arXiv preprint arXiv:2109.01652, 2021. · arXiv
  110. Y. Zheng, R. Zhang, J. Zhang, Y. Ye, and Z. Luo, "Llamafactory: Unified efficient fine-tuning of 100+ language models," arXiv preprint arXiv:2403.13372, 2024. · arXiv
  111. S. Xue, C. Jiang, W. Shi, F. Cheng, K. Chen, H. Yang, Z. Zhang, J. He, H. Zhang, G. Wei, W. Zhao, F. Zhou, D. Qi, H. Yi, S. Liu, and F. Chen, "Db-gpt: Empowering database interactions with private large language models," arXiv preprint arXiv:2312.17449, 2024. · arXiv
  112. T. Wang, H. Lin, X. Han, L. Sun, X. Chen, H. Wang, and Z. Zeng, "Dbcopilot: Scaling natural language querying to massive databases," arXiv preprint arXiv:2312.03463, 2023. · arXiv
  113. H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar et al., "Llama: Open and efficient foundation language models," arXiv preprint arXiv:2302.13971, 2023. · arXiv
  114. H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhos- ale et al., "Llama 2: Open foundation and fine-tuned chat models," arXiv preprint arXiv:2307.09288, 2023. · arXiv
  115. J. Bai, S. Bai, Y. Chu, Z. Cui, K. Dang, X. Deng, Y. Fan, W. Ge, Y. Han, F. Huang et al., "Qwen technical report," arXiv preprint arXiv:2309.16609, 2023. · arXiv
  116. B. Zhang, Y. Ye, G. Du, X. Hu, Z. Li, S. Yang, C. H. Liu, R. Zhao, Z. Li, and H. Mao, "Benchmarking the text-to-sql capability of large language models: A com- prehensive evaluation," arXiv preprint arXiv:2403.02951, 2024. · arXiv
  117. L. Reynolds and K. McDonell, "Prompt programming for large language models: Beyond the few-shot paradigm," in Conference on Human Factors in Computing Systems (CHI), 2021.
  118. X. Ye and G. Durrett, "The unreliability of explana- tions in few-shot prompting for textual reasoning," in Advances in Neural Information Processing Systems (NeurIPS), 2022.
  119. C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, "Exploring the limits of transfer learning with a unified text-to-text transformer," Journal of Machine Learning Research (JMLR), 2020.
  120. M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman et al., "Evaluating large language models trained on code," arXiv preprint arXiv:2107.03374, 2021. · arXiv
  121. P. P. Ray, "Chatgpt: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope," Internet of Things and Cyber-Physical Systems, 2023.
  122. X. Wang, J. Wei, D. Schuurmans, Q. V. Le, E. H. Chi, S. Narang, A. Chowdhery, and D. Zhou, "Self- consistency improves chain of thought reasoning in lan- guage models," in International Conference on Learning Representations (ICLR), 2023.
  123. W. Chen, X. Ma, X. Wang, and W. W. Cohen, "Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks," Transactions on Machine Learning Research (TMLR), 2023.
  124. R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn, "Direct preference optimization: Your language model is secretly a reward model," in Advances in Neural Information Processing Systems (NeurIPS), 2023.
  125. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al., "Training language models to follow instructions with human feedback," in Advances in Neural Informa- tion Processing Systems (NeurIPS), 2022.
  126. Z. Gong and Y. Sun, "Graph reasoning enhanced language models for text-to-sql," in Conference on Information Retrieval (SIGIR), 2024.
  127. R. Cao, L. Chen, J. Li, H. Zhang, H. Xu, W. Zhang, and K. Yu, "A heterogeneous graph to abstract syntax tree framework for text-to-sql," IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023.
  128. D. Erhan, A. Courville, Y. Bengio, and P. Vincent, "Why does unsupervised pre-training help deep learning?" in Artificial Intelligence and Statistics (AISTATS), 2010.
  129. R. Li, L. B. allal, Y. Zi, N. Muennighoff, D. Kocetkov, C. Mou, M. Marone, C. Akiki, J. LI, J. Chim, Q. Liu et al., "Starcoder: may the source be with you!" Trans- actions on Machine Learning Research (TMLR), 2023.
  130. Y. Wang, W. Zhong, L. Li, F. Mi, X. Zeng, W. Huang, L. Shang, X. Jiang, and Q. Liu, "Aligning large lan- guage models with human: A survey," arXiv preprint arXiv:2307.12966, 2023. · arXiv
  131. Z. Cai, X. Li, B. Hui, M. Yang, B. Li, B. Li, Z. Cao, W. Li, F. Huang, L. Si, and Y. Li, "STAR: SQL guided pre-training for context-dependent text-to-SQL parsing," in Findings of Empirical Methods in Natural Language Processing (EMNLP), 2022.
  132. R. Giaquinto, D. Zhang, B. Kleiner, Y. Li, M. Tan, P. Bhatia, R. Nallapati, and X. Ma, "Multitask pretraining with structured knowledge for text-to-SQL generation," in Association for Computational Linguistics (ACL), 2023.
  133. H. Song, M. Kim, D. Park, Y. Shin, and J.-G. Lee, "Learning from noisy labels with deep neural networks: A survey," IEEE Transactions on Neural Networks and Learning Systems (TNNLS), 2023.
  134. T. Scholak, N. Schucher, and D. Bahdanau, "Picard: Parsing incrementally for constrained auto-regressive decoding from language models," in Empirical Methods in Natural Language Processing (EMNLP), 2021.
  135. B. Hui, R. Geng, L. Wang, B. Qin, B. Li, J. Sun, and Y. Li, "Injecting syntax to question-schema interaction graph encoder for text-to-sql parsers," in Findings of Association for Computational Linguistics (ACL), 2022.
  136. Y. Song, R. C.-W. Wong, and X. Zhao, "Speech-to-sql: toward speech-driven sql query generation from natural language question," Journal on Very Large Data Bases (VLDBJ), 2024.
  137. B. Yan, K. Li, M. Xu, Y. Dong, Y. Zhang, Z. Ren, and X. Cheng, "On protecting the data privacy of large language models (llms): A survey," arXiv preprint arXiv:2403.05156, 2024. · arXiv
  138. Y. Zhang, Y. Li, L. Cui, D. Cai, L. Liu, T. Fu, X. Huang, E. Zhao, Y. Zhang, Y. Chen et al., "Siren's song in the ai ocean: a survey on hallucination in large language models," arXiv preprint arXiv:2309.01219, 2023. · arXiv
  139. C. Singh, J. P. Inala, M. Galley, R. Caruana, and J. Gao, "Rethinking interpretability in the era of large language models," arXiv preprint arXiv:2402.01761, 2024. · arXiv
  140. D. Dai, L. Dong, Y. Hao, Z. Sui, B. Chang, and F. Wei, "Knowledge neurons in pretrained transformers," in Association for Computational Linguistics (ACL), 2022.
  141. N. Zhang, Y. Yao, B. Tian, P. Wang, S. Deng, M. Wang, Z. Xi, S. Mao, J. Zhang, Y. Ni et al., "A comprehensive study of knowledge editing for large language models," arXiv preprint arXiv:2401.01286, 2024. · arXiv
  142. K. Meng, A. S. Sharma, A. J. Andonian, Y. Belinkov, and D. Bau, "Mass-editing memory in a transformer," in International Conference on Learning Representations (ICLR), 2023.
  143. K. Meng, D. Bau, A. Andonian, and Y. Belinkov, "Lo- cating and editing factual associations in gpt," Advances in Neural Information Processing Systems (NeurIPS), 2022.
  144. C. Zheng, L. Li, Q. Dong, Y. Fan, Z. Wu, J. Xu, and B. Chang, "Can we edit factual knowledge by in-context learning?" in Empirical Methods in Natural Language Processing (EMNLP), 2023.
  145. J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou et al., "Chain-of-thought prompting elicits reasoning in large language models," in Advances in Neural Information Processing Systems (NeurIPS), 2022.
  146. H. Luo, Z. Tang, S. Peng, Y. Guo, W. Zhang, C. Ma, G. Dong, M. Song, W. Lin et al., "Chatkbqa: A generate- then-retrieve framework for knowledge base question answering with fine-tuned large language models," arXiv preprint arXiv:2310.08975, 2023. · arXiv
  147. Z. Li, S. Fan, Y. Gu, X. Li, Z. Duan, B. Dong, N. Liu, and J. Wang, "Flexkbqa: A flexible llm-powered framework for few-shot knowledge base question answering," in Conference on Artificial Intelligence (AAAI), 2024.
  148. G. Xiong, J. Bao, and W. Zhao, "Interactive-kbqa: Multi-turn interactions for knowledge base question answering with large language models," arXiv preprint arXiv:2402.15131, 2024. · arXiv

Cited by

Sorted by citations

Metadata

Authors
Hong, Z., Yuan, Z., Zhang, Q., Chen, H., Dong, J., Huang, F., & Huang, X.
Affiliations
  • Hong Kong Polytechnic University (HK)
  • City University of Macau (MO)
  • Jinan University (CN)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
Volume
37
Issue
12
Pages
7328-7345
ISSN
1041-4347 (ISSN-L)
DOI
10.48550/arxiv.2406.08426
arXiv
2406.08426
OpenAlex
W7081978817
Cited by
100
References
0
Citations by year
2026: 65 · 2025: 30 · 2024: 5
FWCI
38.10
Type
journal article
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • IEEE Transactions on Knowledge and Data Engineering (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Natural Science Foundation of China (62502008)
  • Hong Kong Polytechnic University
  • Fundo para o Desenvolvimento das Ciências e da Tecnologia
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Information Systems › Advanced Database Systems and Queries
Keywords
text-to-SQL, large language models, natural language interfaces to databases, question comprehension, pre-trained language models, database query generation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review