Logical and Physical Optimizations for SQL Query Execution over Large Language Models
Open accessAbstract
Interacting with Large Language Models (LLMs) via declarative queries is increasingly popular for tasks like question answering and data extraction, thanks to their ability to process vast unstructured data. However, LLMs often struggle with answering complex factual questions, exhibiting low precision and recall in the returned data. This challenge highlights that executing queries on LLMs remains a largely unexplored domain, where traditional data processing assumptions often fall short. Conventional query optimization, typically cost-driven, overlooks LLM-specific quality challenges such as contextual understanding. Just as new physical operators are designed to address the unique characteristics of LLMs, optimization must consider these quality challenges. Our results highlight that adhering strictly to conventional query optimization principles fails to generate the best plans in terms of result quality. To tackle this challenge, we present a novel approach to enhance SQL results by applying query optimization techniques specifically adapted for LLMs. We introduce a database system, GALOIS, that sits between the query and the LLM, effectively using the latter as a storage layer. We design alternative physical operators tailored for LLM-based query execution and adapt traditional optimization strategies to this novel context. For example, while pushing down operators in the query plan reduces execution cost (fewer calls to the model), it might complicate the call to the LLM and deteriorate result quality. Additionally, these models lack a traditional catalog for optimization, leading us to develop methods to dynamically gather such metadata during query execution. Our solution is compatible with any LLM and balances the trade-off between query result quality and execution cost. Experiments show up to 144% quality improvement over questions in Natural Language and 29% over direct SQL execution, highlighting the advantages of integrating database solutions with LLMs.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 16,037 · see citing works
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · see citing works
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101 · see citing works
Language Models as Knowledge Bases?
Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1250 · Cited by 1,834 · see citing works
Lost in the Middle: How Language Models Use Long Contexts
Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00638 · Cited by 1,612 · cited 1× · see citing works
“While research in tabular language models indicates that such a scenario is not yet feasible, primarily due to context size limitations [4] and its issues with long inputs [39], recent advancements are overcoming this constraint [16, 28, 41].”
Yu, T., Zhang, R., Yang, K., Yasunaga, M., Wang, D., Li, Z., Ma, J., Li, I., Yao, Q., Roman, S., et al. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1425 · Cited by 1,034 · see citing works
TruthfulQA: Measuring How Models Mimic Human Falsehoods
Lin, S., Hilton, J., & Evans, O. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.229 · Cited by 904 · cited 1× · see citing works
“Techniques proposed for crowd databases, such as redundancy and validation questions, are also studied to improve the quality of LLM’s responses and could be adopted in our setting [35].”
A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
Fan, W., Ding, Y., Ning, L., Wang, S., Li, H., Yin, D., Chua, T.-S., & Li, Q. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671470 · Cited by 827 · see citing works
Franklin, M. J. M., Kossmann, D., Kraska, T., Ramesh, S., & Xin, R. (2011) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/1989323.1989331 · Cited by 623 · see citing works
Computation of normalized edit distance and applications
Marzal, A., & Vidal, E. (1993) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.232078 · Cited by 346 · see citing works
A survey on deep learning approaches for text-to-SQL
Katsogiannis-Meimarakis, G., & Koutrika, G. (2023) · The VLDB Journal · doi:10.1007/s00778-022-00776-8 · Cited by 203 · see citing works
LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models
Jiang, H., Wu, Q., Lin, C.-Y., Yang, Y., & Qiu, L. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.825 · Cited by 168 · see citing works
Peng, B., Galley, M., He, P., Cheng, H., Xie, Y., Hu, Y., Huang, Q., Liden, L., Yu, Z., Chen, W., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.12813 · Cited by 156 · see citing works
CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving
Liu, Y., Li, H., Cheng, Y., Ray, S., Huang, Y., Zhang, Q., Du, K., Yao, J., Lu, S., Ananthanarayanan, G., et al. (2024) · Proceedings of the ACM SIGCOMM 2024 Conference · doi:10.1145/3651890.3672274 · Cited by 118 · see citing works
Transformers for Tabular Data Representation: A Survey of Models and Applications
Badaro, G., Saeed, M., & Papotti, P. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00544 · Cited by 96 · see citing works
Self-citationLanguage Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes
Arora, S., Yang, B., Eyuboglu, S., Narayan, A., Hojel, A., Trummer, I., & Ré, C. (2023) · Proceedings of the VLDB Endowment · doi:10.14778/3626292.3626294 · Cited by 93 · see citing works
A Survey of Confidence Estimation and Calibration in Large Language Models . -
Geng, J., Cai, F., Wang, Y., Koeppl, H., Nakov, P., & Gurevych, I. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.366 · Cited by 88 · see citing works
The Four Generations of Entity Resolution
Papadakis, G., Ioannou, E., Thanos, E., & Palpanas, T. (2021) · Synthesis lectures on data management · doi:10.1007/978-3-031-01878-7 · Cited by 78 · see citing works
Cultural Bias in Large Language Models: A Comprehensive Analysis and Mitigation Strategies
Liu, Z. (2025) · Journal of Transcultural Communication · doi:10.1515/jtc-2023-0019 · Cited by 77 · see citing works
Sampling-Based Query Re-Optimization
Wu, W., Naughton, J. F., & Singh, H. (2016) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/2882903.2882914 · Cited by 76 · see citing works
Zhang, C., Ré, C., Cafarella, M., De, C., Ratner, A., Shin, J., Wang, F., & Wu, S. (2017) · Communications of the ACM · doi:10.1145/3060586 · Cited by 59 · see citing works
RHEEM: enabling cross-platform data processing
Agrawal, D., Chawla, S., Contreras-Rojas, B., Elmagarmid, A. K., Idris, Y., Kaoudi, Z., Kruse, S., Lucas, J., Mansour, E. A. H., Ouzzani, M., et al. (2018) · Proceedings of the VLDB Endowment · doi:10.14778/3236187.3236195 · Cited by 50 · see citing works
Self-citationThe Seattle report on database research
Abadi, D. J., Ailamaki, A., Andersen, D., Bailis, P. D., Bałazińska, M., Bernstein, P. A., Boncz, P. A., Chaudhuri, S., Cheung, A., Doan, A., et al. (2022) · Communications of the ACM · doi:10.1145/3524284 · Cited by 43 · see citing works
From natural language processing to neural databases
Thorne, J. P., Yazdani, M., Saeidi, M., Silvestri, F., Riedel, S., & Halevy, A. Y. (2021) · Proceedings of the VLDB Endowment · doi:10.14778/3447689.3447706 · Cited by 40 · see citing works
Text-to-Table: A New Way of Information Extraction
Wu, X., Zhang, J., & Li, H. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.180 · Cited by 40 · see citing works
Optimization of Complex Dataflows with User-Defined Functions
Rheinländer, A., Leser, U., & Graefe, G. (2017) · ACM Computing Surveys · doi:10.1145/3078752 · Cited by 40 · see citing works
Natural Language Interfaces to Data
Quamar, A., Efthymiou, V., Lei, C., & Lei, C. (2022) · Foundations and Trends in Databases · doi:10.1561/1900000078 · Cited by 38 · see citing works
Simonini, G., Zecchini, L., Bergamaschi, S., & Naumann, F. (2022) · Proceedings of the VLDB Endowment · doi:10.14778/3523210.3523226 · Cited by 29 · see citing works
Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph
Vashurin, R., Fadeeva, E., Vazhentsev, A., Rvanova, L., Vasilev, D., Tsvigun, A., Petrakov, S., Xing, R., Sadallah, A., Grishchenkov, K., et al. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00737 · Cited by 21 · see citing works
Demonstrating CAESURA: Language Models as Multi-Modal Query Planners
Urban, M., & Binnig, C. (2024) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3626246.3654732 · Cited by 19 · see citing works
Zero-shot Topical Text Classification with LLMs - an Experimental Study
Gretz, S., Halfon, A., Shnayderman, I., Ronen, O., Spector, A., Dankin, L., Katsis, Y., Arviv, O., Katz, Y., Slonim, N., et al. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.647 · Cited by 17 · see citing works
A Method for Optimizing Opaque Filter Queries
He, W., Anderson, M. R., Strome, M. B., & Cafarella, M. (2020) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3318464.3389766 · Cited by 15 · see citing works
Data Ambiguity Profiling for the Generation of Training Examples
Veltri, E., Badaro, G., Saeed, M., & Papotti, P. (2023) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde55515.2023.00041 · Cited by 13 · see citing works
Self-citationThalamusDB: Approximate Query Processing on Multi-Modal Data
Jo, S., & Trummer, I. (2024) · Proceedings of the ACM on Management of Data · doi:10.1145/3654989 · Cited by 13 · see citing works
Unsupervised Quality Estimation for Neural Machine Translation
Fomicheva, M., Sun, S., Yankovskaya, L., Blain, F., Guzmán, F., Fishel, M., Aletras, N., Chaudhary, V., & Specia, L. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00330 · Cited by 10 · see citing works
Semantic Operators: A Declarative Model for Rich, AI-based Data Processing
Patel, L., Jha, S., Pan, M., Gupta, H., Asawa, P., Guestrin, C., & Zaharia, M. (2024) · arXiv · doi:10.48550/arxiv.2407.11418 · Cited by 9 · see citing works
Optimizing LLM Queries in Relational Data Analytics Workloads
Liu, S., Biswal, A., Kamsetty, A., Cheng, A., Schroeder, L. G., Patel, L., Cao, S., Mo, X., Stoica, I., Gonzalez, J. E., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.05821 · Cited by 8 · see citing works
DB-BERT: making database tuning tools “read” the manual
Trummer, I. (2023) · The VLDB Journal · doi:10.1007/s00778-023-00831-y · Cited by 5 · see citing works
FINCH: Prompt-guided Key-Value Cache Compression for Large Language Models
Corallo, G., & Papotti, P. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00716 · Cited by 5 · see citing works
Self-citationSimilarity Measures For Incomplete Database Instances
Glavic, B., Mecca, G., J., M. R., Papotti, P., Santoro, D., & Veltri, E. (2024) · CINECA IRIS Institutional Research Information System (University of Basilicata) · doi:10.48786/edbt.2024.40 · Cited by 3 · see citing works
Self-citationCacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion
Yao, J., Li, H., Liu, Y., Ray, S., Cheng, Y., Zhang, Q., Du, K., Lu, S., & Jiang, J. (2024) · arXiv · doi:10.48550/arxiv.2405.16444 · Cited by 3 · see citing works
Querying Large Language Models with SQL
Saeed, M., Cao, N. D., & Papotti, P. (2024) · OpenProceedings · doi:10.48786/edbt.2024.32 · Cited by 3 · see citing works
Self-citationDetective Gadget: Generic Iterative Entity Resolution Over Dirty Data
Buoncristiano, M., Mecca, G., Santoro, D., & Veltri, E. (2022) · Data · doi:10.2139/ssrn.4272148 · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 30
- Asim Biswal, Liana Patel, Siddarth Jha, Amog Kamsetty, Shu Liu, Joseph E. Gonzalez, Carlos Guestrin, and Matei Zaharia. 2024. Text2SQL is Not Enough: Unifying AI and Databases with TAG. arXiv:2408.14717 [cs.DB] https: //arxiv.org/abs/2408.14717 · arXiv
- Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In NeurIPS. https://proceedings. neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html
- Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. 2023. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712 (2023). · arXiv
- Lihu Chen, Alexandre Perez-Lebel, Fabian M. Suchanek, and Gaël Varoquaux. 2024. Reconfidencing LLMs from the Grouping Loss Perspective. arXiv:2402.04957 [cs.CL] https://arxiv.org/abs/2402.04957 · arXiv
- Lihu Chen, Alexandre Perez-Lebel, Fabian M Suchanek, and Gaël Varoquaux. 2024. Reconfidencing LLMs from the Grouping Loss Perspective. arXiv preprint arXiv:2402.04957 (2024). · arXiv
- shiqi chen, Yiran Zhao, Jinghan Zhang, I-Chun Chern, Siyang Gao, Pengfei Liu, and Junxian He. 2023. FELM: Benchmarking Factuality Evaluation of Large Language Models. In Advances in Neural Information Processing Sys- tems, Vol.
- Curran Associates, Inc., 44502-44523. https://proceedings.neurips.cc/paper_files/paper/2023/file/ 8b8a7960d343e023a6a0afe37eee6022-Paper-Datasets_and_Benchmarks.pdf
- Zui Chen, Lei Cao, Sam Madden, Tim Kraska, Zeyuan Shang, Ju Fan, Nan Tang, Zihui Gu, Chunwei Liu, and Michael Cafarella. 2024. SEED: Domain-Specific Data Curation With Large Language Models. arXiv:2310.00749 [cs.DB] https://arxiv.org/abs/2310.00749 · arXiv
- Zui Chen, Zihui Gu, Lei Cao, Ju Fan, Samuel Madden, and Nan Tang. 2023. Symphony: Towards Natural Language Query Answering over Multi-modal Data Lakes. In 13th Conference on Innovative Data Systems Research, CIDR 2023, Amsterdam, The Netherlands, January 8-11, 2023. www.cidrdb.org. https://www.cidrdb.org/cidr2023/papers/p51-chen.pdf
- Laura Chiticariu, Marina Danilevsky, Yunyao Li, Frederick Reiss, and Huaiyu Zhu. 2018. SystemT: Declarative Text Understanding for Enterprise. In NAACL. Association for Computational Linguistics, 76-83. doi:10.18653/v1/N18-3010 · DOI
- Xiang Deng, Huan Sun, Alyssa Lees, You Wu, and Cong Yu. 2020. TURL: Table Understanding through Representation Learning. Proc. VLDB Endow. 14, 3 (2020), 307-319. doi:10.5555/3430915.3442430 · DOI
- Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024). · arXiv
- Avrilia Floratou, Fotis Psallidas, Fuheng Zhao, Shaleen Deep, Gunther Hagleither, Wangda Tan, Joyce Cahoon, Rana Alotaibi, Jordan Henkel, Abhik Singla, Alex Van Grootel, Brandon Chow, Kai Deng, Katherine Lin, Marcos Campos, K. Venkatesh Emani, Vivek Pandit, Victor Shnayder, Wenjing Wang, and Carlo Curino. 2024. NL2SQL is a solved problem... Not!. In 14th Conference on Innovative Data Systems Research, CIDR 2024, Chaminade, HI, USA, January 14-17, 2024. www.cidrdb.org. https://www.cidrdb.org/cidr2024/papers/p74-floratou.pdf
- Xingyu Ji, Aditya Parameswaran, and Madelon Hulsebos. [n. d.]. TARGET: Benchmarking Table Retrieval for Generative Tasks. In NeurIPS 2024 Third Table Representation Learning Workshop.
- Moe Kayali, Anton Lykov, Ilias Fountalis, Nikolaos Vasiloglou, Dan Olteanu, and Dan Suciu. 2024. CHORUS: Foundation Models for Unified Data Discovery and Exploration. Proc. VLDB Endow. 17, 8 (2024), 2104-2114. https://www.vldb. org/pvldb/vol17/p2104-kayali.pdf
- Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, Xuanhe Zhou, Chenhao Ma, Guoliang Li, Kevin Chen-Chuan Chang, Fei Huang, Reynold Cheng, and Yongbin Li. 2023. Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 -16, 2023. http://papers.nips.cc/paper_files/paper/2023/hash/ 83fc8fab1710363050bbd1d4b8cc0021-Abstract-Datasets_and_Benchmarks.html
- Xianming Li and Jing Li. 2023. AnglE-optimized Text Embeddings. arXiv preprint arXiv:2309.12871 (2023). · arXiv
- Yiming Lin, Madelon Hulsebos, Ruiying Ma, Shreya Shankar, Sepanta Zeigham, Aditya G. Parameswaran, and Eugene Wu. 2024. Towards Accurate and Efficient Document Analytics with Large Language Models. arXiv:2405.04674 [cs.DB] https://arxiv.org/abs/2405.04674 · arXiv
- Chunwei Liu, Matthew Russo, Michael Cafarella, Lei Cao, Peter Baille Chen, Zui Chen, Michael Franklin, Tim Kraska, Samuel Madden, and Gerardo Vitagliano. 2024. A Declarative System for Optimizing AI Workloads. arXiv:2405.14696 [cs.CL] https://arxiv.org/abs/2405.14696 · arXiv
- Jie Liu and Barzan Mozafari. 2024. Query Rewriting via Large Language Models. arXiv:2403.09060 [cs.DB] https: //arxiv.org/abs/2403.09060 · arXiv
- OpenAI. 2024. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL] https://arxiv.org/abs/2303.08774 · arXiv
- Simone Papicchio, Paolo Papotti, and Luca Cagliero. 2023. QATCH: Benchmarking Table Representation Learning Models on Your Data. In NeurIPS (Datasets and Benchmarks).
- Joon Sung Park, Carolyn Q Zou, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Robb Willer, Percy Liang, and Michael S Bernstein. 2024. Generative agent simulations of 1,000 people. arXiv preprint arXiv:2411.10109 (2024). · arXiv
- Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language Models as Knowledge Bases?. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP),
- Shreya Shankar, Tristan Chambers, Tarak Shah, Aditya G. Parameswaran, and Eugene Wu. 2024. DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processing. arXiv:2410.12189 [cs.DB] https://arxiv.org/abs/ 2410.12189 · arXiv
- Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314 (2024). · arXiv
- Kai Sun, Yifan Ethan Xu, Hanwen Zha, Yue Liu, and Xin Luna Dong. 2024. Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs? arXiv:2308.10168 [cs.CL] https: //arxiv.org/abs/2308.10168 · arXiv
- Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. 2024. Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net.
- Fuheng Zhao, Divyakant Agrawal, and Amr El Abbadi. 2024. Hybrid Querying Over Relational Databases and Large Language Models. arXiv:2408.00884 [cs.DB] https://arxiv.org/abs/2408.00884 · arXiv
- Fuheng Zhao, Lawrence Lim, Ishtiyaque Ahmad, Divyakant Agrawal, and Amr El Abbadi. 2024. LLM-SQL-Solver: Can LLMs Determine SQL Equivalence? arXiv:2312.10321 [cs.DB] https://arxiv.org/abs/2312.10321 · arXiv
Cited by
Sorted by citations
Beyond Quacking: Deep Integration of Language Models and RAG into DuckDB
Dorbani, A., Yasser, S., Lin, J., & Mhedhbi, A. (2025) · Proceedings of the VLDB Endowment · doi:10.14778/3750601.3750685 · Cited by 5 · see citing works
Skyline Retrieval meets Set-Cover Chunk Merging: A Cost-Effective RAG-Sketch for Long-Context LLM QA
Zhu, X., Li, H., Zhang, Y., & Chen, L. (2026) · Proceedings of the ACM on Management of Data · doi:10.1145/3802111 · Cited by 2 · see citing works
Relational Deep Dive: Error-Aware Queries Over Unstructured Data
Chao, D., Chen, K., Guan, N., & Koudas, N. (2026) · Proceedings of the VLDB Endowment · doi:10.14778/3811243.3811256 · Cited by 1 · see citing works
Comparative analysis of query optimization techniques in modern relational database systems
Solohub, V., & Pashkevych, V. (2026) · Journal of Computer Sciences Institute · doi:10.35784/jcsi.8944
Indexing and Search Algorithms for Large Language Models on the Edge
Christou, S., Krasovitskiy, K., Konstantinidis, A., & Demetrios Zeinalipour-yazti, A. (2026) · IEEE International Conference on Mobile Data Management (MDM) · doi:10.1109/mdm71479.2026.00015
ELENA: AN Explainability-Aided Online Query Optimization Framework
Dong, Y., Yao, Y., Wu, Y., Chen, L., & Zhu, R. (2026) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde65706.2026.00213
Large language models as digital libraries: a multi-benchmark and multi-model study
Cazzaro, M., & Silvello, G. (2026) · International Journal on Digital Libraries · doi:10.1007/s00799-026-00448-w · Cited by 1 · see citing works
Who Should Own the Loop? Harness Decomposition for Small-Model Repository Repair
Dente, F., Satriani, D., Santoro, D., Veltri, E., & Papotti, P. (2026) · International Workshop on Agentic AI for Next-Generation Software Development · doi:10.1145/3843282.3844442
Self-citation
- Authors
- Satriani, D., Veltri, E., Santoro, D., Rosato, S., Varriale, S., & Papotti, P.
- Affiliations
- University of Basilicata (IT)
- EURECOM (FR)
- Venue
- Proceedings of the ACM on Management of Data
- Published
- 2025-06-17
- Volume
- 3
- Issue
- 3
- Pages
- 1-28
- ISSN
- 2836-6573 (ISSN-L)
- OpenAlex
- W4411403196
- Cited by
- 8
- References
- 41
- Citations by year
- 2026: 7 · 2025: 1
- FWCI
- 3.38
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Proceedings of the ACM on Management of Data (publishedVersion)
- CINECA IRIS Institutional Research Information System (University of Basilicata) (submittedVersion, cc-by-nd)
- HAL (submittedVersion, cc-by)
- Funders
- Tech4You (C43C22000400006)
- Horizon 2020 (101189908)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Advanced Database Systems and Queries
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- large language models, SQL query execution, query optimization, physical operators, Galois, quality of results, execution cost, declarative queries, question answering, data extraction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
