Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study

Sui, Yuan; Zhou, Mengyu; Zhou, Mingjie; Han, Shi; Zhang, Dongmei

doi:10.1145/3616855.3635752

Cited by 156Topic ModelingMultimodal Machine Learning ApplicationsNatural Language Processing Techniques

Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study

Open access

Sui, Yuan, Zhou, Mengyu, Zhou, Mingjie, Han, Shi, Zhang, Dongmei·2024·Proceedings of the 17th ACM International Conference on Web Search and Data Mining

DOI 10.1145/3616855.3635752

Open this paper in CoChat — free account Open PDF

Abstract

Large language models (LLMs) are becoming attractive as few-shot reasoners to solve Natural Language (NL)-related tasks. However, there is still much to learn about how well LLMs understand structured data, such as tables. Although tables can be used as input to LLMs with serialization, there is a lack of comprehensive studies that examine whether LLMs can truly comprehend such data. In this paper, we try to understand this by designing a benchmark to evaluate the structural understanding capabilities (SUC) of LLMs. The benchmark we create includes seven tasks, each with its own unique challenges, \eg, cell lookup, row retrieval, and size detection. We perform a series of evaluations on GPT-3.5 and GPT-4. We find that performance varied depending on several input choices, including table input format, content order, role prompting, and partition marks. Drawing from the insights gained through the benchmark evaluations, we proposeself-augmentation for effective structural prompting, such as critical value / range identification using internal knowledge of LLMs. When combined with carefully chosen input choices, these structural prompting methods lead to promising improvements in LLM performance on a variety of tabular tasks, \eg, TabFact(\uparrow2.31%), HybridQA(\uparrow2.13%), SQA(\uparrow2.72%), Feverous(\uparrow0.84%), and ToTTo(\uparrow5.68%). We believe that our open-source (please find code and data at https://github.com/microsoft/TableProvider) benchmark and proposed prompting methods can serve as a simple yet generic selection for future research.

Integrity signals

Registry

Crossref · 10.1145/3616855.3635752

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

LLMs often struggle with constraints involving numbers, negations, or long instructions with large number of constraints.
“A qualitative examination of responses with unsatisfied constraints suggests that LLMs often struggle with constraints involving numbers, negations, or long instructions with large number of constraints , which is consistent with findings from previous works (Sui et al., 2024; García-Ferrero et al., 2023; Truong et al., 2023; Jiang et al., 2024).”
From LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraints (2024) · cites this paper as Sui et al., 2024

Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,759 · see citing works
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881 · see citing works
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263 · see citing works
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · see citing works
Large language models are zero-shot reasoners
Kojima, T., Gu, S. (., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1613 · Cited by 2,013 · see citing works
Scaling Instruction-Finetuned Language Models
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.11416 · Cited by 1,441 · see citing works
Evaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436 · see citing works
Efficient transformers: A survey
Tay, Y., Dehghani, M., Bahri, D., & Metzler, D. (2023) · ACM Computing Surveys · doi:10.1145/3530811 · Cited by 1,267 · see citing works
Training Verifiers to Solve Math Word Problems
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.14168 · Cited by 575 · see citing works
TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data
Yin, P., Neubig, G., Yih, W.-T., & Riedel, S. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.745 · Cited by 487 · see citing works
Planning with Large Language Models for Code GenerationPreprint
Zhang, S., Chen, Z., Shen, Y., Ding, M., Tenenbaum, J., & Gan, C. (2023) · International Conference on Learning Representations · doi:10.48550/arxiv.2303.05510 · Cited by 277 · see citing works
ToTTo: A Controlled Table-To-Text Generation Dataset
Parikh, A., Wang, X., Gehrmann, S., Faruqui, M., Dhingra, B., Yang, D., & Das, D. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.89 · Cited by 252 · see citing works
Events and their Localisation are Relative to a LabPreprint
Vilasini, V., Chen, L.-Q., Ye, L., & Renner, R. (2025) · Journal of Physics A Mathematical and Theoretical · doi:10.1088/1751-8121/ae2076 · Cited by 245 · see citing works
TabFact: A Large-scale Dataset for Table-based Fact Verification
Chen, W., Wang, H., Chen, J., Zhang, Y., Wang, H., Li, S., Zhou, X., & Wang, W. Y. (2019) · arXiv · doi:10.48550/arxiv.1909.02164 · Cited by 239 · see citing works
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Rae, J. W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R., Young, S., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.11446 · Cited by 235 · see citing works
Search-based Neural Structured Learning for Sequential Question Answering
Iyyer, M., Yih, W.-T., & Chang, M. (2017) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/p17-1167 · Cited by 219 · see citing works
HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data
Chen, W., Zha, H., Chen, Z., Xiong, W., Wang, H., & Wang, W. Y. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.91 · Cited by 216 · see citing works
TAPEX: Table Pre-training via Learning a Neural SQL ExecutorPreprint
Liu, Q., Chen, B., Guo, J., Ziyadi, M., Lin, Z., Chen, W., & Lou, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.07653 · Cited by 145 · see citing works
TableFormer: Table Structure Understanding with Transformers
Nassar, A., Livathinos, N., Lysak, M., & Staar, P. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00457 · Cited by 129 · see citing works
TUTA
Wang, Z., Dong, H., Jia, R., Li, J., Fu, Z., Han, S., & Zhang, D. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467434 · Cited by 127 · see citing works
Self-citation
Large Language Models are few(1)-shot Table Reasoners
Chen, W. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.83 · Cited by 116 · see citing works
TableGPT: Few-shot Table-to-Text Generation with Table Structure Reconstruction and Content Matching
Gong, H., Sun, Y., Feng, X., Qin, B., Bi, W., Liu, X., & Liu, T. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.179 · Cited by 70 · see citing works
GitTables: A Large-Scale Corpus of Relational Tables
Hulsebos, M., Demiralp, Ç., & Groth, P. (2023) · Proceedings of the ACM on Management of Data · doi:10.1145/3588710 · Cited by 58 · see citing works
Table Pre-training: A Survey on Model Architectures, Pre-training Objectives, and Downstream Tasks
Dong, H., Cheng, Z., He, X., Zhou, M., Zhou, A., Zhou, F., Liu, A., Han, S., & Zhang, D. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/761 · Cited by 57 · see citing works
Self-citation
FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured informationPreprint
Aly, R., Guo, Z., Schlichtkrull, M., Thorne, J., Vlachos, A., Christodoulopoulos, C., Cocarascu, O., & Mittal, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05707 · Cited by 57 · see citing works
Generated Knowledge Prompting for Commonsense ReasoningPreprint
Liu, J., Liu, A., Lu, X., Welleck, S., West, P., Bras, R. J. L., Choi, Y., & Hajishirzi, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.08387 · Cited by 48 · see citing works
StruBERT: Structure-aware BERT for Table Search and Matching
Trabelsi, M., Chen, Z., Zhang, S., Davison, B. D., & Heflin, J. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3511972 · Cited by 37 · see citing works
Let’s Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs
Aggarwal, P., Madaan, A., Yang, Y., & Mausam, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.761 · Cited by 29 · see citing works
Web Table Extraction, Retrieval and Augmentation: A SurveyPreprint
Zhang, S., & Balog, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.00207 · Cited by 25 · see citing works
MATE: Multi-view Attention for Table Transformer EfficiencyPreprint
Eisenschlos, J. M., Gor, M., Müller, T., & Cohen, W. W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.04312 · Cited by 14 · see citing works
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language ModelsPreprint
Xie, T., Wu, C. H., Shi, P. J., Zhong, R., Scholak, T., Yasunaga, M., Wu, C.-S., Zhong, M., Yin, P., Wang, S. I., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2201.05966 · Cited by 11 · see citing works
Simulating Users in Interactive Web Table Retrieval
Engelmann, B., Breuer, T., & Schaer, P. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3615187 · Cited by 6 · see citing works
FormLM: Recommending Creation Ideas for Online Forms by Modelling Semantic and Structural InformationPreprint
Shao, Y., Zhou, M., Zhong, Y., Wu, T., Han, H., Shi, H., Huang, G., & Zhang, D. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.05284 · Cited by 1 · see citing works
Self-citation

Flagged first, then by citations

Sorted by citations

Optimization of hepatological clinical guidelines interpretation by large language models: a retrieval augmented generation-based framework
Kresevic, S., Giuffrè, M., Ajcevic, M., Accardo, A., Crocè, L. S., & Shung, D. L. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01091-y · Cited by 207 · see citing works
From text to insight: large language models for chemical data extraction
Schilling-Wilhelmi, M., Ríos-García, M., Shabih, S., Gil, M. V., Miret, S., Koch, C. T., Marquez, J. A., & Jablonka, K. M. (2024) · Chemical Society Reviews · doi:10.1039/d4cs00913d · Cited by 155 · see citing works
LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasters
Chang, C., Wang, W., Peng, W.-C., & Chen, T.-F. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3719207 · Cited by 146 · see citing works
Generative AI for self-adaptive systems: State of the art and research roadmap
Li, J., Zhang, M., Li, N., Weyns, D., Jin, Z., & Tei, K. (2024) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3686803 · Cited by 77 · see citing works
Development of Prompt Templates for Large Language Model–Driven Screening in Systematic Reviews
Cao, C., Sang, J., Arora, R., Chen, D., Kloosterman, R., Cecere, M., Gorla, J., Saleh, R., Drennan, I., Teja, B., et al. (2025) · Annals of Internal Medicine · doi:10.7326/annals-24-02189 · Cited by 67 · see citing works
Large language model for table processing: a survey
Lu, W., Zhang, J., Fan, J., Fu, Z., Chen, Y., & Du, X. (2025) · Frontiers of Computer Science · doi:10.1007/s11704-024-40763-6 · Cited by 77 · see citing works
VisEval: A Benchmark for Data Visualization in the Era of Large Language Models
Chen, N., Zhang, Y., Xu, J., Ren, K., & Yang, Y. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3456320 · Cited by 53 · see citing works
An IoT intrusion detection framework based on feature selection and large language models fine-tuning
Ma, H., Zhang, W., Zhang, D., & Chen, B. (2025) · Scientific Reports · doi:10.1038/s41598-025-08905-3 · Cited by 32 · see citing works
Self-citation
TFGIN: Tight-Fitting Graph Inference Network for Table-based Fact Verification
Wu, L., Wang, K., Nie, K., Guo, S., Gao, C., Wang, Z., & Li, S. (2025) · ACM Transactions on Information Systems · doi:10.1145/3734520 · Cited by 30 · see citing works
A survey of table reasoning with large language models
Zhang, X., Wang, D., Dou, L., Zhu, Q., & Che, W. (2025) · Frontiers of Computer Science · doi:10.1007/s11704-024-40330-z · Cited by 35 · see citing works
Large Language Models for Structured and Semi-Structured Data, Recommender Systems and Knowledge Base Engineering: A Survey of Recent Techniques and Architectures
Smajić, A., Karlović, R., Bobanović, M., & Lorencin, I. (2025) · Electronics · doi:10.3390/electronics14153153 · Cited by 21 · see citing works
ShapefileGPT: a multi-agent large language model framework for automated shapefile processing
Lin, Q., Hu, R., Li, H., Wu, S., Li, Y., Fang, K., Feng, H., Du, Z., & Xu, L. (2025) · International Journal of Digital Earth · doi:10.1080/17538947.2025.2577884 · Cited by 21 · see citing works
Research on Large-scale Structured and Unstructured Data Processing based on Large Language Model
Li, B., Jiang, G., Li, N., & Song, C. (2024) · International Conference on Machine Learning, Pattern Recognition and Automation Engineering (MLPRAE) · doi:10.1145/3696687.3696707 · Cited by 19 · see citing works
Untitled work
(2025) · doi:10.1145/3833415 · Cited by 18 · see citing works
Generating textual explanations for scheduling systems leveraging the reasoning capabilities of large language models
Powell, C., & Riccardi, A. (2025) · Journal of Intelligent Information Systems · doi:10.1007/s10844-025-00940-w · Cited by 17 · see citing works
Assessing the performance of generative artificial intelligence in retrieving information against manually curated genetic and genomic data
Poretsky, E., Blake, V. C., Andorf, C. M., & Sen, T. Z. (2025) · Database · doi:10.1093/database/baaf011 · Cited by 16 · see citing works
A Review of Prompt Engineering Techniques for Large Language Models
Naser, M. Z. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2607553 · Cited by 17 · see citing works
Priv-IQ: A Benchmark and Comparative Evaluation of Large Multimodal Models on Privacy Competencies
Shahriar, S., & Dara, R. A. (2025) · AI · doi:10.3390/ai6020029 · Cited by 18 · see citing works
Can LLMs Answer Investment Banking Questions? Using Domain-Tuned Functions to Improve LLM Performance on Knowledge-Intensive Analytical Tasks
Harvel, N., Haiek, F. B., Ankolekar, A., & Brunner, D. J. (2024) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v3i1.31191 · Cited by 14 · see citing works
Benchmarking TabLM: Evaluating the Performance of Language Models Against Traditional Machine Learning in Structured Data Tasks
Shakir, W. A. (2024) · International Conference on Emerging Technologies for Dependable Internet of Things (ICETI) · doi:10.1109/iceti63946.2024.10777155 · Cited by 13 · see citing works
GIDCL: A Graph-Enhanced Interpretable Data Cleaning Framework with Large Language Models
Yan, M., Wang, Y., Wang, Y., Miao, X., & Li, J. (2024) · Proceedings of the ACM on Management of Data · doi:10.1145/3698811 · Cited by 12 · see citing works
Generating Realistic Tabular Data with Large Language Models
Nguyen, D., Gupta, S., Do, K., Nguyen, T., & Venkatesh, S. (2024) · Proceedings · doi:10.1109/icdm59182.2024.00040 · Cited by 15 · see citing works
GLOSS: Group of LLMs for Open-ended Sensemaking of Passive Sensing Data for Health and Wellbeing
Choube, A., Le, H., Li, J., Ji, K., Swain, V. D., & Mishra, V. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3749474 · Cited by 15 · see citing works
AI in Investment Analysis: LLMs for Equity Stock Ratings
Papasotiriou, K., Sood, S., Reynolds, S., & Balch, T. R. (2024) · ACM International Conference on AI in Finance (ICAIF) · doi:10.1145/3677052.3698694 · Cited by 12 · see citing works
A review of network delay prediction and advances in large language models for air traffic
Sun, M., Tian, Y., Li, J., Wu, C., Peng, L., & Xu, S. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11400-w · Cited by 12 · see citing works
WorkloadGPT: A Large Language Model Approach to Real-Time Detection of Pilot Workload
Gao, Y., Yue, L., Sun, J., Shan, X., Liu, Y., & Wu, X. (2024) · Applied Sciences · doi:10.3390/app14188274 · Cited by 12 · see citing works
Benchmarking Large Language Models on Homework Assessment in Circuit Analysis
Chen, L., Qin, Z., Guo, Y., Rohde, J. A., & Zhang, Y. (2025) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-025-00501-w · Cited by 9 · see citing works
Can Large Language Models Be Query Optimizer for Relational Databases?
Tan, J., Zhao, K., Li, R., Yu, J. X., Piao, C., Cheng, H., Meng, H. M. L., Zhao, D., & Rong, Y. (2025) · Proceedings of the ACM on Management of Data · doi:10.1145/3769771 · Cited by 8 · see citing works
SMUTF: Schema Matching Using Generative Tags and Hybrid Features
Zhang, Y., Mei, D., Luo, H., Xu, C., & Tsai, R. T. (2025) · Information Systems · doi:10.1016/j.is.2025.102570 · Cited by 9 · see citing works
Ontology-conformal recognition of materials entities using language models
Potu, S. T., Murthy, R. N., Thomas, A., Mishra, L., Prange, N., & Durmaz, A. R. (2025) · Scientific Reports · doi:10.1038/s41598-025-03619-y · Cited by 8 · see citing works
Towards Scalable Schema Mapping using Large Language Models
Buss, C., Safari, M., Termehchy, A., Maier, D., & Lee, S. (2025) · doi:10.1145/3737412.3743490 · Cited by 6 · see citing works
Unveiling Challenges for LLMs in Enterprise Data Engineering
Bodensohn, J.-M., Brackmann, U., Vogel, L., Sanghi, A., & Binnig, C. (2025) · Proceedings of the VLDB Endowment · doi:10.14778/3773749.3773758 · Cited by 11 · see citing works
Data Has Entered the Chat: How Data Workers Conduct Exploratory Visual Analytic Conversations with GenAI Agents
Hong, M.-H., & Crisan, A. (2025) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3744750 · Cited by 6 · see citing works
Revisiting Business Process Analysis Through the Lens of Large Language Models: Prompting Experiments With BPMN Process Serializations
Dolha, D. N., Ghiran, A.-M., & Buchmann, R. A. (2025) · Neurosymbolic Artificial Intelligence · doi:10.1177/29498732251377510 · Cited by 5 · see citing works
ClinicRealm: Re-evaluating large language models with conventional machine learning for non-generative clinical prediction tasks
Zhu, Y., Gao, J.-Y., Wang, Z.-X., Liao, W., Zheng, X.-C., Liang, L., Bernabeu, M., Wang, Y.-S., Yu, L.-Q., Pan, C.-W., et al. (2024) · npj Digit. Medicine · doi:10.1038/s41746-026-02539-z · Cited by 16 · see citing works
LLM-empowered literature mining for material substitution studies in sustainable concrete
Duan, Y., Tian, Y., Ghosh, S., Venugopal, V., Chen, J., & Olivetti, E. (2025) · Resources Conservation and Recycling · doi:10.1016/j.resconrec.2025.108379 · Cited by 5 · see citing works
Broadening Access to Simulations for End-Users via Large Language Models: Challenges and Opportunities
Giabbanelli, P. J., Padilla, J. J., & Agrawal, A. (2024) · Winter Simulation Conference · doi:10.1109/wsc63780.2024.10838754 · Cited by 5 · see citing works
Harnessing Large Language Models for question answering over complex tables
Lyu, J., Ying, Z., Shi, N., Xue, J., Pan, Z., Ding, W., & Zhou, W. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111910 · Cited by 4 · see citing works
Large language models are zero-shot point-of-interest recommenders
Kim, J., Seo, Y., Kim, Y., Kang, J.-H., Shin, J., Trirat, P., & Lee, J.-G. (2025) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-025-01148-w · Cited by 4 · see citing works
Agentic LLM Framework for Generating Spatial Intelligence to Support Decision-Making in Smart Cities
Xu, Y., Kibria, G., & Peeta, S. (2025) · doi:10.1145/3764924.3770899 · Cited by 4 · see citing works
TabLinkLLM: An LLM-Based Approach for Entity Linking in Tabular Data
Jayawardene, I., Avogadro, R., Soylu, A., & Roman, D. (2024) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat62293.2024.00035 · Cited by 4 · see citing works
Language-Empowered Conversion for Remote Sensing Image Retrieval With Text Feedback
Yang, J. B., Li, S., Sun, Y., Wang, H., & Zhou, Z. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3572827 · Cited by 4 · see citing works
LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraints
Palmeira Ferraz, T., Mehta, K., Lin, Y.-H., Chang, H.-S., Oraby, S., Liu, S., Subramanian, V., Chung, T., Bansal, M., & Peng, N. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.458 · Cited by 3 · see citing works
“3 (Chiang et al., 2023), and Zephyr 7B β (Tunstall et al., 2024), all top performers on the Open LLM Leaderboard as of February 2024 (Beeching et al., 2023).”
Through the Thicket: A Study of Number-Oriented LLMS Derived from Random Forest Models
Romaszewski, M., Sekuła, P., Głomb, P., Cholewa, M., & Kołodziej, K. (2025) · Journal of Artificial Intelligence and Soft Computing Research · doi:10.2478/jaiscr-2025-0014 · Cited by 6 · see citing works
Zero-Shot Reranking with Large Language Models and Precomputed Ranking Features: Opportunities and Limitations
Movin, M., & Hauff, C. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730119 · Cited by 3 · see citing works
AutoPK: Leveraging LLMs and a Hybrid Similarity Metric for Advanced Retrieval of Pharmacokinetic Data from Complex Tables and Documents
Sholehrasa, H., Ghanaatian, A., Caragea, D., Tell, L. A., Riviere, J. E., & Jaberi-Douraki, M. (2025) · IEEE International Conference on Tools with Artificial Intelligence · doi:10.1109/ictai66417.2025.00051 · Cited by 7 · see citing works
Exploiting large language models in peer review: indirect prompt injection attacks and integrity probes
Torrielli, F., Locci, S., Rapp, A., & Di, L. (2026) · Scientometrics · doi:10.1007/s11192-026-05695-x · Cited by 4 · see citing works
How Our AI-assisted Qualitative Analysis Failed
Thominet, L., Acosta, K., Amorim, J., & Sohan, V. K. (2024) · ACM International Conference on Design of Communication (SIGDOC) · doi:10.1145/3641237.3691672 · Cited by 3 · see citing works
Straight Outta Logs: Can Large Language Models Overcome Preprocessing in Next Event Prediction?
Brennig, K., Kaltenpoth, S., & Müller, O. (2025) · Lecture notes in business information processing · doi:10.1007/978-3-031-78666-2_15 · Cited by 3 · see citing works
Tabular Data-centric AI: Challenges, Techniques and Future Perspectives
Fu, Y., Wang, D., Xiong, H., & Liu, K. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679102 · Cited by 3 · see citing works
Generating Plausible and Context-Appropriate Comments on Social Media Posts: A Large Language Model-Based Approach
Ha, T. (2024) · IEEE Access · doi:10.1109/access.2024.3488903 · Cited by 3 · see citing works
Toward trustworthy and sustainable clinical decision support by training ensembles of specialized logistic regressors
Cuzzocrea, A., Folino, F., Pontieri, L., Sabatino, P., & Samami, M. (2025) · Neural Computing and Applications · doi:10.1007/s00521-025-11360-w · Cited by 3 · see citing works
AInsight: Augmenting Expert Decision-Making with On-the-Fly Insights Grounded in Historical Data
Abolnejadian, M., Amirshahi, S., Brehmer, M., & Crisan, A. (2025) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3719160.3737633 · Cited by 2 · see citing works
LENS: Lightweight and Explainable LLM-Based APT Detection at the Edge for 6G Security
Melhem, S. B., Golec, M., Alwarafy, A., & Khamayseh, Y. (2025) · IEEE Access · doi:10.1109/access.2025.3616235 · Cited by 2 · see citing works
Combination of Large Language Model and Retrieval-augmented Generation for Inference of Traditional Chinese Medicine Prescriptions and Syndrome Differentiations: A Study on Sleeping Disorders
Huang, P., Wang, C., Lu, S., Chan, Y., Chen, Z., Lin, W., Chiu, J.-P., & Chen, Y. (2025) · Journal of Medical and Biological Engineering · doi:10.1007/s40846-025-00988-7 · Cited by 2 · see citing works
A novel multilevel taxonomical approach for describing high-dimensional unlabeled movement data
Tavakoli, Y., Peña‐Castillo, L., & Soares, A. (2025) · International Journal of Data Science and Analytics · doi:10.1007/s41060-025-00934-5 · Cited by 2 · see citing works
Towards an AI-based knowledge assistant for goat farmers based on retrieval-augmented generation
Han, N., Liu, D., & Norton, T. (2026) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2026.111542 · Cited by 4 · see citing works
Large language models in judicial assistance: Empirical insights and domain-specific fine-tuning
Zhu, S., Zhao, J., Chen, Y., Sun, X., & Zheng, X. (2026) · Natural language processing. · doi:10.1017/nlp.2026.10014 · Cited by 2 · see citing works
Can Language Models Enable in-Context Database?
Pan, Y., Yu, H., Zhao, T., & Sun, J. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11401589 · Cited by 2 · see citing works
Data Agents: Levels, State of the Art, and Open Problems
Luo, Y., Li, G., Fan, J., & Tang, N. (2026) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3788853.3801878 · Cited by 2 · see citing works
On the Use of Large Language Models for Table Tasks
Dong, Y., Oyamada, M., Xiao, C., & Zhang, H. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679100 · Cited by 2 · see citing works
TabulaX: Leveraging Large Language Models for Multi-Class Table Transformations
Nobari, A. D., & Rafiei, D. (2025) · Proceedings of the VLDB Endowment · doi:10.14778/3749646.3749657 · Cited by 3 · see citing works
TQAgent: Enhancing Table-Based Question Answering with Knowledge Graphs and Tree-Structured Reasoning
Zhao, J., Zhang, P., Wang, Y., Xin, R., Lu, X., Li, R., Lyu, S., Ou, Z., & Song, M. (2025) · Applied Sciences · doi:10.3390/app15073788 · Cited by 3 · see citing works
2D-TPE: Two-Dimensional Positional Encoding Enhances Table Understanding for Large Language Models
Li, J.-N., Guan, J., Wu, W., Yu, Z., & Yan, R. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714920 · Cited by 2 · see citing works
МЕТОДИКА ФОРМИРОВАНИЯ БАЗЫ ДАННЫХ ХАРАКТЕРИСТИК СЛОЖНОГО ТЕХНОЛОГИЧЕСКОГО ОБЪЕКТА С ИСПОЛЬЗОВАНИЕМ БОЛЬШИХ ЯЗЫКОВЫХ МОДЕЛЕЙ
Матвеева, А. Р., & Антонов, Е. В. (2024) · Вестник НИЯУ МИФИ · doi:10.26583/vestnik.2024.5.7 · Cited by 1 · see citing works
Challenges in Integrating Conversational AI and GUI-Based Applications
Ziegler, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-91760-8_11 · Cited by 1 · see citing works
Towards uncertainty-calibrated structural data enrichment with large language model for few-shot entity resolution
Yan, M., Wang, Y., Jiang, X., Zhou, H. L., & Li, J. (2025) · Frontiers of Computer Science · doi:10.1007/s11704-025-41143-4 · Cited by 1 · see citing works
Advancing Table Understanding of Large Language Models via Feature Re-ordering
Wang, G., Chen, Y., Chen, H., Fan, X., Wang, J., Li, X., Hu, M., Chang, C.-Y., & Hu, X. (2025) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3748239.3748248 · Cited by 1 · see citing works
Leveraging LLMs for Assignment Report Summaries to Support Teacher Insights in Intelligent Tutoring Systems
Lim, W.-C. I., Heffernan, N. T., Eroshenko, I., Khumwang, W., & Chan, P.-C. (2025) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3698205.3733955 · Cited by 1 · see citing works
Evaluating Large Language Models for Table Data Extraction from Annual Reports in PDF
Nguyen, N. B., Kammermann, L., & Hanne, T. (2025) · Journal of Advances in Information Technology · doi:10.12720/jait.16.8.1118-1126 · Cited by 1 · see citing works
Panel on Neural Relational Data: Tabular Foundation Models, LLMs... or Both?
Papotti, P., & Binnig, C. (2025) · Proceedings of the VLDB Endowment · doi:10.14778/3750601.3760519 · Cited by 1 · see citing works
A Model-Agnostic Approach for Semantic Table Augmentation in Digital Twin Government Using LLMs and Domain-Specific Ontologies
Lee, Y., Son, W., Kim, J., & Chun, S. (2025) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case58245.2025.11163817 · Cited by 1 · see citing works
Querying Large Automotive Software Models: Agentic vs. Direct LLM Approaches
Mazur, L., Petrović, N., Miranda, J. P., Radermacher, A., Rasche, R., & Knoll, A. (2025) · International Generative AI and Computational Language Modelling Conference (GACLM) · doi:10.1109/gaclm67198.2025.11232128 · Cited by 1 · see citing works
Boosting large-language models for fact-checking: leveraging verbalized tabular data as evidence
Luu, S. T., Vo, T., Tran, V., Matsui, T., & Nguyen, L.-M. (2026) · International Journal of Data Science and Analytics · doi:10.1007/s41060-025-00998-3 · Cited by 1 · see citing works
From literature to knowledge graphs: automated extraction and retrieval of life cycle assessment data with large language models
Tang, Z., Dreger, M., Jiang, P., Malek, K., & Tu, Q. (2026) · Journal of Industrial Ecology · doi:10.1007/s44498-026-00135-8 · Cited by 1 · see citing works
Evaluating LLM Abilities to Understand Tabular Electronic Health Records: A Comprehensive Study of Patient Data Extraction and Retrieval
Lovón-Melgarejo, J., Mouysset, M., Oleiwan, J., Moreno, J. G., Damase‐Michel, C., & Tamine, L. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88711-6_10 · Cited by 4 · see citing works
FENAP: Foundation Models for EMA-Derived Narrative Analysis and Prediction
Zhao, F., Shang, Y., & Trull, T. J. (2024) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-024-00170-9 · Cited by 1 · see citing works
QATCH: Automatic Evaluation of SQL-Centric Tasks on Proprietary Data
Papicchio, S., Papotti, P., & Cagliero, L. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3712704 · Cited by 1 · see citing works
HiBench: Benchmarking LLMs Capability on Hierarchical Structure Reasoning
Jiang, Z., Wu, P., Liang, Z., Chen, P. Q., Xu, Y., Jia, Y., Tu, J., Li, C., Ng, P. H. F., & Li, Q. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737378 · Cited by 1 · see citing works
Innovation Under Suppression: Censorship's Effect on Cultural Production in Early-Modern EnglandPreprint
Murrell, P., & Grajzl, P. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5382070 · Cited by 1 · see citing works
Evaluating LLM Abilities to Understand Tabular Electronic Health Records: A Comprehensive Study of Patient Data Extraction and RetrievalPreprint
Lovón-Melgarejo, J., Mouysset, M., Oleiwan, J., Moreno, J. G., Damase‐Michel, C., & Tamine, L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.09384 · Cited by 1 · see citing works
Recommendations for the Development of Artificial Intelligence Applications for the Retail Level
Hartman, J. (2024) · Journal of Food Protection · doi:10.1016/j.jfp.2024.100398
Building an extensible, AI-augmented ecological momentary assessment platform
Zhao, F. (2024) · doi:10.32469/10355/105952
A Retrieval-Augmented Framework for Tabular Interpretation with Large Language Model
Yan, M., Ren, W., Wang, Y., & Li, J. (2025) · Lecture notes in computer science · doi:10.1007/978-981-97-5779-4_23
Classification of Table Cells Based on LLM Prompts
Liu, M., Bu, C., Bai, S., Dong, B., & Wu, X. (2024) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc54092.2024.10831285
Continuous Pre-training of TAPAS Model with E-commerce Structured Data
Chen, X., Du, S., & Qiu, Y. (2024) · IEEE International Conference on Information Technology, Big Data and Artificial Intelligence (ICIBA) · doi:10.1109/iciba62489.2024.10868941
EFFICIENCY OF LLM INSTRUCTION FORMATS FOR CLASS IMBALANCE PROBLEMS IN TRAINING DATA FOR PREDICTIVE MONITORING SYSTEMS
Lutsiuk, A. (2025) · Information and communication technologies electronic engineering · doi:10.23939/ictee2025.01.075
Query Answering for Tabular Data Using Large Language Models
Oh, M., & Chun, J. (2024) · International Conference on AI x Data and Knowledge Engineering (AIxDKE) · doi:10.1109/aixdke63520.2024.00037
Towards An Improved Video RAG Workflow With Orchestration Support in A Visual Data Management System
Chatterjee, S., Verma, R., Kumar, A., & Raghunath, A. (2025) · Workshop on Data Management for End-to-End Machine Learning (DEEM) · doi:10.1145/3735654.3735945
Generating Pandas Code for Big Table Question Answering Using Large Language Models
Vyatkin, A. A., & Oliseenko, V. D. (2025) · IEEE International Conference on Soft Computing and Measurements (SCM) · doi:10.1109/scm66446.2025.11060118
Scalable Tabular Hierarchical Metadata Classification in Heterogeneous Structured Large-Scale Datasets Using Contrastive Learning
Kandibedala, B., Shrestha, G., Pyayt, A. L., Ivanov, T., & Gubanov, M. N. (2025) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde65448.2025.00141
TabHealer: Structure-Aware Table Recovery from OCR Errors Using LLM EnsemblesPreprint
jang, Y., & Oh, H. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5457005
TableCall: Boosting Table Question Answering with Tool-Driven Training-Free LLMs
Xu, C.-B., Chen, Y., Li, X.-H., Yin, F., & Liu, C. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04630-7_8
Late Breaking Results: Fine-Tuning LLMs for Test Stimuli Generation
Park, H., Park, S., & Kang, S. (2025) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac63849.2025.11132967
Enhancing Table Recognition Using Vision Language Models (VLM)
Liu, Y., & Zhu, Y. (2025) · International Conference on Computer Communication and Artificial Intelligence (CCAI) · doi:10.1109/ccai65422.2025.11189648
A Survey of Large Language Models for Tabular Data Imputation: Tuning Paradigms and Challenges
Jha, S., Goyal, V., & Meena, S. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-032-03740-4_19
Building Trustworthy Knowledge Retrieval for Nuclear Chemical Engineering: A Multi-Dimensional Trust Evaluation and Propagation FrameworkPreprint
Yu, T., Wang, H., Li, S., Luo, Y., Xian, L., He, H. X., Gong, H., Zhu, T., & Ye, G. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5704125
Enhancing Microsoft Excel with AI-Driven Natural Language Processing for Automated Spreadsheet Operations
Punetha, D., & Baidya, D. (2025) · doi:10.38124/ijisrt/25nov482
A Preliminary Fundamental Financial Analysis Framework Using Structured LLM Prompting - A Case Study
Gupta, I., Sharma, N., Kaushal, A., & Rajeswara Rao, K. V. S. (2025) · International Conference on Computation System and Information Technology for Sustainable Solutions (CSITSS) · doi:10.1109/csitss67709.2025.11294475
Maintaining Academic Integrity in the Era of Large Language Models
Hidayati, N. N., Santosa, A., Nurfadhilah, E., Latief, A. D., Komariah, K. S., Jarin, A., Pebiana, S., Wabula, Y., Fajri, R., & Sampurno, T. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-0250-8.ch005

Tables as Texts or Images: Evaluating the Table Reasoning Ability of LLMs and MLLMs
Deng, N.-H., Sun, Z., He, R., Sikka, A., Chen, Y.-L., Ma, L., Zhang, Y., & Mihalcea, R. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2024.findings-acl.23 · Cited by 64 · see citing works
Adaptive multi-format synergistic table understanding framework for large language models
Zhang, F., Zhou, G., Li, Z., Wang, J., & Lu, J. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-01081-y
Struc-Bench: Are Large Language Models Really Good at Generating Complex Structured Data?Preprint
Tang, X., Zong, Y., Phang, J., Zhao, Y., Zhou, W., Cohan, A., & Gerstein, M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.08963 · Cited by 5 · see citing works
OpenTab: Advancing Large Language Models as Open-domain Table ReasonersPreprint
Kong, K., Zhang, J., Shen, Z., Srinivasan, B., Lei, C., Faloutsos, C., Rangwala, H., & Karypis, G. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.14361 · Cited by 1 · see citing works
TReB: A Comprehensive Benchmark for Evaluating Table Reasoning Capabilities of Large Language Models
Li, C., Liu, X., Song, Z., Chi, C., Shi, B., Zhao, C., Chang, G., Wang, Z., Yang, K., Wang, X., et al. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808617 · Cited by 1 · see citing works

Authors: Sui, Y., Zhou, M., Zhou, M., Han, S., & Zhang, D.
Affiliations: National University of Singapore (SG)
Microsoft Research Asia (China) (CN)
ORCID: Sui, Yuan
Zhou, Mengyu
Zhou, Mingjie
Han, Shi
Zhang, Dongmei
Venue: Proceedings of the 17th ACM International Conference on Web Search and Data Mining
Publisher: ACM
Published: 2024-03-04
Pages: 645-654
DOI: 10.1145/3616855.3635752
OpenAlex: W4392366650
Cited by: 155
References: 33
Citations by year: 2026: 59 · 2025: 72 · 2024: 24
FWCI: 45.74
Type: conference paper
Language: en
Open access: bronze
doi.org
Licence: www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations: ACM International Conference on Web Search and Data Mining (WSDM) (publishedVersion)
arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
arXiv (Cornell University)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords: large language models, self-augmentation, GPT-3.5, GPT-4

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks