Cited by 542Topic ModelingNatural Language Processing TechniquesMultimodal Machine Learning Applications

The DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index.

TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data

Open access

Yin, Pengcheng, Neubig, Graham, Yih, Wen-tau, Riedel, Sebastian·2020·Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

DOI 10.18653/v1/2020.acl-main.745

Open this paper in CoChat — free account Open PDF

Abstract

Recent years have witnessed the burgeoning of pretrained language models (LMs) for textbased natural language (NL) understanding tasks.Such models are typically trained on free-form NL text, hence may not be suitable for tasks like semantic parsing over structured data, which require reasoning over both free-form NL questions and structured tabular data (e.g., database tables).In this paper we present TABERT, a pretrained LM that jointly learns representations for NL sentences and (semi-)structured tables.TABERT is trained on a large corpus of 26 million tables and their English contexts.In experiments, neural semantic parsers using TABERT as feature representation layers achieve new best results on the challenging weakly-supervised semantic parsing benchmark WIKITABLEQUESTIONS, while performing competitively on the text-to-SQL dataset SPIDER. 1

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.18653/v1/2020.acl-main.745

Cites

59 of 59 references verified0 citation errors · 0 not checkable

Statistics

Figure 1 Overview of TABERT for learning representations of utterances and table schemas with an example from WIKITABLEQUESTIONS 3 . (A) A content snapshot of the table is created based on the input NL utterance. (B) Each row in the snapshot is encoded by a Transformer (only R 2 is shown), producing row-wise encodings for utterance tokens and cells. (C) All row-wise encodings are aligned and processed by V vertical self-attention layers, generating utterance and column representations.CC-BY · From the PDF

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Attention Is All You Need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
MizAR 60 for Mizar 50
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
Gaussian Error Linear Units (GELUs)
Hendrycks, D., & Gimpel, K. (2016) · arXiv · doi:10.48550/arxiv.1606.08415 · Cited by 3,537
SpanBERT: Improving Pre-training by Representing and Predicting Spans
Joshi, M., Chen, D., Liu, Y., Weld, D. S., Zettlemoyer, L., & Levy, O. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00300 · Cited by 1,827

Show all 58 references

Research works printed but not linked: 3

Jonathan Berant and Percy Liang. 2014. Semantic pars- ing via paraphrasing. In Proceedings of ACL.
Jindrich Libovický and Jindrich Helcl. 2017. Attention strategies for multi-source sequence-to-sequence learning. In Proceedings of ACL.
Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. 2019b. Ernie: En- hanced language representation with informative en- tities. In Proceedings of ACL.

Sorted by citations

Items 501–542 of 542

LedgerRAG: Faithful Retrieval-Augmented Numerical Reasoning Over Financial Filings
Gudipuri, N. (2026) · IEEE International Conference on Control System, Computing and Engineering · doi:10.1109/iccsce69880.2026.11750725
Feature-interpretable disease prediction from tabular data via dynamic GCN and LLMs
Pang, S., Dong, Y., Wang, R., Wang, J., Lyu, H., Chen, W., Li, Q., Song, H., & Shi, X. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.114137
CHART: Clustering and Hardness-Aware Data Selection for Table Understanding
Wang, D., Wang, X., Zheng, H., & Qi, Y. (2026) · International Conference on Intelligent Computing and Signal Processing (ICSP) · doi:10.1109/icsp69961.2026.11540564
ESGTabQA: Open-Domain Question Answering on High-Cardinality ESG Tables
Binks, A., Suleiman, B., & Sun, J. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-7251-9_22
AmbiTab-Sports: An Ambiguity-Aware Benchmark for Tabular Question AnsweringPreprint
Billah, M. A. M., & Sarker, T. (2026) · Research Square · doi:10.21203/rs.3.rs-9539202/v1
Towards Interpretable Tabular Reasoning: Enhancing LLM Reasoning on Tabular Data with Pre-Constructed Logic Graph
Gao, L., Yu, Z., Yin, Z., Zhang, Q., Zhu, Y., Zheng, B., Wang, H., Zhao, J., Chen, G., & Guo, S. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.1396
SchemaAware-Cache: A Context- and Schema-Aware Semantic Caching Framework for Efficient Text-to-SQL Systems
Wang, Q., Li, C., Li, X., Wang, C., Ma, J., Zhang, Y., Yang, W., & Zhao, Z. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11582648
AsTQA: A Benchmark Dataset and Generative Language Model for Tabular Question Answering in Assamese
Lahkar, M. P., Sharma, U., & Pradhan, T. (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.06.280
Beyond creative generation: a deep generative framework for high-fidelity pseudo-virtual prototyping and design evaluation
Lee, S., Hong, S., Shin, J., & Yi, M. Y. (2026) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-026-02913-4
Multi-Table Retrieval Method Based on Implicit Association Reasoning in the Petroleum Domain
Liu, C., Cai, M., Yang, Z., Wang, B., & Wang, C. (2026) · Applied Sciences · doi:10.3390/app16147043
Code Generation for Open Data Statistics: Case Study on the Genesis Database
Schmidberger, V., Lapp, J., Melan, A., & Maucher, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-32335-4_10
Text-to-SQL via Model Context Protocol: Structured Context Orchestration for Reliable Database Query Generation
Yang, G., Wei, V. J., Ling, Y., & Zhang, T. (2026) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2026.3716724
Tabular Data Wrangling in the Era of Large Language Model: A Survey
He, Y. Q., Yang, Q., Yang, Y., Zhou, Y., Hu, Z., Yuan, S., Xiao, Y. H., & Li, Z. (2026) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2026.3716977
Pandora: Leveraging Code-Driven Knowledge Transfer for Unified Structured Knowledge Reasoning
Chen, Y., He, J., Fu, L., Zhang, S., Jin, R., Dai, X., Li, J., Min, D., Hu, N., Zhang, Y., et al. (2026) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2026.3718049 · Cited by 1
TabHealth: Towards a Self-Supervised Tabular Foundation Model for Smart Healthcare
Campana, M. G., Anastasi, G., Pieroni, S., Franchini, M., Molinaro, S., & Delmastro, F. (2026) · IEEE International Conference on Smart Computing · doi:10.1109/smartcomp69968.2026.00015
PIPER: Content-Based Table Search via Profiling and LLM-Generated Pseudoqueries
Terrenzi, R., Falconi, M., Ayvaz, S., & Plebani, P. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-34374-1_14
Bridging the Semantic Gap for Categorical Data Clustering via Large Language Models
Yang, Z., Liao, X., Zhang, Y., & Cheung, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31583-0_14
ZeroRel: Relational Reasoning via Graph-guided Large Language Models
Tian, Y., Zhang, K., Li, Q., Wu, L., & Hong, R. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3818115
SelPE: Progressive Selection for Private Structured Text Synthesis
Zhu, X., Nan, G., Zhang, H., Niu, B., Yue, Y., Wang, Z., Liu, Y., Lei, M., & Tao, X. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817858
Insight Bridge: Connecting Business Scenarios to Data with Large Language Models for Automated SQL Generation
Kumar, R., V, Rajalakshmi, A., & Narayan, A., V (2026) · International Conference on Data Science and Business Systems (ICDSBS) · doi:10.1109/icdsbs69077.2026.11634656
Empowering Large Language Models for Relational Data Cleaning and Integration
Shi, X., Chung, G., Julian, D., & Qiao, L. (2026) · Big Data Mining and Analytics · doi:10.26599/bdma.2025.9020073
Advanced Natural Language to SQL Query Conversion: A Hybrid Approach Leveraging Deep Learning and Semantic Analysis
Nair, U. N., & Velmurugan, L. (2026) · SN Computer Science · doi:10.1007/s42979-026-05292-y
AI-Driven Discovery of Cybercrime Insights from Diverse Web Data
Kumar, S., Rajkumar, Kumar, Y., Khurana, D., Kumar, N., & Garg, P. (2026) · International Conference on Emerging VLSI and Semiconductor Technology for AI and Computing Applications (EVST) · doi:10.1109/evst69093.2026.11660511
Unsupervised column relevance extraction for table-based fact verification via structured and explainable large language model reasoning
Võ, T., Luu, S. T., Matsui, T., & Nguyen, L.-M. (2026) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2026.111473
Token-Aligned Relational Prior Guidance for Semantic Learning on Small-Sample Heterogeneous Tabular DataPreprint
Miao, L., Wang, B., Liu, J., & Yang, L. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7429835
Unified Table Understanding via Context-Aware Cell Embedding Learning
Tchuitcheu, W. C., Lu, T., Beersel, A. V., Jordens, J., & Dooms, A. (2026) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2026.3731233
ViPanelTR: A Multi-Agent Framework for Vietnamese Table Question Answering
Khoi, N. D., Kiet, V. T., & Thin, D. V. (2026) · International Conference on Multimedia Analysis and Pattern Recognition (MAPR) · doi:10.1109/mapr72750.2026.11685757
Table Understanding Enhanced Large Language Models for Chinese Table-to-Text Generation
Ni, Y., Li, Y., Li, S., He, Y., Li, Y., & Zeng, J. (2026) · International Conference on Big Data and Information Analytics (BigDIA) · doi:10.1109/bigdia71496.2026.11684533
Structured Important Data Identification Framework
Di, Y., Cao, Z., & Lai, R. (2026) · IEEE International Conference on Energy and Electrical Engineering (EEE) · doi:10.1109/eee69389.2026.11679388
Attract & Engage Visitors for Tabular Data Maintenance: a Longitudinal Naturalistic Field Study
Wallace, S., Lee, N. K., Peng, Z., Massachi, T., Do, L., Rathore, S., Brown, S. M., & Huang, J. J. S. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3816901
Learning Joinable Column Representations via Set Encoding
Hong, H., Hatano, K., & Kim, K. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39820-8_23
SemDistill: Bootstrapping a Low-Latency, Low-Cost Semantic Table Annotator from Noisy LLM
Ge, Y., Ye, Z., Mao, Y., & Gao, Y. (2026) · Proceedings of the ACM on Management of Data · doi:10.1145/3837124
Language Model Analysis for Text-to-SQL: A Case Study in Public Transparency
de Santana, A. S., da Costa, L. J. F., Baptista, C. D. S., & Alves, A. L. F. (2026) · International Conference on eDemocracy & eGovernment · doi:10.1109/icedeg70169.2026.11695387
Structuring Intelligence: Agent-Based Decomposition for Tabular Reasoning with Small Language Models
de Campos, G. A. L., Barros, A. L. B. P., da Costa, L. F., Alves, H. B., Fernandes, D. D., Dias, G. A. G. S., Cavalcanti, J. A. M. B., Pinto, A. Í. M., Silva, L. S., Santos, R. S., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39904-5_12
Recuperação de documentos para enriquecimento de dados tabulares
Delbone, F. R., & Pena, E. H. M. (2026) · Simpósio Brasileiro de Bancos de Dados (SBBD) · doi:10.5753/sbbd.2026.249450
FROG: Fuzzy Reasoning Over Graphs for Robust Visual Question Answering
Le, X. T., Vo, Q., Dang, N. V. M., Tran, T. H. M., Nguyen, N., Nguyen, T. T., & Quan, T. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-5641-9_5
RoGTR: Robust Table Question Answering via Graph-based Textual Reasoning
Guo, X., & Zhang, W. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11728891
An Enhanced Table Retrieval-Augmented Generation Method for Intelligent Question Answering over Large and Complex Enterprise SpreadsheetsPreprint
Li, M., & Wang, S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7634816
Chapter-Aware Retrieval-Augmented Generation for Complex ESG Data Extraction and Traceability
Tao, X., Lyu, H., Ok, S.-Y., Oh, J., & Lee, S. (2026) · IEEE Access · doi:10.1109/access.2026.3745456
Evaluating multimodal fusion strategies for early postoperative delirium prediction using structured electronic health record data and clinical narratives
Ara, L., Miled, Z. B., Boustani, M. A., & Mohanty, S. (2026) · JAMIA Open · doi:10.1093/jamiaopen/ooag206
All Relations at Once: Efficient Open-World Tabular Relation Discovery with Hierarchy-Aware Reward Shaping
Zhang, X., 宋爱波, Qiu, J., & Zhang, T. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.661
AdaCAS: Balancing Efficiency and Effectiveness for Table Union Search via Adaptive Column Aspect Selection
Qiu, E., Chen, T., Xu, C., Tu, Y., & Gao, J. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3840958

Authors: Yin, P., Neubig, G., Yih, W.-T., & Riedel, S.
Affiliations: Meta (United States) (US)
ORCID: Yin, Pengcheng
Neubig, Graham
Riedel, Sebastian
Venue: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Publisher: Association for Computational Linguistics
Published: 2020
Pages: 8413-8426
ISSN: 0736-587X (ISSN-L)
DOI: 10.18653/v1/2020.acl-main.745
OpenAlex: W3035231859
Cited by: 487
References: 58
Citations by year: 2026: 68 · 2025: 86 · 2024: 72 · 2023: 89 · 2022: 76 · 2021: 84 · 2020: 12
FWCI: 24.93
Type: conference paper
Language: en
Open access: hybrid
doi.org
Licence: cc-by
Locations: Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University) (cc-by)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords: pre-trained language models, tabular data, semantic parsing, structured data, natural language understanding, text-to-SQL, spiders, database tables

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data

Abstract

Overview

Integrity signals

Figures

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks