The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Handling Divergent Reference Texts when Evaluating Table-to-Text Generation
Open accessAbstract
Automatically constructed datasets for generating text from semi-structured data (tables), such as WikiBio (Lebret et al., 2016), often contain reference texts that diverge from the information in the corresponding semistructured data.We show that metrics which rely solely on the reference texts, such as BLEU and ROUGE, show poor correlation with human judgments when those references diverge.We propose a new metric, PAR-ENT, which aligns n-grams from the reference and generated texts to the semi-structured data before computing their precision and recall.Through a large scale human evaluation study of table-to-text models for WikiBio, we show that PARENT correlates with human judgments better than existing text generation metrics.We also adapt and evaluate the information extraction based evaluation proposed in Wiseman et al. (2017), and show that PAR-ENT has comparable correlation to it, while being easier to use.We show that PARENT is also applicable when the reference texts are elicited from humans using the data from the WebNLG challenge.1 * Work done during an internship at Google.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
(2004) · Cited by 9,718
CIDEr: Consensus-based image description evaluation
Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845
Get To The Point: Summarization with Pointer-Generator Networks
See, A., Liu, P. J., & Manning, C. D. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1099 · Cited by 3,775
Meteor Universal: Language Specific Translation Evaluation for Any Target Language
Denkowski, M., & Lavie, A. (2014) · WMT@ACL · doi:10.3115/v1/w14-3348 · Cited by 2,108
Automatic evaluation of machine translation quality using n-gram co-occurrence statistics
Doddington, G. R. (2002) · International Conference on Human Language Technology Research · doi:10.3115/1289189.1289273 · Cited by 1,624
Liu, C., Lowe, R., Serban, I. V., Noseworthy, M., Charlin, L., & Pineau, J. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1230 · Cited by 1,377
Gatt, A., & Krahmer, E. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.5477 · Cited by 776
McKeown, K. R. (1985) · Cambridge University Press eBooks · doi:10.1017/cbo9780511620751 · Cited by 753
Research works printed but not linked: 4
- Boxing Chen and Colin Cherry. 2014. A systematic comparison of smoothing techniques for sentence- level bleu. In Proc. of WMT.
- Claire Gardent, Anastasia Shimorina, Shashi Narayan, and Laura Perez-Beltrachini. 2017. Creating train- ing corpora for micro-planners. In Proc. of ACL.
- Jekaterina Novikova, Ondřej Dušek, and Verena Rieser. 2017b. The E2E dataset: New challenges for end- to-end generation. In Proc. of SIGDIAL.
- Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Å Ąukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2016. Google's neural machine translation system: Bridging the gap between human and machine translation. CoRR, abs/1609.08144. · arXiv
Cited by
Sorted by citations
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025
“PARENT [22] measures the accuracy of table-to-text generation by aligning n-grams from the reference description R and generated texts G to the table T .”
BackgroundMethodInfluentialOn Faithfulness and Factuality in Abstractive Summarization
Maynez, J., Narayan, S., Bohnet, B., & McDonald, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.173 · Cited by 1,185
What Makes Good In-Context Examples for GPT-3?
Liu, J., Shen, D., Zhang, Y., Dolan, B., Carin, L., & Chen, W. (2022) · Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures · doi:10.18653/v1/2022.deelio-1.10 · Cited by 563
HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models
Li, J., Cheng, X., Zhao, X., Nie, J.-Y., & Wen, J.-R. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.397 · Cited by 343
ToTTo: A Controlled Table-To-Text Generation Dataset
Parikh, A., Wang, X., Gehrmann, S., Faruqui, M., Dhingra, B., Yang, D., & Das, D. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.89 · Cited by 252
Self-citationIs chatGPT a good nlg evaluator? A preliminary study
Wang, J., Liang, Y., Meng, F., Sun, Z., Shi, H., Li, Z., Xu, J., Qu, J., & Zhou, J. (2023) · Proceedings of the 4th New Frontiers in Summarization Workshop · doi:10.18653/v1/2023.newsum-1.1 · Cited by 235
A Survey of Evaluation Metrics Used for NLG Systems
Sai, A. B., Mohankumar, A. K., & Khapra, M. M. (2023) · ACM Computing Surveys · doi:10.1145/3485766 · Cited by 235
Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports
Zhang, Y., Merck, D., Tsai, E., Manning, C. D., & Langlotz, C. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.458 · Cited by 171
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
Gehrmann, S., Adewumi, T., Aggarwal, K., Ammanamanchi, P. S., Aremu, A., Bosselut, A., Chandu, K. R., Clinciu, M.-A., Das, D., Dhole, K., et al. (2021) · Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021) · doi:10.18653/v1/2021.gem-1.10 · Cited by 159
Self-citationText-to-Text Pre-Training for Data-to-Text Tasks
Kale, M., & Rastogi, A. (2020) · International Conference on Natural Language Generation · doi:10.18653/v1/2020.inlg-1.14 · Cited by 145
Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models
Lin, Z., Guan, S., Zhang, W., Zhang, H., Li, Y., & Zhang, H. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10896-y · Cited by 141
Few-Shot NLG with Pre-Trained Language Model
Chen, Z., Eavani, H., Chen, W., Liu, Y., & Wang, W. Y. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.18 · Cited by 142
Logical Natural Language Generation from Open-Domain Tables
Chen, W., Chen, J., Su, Y., Chen, Z., & Wang, W. Y. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.708 · Cited by 142
Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text
Gehrmann, S., Clark, E., & Sellam, T. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13715 · Cited by 123
Planning with Learned Entity Prompts for Abstractive Summarization
Narayan, S., Zhao, Y., Maynez, J., Simões, G., Nikolaev, V., & McDonald, R. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00438 · Cited by 124
DART: Open-Domain Structured Data Record to Text Generation
Nan, L., Radev, D., Zhang, R., Rau, A., Sivaprasad, A., Hsieh, C., Tang, X., Vyas, A., Verma, N., Krishna, P., et al. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.37 · Cited by 98
Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding
Dziri, N., Madotto, A., Zaïane, O., & Bose, A. J. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.168 · Cited by 81
INFOTABS: Inference on Tables as Semi-structured Data
Gupta, V., Mehta, M., Nokhiz, P., & Srikumar, V. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.210 · Cited by 95
Controlled Hallucinations: Learning to Generate Faithfully from Noisy Data
Filippova, K. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.76 · Cited by 79
Improved Natural Language Generation via Loss Truncation
Kang, D. D., & Hashimoto, T. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.66 · Cited by 85
Plan-then-Generate: Controlled Data-to-Text Generation via Planning
Su, Y., Vandyke, D., Wang, S., Fang, Y., & Collier, N. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.76 · Cited by 71
A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation
Liu, T., Zhang, Y., Brockett, C., Mao, Y., Sui, Z., Chen, W., & Dolan, B. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.464 · Cited by 63
HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generation
Cheng, Z., Dong, H., Wang, Z., Jia, R., Guo, J., Gao, Y., Han, S., Lou, J., & Zhang, D. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.78 · Cited by 86
Sticking to the Facts: Confident Decoding for Faithful Data-to-Text Generation
Tian, R., Narayan, S., Sellam, T., & Parikh, A. P. (2019) · arXiv · doi:10.48550/arxiv.1910.08684 · Cited by 60
Self-citationTowards Table-to-Text Generation with Numerical Reasoning
Suadaa, L. H., Kamigaito, H., Funakoshi, K., Okumura, M., & Takamura, H. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.115 · Cited by 52
(2020) · doi:10.18653/v1/2020.findings-emnlp.190 · Cited by 51
Controlling hallucinations at word level in data-to-text generation
Rebuffel, C., Roberti, M., Soulier, L., Scoutheeten, G., Cancelliere, R., & Gallinari, P. (2022) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-021-00801-4 · Cited by 50
Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark
Dziri, N., Rashkin, H., Linzen, T., & Reitter, D. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00506 · Cited by 79
AI Alignment: A Contemporary Survey
Ji, J., Qiu, T., Chen, B., Zhou, J., Zhang, B., Hong, D., Lou, H., Wang, K., Duan, Y., He, Z., et al. (2025) · ACM Computing Surveys · doi:10.1145/3770749 · Cited by 772
A Gold Standard Methodology for Evaluating Accuracy in Data-To-Text Systems
Thomson, C., & Reiter, E. (2020) · Proceedings of the 13th International Conference on Natural Language Generation · doi:10.18653/v1/2020.inlg-1.22 · Cited by 38
Robust (Controlled) Table-to-Text Generation with Structure-Aware Equivariance Learning
Wang, F., Xu, Z., Szekely, P. A., & Chen, M. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.371 · Cited by 42
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Aralikatte, R., Narayan, S., Maynez, J., Rothe, S., & McDonald, R. (2021) · doi:10.18653/v1/2021.acl-long.474 · Cited by 45
Perturbation CheckLists for Evaluating NLG Evaluation Metrics
Sai, A. B., Dixit, T., Sheth, D. Y., Mohan, S., & Khapra, M. M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.575 · Cited by 39
Data-QuestEval: A Referenceless Metric for Data-to-Text Semantic Evaluation
Rebuffel, C., Scialom, T., Soulier, L., Piwowarski, B., Lamprier, S., Staiano, J., Scoutheeten, G., & Gallinari, P. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.633 · Cited by 24
A Well-Composed Text is Half Done! Composition Sampling for Diverse Conditional Generation
Narayan, S., Simões, G., Zhao, Y., Maynez, J., Das, D., Collins, M., & Lapata, M. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.94 · Cited by 27
Self-citationWikiTableT: A Large-Scale Data-to-Text Dataset for Generating Wikipedia Article Sections
Chen, M., Wiseman, S., & Gimpel, K. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.17 · Cited by 30
ENT-DESC: Entity Description Generation by Exploring Knowledge Graph
Cheng, L., Wu, D., Bing, L., Zhang, Y., Jie, Z., Lu, W., & Si, L. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.90 · Cited by 22
A Survey on Neural Data-to-Text Generation
Lin, Y., Ruan, T., Liu, J., & Wang, H. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3304385 · Cited by 25
Towards Faithfulness in Open Domain Table-to-text Generation from an Entity-centric View
Liu, T., Zheng, X., Chang, B., & Sui, Z. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17583 · Cited by 21
Unifying Structured Data as Graph for Data-to-Text Pre-Training
Li, S., Li, L., Geng, R., Yang, M., Li, B., Yuan, G., He, W., Yuan, S. F., Ma, C., Huang, F., et al. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00641 · Cited by 18
Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story GenerationPreprint
Chhun, C., Colombo, P., Clavel, C., & Suchanek, F. M. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.11646 · Cited by 38
ChatGPT Incorrectness Detection in Software Reviews
Tanzil, M. H., Khan, J. Y., & Uddin, G. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639194 · Cited by 23
Stage-wise Fine-tuning for Graph-to-Text Generation
Wang, Q., Yavuz, S., Lin, X., Ji, H., & Rajani, N. F. (2021) · doi:10.18653/v1/2021.acl-srw.2 · Cited by 14
Search and Learn: Improving Semantic Coverage for Data-to-Text Generation
Jolly, S., Zhang, Z. X., Dengel, A. R., & Mou, L. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21332 · Cited by 14
TWT: Table with Written Text for Controlled Data-to-Text Generation
Li, T., Fang, L., Lou, J., & Li, Z. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.107 · Cited by 12
Attend, Memorize and Generate: Towards Faithful Table-to-Text Generation in Few Shots
Zhao, W., Ye, L., Wan, Y., & Yu, P. S. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.347 · Cited by 12
R2D2: Robust Data-to-Text with Replacement Detection
Nan, L., Flores, L. J., Zhao, Y., Liu, Y., Benson, L., Zou, W., & Radev, D. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.464 · Cited by 14
Proceedings of the 4th New Frontiers in Summarization Workshop
(2023) · doi:10.18653/v1/2023.newsum-1 · Cited by 12
FaMeSumm: Investigating and Improving Faithfulness of Medical Summarization
Zhang, N., Zhang, Y., Guo, W., Mitra, P., & Zhang, R. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.673 · Cited by 18
Generative adversarial network for Table-to-Text generation
Zhao, J., Zhan, Z., Li, T., Li, R., Hu, C., Wang, S., & Zhang, Y. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.04.036 · Cited by 11
(2021) · doi:10.18653/v1/2021.findings-acl.427 · Cited by 10
Truth-Conditional Captions for Time Series Data
Jhamtani, H., & Berg-Kirkpatrick, T. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.55 · Cited by 10
Montella, S., Fabre, B., Urvoy, T., Heinecke, J., & Rojas-Barahona, L. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.00571 · Cited by 12
Data-to-text Generation by Splicing Together Nearest Neighbors
Wiseman, S., Bačkurs, A., & Stratos, K. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.352 · Cited by 12
Generation-Focused Table-Based Intermediate Pre-training for Free-Form Question Answering
Shi, P. J., Ng, P., Nan, F., Zhu, H., Wang, J., Jiang, J., Li, A. H., Chakravarti, R., Weidner, D., Xiang, B., et al. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21382 · Cited by 11
Yu, L., Alégroth, E., Chatzipetrou, P., & Gorschek, T. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107802 · Cited by 14
SynthBio: A Case Study in Human-AI Collaborative Curation of Text DatasetsPreprint
Yuan, A., Ippolito, D., Nikolaev, V., Callison-Burch, C., Coenen, A., & Gehrmann, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.06467 · Cited by 18
MedT2T: An adaptive pointer constrain generating method for a new medical text-to-table task
Zhao, W., Gu, D., Yang, X., Jia, M., Liang, C., Wang, X., & Золотарев, О. (2024) · Future Generation Computer Systems · doi:10.1016/j.future.2024.07.030 · Cited by 7
semiPQA: A Study on Product Question Answering over Semi-structured Data
Shen, X., Barlacchi, G., Tredici, M. D., Cheng, W., & de Gispert, A. (2022) · Workshop on e-Commerce and NLP (ECNLP) · doi:10.18653/v1/2022.ecnlp-1.14 · Cited by 8
Benchmarking and Assessing Transformations Between Text and Causal Maps via Large Language Models
Giabbanelli, P. J., Gandee, T. J., Agrawal, A., & Hosseinichimeh, N. (2025) · Applied Ontology · doi:10.1177/15705838241304102 · Cited by 8
Entity-Based Semantic Adequacy for Data-to-Text Generation
Faille, J., Albert Gatt, A., & Gardent, C. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.132 · Cited by 7
XWikiGen: Cross-lingual Summarization for Encyclopedic Text Generation in Low Resource Languages
Taunk, D., Sagare, S., Patil, A., Subramanian, S., Gupta, M., & Varma, V. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583405 · Cited by 10
TaTA: A Multilingual Table-to-Text Dataset for African Languages
Gehrmann, S., Ruder, S., Nikolaev, V. A., Botha, J., Chavinda, M., Parikh, A. P., & Rivera, C. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.118 · Cited by 10
Self-citationNeural Methods for Data-to-text Generation
Sharma, M., Gogineni, A. K., & Ramakrishnan, N. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3660639 · Cited by 23
A Tutorial on Evaluation Metrics used in Natural Language Generation
Khapra, M. M., & Sai, A. B. (2021) · doi:10.18653/v1/2021.naacl-tutorials.4 · Cited by 7
ASDOT: Any-Shot Data-to-Text Generation with Pretrained Language Models
Xiang, J., Liu, Z., Zhou, Y., Xing, E. P., & Hu, Z. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.136 · Cited by 7
Directed Acyclic Transformer Pre-training for High-quality Non-autoregressive Text Generation
Huang, F., Ke, P., & Huang, M. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00582 · Cited by 7
Localize, Retrieve and Fuse: A Generalized Framework for Free-Form Question Answering over Tables
Zhao, W., Liu, Y., Wan, Y., Wang, Y., Deng, Z., & Yu, P. S. (2023) · doi:10.18653/v1/2023.findings-ijcnlp.1 · Cited by 7
Enhancing consistency with the fusion of paralleled decoders for text generation
Li, Y., Huang, H., Bai, Y., & Gao, Y. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102652 · Cited by 7
Language Model Augmented Relevance Score
Liu, R., Wei, J., & Vosoughi, S. (2021) · doi:10.18653/v1/2021.acl-long.521 · Cited by 10
Semantic Verification in Large Language Model-based Retrieval Augmented Generation
Martin, A., Witschel, H. F., Mandl, M. M., & Stockhecke, M. (2024) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v3i1.31199 · Cited by 7
Bootstrapping Generators from Noisy Data
Perez-Beltrachini, L., & Lapata, M. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1137 · Cited by 10
Rebuffel, C., Soulier, L., Scoutheeten, G., & Gallinari, P. (2020) · International Conference on Natural Language Generation · doi:10.18653/v1/2020.inlg-1.18 · Cited by 7
MiRANews: Dataset and Benchmarks for Multi-Resource-Assisted News Summarization
Xu, X., Dušek, O., Narayan, S., Rieser, V., & Konstas, I. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.133 · Cited by 7
Faithful Low-Resource Data-to-Text Generation through Cycle Training
Wang, Z., Collins, M., Vedula, N., Filice, S., Malmasi, S., & Rokhlenko, O. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.160 · Cited by 8
FALTE: A Toolkit for Fine-grained Annotation for Long Text Evaluation
Goyal, T., Li, J. J., & Durrett, G. (2022) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2022.emnlp-demos.35 · Cited by 5
CogNLG: Cognitive graph for KG‐to‐text generation
Lai, P., Ye, F., Fu, Y.-G., Chen, Z., Wu, Y., Wang, Y., & Chang, V. I. C. (2023) · Expert Systems · doi:10.1111/exsy.13461 · Cited by 5
FactSpotter: Evaluating the Factual Faithfulness of Graph-to-Text Generation
Zhang, K., Balalau, O., & Manolescu, I. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.672 · Cited by 5
Table-to-Text Generation With Pretrained Diffusion Models
Krylov, A. S., & Somov, O. (2024) · IEEE Access · doi:10.1109/access.2024.3440006 · Cited by 5
Semantic Matching for Sequence-to-Sequence Learning
Zhang, R., Chen, C., Zhang, X., Bai, K., & Carin, L. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.21 · Cited by 5
Precisely the Point: Adversarial Augmentations for Faithful and Informative Text Generation
Wu, W., Li, W., Liu, J., Xiao, X., Li, S., & Lyu, Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.482 · Cited by 6
XF2T: Cross-lingual Fact-to-Text Generation for Low-Resource Languages
Sagare, S., Abhishek, T., Singh, B., Sharma, A., Gupta, M., & Varma, V. (2023) · International Conference on Natural Language Generation · doi:10.18653/v1/2023.inlg-main.2 · Cited by 5
Time-aware Prompting for Text Generation
Cao, S., & Wang, L. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.535 · Cited by 5
Enhancing diversity for logical table‐to‐text generation with mixture of experts
Wu, J., & Hou, M. (2024) · Expert Systems · doi:10.1111/exsy.13533 · Cited by 4
Scrutinizing the foundations: could large language models be solipsistic?
Esanu, A. (2024) · Synthese · doi:10.1007/s11229-024-04589-w · Cited by 5
OpenRT: An Open-source Framework for Reasoning Over Tabular Data
Zhao, Y., Mi, B., Qi, Z., Nan, L., Guo, M., Cohan, A., & Radev, D. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-demo.32 · Cited by 5
Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning
Li, A. H., Shang, M., Spiliopoulou, E., Ma, J., Ng, P., Wang, Z., Min, B., Wang, W. Y., McKeown, K. R., Castelli, V., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.894 · Cited by 4
Active Evaluation: Efficient NLG Evaluation with Few Pairwise Comparisons
Mohankumar, A. K., & Khapra, M. M. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.600 · Cited by 7
Text-in-Context: Token-Level Error Detection for Table-to-Text Generation
Kasner, Z., Mille, S., & Dušek, O. (2021) · International Conference on Natural Language Generation · doi:10.18653/v1/2021.inlg-1.25 · Cited by 3
Fact Checking Machine Generated Text with Dependency Trees
Estes, A., Vedula, N., Collins, M., Cecil, M., & Rokhlenko, O. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-industry.46 · Cited by 4
Unsupervised Token-level Hallucination Detection from Summary Generation By-products
Marfurt, A., & Henderson, J. (2022) · Workshop on Generation, Evaluation and Metrics (GEM) · doi:10.18653/v1/2022.gem-1.21 · Cited by 5
Plan-then-Seam: Towards Efficient Table-to-Text Generation
Li, L., Geng, R., Fang, C., Li, B., Ma, C., Li, B., & Li, Y. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.15 · Cited by 5
ReTAG: Reasoning Aware Table to Analytic Text Generation
Ghosal, D., Nema, P., & Raghuveer, A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.389 · Cited by 4
Adapting Knowledge for Few-Shot Table-to-Text Generation
Guo, Z., Yan, M., Qi, J., Zhou, J., He, Z., Zheng, G., Wang, X., & Zhou, C. (2024) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3430480 · Cited by 5
Trainable Ranking Models to Evaluate the Semantic Accuracy of Data-to-Text Neural Generator
Garneau, N., & Lamontagne, L. (2021) · Workshop on Evaluation and Comparison of NLP Systems (Eval4NLP) · doi:10.18653/v1/2021.eval4nlp-1.6 · Cited by 3
Lan, Z., Li, W., Su, J., Xiao, X., Liu, J., Wu, W., & Lyu, Y. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14267 · Cited by 3
Plan-then-Generate: Controlled Data-to-Text Generation via PlanningPreprint
Su, Y., Vandyke, D., Wang, S., Fang, Y., & Collier, N. (2021) · arXiv (Cornell University) · doi:10.17863/cam.84984 · Cited by 12
A comprehensive study on factual consistency in LLMs for data-to-text generation
Mahapatra, J., & Garain, U. (2025) · Sadhana · doi:10.1007/s12046-025-02860-5 · Cited by 6
A framework to synthetically generate fine-grained hallucinated data
Saxena, A., Aggarwal, D., Badathala, N., & Bhattacharyya, P. (2025) · Language Resources and Evaluation · doi:10.1007/s10579-025-09864-x · Cited by 5
Metadata
- Authors
- Dhingra, B., Faruqui, M., Parikh, A., Chang, M.-W., Das, D., & Cohen, W.
- Affiliations
- Google (United States) (US)
- Venue
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Published
- 2019
- Pages
- 4884-4895
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2948628052
- Cited by
- 150
- References
- 40
- Citations by year
- 2026: 7 · 2025: 17 · 2024: 13 · 2023: 33 · 2022: 26 · 2021: 35 · 2020: 17 · 2019: 1
- FWCI
- 7.56
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- table-to-text generation, evaluation metrics, semi-structured data, human evaluation correlation, WebNLG, BLEU, ROUGE
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
