The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Reasoning Over Semantic-Level Graph for Fact Checking
Open accessAbstract
Fact checking is a challenging task because verifying the truthfulness of a claim requires reasoning about multiple retrievable evidence.In this work, we present a method suitable for reasoning about the semantic-level structure of evidence.Unlike most previous works, which typically represent evidence sentences with either string concatenation or fusing the features of isolated evidence sentences, our approach operates on rich semantic structures of evidence obtained by semantic role labeling.We propose two mechanisms to exploit the structure of evidence while leveraging the advances of pre-trained models like BERT, GPT or XLNet.Specifically, using XLNet as the backbone, we first utilize the graph structure to re-define the relative distances of words, with the intuition that semantically related words should have short distances.Then, we adopt graph convolutional network and graph attention network to propagate and aggregate information from neighboring nodes on the graph.We evaluate our system on FEVER, a benchmark dataset for fact checking, and find that rich structural information is helpful and both our graph-based mechanisms improve the accuracy.Our model is the state-of-the-art system in terms of both official evaluation metrics, namely claim verification accuracy and FEVER score.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Existing fact-verification approaches follow a multi-stage pipeline of claim detection, evidence retrieval and verdict prediction (Guo et al., 2022; Zhong et al., 2020). Such methods, however, require access to external databases and can have considerable inference costs.”
From SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models (2023) · cites this paper as Zhong et al., 2020
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Semi-supervised classification with graph convolutional networks
Kipf, T. N., & Welling, M. (2016) · arXiv · doi:10.48550/arxiv.1609.02907 · Cited by 24,853
Graph Attention NetworksPreprint
Veličković, P., Casanova, A., Liò, P., Cucurull, G., Romero, A., & Bengio, Y. (2017) · arXiv (Cornell University) · doi:10.17863/cam.48429 · Cited by 19,745
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., & Bengio, Y. (2017) · arXiv · doi:10.48550/arxiv.1710.10903 · Cited by 12,107
The spread of true and false news online
Vosoughi, S., Roy, D., & Aral, S. (2018) · Science · doi:10.1126/science.aap9559 · Cited by 8,847 · cited 1×
“Vosoughi et al. (2018) show that false news reaches more people than the truth.”
BackgroundXLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687
Open Information Extraction from the Web
Etzioni, O., Banko, M., Cafarella, M. J., Soderland, S., & Weld, D. S. (2007) · CACM · doi:10.1145/1409360.1409378 · Cited by 2,620
Self-Attention with Relative Position Representations
Shaw, P., Uszkoreit, J., & Vaswani, A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2074 · Cited by 2,505
ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
Speer, R., Chin, J., & Havasi, C. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11164 · Cited by 2,312
"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection
Wang, W. Y. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-2067 · Cited by 2,010
Open information extraction from the web
Banko, M., Cafarella, M., Soderland, S. G., Broadhead, M., & Etzioni, O. (2007) · Cited by 1,410
FEVER: a Large-scale Dataset for Fact Extraction and VERification
Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1074 · Cited by 1,343
Enhanced LSTM for Natural Language Inference
Chen, Q., Zhu, X., Ling, Z.-H., Wei, S., Jiang, H., & Inkpen, D. Z. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1152 · Cited by 1,315
Truth of Varying Shades: Analyzing Language in Fake News and Political Fact-Checking
Rashkin, H., Choi, E., Jang, J. Y., Volkova, S., & Choi, Y. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1317 · Cited by 965
ConceptNet 5.5: An Open Multilingual Graph of General KnowledgePreprint
Speer, R., Chin, J., & Havasi, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.03975 · Cited by 866
Introduction to the CoNLL-2005 shared task
Carreras, X., & Màrquez, L. (2005) · Conference on Computational Natural Language Learning · doi:10.3115/1706543.1706571 · Cited by 646
Simple BERT Models for Relation Extraction and Semantic Role LabelingPreprint
Shi, P., & Lin, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.05255 · Cited by 438
Fact Checking: Task definition and dataset construction
Vlachos, A., & Riedel, S. (2014) · Proceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Science · doi:10.3115/v1/w14-2508 · Cited by 402
Introduction to the CoNLL-2004 Shared Task: Semantic Role Labeling
Carreras, X., & Màrquez, L. (2004) · Cited by 395
(2017) · Digital Access to Scholarship at Harvard (DASH) (Harvard University) · Cited by 362
Emergent: a novel data-set for stance classification
Ferreira, W., & Vlachos, A. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1138 · Cited by 360
Combining Fact Extraction and Verification with Neural Semantic Matching Networks
Nie, Y., Chen, H., & Bansal, M. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016859 · Cited by 286
Recognizing Textual Entailment
Dagan, I., Roth, D., Sammons, M., & Zanzotto, F. M. (2013) · Synthesis lectures on human language technologies · doi:10.2200/s00509ed1v01y201305hlt023 · Cited by 283
GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification
Zhou, J., Han, X., Yang, C. H., Liu, Z., Wang, L., Li, C., & Sun, M. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1085 · Cited by 225
The Fact Extraction and VERification (FEVER) Shared Task
Thorne, J., Vlachos, A., Cocarascu, O., Christodoulopoulos, C., & Mittal, A. (2018) · Proceedings of the First Workshop on Fact Extraction and VERification (FEVER) · doi:10.18653/v1/w18-5501 · Cited by 212
Recognizing textual entailment models and applications
Dagan, I. (2013) · Cited by 205
UKP-Athene: Multi-Sentence Textual Entailment for Claim Verification
Hanselowski, A., Zhang, H., Li, Z., Sorokin, D., Schiller, B., Schulz, C., & Gurevych, I. (2018) · Fact Extraction and VERification Workshop (FEVER) · doi:10.18653/v1/w18-5516 · Cited by 184
Assessing The Factual Accuracy of Generated Text
Goodrich, B., Rao, V., Liu, P. J., & Saleh, M. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330955 · Cited by 162
UCL Machine Reading Group: Four Factor Framework For Fact Finding (HexaF)
Yoneda, T., Mitchell, J., Welbl, J., Stenetorp, P., & Riedel, S. (2018) · Proceedings of the First Workshop on Fact Extraction and VERification (FEVER) · doi:10.18653/v1/w18-5515 · Cited by 101
NaturalLI: Natural Logic Inference for Common Sense Reasoning
Angeli, G., & Manning, C. D. (2014) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1059 · Cited by 99
Language-Aware Truth Assessment of Fact Candidates
Nakashole, N., & Mitchell, T. M. (2014) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/v1/p14-1095 · Cited by 90
TwoWingOS: A Two-Wing Optimization Strategy for Evidential Claim Verification
Yin, W., & Roth, D. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1010 · Cited by 77
Improved Semantic-Aware Network Embedding with Fine-Grained Word Alignment
Shen, D., Zhang, X., Henao, R., & Carin, L. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1209 · Cited by 26
UKP-Athene: Multi-Sentence Textual Entailment for Claim VerificationPreprint
Hanselowski, A., Zhang, H., Li, Z., Sorokin, D., Schiller, B., Schulz, C., & Gurevych, I. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.01479 · Cited by 9
Team UMBC-FEVER : Claim verification using Semantic Lexical Resources
Padia, A., Ferraro, F., & Finin, T. (2018) · Fact Extraction and VERification Workshop (FEVER) · doi:10.18653/v1/w18-5527 · Cited by 5
Kernel Graph Attention Network for Fact VerificationPreprint
Liu, Z., Xiong, C., & Sun, M. (2019) · arXiv (Cornell University) · Cited by 5
Improved Semantic-Aware Network Embedding with Fine-Grained Word AlignmentPreprint
Shen, D., Zhang, X., Henao, R., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.09633 · Cited by 3
Chen, T., Wang, Z., Li, L., Li, D., Li, Z., Chang, X., Bian, P., Liang, G., Wang, Q., & Xie, T. (2025) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.ecoop.2025.6
FEVER: a large-scale dataset for Fact Extraction and VERification
Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2018) · arXiv
Self-Attention with Relative Position Representations
Shaw, P., Uszkoreit, J., & Vaswani, A. (2018) · arXiv · doi:10.48550/arxiv.1803.02155
Enhanced LSTM for Natural Language Inference
Chen, Q., Zhu, X., Ling, Z., Wei, S., Jiang, H., & Inkpen, D. (2016) · arXiv · doi:10.48550/arxiv.1609.06038
Fine-grained Fact Verification with Kernel Graph Attention Network
Liu, Z., Xiong, C., Sun, M., & Liu, Z. (2019) · arXiv · doi:10.48550/arxiv.1910.09796
The Fact Extraction and VERification (FEVER) Shared Task
Thorne, J., Vlachos, A., Cocarascu, O., Christodoulopoulos, C., & Mittal, A. (2018) · arXiv · doi:10.48550/arxiv.1811.10971
"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection
Wang, W. Y. (2017) · arXiv · doi:10.48550/arxiv.1705.00648
TwoWingOS: A Two-Wing Optimization Strategy for Evidential Claim Verification
Yin, W., & Roth, D. (2018) · arXiv · doi:10.48550/arxiv.1808.03465
Flagged first, then by citations
Cited by
Sorted by citations
Graph neural networks: A review of methods and applications
Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., & Sun, M. (2020) · AI Open · doi:10.1016/j.aiopen.2021.01.001 · Cited by 6,880
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models
Manakul, P., Liusie, A., & Gales, M. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.557 · Cited by 557
A Survey on Automated Fact-Checking
Guo, Z., Schlichtkrull, M., & Vlachos, A. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00454 · Cited by 500
Compare to The Knowledge: Graph Neural Fake News Detection with External Knowledge
Hu, L., Yang, T., Zhang, L., Zhong, W., Tang, D., Shi, C., Duan, N., & Zhou, M. (2021) · doi:10.18653/v1/2021.acl-long.62 · Cited by 233
Self-citation
Metadata
- Authors
- Zhong, W., Xu, J., Tang, D., Xu, Z., Duan, N., Zhou, M., Wang, J., & Yin, J.
- Affiliations
- Sun Yat-sen University (CN)
- Peking University (CN)
- Microsoft (United States) (US)
- Venue
- Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
- Published
- 2020
- Pages
- 6170-6180
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W3034808961
- Cited by
- 151
- References
- 36
- Citations by year
- 2026: 4 · 2025: 15 · 2024: 22 · 2023: 26 · 2022: 36 · 2021: 38 · 2020: 10
- FWCI
- 58.32
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- Funders
- National Natural Science Foundation of China (U1611264, U1711262, U1811261, U1811264, U1711261, U1911203)
- Basic and Applied Basic Research Foundation of Guangdong Province (2019B15 15130001)
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- fact-checking, claim verification, semantic role labeling, graph convolutional networks, graph attention networks, XLNet, FEVER dataset, word distance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
