Learning from Fact-checkers: Analysis and Generation of Fact-checking Language
Open accessThe DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
In fighting against fake news, many fact-checking systems comprised of human-based fact-checking sites (e.g., snopes.com and politifact.com) and automatic detection systems have been developed in recent years. However, online users still keep sharing fake news even when it has been debunked. It means that early fake news detection may be insufficient and we need another complementary approach to mitigate the spread of misinformation. In this paper, we introduce a novel application of text generation for combating fake news. In particular, we (1) leverage online users named fact-checkers, who cite fact-checking sites as credible evidences to fact-check information in public discourse; (2) analyze linguistic characteristics of fact-checking tweets; and (3) propose and build a deep learning framework to generate responses with fact-checking intention to increase the fact-checkers' engagement in fact-checking activities. Our analysis reveals that the fact-checkers tend to refute misinformation and use formal language (e.g. few swear words and Internet slangs). Our framework successfully generates relevant responses, and outperforms competing models by achieving up to 30% improvements. Our qualitative study also confirms that the superiority of our generated responses compared with responses generated from the existing models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The spread of true and false news online
Vosoughi, S., Roy, D., & Aral, S. (2018) · Science · doi:10.1126/science.aap9559 · Cited by 8,847 · cited 4×
“The dataset in [54] is large but fully anonymized, and the dataset in [52] does not contain original tweets.”
BackgroundMethodInfluentialWhen Corrections Fail: The Persistence of Political Misperceptions
Nyhan, B., & Reifler, J. (2010) · Political Behavior · doi:10.1007/s11109-010-9112-2 · Cited by 3,173 · cited 1×
“Furthermore, recent work showed that individuals tend to selectively consume news that have ideologies similar to what they believe while disregarding contradicting arguments [8, 35].”
BackgroundCharacter-Aware Neural Language Models
Kim, Y., Jernite, Y., Sontag, D., & Rush, A. M. (2015) · arXiv · doi:10.48550/arxiv.1508.06615 · Cited by 1,530
Vinyals, O., & Le, Q. (2015) · arXiv · doi:10.48550/arxiv.1506.05869 · Cited by 1,400
FEVER: a Large-scale Dataset for Fact Extraction and VERification
Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1074 · Cited by 1,343
Uncovering social spammers: social honeypots + machine learning
Lee, K., Caverlee, J., & Webb, S. (2010) · Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1835449.1835522 · Cited by 619 · cited 1×
“nsion (e.g. original posters may have other abusing behaviors that triggered Twitter security system), one obvious observation is that fact-checkers tended to target bad users (e.g. content polluters [24]), who usually have abusing behavior on social platform. It means that fact-checkers are enthusiastic about checkin…”
BackgroundSelf-citationEnquiring Minds: Early Detection of Rumors in Social Media from Enquiry Posts
Zhao, Z., Resnick, P., & Mei, Q. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2736277.2741637 · Cited by 583 · cited 2×
“However, falsified news is still disseminated like wild fire [31, 59] despite dramatic rise of fact-checking sites worldwide [21].”
BackgroundGupta, A., Lamba, H., Kumaraguru, P., & Joshi, A. (2013) · Proceedings of the 22nd International Conference on World Wide Web Companion · doi:10.1145/2487788.2488033 · Cited by 573 · cited 3×
“gth of a generated response. Figure 4(b) shows a scatter plot of FC-tweets’ |tokens|and |shares|(i.e., quotes and retweets). Generally, most FC-tweets had |shares|=0. However, FC-tweets with |tokens|∈[10;20]usually received more attention. To verify this, we created two lists – one containing |shares|of FC-tweets with…”
BackgroundMethodInfluentialFriggeri, A., Adamic, L., Eckles, D., & Cheng, J. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14559 · Cited by 382 · cited 4×
“Since our work utilizes FC-tweets, analyzing users’ replies [9, 15, 40, 54] are closely related to ours.”
BackgroundMethodInfluentialVolkova, S., Shaffer, K., Jang, J. Y., & Hodas, N. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p17-2102 · Cited by 381 · cited 1×
“hoaxies, clickbait, satire and disinformation) [42, 53].”
Cited by
Park, H., & Ahn, D. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642785 · Cited by 84
“Moreover, the HCI community has long designed and studied fact-checking algorithms [4, 26, 35, 39, 89, 103] to identify fake news, false information, and rumors [47, 56–58].”
BackgroundGiachanou, A., Ríssola, E. A., Ghanem, B., Crestani, F., & Rosso, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-51310-8_17 · Cited by 58
“There are different collections built in the field of fake news [27,28,30].”
BackgroundInfluentialDataset of Fake News Detection and Fact Verification: A Survey
Murayama, T. (2021) · arXiv · doi:10.48550/arxiv.2111.03299 · Cited by 40
“Generate-Factcheck [155], which is the extension of Rise-Guardian, includes 73,203 fact-checking tweets including fact-checking articles’ URLs with true or false labels and 64,110 original tweets, to generate fact-checking posts relevant to original posts.”
Background
Metadata
- Authors
- Vo, N., & Lee, K.
- Venue
- Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval
- Publisher
- ACM
- Published
- 2019-07-18
- Pages
- 335-344
- OpenAlex
- W2955681503
- Cited by
- 53
- References
- 18
- FWCI
- 17.50
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Learning from Fact-checkers: Analysis and Generation of Fact-checking Language. https://app.cochat.ai/papers/10.1145/3331184.3331248
- The paper
- Vo, N., & Lee, K. (2019). Learning from Fact-checkers: Analysis and Generation of Fact-checking Language. Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. https://doi.org/10.1145/3331184.3331248
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.