InfoSurgeon: Cross-Media Fine-grained Information Consistency Checking for Fake News Detection
Open accessAbstract
Yi Fung, Christopher Thomas, Revanth Gangi Reddy, Sandeep Polisetty, Heng Ji, Shih-Fu Chang, Kathleen McKeown, Mohit Bansal, Avi Sil. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prior Seq2Seq fake article generators occasionally make grammar and commonsense mistakes, making them not deceptive enough to humans.
“However, the articles generated by these approaches occasionally make grammar and commonsense mistakes, making them not deceptive enough to humans.”
From On the Risk of Misinformation Pollution with Large Language Models (2023) · cites this paper as Fung et al., 2021
The Real Article Fake Article passage says an earlier GPT-2 generator produced propaganda that could fool humans, while §5.5 reports that InfoSurgeon’s own generated articles were hard for human judges to detect. Neither passage addresses whether prior Seq2Seq generators made grammatical or commonsense mistakes that reduced their deceptiveness. Searches for “grammar,” “commonsense,” and the claim’s phrasing found no such finding in our copy; page numbers were not provided. (LLM, conf=0.72)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821 · see citing works
StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation
Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., & Choo, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00916 · Cited by 4,163 · see citing works
The Curious Case of Neural Text Degeneration
Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09751 · Cited by 1,304 · see citing works
TabFact: A Large-scale Dataset for Table-based Fact Verification
Chen, W., Wang, H., Chen, J., Zhang, Y., Wang, H., Li, S., Zhou, X., & Wang, W. Y. (2019) · arXiv · doi:10.48550/arxiv.1909.02164 · Cited by 239 · see citing works
Good News, Everyone! Context Driven Entity-Aware Captioning for News Images
Biten, A. F., Gomez, L., Rusiñol, M., & Karatzas, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01275 · Cited by 136 · see citing works
The limitations of stylometry for detecting machine-generated fake news
Schuster, T., Schuster, R., Shah, D. J., & Barzilay, R. (2020) · Computational Linguistics · doi:10.1162/coli_a_00380 · Cited by 109 · see citing works
Flagged first, then by citations
Research works printed but not linked: 46
- Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. 2018. Bottom-up and top-down attention for image captioning and visual question answering. In Proceedings of the IEEE conference on computer vi- sion and pattern recognition, pages 6077-6086.
- Gabor Angeli, Melvin Jose Johnson Premkumar, and Christopher D Manning. 2015. Leveraging linguis- tic structure for open domain information extraction. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Lan- guage Processing (Volume 1: Long Papers), pages 344-354.
- Ramón Fernandez Astudillo, Miguel Ballesteros, Tahira Naseem, Austin Blodgett, and Radu Flo- rian. 2020. Transition-based parsing with stack- transformers. In Proceedings of the 2020 Confer- ence on Empirical Methods in Natural Language Processing: Findings, pages 1001-1007.
- Pepa Atanasova, Jakob Grue Simonsen, Christina Li- oma, and Isabelle Augenstein. 2020. Generat- ing fact checking explanations. In Proceedings of the 2020 Association of Computational Linguistics (ACL), pages 417-422.
- Ramy Baly, Georgi Karadzhov, Dimitar Alexandrov, James Glass, and Preslav Nakov. 2018. Predict- ing factuality of reporting and bias of news media sources. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9054-9065.
- Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider. 2013. Abstract meaning representation for sembanking. In Proceedings of the 7th Linguis- tic Annotation Workshop and Interoperability with Discourse, LAW-ID@ACL 2013, August 8-9, 2013, Sofia, Bulgaria, pages 178-186.
- Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: a col- laboratively created graph database for structuring human knowledge. Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 1247-1250. · DOI · Paper page
- Wenhu Chen, Yu Su, Xifeng Yan, and William Yang Wang. 2020. KGPT: Knowledge-grounded pre- training for data-to-text generation. In Proceed- ings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8635-8648, Online. Association for Computational Linguistics. · DOI
- Giovanni Luca Ciampaglia, Prashant Shiralkar, Luis M Rocha, Johan Bollen, Filippo Menczer, and Alessandro Flammini. 2015. Computational fact checking from knowledge networks. PloS one, 10(6):e0128193. · DOI · Paper page
- Giovanni Da San Martino, Alberto Barrón-Cedeno, Henning Wachsmuth, Rostislav Petrov, and Preslav Nakov. 2020. Semeval-2020 task 11: Detection of propaganda techniques in news articles. In Proceed- ings of the Fourteenth Workshop on Semantic Evalu- ation, pages 1377-1414.
- Sohan De Sarkar, Fan Yang, and Arjun Mukherjee. 2018. Attending sentences to detect satirical fake news. In Proceedings of the 27th International Con- ference on Computational Linguistics, pages 3371- 3380.
- Christiane Fellbaum. 1998. WordNet: An Electronic Lexical Database. Cambridge, MA: MIT Press.
- Xinyu Fu, Jiani Zhang, Ziqiao Meng, and Irwin King. 2020. Magnn: metapath aggregated graph neural network for heterogeneous graph embedding. In Proceedings of The Web Conference 2020, pages 2331-2341.
- Felix A Gers, Jürgen Schmidhuber, and Fred Cummins. 2000. Learning to forget: Continual prediction with lstm. Neural computation, 12(10):2451-2471.
- Andreas Hanselowski, Avinesh PVS, Benjamin Schiller, Felix Caspelherr, Debanjan Chaudhuri, Christian M. Meyer, and Iryna Gurevych. 2018. A retrospective analysis of the fake news challenge stance-detection task. In Proceedings of the 27th International Conference on Computational Lin- guistics, pages 1859-1874, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
- Minyoung Huh, Andrew Liu, Andrew Owens, and Alexei A Efros. 2018. Fighting fake news: Image splice detection via learned self-consistency. Eu- ropean Conference on Computer Vision (ECCV), pages 101-117.
- Rohit Kumar Kaliyar, Anurag Goswami, and Pratik Narang. 2020. Deepfake: Improving fake news de- tection using tensor decomposition-based deep neu- ral network. The Journal of Supercomputing. · DOI · Paper page
- Hamid Karimi, Proteek Roy, Sari Saba-Sadiya, and Jiliang Tang. 2018. Multi-source multi-class fake news detection. In Proceedings of the 27th Inter- national Conference on Computational Linguistics, pages 1546-1557, Santa Fe, New Mexico, USA. As- sociation for Computational Linguistics.
- Hamid Karimi and Jiliang Tang. 2019. Learning hier- archical discourse-level structure for fake news de- tection. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, pages 3432-3442.
- Tuan Lai, Heng Ji, Trung Bui, Quan Hung Tran, Franck Dernoncourt, and Walter Chang. 2021. A context- dependent gated module for incorporating symbolic semantics into event coreference resolution. arXiv preprint arXiv:2104.01697. · arXiv
- Manling Li, Alireza Zareian, Qi Zeng, Spencer White- head, Di Lu, Heng Ji, and Shih-Fu Chang. 2020. Cross-media structured common space for multime- dia event extraction. In Proceedings of the 58th An- nual Meeting of the Association for Computational Linguistics, pages 2557-2568, Online. Association for Computational Linguistics. · DOI
- Ying Lin, Heng Ji, Fei Huang, and Lingfei Wu. 2020. A joint end-to-end neural model for information ex- traction with global features. In Proc. The 58th An- nual Meeting of the Association for Computational Linguistics (ACL2020).
- Yang Liu and Mirella Lapata. 2019. Text summariza- tion with pretrained encoders. In Proceedings of the 2019 Conference on Empirical Methods in Nat- ural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3730-3740, Hong Kong, China. Association for Computational Linguistics. · DOI
- Jing Ma, Wei Gao, and Kam-Fai Wong. 2018. Ru- mor detection on Twitter with tree-structured recur- sive neural networks. In Proceedings of the 56th An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1980- 1989, Melbourne, Australia. Association for Compu- tational Linguistics. · DOI
- Moin Nadeem, Wei Fang, Brian Xu, Mitra Mohtarami, and James Glass. 2019. FAKTA: An automatic end- to-end fact checking system. In Proceedings of the 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics (Demonstrations), pages 78-83, Minneapolis, Min- nesota. Association for Computational Linguistics. · DOI
- Preslav Nakov, Lluís Màrquez, Alberto Barrón- Cedeño, Pepa Gencheva, Georgi Karadzhov, Tsve- tomila Mihaylova, Mitra Mohtarami, and James Glass. 2019. Automatic fact checking using context and discourse information. ACM Journal of Data and Information Quality. · DOI
- Martha Palmer, Dan Gildea, and Paul Kingsbury. 2005. The proposition bank: A corpus annotated with se- mantic roles. Computational Linguistics Journal, 31(1):71-106.
- Jeff Z Pan, Siyana Pavlova, Chenxi Li, Ningxi Li, Yang- mei Li, and Jinshuo Liu. 2018. Content based fake news detection using knowledge graphs. In Inter- national semantic web conference, pages 669-683. Springer. · DOI
- Xiaoman Pan, Boliang Zhang, Jonathan May, Joel Nothman, Kevin Knight, and Heng Ji. 2017. Cross- lingual name tagging and linking for 282 languages. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1946-1958, Vancouver, Canada. Association for Computational Linguistics. · DOI
- Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Confer- ence on Empirical Methods in Natural Language Processing (EMNLP), pages 1532-1543, Doha, Qatar. Association for Computational Linguistics. · DOI · Paper page
- Verónica Pérez-Rosas, Bennett Kleinberg, Alexandra Lefevre, and Rada Mihalcea. 2018. Automatic de- tection of fake news. In Proceedings of the 27th International Conference on Computational Linguis- tics, pages 3391-3401, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
- Matt Post and David Vilar. 2018. Fast lexically con- strained decoding with dynamic beam allocation for neural machine translation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, Volume 1 (Long Pa- pers), pages 1314-1324, New Orleans, Louisiana. Association for Computational Linguistics. · DOI
- Sarah Pratt, Mark Yatskar, Luca Weihs, Ali Farhadi, and Aniruddha Kembhavi. 2020. Grounded situa- tion recognition. In European Conference on Com- puter Vision, pages 314-332. Springer. · DOI
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
- Shaden Shaar, Nikolay Babulkov, Giovanni Da San Martino, and Preslav Nakov. 2020. That is a known lie: Detecting previously fact-checked claims. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguis- tics, pages 3607-3618, Online. Association for Computational Linguistics. · DOI
- Baoxu Shi and Tim Weninger. 2016. Discriminative predicate path mining for fact checking in knowl- edge graphs. Knowledge-based systems, 104:123- 133. · DOI
- Linfeng Song, Yue Zhang, Zhiguo Wang, and Daniel Gildea. 2018. A graph-to-sequence model for amr- to-text generation. Proceedings of the 56th Annual Meeting of the Association for Computational Lin- guistics, pages 1616-1626.
- Reuben Tan, Bryan Plummer, and Kate Saenko. 2020. Detecting cross-modal inconsistency to de- fend against neural fake news. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2081-2106, Online. Association for Computational Linguistics. · DOI
- James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. Fever: a large-scale dataset for fact extraction and verification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 809-819.
- Sheng-Yu Wang, Oliver Wang, Andrew Owens, Richard Zhang, and Alexei A Efros. 2019. De- tecting photoshopped faces by scripting photoshop. ICCV, pages 10072-10081.
- William Yang Wang. 2017. "liar, liar pants on fire": A new benchmark dataset for fake news detection. In Proceedings of the 55th Annual Meeting of the As- sociation for Computational Linguistics (Volume 2: Short Papers), pages 422-426, Vancouver, Canada. Association for Computational Linguistics. · DOI
- Yaqing Wang, Fenglong Ma, Zhiwei Jin, Ye Yuan, Guangxu Xun, Kishlay Jha, Lu Su, and Jing Gao. 2018. Eann: Event adversarial neural networks for multi-modal fake news detection. In Proceed- ings of the 24th acm sigkdd international conference on knowledge discovery & data mining, pages 849- 857. · DOI
- Haoyang Wen, Ying Lin, Tuan Lai, Xiaoman Pan, Sha Li, Xudong Lin, Ben Zhou, Manling Li, Haoyu Wang, Hongming Zhang, et al. 2021. Resin: A dock- erized schema-guided cross-document cross-lingual cross-media information extraction and event track- ing system. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Demonstrations, pages 133-143.
- Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. In Advances in Neural Information Process- ing Systems, pages 9054-9065.
- Dimitrina Zlatkova, Preslav Nakov, and Ivan Koychev. 2019. Fact-checking meets fauxtography: Verify- ing claims about images. In Proceedings of the 2019 Conference on Empirical Methods in Natu- ral Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2099-2108, Hong Kong, China. Association for Computational Linguistics. References · DOI
- Reuben Tan, Bryan Plummer, and Kate Saenko. 2020. Detecting cross-modal inconsistency to de- fend against neural fake news. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2081-2106, Online. Association for Computational Linguistics. · DOI
Cited by
Sorted by citations
On the Risk of Misinformation Pollution with Large Language Models
Pan, Y., Pan, L., Chen, W., Nakov, P., Kan, M.-Y., & Wang, W. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.97 · Cited by 70 · see citing works
Faking Fake News for Real Fake News Detection: Propaganda-loaded Training Data Generation
Huang, K.-H., McKeown, K., Nakov, P., Choi, Y., & Ji, H. (2022) · arXiv · doi:10.48550/arxiv.2203.05386 · Cited by 46 · see citing works
Self-citation
- Authors
- Fung, Y., Thomas, C., Gangi Reddy, R., Polisetty, S., Ji, H., Chang, S.-F., McKeown, K., Bansal, M., & Sil, A.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- Columbia University (US)
- University of Massachusetts Amherst (US)
- University of North Carolina at Chapel Hill (US)
- Venue
- Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
- Published
- 2021
- Pages
- 1683-1698
- OpenAlex
- W3174408325
- Cited by
- 52
- References
- 6
- Citations by year
- 2026: 4 · 2025: 6 · 2024: 6 · 2023: 18 · 2022: 16 · 2021: 2
- FWCI
- 28.01
- Type
- conference paper
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Funders
- Defense Advanced Research Projects Agency (FA8750, FA8750-18-2-0014, HR001120C0123)
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Keywords
- fake news detection, misinformation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
