The DOI matches its Crossref record; 53 of the 57 references it cites were found in a registry or an index.
The Role of the Crowd in Countering Misinformation: A Case Study of the COVID-19 Infodemic
Open accessAbstract
Fact checking by professionals is viewed as a vital defense in the fight against misinformation. While fact checking is important and its impact has been significant, fact checks could have limited visibility and may not reach the intended audience, such as those deeply embedded in polarized communities. Concerned citizens (i.e., the crowd), who are users of the platforms where misinformation appears, can play a crucial role in disseminating fact-checking information and in countering the spread of misinformation. To explore if this is the case, we conduct a data-driven study of misinformation on the Twitter platform, focusing on tweets related to the COVID-19 pandemic, analyzing the spread of misinformation, professional fact checks, and the crowds response to popular misleading claims about COVID-19.In this work, we curate a dataset of false claims and statements that seek to challenge or refute them. We train a classifier to create a novel dataset of 155,468 COVID-19-related tweets, containing 33,237 false claims and 33,413 refuting arguments. Our findings show that professional fact-checking tweets have limited volume and reach. In contrast, we observe that the surge in misinformation tweets results in a quick response and a corresponding increase in tweets that refute such misinformation. More importantly, we find contrasting differences in the way the crowd refutes tweets, some tweets appear to be opinions, while others contain concrete evidence, such as a link to a reputed source. Our work provides insights into how misinformation is organically countered in social platforms by some of their users and the role they play in amplifying professional fact checks. These insights could lead to development of tools and mechanisms that can empower concerned citizens in combating misinformation. The code and data can be found in this link.1
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“as disinformation actors change their tactics (Kumar and Shah 2018; Micallef et al. 2020).”
From Fake news, disinformation and misinformation in social media: a review (2023) · cites this paper as Micallef et al. 2020
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Politeness : Some Universals in Language Usage
Brown, P. (1987) · Cited by 11,901
The spread of true and false news online
Vosoughi, S., Roy, D., & Aral, S. (2018) · Science · doi:10.1126/science.aap9559 · Cited by 8,847
Lazer, D. M. J., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F., Metzger, M. J., Nyhan, B., Pennycook, G., Rothschild, D., et al. (2018) · Science · doi:10.1126/science.aao2998 · Cited by 4,160
The COVID-19 social media infodemic
Cinelli, M., Quattrociocchi, W., Galeazzi, A., Valensise, C. M., Brugnoli, E., Schmidt, A. L., Zola, P., Zollo, F., & Scala, A. (2020) · Scientific Reports · doi:10.1038/s41598-020-73510-5 · Cited by 2,313
Microblogging during two natural hazards events
Vieweg, S. E., Hughes, A., Starbird, K., & Palen, L. A. (2010) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753486 · Cited by 1,485
Wang, Y., Ma, F., Jin, Z., Yuan, Y., Xun, G., Jha, K., Su, L., & Gao, J. (2018) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3219819.3219903 · Cited by 1,181
COVID-19–Related Infodemic and Its Impact on Public Health: A Global Social Media Analysis
Islam, M. S., Sarkar, T., Khan, S. H., Mostofa Kamal, A.-H., Hasan, S. M. M., Kabir, A., Yeasmin, D., Islam, M. A., Amin Chowdhury, K. I., Anwar, K. S., et al. (2020) · The American Journal of Tropical Medicine and Hygiene · doi:10.4269/ajtmh.20-0812 · Cited by 1,178 · cited 1×
“Some of the false claims related to COVID-19 have already had severe harmful consequences, including violence [9] and over 800 deaths [10].”
BackgroundTwitter under crisis: can we trust what we RT?
Mendoza, M., Poblete, B., & Castillo, C. (2010) · Proceedings of the First Workshop on Social Media Analytics · doi:10.1145/1964858.1964869 · Cited by 955
Coronavirus Goes Viral: Quantifying the COVID-19 Misinformation Epidemic on Twitter
Kouzy, R., Jaoude, J. A., Kraitem, A. M., Alam, M. B. E., Karam, B. S., Adib, E., Zarka, J. G., Traboulsi, C., Akl, E. A., & Baddour, K. (2020) · Cureus · doi:10.7759/cureus.7255 · Cited by 950
Chen, E., Lerman, K., & Ferrara, E. (2020) · JMIR Public Health and Surveillance · doi:10.2196/19273 · Cited by 852
(2011) · Cited by 724
Types, sources, and claims of Covid-19 misinformation
Brennen, J. S., Simon, F., Howard, P., & Nielsen, R. (2023) · Reuters Institute for the Study of Journalism · doi:10.60625/risj-awvq-sr55 · Cited by 663
Using Expert Sources to Correct Health Misinformation in Social Media
Vraga, E. K., & Bode, L. (2017) · Science Communication · doi:10.1177/1075547017731776 · Cited by 627
Freeman, E. A., & Moisen, G. G. (2008) · Ecological Modelling · doi:10.1016/j.ecolmodel.2008.05.015 · Cited by 601
van der Meer, T. G. L. A., & Jin, Y. (2019) · Health Communication · doi:10.1080/10410236.2019.1573295 · Cited by 432
(2014) · iConference 2014 Proceedings · doi:10.9776/14308 · Cited by 423
An exploratory study of COVID-19 misinformation on Twitter
Shahi, G. K., Dirkson, A. R., & Majchrzak, T. A. (2021) · Online Social Networks and Media · doi:10.1016/j.osnem.2020.100104 · Cited by 397
Friggeri, A., Adamic, L., Eckles, D., & Cheng, J. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14559 · Cited by 382
Ebola, Twitter, and misinformation: a dangerous combination?
Oyeyemi, S. O., Gabarrón, E., & Wynn, R. (2014) · BMJ · doi:10.1136/bmj.g6178 · Cited by 362
Disinformation on the Web: Impact, Characteristics, and Detection of Wikipedia Hoaxes
Kumar, S., West, R., & Leskovec, J. (2016) · Proceedings of the 25th International Conference on World Wide Web · doi:10.1145/2872427.2883085 · Cited by 310 · cited 1×
“Social media contains many types of misinformation, such as rumors [19], hoaxes [24], false news [20] and false information [14].”
BackgroundSelf-citationFalse Information on Web and Social Media: A Survey
Kumar, S., & Shah, N. (2018) · arXiv · doi:10.48550/arxiv.1804.08559 · Cited by 289
Self-citationAnatomy of an online misinformation network
Shao, C., Hui, P.-M., Wang, L., Jiang, X., Flammini, A., Menczer, F., & Ciampaglia, G. L. (2018) · PLoS ONE · doi:10.1371/journal.pone.0196087 · Cited by 282
Tweeting the terror: modelling the social media reaction to the Woolwich terrorist attack
Burnap, P., Williams, M. L., Sloan, L., RANA, O. F., Housley, W., Edwards, A., Knight, V., Procter, R., & Alex Voss, L. (2014) · Social Network Analysis and Mining · doi:10.1007/s13278-014-0206-4 · Cited by 242
COVID-19 on Social Media: Analyzing Misinformation in Twitter ConversationsPreprint
Sharma, K., Seo, S., Meng, C., Rambhatla, S., & Liu, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.12309 · Cited by 235
Sentiment Aware Fake News Detection on Online Social Networks
Ajao, O., Bhowmik, D., & Zargari, S. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683170 · Cited by 228
COVIDLies: Detecting COVID-19 Misinformation on Social Media
Hossain, T., Logan, R. L., Ugarte, D. A., Matsubara, Y., Young, S. D., & Singh, S. (2020) · Workshop on NLP for COVID-19 (NLP-COVID19) · doi:10.18653/v1/2020.nlpcovid19-2.11 · Cited by 214
Weak Supervision for Fake News Detection via Reinforcement Learning
Wang, Y., Yang, W., Ma, F., Xu, J., Zhong, B., Deng, Q., & Gao, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5389 · Cited by 157
Schuster, C. (2004) · Educational and Psychological Measurement · doi:10.1177/0013164403260197 · Cited by 139
Jiang, S., & Wilson, C. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274351 · Cited by 125
Wintersieck, A. (2017) · American Politics Research · doi:10.1177/1532673x16686555 · Cited by 118
Miller, M., Banerjee, T., Muppalla, R., Romine, W. L., & Sheth, A. (2017) · JMIR Public Health and Surveillance · doi:10.2196/publichealth.7157 · Cited by 101
Characterizing COVID-19 Misinformation Communities Using a Novel Twitter DatasetPreprint
Memon, S. A., & Carley, K. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.00791 · Cited by 96
Brown, P., Levinson, S., & Gumperz, J. (2021) · Cross-Cultural Pragmatics · doi:10.1017/cbo9780511813085 · Cited by 87
Exposure to social engagement metrics increases vulnerability to misinformation
Avram, M., Micallef, N., Patil, S., & Menczer, F. (2020) · Harvard Kennedy School Misinformation Review · doi:10.37016/mr-2020-033 · Cited by 83
Self-citationConvoKit: A Toolkit for the Analysis of Conversations
Chang, J. P., Chiam, C., Fu, L., Wang, A., Zhang, J., & Danescu-Niculescu-Mizil, C. (2020) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2020.sigdial-1.8 · Cited by 83
Towards Detecting Rumours in Social Media
Zubiaga, A., Liakata, M., Procter, R., Bontcheva, K., & Tolmie, P. (2015) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · doi:10.48550/arxiv.1504.04712 · Cited by 72
Neutral bots probe political bias on social media
Chen, W., Pacheco, D. F., Yang, K.-C., & Menczer, F. (2021) · RePEc: Research Papers in Economics · Cited by 68
Politeness Strategies in Twitter Updates of Female English Language Studies Malaysian Undergraduates
Maros, M., & Rosli, L. (2017) · 3L The Southeast Asian Journal of English Language Studies · doi:10.17576/3l-2017-2301-10 · Cited by 57
Disinformation and Misinformation on Twitter during the Novel Coronavirus OutbreakPreprint
Huang, B., & Carley, K. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.04278 · Cited by 52
Yellow fever outbreaks and Twitter: Rumors and misinformation
Ortiz-Martínez, Y., & Jiménez-Arcia, L. F. (2017) · American Journal of Infection Control · doi:10.1016/j.ajic.2017.02.027 · Cited by 51
Do the right thing: Tone may not affect correction of misinformation on social media
Bode, L., Vraga, E. K., & Tully, M. (2020) · Harvard Kennedy School Misinformation Review · doi:10.37016/mr-2020-026 · Cited by 47
Roitero, K., Soprano, M., Portelli, B., Spina, D., Mea, V. D., Serra, G., Mizzaro, S., & Demartini, G. (2020) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3340531.3412048 · Cited by 46
A Model for Identifying Misinformation in Online Social Networks
Antoniadis, S., Litou, I., & Kalogeraki, V. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-26148-5_32 · Cited by 33
Dharawat, A., Lourentzou, I., Morales, A., & Zhai, C. (2020) · arXiv (Cornell University) · Cited by 28
Alam, F., Shaar, S., Dalvi, F., Sajjad, H., Nikolov, A., Mubarak, H., Martino, G. D. S., Abdelalí, A., Durrani, N., Darwish, K. M., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.00033 · Cited by 28
CMU-MisCov19: A Novel Twitter Dataset for Characterizing COVID-19 Misinformation
Memon, S. A., & Carley, K. M. (2020) · Figshare · doi:10.5281/zenodo.4024153 · Cited by 28
Efficient and timely misinformation blocking under varying cost constraints
Litou, I., Kalogeraki, V., Katakis, I., & Gunopulos, D. (2017) · Online Social Networks and Media · doi:10.1016/j.osnem.2017.07.001 · Cited by 27
DETECTING COVID-19 MISINFORMATION ON SOCIAL MEDIA
Michaud, J. (2023) · Issues in Information Systems · doi:10.48009/3_iis_2023_124 · Cited by 12
Introduction to the Special Issue on Combating Digital Misinformation and Disinformation
Hassan, N., Li, C., Yang, J., & Yu, C. (2019) · Journal of Data and Information Quality · doi:10.1145/3321484 · Cited by 4
Investigating “Who” in the Crowdsourcing of News Credibility
Bhuiyan, M. M., Zhang, A. X., Sehat, C. M., & Mitra, T. (2020) · Cited by 3
Flagged first, then by citations
4 of the 57 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- 2020, Novel coronavirus (2019-nCoV)
- pennebaker, 2001, Linguistic inquiry and word count: Liwc 2001, Mahway Lawrence Erlbaum Associates, 71, 2001
- 2020, Coronavirus ‘Murder threats‘ to telecoms engineers over 5G
Cited by
Sorted by citations
Fake news, disinformation and misinformation in social media: a review
Aïmeur, E., Amri, S., & Brassard, G. (2023) · Social Network Analysis and Mining · doi:10.1007/s13278-023-01028-5 · Cited by 772
Zhou, J., Zhang, Y., Luo, Q., Parker, A. G., & De Choudhury, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581318 · Cited by 288
Jungherr, A. M., & Schroeder, R. (2021) · Social Media + Society · doi:10.1177/2056305121988928 · Cited by 198
Examining the impact of sharing COVID-19 misinformation online on mental health
Verma, G., Bhardwaj, A., Aledavood, T., Choudhury, M. D., & Kumar, S. (2022) · Scientific Reports · doi:10.1038/s41598-022-11488-y · Cited by 133
Self-citationCrowds Can Effectively Identify Misinformation at Scale
Martel, C., Allen, J., Pennycook, G., & Rand, D. G. (2024) · Perspectives on Psychological Science · doi:10.1177/17456916231190388 · Cited by 84
Metadata
- Authors
- Micallef, N., He, B., Kumar, S., Ahamad, M., & Memon, N.
- Affiliations
- New York University Abu Dhabi (AE)
- Georgia Institute of Technology (US)
- New York University (US)
- Venue
- 2020 IEEE International Conference on Big Data (Big Data)
- Publisher
- IEEE
- Published
- 2020-12-10
- Pages
- 748-757
- OpenAlex
- W3100943362
- Cited by
- 116
- References
- 57
- Citations by year
- 2026: 8 · 2025: 40 · 2024: 21 · 2023: 23 · 2022: 16 · 2021: 8
- FWCI
- 24.92
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE International Conference on Big Data (BigData) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (2027689)
- New York University Abu Dhabi
- York University
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Complex Network Analysis Techniques
- Social Sciences › Communication › Communication and COVID-19 Impact
- Keywords
- misinformation, COVID-19 infodemic, fact-checking, Twitter, false claims, misinformation spread
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
