Analysing How People Orient to and Spread Rumours in Social Media by Looking at Conversational Threads
Open accessAbstract
As breaking news unfolds people increasingly rely on social media to stay abreast of the latest updates. The use of social media in such situations comes with the caveat that new information being released piecemeal may encourage rumours, many of which remain unverified long after their point of release. Little is known, however, about the dynamics of the life cycle of a social media rumour. In this paper we present a methodology that has enabled us to collect, identify and annotate a dataset of 330 rumour threads (4,842 tweets) associated with 9 newsworthy events. We analyse this dataset to understand how users spread, support, or deny rumours that are later proven true or false, by distinguishing two levels of status in a rumour life cycle i.e., before and after its veracity status is resolved. The identification of rumours associated with each event, as well as the tweet that resolved each rumour as true or false, was performed by journalist members of the research team who tracked the events in real time. Our study shows that rumours that are ultimately proven true tend to be resolved faster than those that turn out to be false. Whilst one can readily see users denying rumours once they have been debunked, users appear to be less capable of distinguishing true from false rumours when their veracity remains in question. In fact, we show that the prevalent tendency for users is to support every unverified rumour. We also analyse the role of different types of users, finding that highly reputable users such as news organisations endeavour to post well-grounded statements, which appear to be certain and accompanied by evidence. Nevertheless, these often prove to be unverified pieces of information that give rise to false rumours. Our study reinforces the need for developing robust machine learning techniques that can provide assistance in real time for assessing the veracity of rumours. The findings of our study provide useful insights for achieving this aim.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Zubiaga et al. (2016c) concluded that users tend to support unverified rumours early on, then shift toward supporting true rumours and debunking false ones over time.
“This study concludes that, while the overall tendency is for users to support unverified rumours in the early stages, there is a shift toward supporting true rumours and debunking false rumours as time goes on.”
From Detection and Resolution of Rumours in Social Media (2019) · cites this paper as Zubiaga et al. 2016c
The Results (Rumour Support and Denial; page not supplied) say users tend to support unverified rumours and that denial increases after false rumours are debunked. But both the Results and Discussion and Conclusions say support drops after resolution even for true rumours, rather than shifting toward greater support for them. (LLM, conf=0.88)
Experimental — accuracy not yet measuredRumours can be classified by various factors, including eventual veracity and degree of credibility.
“Many different factors are available for classifying rumours by type, including its eventual veracity value (true, false, or unresolved) (Zubiaga et al. 2016c) or its degree of credibility (e.g., high or low) (Jaeger et al. 1980).”
From Detection and Resolution of Rumours in Social Media (2019) · cites this paper as Zubiaga et al. 2016c
Bottom-up sampling strategies emerged more recently to collect a wider range of rumours by sifting through data to identify rumours rather than using known ones.
“Bottom-up sampling strategies have emerged more recently in studies that aimed at collecting a wider range of rumours, i.e., sifting through data to identify rumours, rather than rumours that are already known.”
From Detection and Resolution of Rumours in Social Media (2019) · cites this paper as Zubiaga et al. 2016c
Veracity annotation has sometimes been operationalised by enlisting journalists with verification expertise.
“This annotation process has been sometimes operationalised by enlisting the help of journalists with expertise in verification (Zubiaga et al. 2016c).”
From Detection and Resolution of Rumours in Social Media (2019) · cites this paper as Zubiaga et al. 2016c
In one example, journalists analysed rumourous claims during breaking news to determine if a rumour was later confirmed true or debunked false; some rumours were marked unverified because no reliable resolution was found.
“journalists analysed rumourous claims spreading on social media during breaking news to determine, where possible, if a rumour had later been confirmed as true or debunked and proven false; this is, however, not always possible and some rumours were marked as unverified as no reliable resolution could be found.”
From Detection and Resolution of Rumours in Social Media (2019) · cites this paper as Zubiaga et al. 2016c
Zubiaga et al. (2016c) annotated rumourous tweets for certainty, evidentiality, and support.
“For example, Zubiaga et al. (2016c) annotated rumourous tweets for certainty (certain, somewhat certain, uncertain) and evidentiality (first-hand experience, inclusion of URL, quotation of person/organisation, link to an image, quotation of unverifiable source, employment of reasoning, no evidence), along with support.”
From Detection and Resolution of Rumours in Social Media (2019) · cites this paper as Zubiaga et al. 2016c
The dataset by Zubiaga et al. (2016) provides over 4,500 tweets categorised by rumour, but unlike Qazvinian's it does not include unrelated tweets with similar characteristics.
“While not specifically intended for rumour tracking, the dataset produced by Zubiaga et al. (2016) provides over 4,500 tweets categorised by rumour.This dataset is different as it does not include tweets with similar characteristics that are actually unrelated.”
From Detection and Resolution of Rumours in Social Media (2019) · cites this paper as Zubiaga et al. 2016
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Individual Comparisons by Ranking Methods
Wilcoxon, F. (1945) · Biometrics Bulletin · doi:10.2307/3001968 · Cited by 15,354 · see citing works
What is Twitter, a social network or a news media?
Kwak, H., Lee, C., Park, H., & Moon, S. (2010) · Proceedings of the 19th international conference on World wide web · doi:10.1145/1772690.1772751 · Cited by 6,652 · see citing works
Rumor Diffusion and Convergence during the 3.11 Earthquake: A Twitter Case Study
Takayasu, M., Sato, K., Sano, Y., Yamada, K., Miura, W., & Takayasu, H. (2015) · PLOS ONE · doi:10.1371/journal.pone.0121443 · Cited by 132 · see citing works
Tweet, but verify: epistemic study of information verification on Twitter
Zubiaga, A., & Ji, H. (2014) · Social Network Analysis and Mining · doi:10.1007/s13278-014-0163-y · Cited by 125 · see citing works
Self-citationHannak, A., Margolin, D., Keegan, B., & Weber, I. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14555 · Cited by 90 · see citing works
Crowdsourcing the Annotation of Rumourous Conversations in Social Media
Zubiaga, A., Liakata, M., Procter, R., Bontcheva, K., & Tolmie, P. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2740908.2743052 · Cited by 67 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 24
- [6]Derczynski L, Bontcheva K. Pheme: Veracity in digital social networks Proceedings of ISA. 2014;.
- [7]DiFonzo N, Bordia P. Rumor, gossip and urban legends. Diogenes. 2007; 54(1): 19– 35. 10.1177/0392192107073433 · DOI · Paper page
- [8]Zubiaga A, Liakata M, Procter R, Bontcheva K, Tolmie P. Towards Detecting Rumours in Social Media. In: Proceedings of the AAAI Workshop on AI for Cities; 2015.
- [9]Donovan P. How idle is idle talk? One hundred years of rumor research. Diogenes. 2007; 54(1): 59– 82. 10.1177/0392192107073434 · DOI
- [11]Qazvinian V, Rosengren E, Radev DR, Mei Q. Rumor has it: Identifying misinformation in microblogs In: Proceedings of EMNLP; 2011 p. 1589– 1599.
- [12]Allport GW, Postman L. An analysis of rumor. Public Opinion Quarterly. 1946; 10(4): 501– 517.
- [13]Allport GW, Postman L. The psychology of rumor. 1947;. 21012248
- [14]Prasad J. The psychology of rumour: A study relating to the great Indian earthquake of 1934. British Journal of Psychology General Section. 1935; 26(1): 1– 15. 10.1111/j.2044-8295.1935.tb00770.x · DOI
- [15]Sinha D. Behaviour in a catastrophic situation: A psychological study of reports and rumours. British Journal of Psychology General Section. 1952; 43(3): 200– 209. 10.1111/j.2044-8295.1952.tb00343.x · DOI
- [16]Jaeger ME, Anthony S, Rosnow RL. Who Hears What from Whom and with What Effect A Study of Rumor. Personality and Social Psychology Bulletin. 1980; 6(3): 473– 478. 10.1177/014616728063024 · DOI · Paper page
- [17]Scanlon TJ. Post-disaster rumor chains: A case study. Mass Emergencies. 1977; 2(126): 22– 27.
- [18]Knapp RH. A psychology of rumor. Public Opinion Quarterly. 1944; 8(1): 22– 37.
- [19]Bordia P. Studying verbal interaction on the Internet: The case of rumor transmission research. Behavior Research Methods, Instruments, & Computers. 1996; 28(2): 149– 151. 10.3758/BF03204753 · DOI
- [21]Castillo C, Mendoza M, Poblete B. Predicting information credibility in time-sensitive social media. Internet Research. 2013; 23(5): 560– 588. 10.1108/IntR-05-2012-0095 · DOI · Paper page
- [22]Procter R, Vis F, Voss A. Reading the riots on Twitter: methodological innovation for the analysis of big data. International Journal of Social Research Methodology. 2013; 16(3): 197– 214. 10.1080/13645579.2013.774172 · DOI
- [23]Starbird K, Maddock J, Orand M, Achterman P, Mason RM. Rumors, False Flags, and Digital Vigilantes: Misinformation on Twitter after the 2013 Boston Marathon Bombing. In: Proceedings of iConference. iSchools; 2014.
- [24]Friggeri A, Adamic LA, Eckles D, Cheng J. Rumor cascades. In: Proceedings of the AAAI Conference on Weblogs and Social Media; 2014.
- [27]Lukasik M, Cohn T, Bontcheva K. Classifying Tweet Level Judgements of Rumours in Social Media. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal: Association for Computational Linguistics; 2015. p. 2590–2595.
- [32]Tolmie P, Procter R, Rouncefield M, Liakata M, Zubiaga A. Microblog Analysis as a Programme of Work. ArXiv e-prints. 2015 Nov;.
- [33]Saurí R, Pustejovsky J. FactBank: A corpus annotated with event factuality. Language resources and evaluation. 2009; 43(3): 227– 268. 10.1007/s10579-009-9089-9 · DOI
- [34]Procter R, Housley W, Williams M, Edwards A, Burnap P, Morgan J, et al Enabling social media research through citizen social science. ECSCW 2013 Adjunct Proceedings. 2013; 3.
- [38]Cotter C. News talk: Investigating the language of journalism. Cambridge University Press; 2010.
- [40]Saunders KW. Mythic Difficulty in Proving a Negative, The. Seton Hall Law Review. 1984; 15: 276.
- [41]Sacks H. Lectures on conversation. Oxford: Basil Blackwell; 1992.
Cited by
Sorted by citations
COVID-19–Related Infodemic and Its Impact on Public Health: A Global Social Media Analysis
Islam, M. S., Sarkar, T., Khan, S. H., Mostofa Kamal, A.-H., Hasan, S. M. M., Kabir, A., Yeasmin, D., Islam, M. A., Amin Chowdhury, K. I., Anwar, K. S., et al. (2020) · The American Journal of Tropical Medicine and Hygiene · doi:10.4269/ajtmh.20-0812 · Cited by 1,178 · see citing works
“Rumor is defined as unverified information that canbe foundas true, fabricated, or entirely false after verification.(14) Stigma is a socially constructed process through which a person with stigma can experience discrimination and devaluation in society.”
BackgroundA survey on fake news and rumour detection techniques
Bondielli, A., & Marcelloni, F. (2019) · Information Sciences · doi:10.1016/j.ins.2019.05.035 · Cited by 658 · see citing works
Detection and Resolution of Rumours in Social Media
Zubiaga, A., Aker, A., Bontcheva, K., Liakata, M., & Procter, R. (2019) · ACM Computing Surveys · doi:10.1145/3161603 · Cited by 608 · see citing works
“This is an approach that was used first by [Zubiaga et al. 2016] and subsequently by [Giasemidis et al. 2016].”
BackgroundMethodResultInfluentialSelf-citationCombating Fake News: A Survey on Identification and Mitigation Techniques
Sharma, K., Qian, F., Jiang, H., Ruchansky, N., Zhang, M., & Liu, Y. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3305260 · Cited by 514 · see citing works
Applications of artificial intelligence for disaster management
Sun, W., Bocchini, P., & Davison, B. D. (2020) · Natural Hazards · doi:10.1007/s11069-020-04124-3 · Cited by 491 · see citing works
Social Media, Echo Chambers, and Political Polarization
Barberá, P. (2020) · Social Media and Democracy · doi:10.1017/9781108890960.004 · Cited by 373 · see citing works
SemEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumours
Derczynski, L., Bontcheva, K., Liakata, M., Procter, R., Wong Sak Hoi, G., & Zubiaga, A. (2017) · Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017) · doi:10.18653/v1/s17-2006 · Cited by 323 · see citing works
Self-citationFalse Information on Web and Social Media: A Survey
Kumar, S., & Shah, N. (2018) · arXiv · doi:10.48550/arxiv.1804.08559 · Cited by 289 · see citing works
Jahanbakhsh, F., Zhang, A. X., Berinsky, A. J., Pennycook, G., Rand, D. G., & Karger, D. R. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449092 · Cited by 112 · see citing works
The Web of False Information: Rumors, Fake News, Hoaxes, Clickbait, and Various Other Shenanigans
Zannettou, S., Sirivianos, M., Blackburn, J., & Kourtellis, N. (2019) · Journal of Data and Information Quality · doi:10.1145/3309699 · Cited by 86 · see citing works
Digital Wildfires: Propagation, Verification, Regulation, and Responsible Innovation
Webb, H., Burnap, P., Procter, R., Rana, O., Stahl, B. C., Williams, M., Housley, W., Edwards, A., & Jirotka, M. (2016) · ACM Transactions on Information Systems · doi:10.1145/2893478 · Cited by 64 · see citing works
Self-citationPADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domains
Ben-David, E., Oved, N., & Reichart, R. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00468 · Cited by 54 · see citing works
Mendoza, M., Valenzuela, S., Núñez-Mussa, E., Padilla, F., Providel, E., Campos, S., Bassi, R., Riquelme, A., Aldana, V., & López, C. (2023) · Applied Sciences · doi:10.3390/app13095347 · Cited by 13 · see citing works
Keeping Rumors in Proportion: Managing Uncertainty in Rumor Systems
Krafft, P. M., & Spiro, E. S. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300876 · Cited by 9 · see citing works
- Authors
- Zubiaga, A., Liakata, M., Procter, R., Wong Sak Hoi, G., & Tolmie, P.
- Affiliations
- University of Warwick (GB)
- Corresponding author
- Zubiaga, Arkaitz
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-03-04
- Volume
- 11
- Issue
- 3
- Pages
- e0150989
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4778911
- OpenAlex
- W2281420995
- Cited by
- 722
- References
- 6
- Citations by year
- 2026: 42 · 2025: 51 · 2024: 52 · 2023: 64 · 2022: 99 · 2021: 109 · 2020: 101 · 2019: 96 · 2018: 56 · 2017: 40 · 2016: 9 · 2013: 1
- FWCI
- 191.58
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Warwick Research Archive Portal (University of Warwick) (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Issue Lab (Candid) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- European Commission (611233)
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Complex Network Analysis Techniques
- Social Sciences › Communication › Public Relations and Crisis Communication
- Keywords
- rumor propagation, social media, breaking news, user reputation, news organizations, misinformation
- MeSH terms
- Communication, Denial, Psychological, Social Support, Social Media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review