Cited by 591Misinformation and Its ImpactsCommunication and COVID-19 ImpactComplex Network Analysis Techniques

Assessing the risks of ‘infodemics’ in response to COVID-19 epidemics

Open access
Gallotti, Riccardo, Valle, Francesco, Castaldo, Nicola, Sacco, Pierluigi, De Domenico, Manlio·2020·Nature Human Behaviour

Published version of a preprint

Abstract

During COVID-19, governments and the public are fighting not only a pandemic but also a co-evolving infodemic—the rapid and far-reaching spread of information of questionable quality. We analysed more than 100 million Twitter messages posted worldwide during the early stages of epidemic spread across countries (from 22 January to 10 March 2020) and classified the reliability of the news being circulated. We developed an Infodemic Risk Index to capture the magnitude of exposure to unreliable news across countries. We found that measurable waves of potentially unreliable information preceded the rise of COVID-19 infections, exposing entire countries to falsehoods that pose a serious threat to public health. As infections started to rise, reliable information quickly became more dominant, and Twitter content shifted towards more credible informational sources. Infodemic early-warning signals provide important cues for misinformation mitigation by means of adequate communication strategies. An analysis of news shared on Twitter estimates the level of infodemic risk associated with COVID-19 across countries. Epidemic spread and infodemic risk co-evolve, with reliable information becoming more dominant as infection rates rise locally.

AI snapshot

Written from the paper’s full text.

TL;DR

Analyzing more than 112 million Twitter messages in 64 languages from 21 January to 10 March 2020, the authors find that waves of unreliable and low-quality COVID-19 information often anticipate epidemic waves, while reliable information spreads after outbreaks hit. They introduce an Infodemic Risk Index and report that global infodemic risk tends to decline as epidemics expand, though country patterns vary substantially.

Key findings

  • An estimated 40.4% of online messages during the period were due to automated agents (bots), doubling activity relative to estimates from only four years earlier.
  • The authors extracted about 20.7 million links (3.3 million unique); about 50% of URLs were fact-checked by screening almost 4,000 expert-curated web domains, yielding a final database of 3,892 entries: 1,434 reliable, 2,264 unreliable, and 194 unknown.
  • Infodemic waves of unreliable/low-quality information anticipate epidemic waves; when epidemics hit an area, reliable information is “inoculated” and focus shifts toward certified sources. The global Infodemic Risk Index tends to decrease as epidemics spread, with average risk on days with no confirmed cases at 0.276, a drop at first reported case, a rebound at 3 confirmed cases, then a decrease as cases grow.
  • Country profiles differ: South Korea remains low-risk apart from an early spike; Canada is slightly higher; Venezuela is high-risk throughout; Russia shows erratic, pronounced spikes; Italy’s misinformation collapses after outbreak; and in the USA misinformation is mainly driven by verified sources and remains basically constant. The paper also finds G9 countries with remarkable infodemic risk and developing countries with far lower risk levels.

Methods

  • Twitter messages were collected via the Filter API (24 Jan–10 Mar 2020) and Search API (21–24 Jan 2020) using hashtags/keywords including coronavirus, ncov, #Wuhan, covid19, covid-19, sarscov2, and covid. The authors estimate 161.2 million tweets were posted until 10 Mar 2020 and collected 112.6 million; more than 60 million geolocated messages containing more than 9 million news items were analyzed.
  • Users were classified as human or non-human (bots) using a deep-learning algorithm with >90% accuracy and >95% precision for bots, while keeping verified/unverified distinctions. URLs were extracted and assigned reliability through fact-checking databases and a Harm Score scheme.
  • The authors defined exposure, reliability/unreliability, and an Infodemic Risk Index (IRI) ranging from 0 to 1, aggregated daily and by country. Figure 3 considers 178 countries; Supplementary Fig. 5 uses 177 countries excluding China.

Limitations

  • Paper-stated: only about 50% of URLs were fact-checked, and unassessable links were discarded; exposure is a lower bound because higher-order transmission pathways and users who read without resharing are not tracked; China was removed as a major outlier; the relationship with biological contagion patterns cannot be checked in some cases due to lack of reliable data; and whether the USA will show a similar readjustment remains to be checked.
  • Paper-stated/evident: Twitter API restrictions after 25 Feb 2020 allowed no more than 4.5 million messages per day on average, so an increasing fraction of tweets was missed (161.2 million estimated posted vs. 112.6 million collected); geolocated analysis relies on about 56% of the total observed volume after geocoding profiles.
  • Evident: source reliability inherits labels and Harm Scores from external databases and expert classification; the paper also notes that countries with high infodemic risk might be more unreliable in reporting epidemic data, which could alter perceptions and indirectly mislead people.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41562-020-00994-6

Cites

54 of 54 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 18 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Waves of unreliable and low-quality information occur before epidemic waves.

    “We found that waves of unreliable and low-quality information anticipate the epidemic ones, exposing entire countries to irrational social behavior and serious threats for public health.”
    Quote in the paperSubstantial claim
  • Findingown data

    Machine learning analysis of online user behavior revealed extraordinary activity by automated agents called social bots.

    “By using state-of-the-art machine learning techniques to analyze the online behavior of users (see Methods for details), we have discovered an extraordinary activity of automated agents, referred to as social bots 14,15,35,39 .”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [14] De Domenico, M., Granell, C., Porter, M. A. & Arenas, A. The physics of spreading processes in multilayer networks. Nat. Phys. 12, 901–906 (2016). source

      Not checked in the free run.

    • cites [15] Brockmann, D. & Helbing, D. The hidden geometry of complex, network-driven contagion phenomena. Science 342, 1337–1342 (2013). source

      Not checked in the free run.

    • cites [35] Watts, D. J. A simple model of global cascades on random networks. Proc. Natl Acad. Sci. USA 99, 5766–5771 (2002). source

      Not checked in the free run.

    • cites [39] Hébert-Dufresne, L., Scarpino, S. V. & Young, J.-G. Macroscopic patterns of interacting contagions are indistinguishable from social reinforcement. Nat. Phys. https://doi.org/10.1038/s41567-020-0791-2 (2020). source

      Not checked in the free run.

  • Findingcitation

    During the SARS-CoV-2 emergency, a significant part of human communication activity involved intentional production of informational noise and misleading or false information.

    “Contrary to what it could be expected in principle, what this natural experiment reveals is that, on the verge of a threatening global pandemic emergency due to SARS-CoV-2 [23][24][25] , human communication activity is to a significant extent characterized by the intentional production of informational noise and even of misleading or false information 26 .”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [26] Weekly Epidemiological Record Vol. 95, 16 (WHO, 2020); https://www.who.int/wer/2020/wer9516/en/

      Not checked in the free run.

  • Findingown data

    The average infodemic risk index for days with no confirmed cases was 0.276.

    “We first compute the average value for days with no confirmed cases, which is 0.276”
    Quote in the paperNot a substantial claim
  • Findingreasoning

    Globally, infodemic risk tended to decrease as the epidemics spread worldwide.

    “Overall, the global level of infodemic risk tends to decrease as the epidemics spread globally”
    Quote in the paperSubstantial claim
  • May combine two claims. 36
  • Classified as probably not a substantial claim. 13
  • The quote only partly states this claim. 1
  • May combine two claims.

    Findingown data

    When an epidemic reaches an area, reliable information spreads rapidly and attention shifts to certified sources.

    “When the epidemics hit the same area, reliable information is quickly inoculated, like antibodies, and the system shifts focus towards certified informational sources.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingcitation

    An estimated 40.4% of online messages in the period came from automated agents, twice the estimate from four years earlier.

    “Specifically, we estimate that 40.4% of online messages during this period were due to such automated agents, doubling the activity with respect to estimates of only four years ago 35 .”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [35] Watts, D. J. A simple model of global cascades on random networks. Proc. Natl Acad. Sci. USA 99, 5766–5771 (2002). source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    Average follower counts were 660 for unverified humans, 1,400 for unverified bots, 51,000 for verified humans, and 240,000 for verified bots.

    “The average values are: 660 for Unverified Humans (circles), 1400 for Unverified Bots (squares, 51k for Verified Humans (stars) and 240k for Verified Bots (diamonds).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    The study found 4,988 domains, reduced to 4,417 after removing hard duplicates across databases.

    “We have found a total of 4,988 domains, reduced to 4,417 after removing hard duplicates across databases.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    In South Korea, infodemic risk decreased as contagion spread, indicating growing public focus on reliable news.

    “As the contagion spreads to significant levels, the infodemic risk further decreases, signalling an increasing focus of the public opinion toward reliable news sources.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    The probability distributions of follower counts for the four user classes all display a fat tail, but different categories have different outreach.

    “All distributions display a fat-tail, but different categories of users have a different outreach.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    About 0.84% of collected posts were geo-tagged by users, giving highly accurate location information.

    “About 0.84% of collected posts were geo-tagged by the user, providing highly accurate information about their geographic location.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Canada had slightly higher infodemic risk than South Korea; risk rose as epidemics spread but remained low.

    “Canada presents a slightly higher level of infodemic risk, and unlike South Korea, we see that the risk level increases as the epidemics spread, but stays at low levels.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingreasoning

    The geolocated data used in the paper slightly overestimated the fraction of unreliable news for unverified accounts and underestimated the fraction of fake news for verified profiles compared with the whole sample of Tweets.

    “The difference, possibly caused by the correlation of this kind of behaviour with the choice of how customising the field”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    In Venezuela, infodemics was fully active throughout observation, and verified sources contributed substantially to it.

    “At the opposite, in a high-risk country such as Venezuela, the infodemics is in full swing throughout the period of observation, and in addition to the expected activity from unverified sources one notices that even verified ones contribute to a large extent to the infodemics.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    The relationship between declining infodemic risk and expanding epidemics appears general, with a nonlinearly negative association between confirmed cases and infodemic risk.

    “Fig. 4 shows, however, that the relationship between reduction of infodemic risk and expansion of the epidemics seems to be a rather general trend, as the relationship between number of confirmed cases and infodemic risk is (nonlinearly) negative, confirming the result shown in Fig. 3.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Iran, the United States, Germany, the Netherlands, Austria, and Norway maintained their relative infodemic risk levels, while Italy, South Korea, and Japan significantly reduced it as the epidemics progressed.

    “with major roles played not only by countries such as Iran, but also United States, Germany, the Netherlands, Austria and Norway maintaining their relative levels, and other countries like Italy, South Korea and Japan significantly reducing it with the progression of the epidemics.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Processing reduced the overall database to 3,920 unique domains.

    “This phase of processing reduced the overall database to unique 3,920 domains.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Of the 3,892 entries, 1,434 were reliable, 2,264 were unreliable, and 194 were unknown.

    “Type Count RELIABLE 1434 UNRELIABLE 2264 UNKNOWN 194 Total 3892”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    In the USA, misinformation was mainly driven by verified sources and remained basically constant after the epidemic outbreak.

    “For the USA, misinformation is mainly driven by verified sources, and it remains basically constant even after the epidemics outbreak.”
    Quote in the paperSubstantial claim
  • The quote only partly states this claim.

    Findingown data

    Reporting the first confirmed COVID-19 case in a country was associated with a drop in the infodemic risk index, followed by a return to the natural level at three confirmed cases.

    “We observe that reporting of the first case is associated with a drop in the risk index, followed by an increase back to the”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Collective attention about COVID-19 first increased after the outbreak in Wuhan between 24 Jan and 02 Feb 2020, and rose strongly again after the epidemics spread in northern Italy from 20 Feb 2020 onward.

    “We can observe a first increase in collective attention after the outbreak in Wuhan, China (between 24 Jan and 02 Feb 2020) and a second strong rise after the epidemics began to spread in northern Italy (20 Feb 2020 onwards).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    In Italy, infodemic risk was mostly from unverified sources, but misinformation collapsed and shifted suddenly to reliable sources after the epidemic outbreak.

    “In the case of Italy the risk is mostly due to the activity of unverified sources, but we notice that with the outbreak of the epidemics, the production of misinformation literally collapses and there is a sudden shift to reliable sources.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    There are G9 countries with remarkable infodemic risk and developing countries with far lower risk levels.

    “In fact, we find that there are G9 countries with remarkable infodemic risk and developing countries with far lower risk levels.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    In the COVID-19 crisis, complex infodemic effects are at work with significant cross-country variation, and socio-economic development is not the key discriminant between high and low infodemic risk.

    “Here, we have shown that in the context of the COVID-19 crisis, complex infodemic effects are indeed at work, with significant variations across countries, where level of socio-economic development is not the key discriminant to separate countries with high vs. low infodemic risk.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The fraction of geolocated messages was constantly about 56% of the total volume recorded.

    “The fraction of Geoolocated (messages with shared locations, or geonamed, indicated in green) is constantly about 56% of the total volume recorded”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    In Russia, infodemic risk was erratic with sudden pronounced spikes, verified sources played a major role, and epidemic information was fragmented and mostly unreliable.

    “Finally, in a relatively high-risk country such as Russia we notice that infodemic risk is erratic with sudden, very pronounced spikes, and again also verified sources play a major role.Here too, information about the epidemics is fragmented and mostly unreliable.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    After filtering poorly defined domains, the final database contained 3,892 entries.

    “This action reduced the database to the final number of 3,892 entries”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    About half of extracted URLs were fact-checked using nearly 4,000 expert-curated web domains; the rest pointed to vanished pages or content not automatically classifiable.

    “About 50% of URLs have been fact-checked by screening almost 4,000 expert-curated web domains, whereas the remaining corpus was pointing to disappeared web pages or to content not classifiable automatically (eg, videos on YouTube) and unpopular sources.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The reliability of a URL can be inherited from the reliability of its source domain.
  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Repeated exposure to a message from different sources acts as indirect validation of its reliability and relevance, leading users to spread it and become informationally infectious.

    “As users are repeatedly hit by a given message from different sources, this works as an indirect validation of its reliability and relevance, leading the user to spread it in turn, and to become an informationally infectious agent.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationown data

    Human response to falsehood shows early-warning signals that adequate communication strategies might mitigate.

    “Contrary to mainstream beliefs, we show that human response to falsehood exhibits early-warning signals that might be mitigated with adequate communication strategies.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).

    Premisecitation

    It has been proposed that at population level, dynamics of multiple interacting contagions are indistinguishable from social reinforcement.

    “Recently, it has been suggested that this challenging phenomenon might be due to the fact that, at population level, the dynamics of multiple interacting contagions are indistinguishable from social reinforcement 38 .”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [38] Stella, M., Cristoforetti, M. & De Domenico, M. Influence of augmented humans in online interactions during voting events. PLoS ONE 14, e0214210 (2019). source

      Not checked in the free run.

  • May combine two claims.

    Premiseown data

    Information dynamics intended to change perceptions can direct collective attention to false or inflammatory content, a phenomenon called infodemics that resembles epidemics.

    “In this study, we show that information dynamics tailored to alter individuals' perceptions and, consequently, their behavioral response, is able to drive collective attention 12 towards false 13,14 or inflammatory 15 content, a phenomenon named infodemics [16][17][18][19] , sharing similarities with more traditional epidemics and spreading phenomena [20][21][22] .”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [12] Zarocostas, J. How to fight an infodemic. Lancet 395, 676 (2020). source

      Not checked in the free run.

    • cites [13] Pastor-Satorras, R., Castellano, C., Van Mieghem, P. & Vespignani, A. Epidemic processes in complex networks. Rev. Mod. Phys. 87, 925–979 (2015). source

      Not checked in the free run.

    • cites [14] De Domenico, M., Granell, C., Porter, M. A. & Arenas, A. The physics of spreading processes in multilayer networks. Nat. Phys. 12, 901–906 (2016). source

      Not checked in the free run.

    • cites [15] Brockmann, D. & Helbing, D. The hidden geometry of complex, network-driven contagion phenomena. Science 342, 1337–1342 (2013). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Infodemics arise from simultaneous activity of multiple human and non-human sources of fake or unreliable news.

    “As shown in Fig. 1, an infodemics is the result of the simultaneous action of multiple human and non-human sources of fake or unreliable news.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Premisecitation

    The danger posed by infodemics can be comparable to and additive with that of the epidemic itself.

    “In fact, the dangerousness of infodemics can compare and sum up to a large extent to that of the epidemics itself 29 .”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [29] Ferrara, E., Varol, O., Davis, C., Menczer, F. & Flammini, A. The rise of social bots. Commun. ACM 59, 96–104 (2016). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationnone given

    Perceived and actual biological and social risks feed upon one another and may co-evolve in complex ways.

    “Perceived and actual biological and social risks feed upon one another, and may co-evolve in complex ways.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    Human and non-human accounts create unreliable COVID-19 content, and their followers are exposed to it and reshare it, becoming infectious agents.

    “Human and non-human accounts forge unreliable content -such as fake or untrustworthy news -about the COVID19, a topic attracting the collective attention of the whole world.Their followers are exposed to such content, and reshare it, becoming infectious agents:”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Study design and conductnone given

    The method has the highest accuracy (>90%) and precision (>95%) in identifying bots compared with state-of-the-art methods.

    “More in detail, our method has the highest accuracy (>90%) and precision in identifying bots (>95%) when compared with state-of-the-art methods.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationreasoning

    The global decrease in infodemic risk suggests that contagion expansion led people to seek more reliable sources and that verified influencers began spreading more reliable news.

    “suggesting that evidence of the expansion of the contagion leads people to look for relatively more reliable sources, and that verified influencers with many followers started inoculating the system with more reliable news (see Supplementary Figures 3 and 4), playing a role that presents interesting analogies to that of antibodies in the treatment of an infectious disease.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim.

    Interpretationreasoning

    When effective therapies are unavailable, coordination of behaviors and diffusion of pro-social orientations driven by reliable information are key resources for mitigating adverse effects.

    “Especially in situations where effective therapies to contrast the diffusion of the pandemic are not readily available, coordination of behaviors and diffusion of pro-social orientations driven by reliable information at all scales are the key resources for the mitigation of adverse effects.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Study design and conductown data

    Until 24 Feb 2020, about 60,000 tweets were lost out of millions, capturing more than 99.5% of all posted messages.

    “We estimate that until 24 Feb 2020 we lost about 60,000 tweets out of millions, capturing more than 99.5% of all messages posted (see Supplementary Fig. 1).”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    Countries with high infodemic risk might also report epidemic data less reliably, altering perceptions and misleading people seeking reliable information.

    “It is also interesting to observe though that countries with high infodemic risk might also be more unreliable in terms of reporting of epidemic data, thus altering the perceptions of people and indirectly misleading them in their search for reliable information.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationown data

    Escalating epidemics lead people to pay more attention to reliable sources, potentially limiting infodemics, but the speed of adjustment may strongly affect social outcomes.

    “The escalation of the epidemics leads people to progressively pay attention to more reliable sources thus potentially limiting the impact of the infodemics, but the actual speed of adjustment may make a major difference in determining the social outcome, and in particular between a controlled epidemics and a global pandemics.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Study design and conductnone given

    The Infodemic Risk Index is robust to user classification and is not sensitive to the performance of bot detection algorithms.

    “IRI is robust to user classification, making it an indicator not sensitive to performances of bot detection algorithms.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The reliability classification used manually checked web domains collected from multiple publicly available databases.

    “We have collected manually-checked web domains from multiple publicly available databases, including scientific and journalistic ones.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Study design and conductcitation

    The choice of Twitter API was the safest to date because biases affecting the Sample API are not found in REST and Filter APIs.

    “Note that this choice is the safest as to date: in fact, it has been recently shown that biases affecting Sample API (which samples data based on rate limits), for instance, are not found in REST and Filter APIs 40 .”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [40] Eysenbach, G. How to fight an infodemic: the four pillars of infodemic management. J. Med. Internet Res. 22, e21820 (2020). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    The findings cast new light on the social mechanics of infodemics-epidemics interaction and may help policy makers design a more integrated strategic approach.

    “This casts new light on the social mechanics of the infodemics-epidemics interaction, and may be of help to policy makers to design a more integrated strategic approach, by suitably embedding communication and information management into a comprehensive, extended public health perspective.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Study design and conductown data

    An estimated 161.2 million tweets were posted until 10 Mar 2020, and 112.6 million were collected.

    “We have estimated a total of 161.2 millions tweets posted until 10 Mar 2020: we have successfully collected 112.6 millions of them, providing an unprecedented opportunity for infodemics analysis.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    Reliable and unreliable sources circulate in different ways and reach different targets.

    “however, they circulate in different ways and reach different targets, a feature that is perfectly captured by the infodemic risk index introduced in this study.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning

    The decrease in infodemic risk was especially pronounced as epidemics escalated, suggesting mediation by perceived social alarm.

    “The effect is particularly pronounced with the escalation of the epidemics, suggesting that this effect could be mediated by levels of perceived social alarm.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim.

    Study design and conductnone given

    When two or more domains are soft duplicates, the classification with the highest Harm Score is kept as a conservative choice.

    “When two or more domains are soft duplicates, we keep the classification with the highest Harm Score, as a conservative choice.”
    Quote in the paperSubstantial claim
46 of 46
  • The paper relies on this without support.

    Verified users are more authentic than unverified users because Twitter applies strict verification criteria.

    “In fact, verified users should be considered as more authentic than unverified ones, because Twitter makes use of strict criteria for verification.”
  • The paper relies on this without support.

    Twitter messages are a valid proxy for global human communication activity during the pandemic.

    “We focus our attention on the analysis of messages posted on a popular microblogging platform 30 , an online social network characterized by heterogeneous connectivity 31 and topological shortcuts typical of small-world systems 32 .”
  • The paper relies on this without support.

    The reliability of a URL can be inherited from the reliability of its source domain.

    “Each URL is therefore used for fact-checking, inheriting the reliability of its source (see Methods).”
  • The paper relies on this without support.

    The machine learning classifier used to distinguish human from bot accounts is accurate.

    “we have used an accurate machine learning approach to classify the author as human or non-human (i.e., bot)”
  • The authors state this as an assumption.

    Exposure can be approximated by the number of followers of the posting user at first order.

    “The number of followers K u of a single user u defines the exposure, in terms of potential visualizations at first-order approximation, of a single message m posted by user u at time t .”
  • The paper relies on this without support.

    The geolocated subset of tweets is representative enough of the full Twitter sample for estimating infodemic risk.

    “The fraction of Geoolocated (messages with shared locations, or geonamed, indicated in green) is constantly about 56% of the total volume recorded”
  • The authors state this as an assumption.

    Message reliability can be adequately represented as a binary value after discarding links that cannot be easily assessed.

    “The reliability r m of a single message m is either 0 or 1, because we discard all web links that can not be easily assessed”
7 of 7

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Bots are present in online cryptocurrency discussions and in the infodemics about the COVID-19 pandemic.

    “Similar results were obtained about the presence of bots in online cryptocurrency discussions [24] and as part of the 'infodemics' about the COVID-19 pandemic [14].”

    From A decade of social bot detection (2020) · cites this paper as [14]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Unknown source categories are also described as unreliable

    Minor

    The supplementary figure description classifies every category other than MSM and SCIENCE as unreliable, but the classification table labels OTHER and SHADOW as UNKNOWN.

1 of 1

Cites

Flagged first, then by citations

7 of the 61 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Gallotti, R., Valle, F., Castaldo, N., Sacco, P., & De Domenico, M.
Affiliations
Fondazione Bruno Kessler (IT)
Venue
Nature Human Behaviour
Published
2020-10-29
Volume
4
Issue
12
Pages
1285-1293
ISSN
2397-3374 (ISSN-L)
OpenAlex
W3015622078
Cited by
591
References
61
Citations by year
2026: 39 · 2025: 69 · 2024: 80 · 2023: 90 · 2022: 129 · 2021: 137 · 2020: 46 · 2018: 1
FWCI
134.67
Type
journal article
Language
en
Open access
Locations
  • Nature Human Behaviour (publishedVersion, public-domain)
  • medRxiv (acceptedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • Padua Research Archive (University of Padova) (submittedVersion, public-domain)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Social Sciences › Communication › Communication and COVID-19 Impact
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Complex Network Analysis Techniques
Keywords
infodemic, COVID-19 pandemic, misinformation, Twitter, news reliability, early warning indicators, epidemic spread, misinformation mitigation, public health communication, social media
MeSH terms
COVID-19, Humans, Mass Media, Models, Theoretical, Risk Assessment, Consumer Health Information, Social Networking, Social Media

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review