Influence of augmented humans in online interactions during voting events
Open accessAbstract
The advent of the digital era provided a fertile ground for the development of virtual societies, complex systems influencing real-world dynamics. Understanding online human behavior and its relevance beyond the digital boundaries is still an open challenge. Here we show that online social interactions during a massive voting event can be used to build an accurate map of real-world political parties and electoral ranks for Italian elections in 2018. We provide evidence that information flow and collective attention are often driven by a special class of highly influential users, that we name "augmented humans", who exploit thousands of automated agents, also known as bots, for enhancing their online influence. We show that augmented humans generate deep information cascades, to the same extent of news media and other broadcasters, while they uniformly infiltrate across the full range of identified groups. Digital augmentation represents the cyber-physical counterpart of the human desire to acquire power within social systems.
AI snapshot
Written from the paper’s full text.
TL;DR
Online Twitter interactions during the 2018 Italian elections reflected real-world political parties and electoral ranks, with interaction volume correlating strongly with vote share. A class of “augmented humans” used bots to gain influence comparable to media and political broadcasters, producing deep information cascades and infiltrating across social groups.
Key findings
- The social bulk identified 8 main communities: five mapped to Italian political ecosystems, two to news media, and one to a single augmented human plus bots; group interaction volume ranked with election outcome (Spearman ρ = 0.9, p = 0.039).
- Bot classification labeled 13.4% of users as bots; humans interacted with bots in 46% of social interactions overall (56% of mentions, 41% of replies, 43% of retweets), while bots interacted with humans in roughly 4%; bots contributed 6% of total social interactions, and 9 out of 10 hubs were bots.
- The social bulk was polarized (ΦF = 0.452), fragmented (ΦL = 0.812), and segregated (Σ = 0.476 vs random expectation 0.172, p < 10−5). Excluding the augmented-human community (97.9% bots), mean bot infiltration was 29.2% and mean augmented-human infiltration was 15.7%.
- Augmented humans were defined as human users with at least 50% bot neighbors in the social bulk; 1,010 accounts (12.7% of humans in the social bulk) were identified. The most central, User01, interacted with 2,700 bots and 55 humans; 67% of augmented humans were influentials, hidden influentials, or broadcasters. Cascades numbered 83,593, with sizes 2–4,313 and a power-law exponent γ = −2.33 ± 0.04.
Methods
- Collected 966,483 tweets from 194,273 users between 24 February and 7 March 2018 using Twitter’s streaming API, filtered by election-related keywords/hashtags; no Twitter warnings were received, so the authors report collecting 100% of tweets containing the specified keywords.
- Classified users as humans or bots with a deep neural network using 10 account features, trained and validated on 22,993 manually annotated accounts (14,218 bots, 8,775 humans) with an 80/20 split; random forest and deep neural network achieved >90% accuracy and >95% bot precision.
- Built interaction networks from retweets, mentions, and replies; constructed a “social bulk” by requiring at least one retweet plus either a mention or reply between user pairs. Analyzed polarization via Fiedler partitioning and modularity, fragmentation via Louvain modularity, segregation via connected components, infiltration by user class, and information cascades via retweet counts and cascade rate.
Limitations
- The paper states that Twitter metadata does not allow full reconstruction of information cascades because of missing intermediary retweets, so cascades were measured as star networks by total retweets received. It also states that further research is needed to generalize the correlation between online interaction volume and electoral outcomes and confirm predictive power.
- Evident from the text: the study is an observational, single-country, single-election case study, so causal claims about bots or augmented humans influencing real-world voting are not established. Bot and augmented-human classifications depend on the trained model and thresholds (e.g., augmented humans defined as ≥50% bot neighbors, users with fewer than 3 bulk interactions discarded), and the data are limited to keyword-filtered tweets collected through Twitter’s streaming API.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 31 cited sources checked · 0 supported · 0 flagged
- Findingown data
Online social interactions during a massive voting event can be used to build an accurate map of real-world political parties and electoral ranks for Italian elections in 2018.
“Here we show that online social interactions during a massive voting event can be used to build an accurate map of real-world political parties and electoral ranks for Italian elections in 2018.”
Quote in the paperSubstantial claim - Findingown data
Information flow and collective attention are often driven by a special class of highly influential users, named 'augmented humans', who exploit thousands of automated agents (bots) for enhancing their online influence.
“We provide evidence that information flow and collective attention are often driven by a special class of highly influential users, that we name “augmented humans”, who exploit thousands of automated agents, also known as bots, for enhancing their online influence.”
Quote in the paperSubstantial claim - Findingown data
Augmented humans generate deep information cascades, to the same extent as news media and other broadcasters.
“We show that augmented humans generate deep information cascades, to the same extent of news media and other broadcasters, while they uniformly infiltrate across the full range of identified groups.”
Quote in the paperSubstantial claim - Findingnone given
Augmented humans uniformly infiltrate across the full range of identified groups.
“We show that augmented humans generate deep information cascades, to the same extent of news media and other broadcasters, while they uniformly infiltrate across the full range of identified groups.”
Quote in the paperSubstantial claim - Findingown data
Bots interact with humans roughly in 4% of the interactions, independently on interaction type.
“Bots interact with humans roughly in 4% of the interactions, independently on interaction type.”
Quote in the paperSubstantial claim
- Findingown data
The deep neural network model provided a more stable classification of certain users playing the role of broadcasters.
“We chose the deep neural network model because it also provided a more stable classification of certain users playing the role of broadcasters (see Supplementary Information).”
Quote in the paperSubstantial claim - Findingown data
Most of the social interactions are from humans to bots (46%).
“Most of the social interactions are from humans to bots (46%);”
Quote in the paperSubstantial claim - Findingown data
Through the multilevel approach, 8 main opposing communities (each having more than 2% of the total nodes in the network) were identified in the social bulk.
“Through the multilevel approach, we identified 8 main opposing communities (i.e. having more than 2% of the total nodes in the network), as reported in Table 2.”
Quote in the paperNot a substantial claim - Findingown data
The expected polarization of a null model is significantly different from the observed network (p-value < 10−5).
“The expected polarization of a null model—where social relationships are uniformly randomized while preserving the individual degrees and the distribution of strengths—is , significantly different from the observed network (p-value < 10−5).”
Quote in the paperSubstantial claim- The authors state this as an assumption: If two users share similar political ideologies, they can endorse and subsequently share (i.e. retweet) the content of each other.
- Findingown data
Bot infiltration and augmented-human infiltration do not correlate with each other (Kendall Tau 0.07, p-value 0.8).
“Instead, bot and augmented human infiltration do not correlate with each other (Kendall Tau 0.07, p-value 0.8), indicating that augmented humans tend to interact selectively with the bots available in their groups rather than creating more.”
Quote in the paperSubstantial claim - Findingown data
On average, larger the number of followers larger the cascade size, with very few exceptions.
“As expected, on average, larger the number of followers larger the cascade size, with very few exceptions.”
Quote in the paperNot a substantial claim - Findingown data
For the largest connected component of the social bulk, the polarization ΦF = 0.452.
“For the largest connected component of the social bulk we calculate the polarization ΦF = 0.452.”
Quote in the paperSubstantial claim- The authors state this as an assumption: If two users share similar political ideologies, they can endorse and subsequently share (i.e. retweet) the content of each other.
- Findingown data
The infiltration of augmented humans is approximately uniformly distributed across all identified groups, with the exception of C8, the augmented human's community.
“The infiltration of augmented humans is approximately uniformly distributed across all identified groups, with the remarkable exception of C8, the augmented human’s community.”
Quote in the paperSubstantial claim - Findingown data
Infiltration of bots is higher in the group representing the augmented human and his/her automatic entourage of interacting social bots.
“Unsurprisingly, infiltration of bots is higher in the group representing the augmented human and his/her automatic entourage of interacting social bots.”
Quote in the paperNot a substantial claim - Findingown data
Bots are almost twice as central as humans.
“Furthermore, if we quantify the centrality of a user in terms of the probability of finding it by exploring the web of interactions at random, then we find that bots are almost twice as central as humans (see SI).”
Quote in the paperSubstantial claim - Findingown data
The most central augmented human by number of social interactions is User01, who interacts with 2,700 bots and 55 humans in the social bulk.
“The most central augmented human in terms of number of social interactions is User01 which interacts with 2,700 bots and 55 humans in the social bulk.”
Quote in the paperNot a substantial claim - Findingown data
We have tracked 83,593 information cascades during Italian elections.
“We have tracked 83,593 information cascades during Italian elections”
Quote in the paperSubstantial claim - Findingown data
The analysis of observed social interactions before, during and after the voting day revealed bot homophily, i.e., automated agents tend to interact more with other bots rather than with humans compared to random expectation.
“The analysis of observed social interactions (links) between users (nodes) before, during and after the voting day revealed bot homophily, i.e., automated agents tend to interact more with other bots rather than with humans compared to random expectation (see SI).”
Quote in the paperSubstantial claim - Findingown data
Bots contribute to 6% of the total number of social interactions occurred during the voting event (March 4 2018).
“In general, bots contribute to 6% of the total number of social interactions occurred during the voting event (March 4 2018).”
Quote in the paperNot a substantial claim - Findingown data
Cascade size ranges between 2 and 4,313.
“Cascade size ranges between 2 and 4,313.”
Quote in the paperNot a substantial claim - Findingown data
There is a positive correlation between cascade rate and size: Cascades involving more users tend also to flow over the interactions web at faster rates.
“Fig 2(c) (right) highlights a positive correlation between cascade rate and size: Cascades involving more users tend also to flow over the interactions web at faster rates.”
Quote in the paperSubstantial claim - Findingreasoning
The map of followers vs cascade growth rate identifies four categories of individuals: hidden influentials, influentials, broadcasters, common users.
“This map allows to easily identify four categories of individuals in the social dynamics”
Quote in the paperSubstantial claim - Findingreasoning
The positive trend between cascade rate and size is stronger for cascades of sizes larger than 102.
“This positive trend is stronger for cascades of sizes larger than 102.”
Quote in the paperSubstantial claim - Findingown data
1,010 user accounts (12.7% of humans in the social bulk) were systematically identified as augmented humans.
“We systematically identified 1,010 user accounts (12.7% of humans in the social bulk) corresponding to augmented humans.”
Quote in the paperSubstantial claim - Findingcitation
The social bulk analyzed here displays a higher modularity than other well studied social networks such as the Zachary Karate Club (ΦF = 0.371) and the dolphin's social network (ΦF = 0.401).
“Notice that the social bulk analyzed here displays a higher modularity than other well studied social networks such as the Zachary Karate Club (ΦF = 0.371) and the dolphin’s social network (ΦF = 0.401)(cf. [37]).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [37] M Newman, 2010, Networks: an introduction
Not checked in the free run.
- The authors state this as an assumption: If two users share similar political ideologies, they can endorse and subsequently share (i.e. retweet) the content of each other.
- Findingown data
Augmented humans played a special role for information spreading, by triggering deep information cascades with the help of bots.
“Like several automated agents identified in our data set, augmented humans played a special role for information spreading, by triggering deep information cascades with the help of bots.”
Quote in the paperSubstantial claim - Findingown data
Groups relative to news media are richer in bots compared to groups representing political parties.
“Furthermore, we find that groups relative to news media are richer in bots compared to groups representing political parties, which is compatible with our previous finding of bots being preferentially news media broadcaster in the observed data.”
Quote in the paperNot a substantial claim - Findingown data
The outcome of the Italian elections (fraction of votes received by each political group) strongly correlates with the group volume (Spearman rank correlation coefficient ρ = 0.9, p-value = 0.039).
“the outcome of Italian elections (i.e. the fraction of votes received by each political group) strongly correlates with the group volume (Spearman rank correlation coefficient ρ = 0.9, p-value = 0.039).”
Quote in the paperSubstantial claim - Findingown data
67% of augmented humans were either influentials, hidden influentials or broadcasters, i.e. almost two out of three played an important role in the flow of online content.
“The analysis of information cascades revealed that almost 2 out 3 augmented humans played an important role in the flow of online content: 67% of this class of users were either influentials or hidden influentials or broadcasters.”
Quote in the paperSubstantial claim - Premisecitation
Bots can produce credible content with human-like temporal patterns.
“Nowadays, bots can produce credible content with human-like temporal patterns [10, 11, 12].”
Quote in the paperSubstantial claim0 of 3 sources checked- cites [10] E Ferrara, 2016, The rise of social bots, Comm of the ACM source
Not checked in the free run.
- cites [11] E Ferrara, 2017, Disinformation and social bot operations in the run up to the 2017 French presidential election, First Monday source
Not checked in the free run.
- cites [12] M Stella, 2018, Bots increase exposure to negative and inflammatory content in online social systems, Proceedings of the National Academy of Sciences
Not checked in the free run.
- Study design and conductreasoning
The restrictive selection (requiring at least one retweet and another social action) allows one to filter out all spurious interactions among users.
“This restrictive selection allows one to filter out all spurious interactions among users with the advantage of not requiring any threshold with respect to the frequency of interactions themselves.”
Quote in the paperNot a substantial claim - Interpretationreasoning
Bots play the role of sinks for information flow.
“The above findings indicate that bots play the role of sinks for information flow.”
Quote in the paperSubstantial claim - Premisecitation
Manoeuvring social platforms can deeply affect real-world dynamics.
“Since manoeuvring social platforms can deeply affect real-world dynamics [14, 15]”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [14] S González-Bailón, 2011, The dynamics of protest recruitment through an online network, Scientific reports source
Not checked in the free run.
- cites [15] S González-Bailón, 2013, Broadcasters and hidden influentials in online protest diffusion, American Behavioral Scientist source
Not checked in the free run.
- Interpretationown data
Augmented humans tend to interact selectively with the bots available in their groups rather than creating more bots.
“indicating that augmented humans tend to interact selectively with the bots available in their groups rather than creating more.”
Quote in the paperSubstantial claim - Interpretationreasoning
The above comparison further highlights the polarization of social interactions on the social media platform.
“Considering that especially the Zachary Karate Club considers social interactions among two different groups, led by two different leaders and hence already highly polarized, then the above comparison further highlights the polarization of social interactions on the social media platform.”
Quote in the paperSubstantial claim- The authors state this as an assumption: If two users share similar political ideologies, they can endorse and subsequently share (i.e. retweet) the content of each other.
- Interpretationown data
Polarization, fragmentation and segregation analyses together constitute evidence that the social bulk displays densely connected groups in opposition with each other.
“Polarization, fragmentation and segregation analyses all constitute evidence that the social bulk displays densely connected groups in opposition with each other.”
Quote in the paperSubstantial claim - Premisecitation
By promoting online activity, bots can interact with humans and influence their standing against specific topics such as political issues.
“By promoting online activity, bots can interact with humans and influence their standing against specific topics such as political issues [10, 7, 12, 13].”
Quote in the paperSubstantial claim0 of 4 sources checked- cites [10] E Ferrara, 2016, The rise of social bots, Comm of the ACM source
Not checked in the free run.
- cites [7] Cresci S, Di Pietro R, Petrocchi M, Spognardi A, Tesconi M. The paradigm-shift of social spambots: Evidence, theories, and tools for the arms race. In: Proceedings of the 26th International Conference on World Wide Web Companion. International World Wide Web Conferences Steering Committee; 2017. p. 963–972. source
Not checked in the free run.
- cites [12] M Stella, 2018, Bots increase exposure to negative and inflammatory content in online social systems, Proceedings of the National Academy of Sciences
Not checked in the free run.
- cites [13] S Vosoughi, 2018, The spread of true and false news online, Science source
Not checked in the free run.
- Interpretationown data
The observed segregation is additional evidence for the presence of antagonism in the considered social ecosystem.
“Hence, the observed segregation represents additional evidence for the presence of antagonism in the considered social ecosystem.”
Quote in the paperSubstantial claim - Interpretationreasoning
Larger cascades contain specific semantic content, in this case political-related topics, which accelerate spreading.
“The stronger correlation for larger cascades suggests that they differ qualitatively from smaller cascades: Larger cascades contain specific semantic content, in this case political-related topics, which accelerate spreading.”
Quote in the paperSubstantial claim - Premisecitation
It is plausible to relate individuals' social influence to the size of information cascades they generate with their content.
“Here, we argue that it is also plausible to relate individuals’ social influence to the size of information cascades they generates with their content [35].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [35] Bakshy E, Hofman JM, Mason WA, Watts DJ. Everyone’s an influencer: quantifying influence on twitter. In: Proceedings of the fourth ACM international conference on Web search and data mining. ACM; 2011. p. 65–74. source
Not checked in the free run.
- Interpretationreasoning
A considerable amount of social influence was obtained by users that preferentially interacted with bots during the considered event.
“indicating that a considerable amount of social influence was obtained by users that preferentially interacted with bots during the considered event.”
Quote in the paperSubstantial claim - Interpretationown data
In online social platforms, the augmentation for achieving social influence is represented by an exploitation of social bots by human accounts.
“our work presents compelling evidence that in online social platforms the augmentation for achieving social influence is represented by an exploitation of social bots by human accounts.”
Quote in the paperSubstantial claim - Interpretationown data
Via augmentation, even common users can become social influencers without having a large social neighborhood, by recurring to armies of bots or the selection of a few key helping bots.
“Our results strongly support the idea that via augmentation even common users can become social influencers without having a large social neighborhood but rather by recurring to the aid of either armies of bots or the selection of a few key helping bots.”
Quote in the paperSubstantial claim - Interpretationown data
The strong correlation between the volume of online interactions in the social bulks and the electoral outcomes highlights the role potentially played by online social systems during the voting process.
“Furthermore, the strong correlation between the volume of online interactions in the social bulks and the electoral outcomes highlights the role potentially played by online social systems during the voting process.”
Quote in the paperSubstantial claim
- May combine two claims. 33
- Classified as probably not a substantial claim. 6
May combine two claims.
Findingown dataMost users are located in Italy, but there are significant fractions of human users also located in the United States and in Europe.
“Although most of the users are located in Italy, there are significant fractions of human users also located in the United States and in Europe”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe social fragmentation of the original system is stable across all phases of voting and significantly larger than random expectation, confirming that results from the social bulk are consistent.
“Once again, we observe that social fragmentation is stable across time and significantly larger than random expectation, confirming that results obtained from the social bulk are consistent.”
Quote in the paperSubstantial claimMay combine two claims.
FindingcitationThe deep neural network model classified 13.4% of users as bots, a value compatible to estimations during other voting events.
“To identify automated agents in the data set, we developed a deep neural network model (see Methods and SI), which classified 13.4% of users as bots, a value compatible to estimations during other voting events [10, 11, 12].”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites [10] E Ferrara, 2016, The rise of social bots, Comm of the ACM source
Not checked in the free run.
- cites [11] E Ferrara, 2017, Disinformation and social bot operations in the run up to the 2017 French presidential election, First Monday source
Not checked in the free run.
- cites [12] M Stella, 2018, Bots increase exposure to negative and inflammatory content in online social systems, Proceedings of the National Academy of Sciences
Not checked in the free run.
May combine two claims.
Findingown dataHumans tend to interact with bots in 56% of mentions, 41% of replies and 43% of retweets.
“Humans tend to interact with bots in 56% of mentions, 41% of replies and 43% of retweets.”
Quote in the paperSubstantial claimMay combine two claims.
Findingreasoning9 out of 10 hubs-i.e., highly interacting users-are bots and they are mainly news media and public profiles of politicians.
“In fact, 9 out of 10 hubs—i.e., highly interacting users—are bots and they are mainly news media and public profiles of politicians”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataDuring Italian elections, bots are mostly broadcasters (mostly media) and influentials (mostly political leaders).
“Remarkably, the topological and dynamical behavior of humans and bots is very different: during Italian elections, bots are mostly broadcasters (mostly media) and influentials (mostly political leaders).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe segregation of the social bulk is Σ = 0.476, significantly stronger than the random expectation 〈Σrand〉 = 0.172 (p-value < 10−5) from a configuration model preserving the connectivity distribution.
“The segregation of the social bulk is Σ = 0.476, significantly stronger than random expectation 〈Σrand〉 = 0.172 (p-value < 10−5) based on a configuration model preserving the connectivity distribution.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataDeeper information cascades are generated mostly by humans with a high number of followers, with the remarkable exception of one, User01, who produced the largest cascade among humans and bots despite having less than 100 followers.
“Deeper information cascades are generated mostly by humans with a high number of followers, with the remarkable exception of one, User01, who produced the largest cascade among humans and bots despite having less than 100 followers.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataExcluding the augmented human's community (97.9% bots), the mean bot infiltration in the bulk network is 29.2% and the mean augmented-human infiltration is 15.7%.
“Excluding the community corresponding to the augmented human (made for 97.9% of bots), the mean bot infiltration in the bulk network is 29.2% while the mean augmented infiltration is 15.7%.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataHub analysis of the social bulk groups shows that one group corresponds to a single augmented human and his/her bots, five groups map the ecosystems of the main Italian political parties, and two groups encode the news media universe (traditional or online).
“The analysis of hubs in each group of the social bulk indicates that i) one group corresponds to a single augmented human and his/her bots; ii) five groups directly map the ecosystems of the main Italian political parties; iii) two groups encode news media universe, either traditional or online news organisations.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataAt night the volume of messages generated by humans drops down (circadian rhythm).
“The circadian rhythm is evident, i.e. at night the volume of messages generated by humans drops down.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationBots display a circadian rhythm similar to humans.
“Also bots display a similar circadian rhythm, in agreement with previous observations [10, 12].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [10] E Ferrara, 2016, The rise of social bots, Comm of the ACM source
Not checked in the free run.
- cites [12] M Stella, 2018, Bots increase exposure to negative and inflammatory content in online social systems, Proceedings of the National Academy of Sciences
Not checked in the free run.
May combine two claims.
Findingown dataThe distribution of observed cascade sizes is heavy-tailed and compatible with a power law characterized by a scaling exponent γ = −2.33±0.04.
“As expected for complex networks with highly heterogeneous connectivity [27], the distribution of observed cascade sizes is heavy-tailed and compatible with a power law characterized by a scaling exponent γ = −2.33±0.04, similarly to size distribution in percolation theory or avalanches in self-organized criticality [32].”
Quote in the paperNot a substantial claim0 of 2 sources checkedMay combine two claims.
Findingown dataThe augmented human's group consists of more than 2,500 automated agents interacting with the augmented human user, producing a star-like structure for that community (network C8).
“First, we focused our attention on the augmented human’s group, consisting of more than 2,500 automated agents artificially interacting with the augmented human user. This peculiar activity leads to a star-like structure for the corresponding community, as shown in Fig 3c, network C8.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataBots act as broadcasters by repeating the same political content of human users, boosting the spread of hashtags related to the electoral process.
“The analysis of topic frequency and associations in bot-generated messages confirms this trend: Bots act as broadcasters by repeating the same political content of human users, boosting the spread of hashtags related to the electoral process (see SI).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe models based on random forest and deep neural network provided the highest accuracy (>90%) and precision in identifying bots (>95%).
“The models based on random forest and deep neural network provided us with the highest accuracy (>90%) and precision in identifying bots (>95%).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataIn the whole social bulk, the measured fragmentation is ΦL = 0.812, significantly larger than the null model expectation (p-value < 10−5), indicating several factions in stronger opposition than expected.
“In the whole social bulk we measure a fragmentation ΦL = 0.812, indicating the presence of several factions in the bulk network that are in a stronger opposition when compared to the null model (, p-value < 10−5).”
Quote in the paperSubstantial claimMay combine two claims.
Findingnone givenBots' locations are distributed worldwide and they are present in areas where no human users are geo-localized such as Morocco, Peru, Finland or Indonesia.
“Similarly, bots’ locations are distributed worldwide and they are present in areas where no human users are geo-localized such as Morocco, Peru, Finland or Indonesia.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAll the augmented humans identified in this study have, on average, less than 9,000 followers and 1,500 friends.
“All the augmented humans identified in this study have, on average, less than 9,000 followers and 1,500 friends, indicating that a considerable amount of social influence was obtained by users that preferentially interacted with bots during the considered event.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataRemoving all users in the two news media communities led to negligible fluctuations (around 0.02) in the fractions of human-bot interactions and in the total volume of tweets produced by bots (around 0.4%).
“The removal of news media accounts led to negligible fluctuations (around 0.02) in the fractions of human-bot interactions (cfr. Fig 1(a)) and in the total volume of tweets produced by bots (around 0.4%).”
Quote in the paperSubstantial claimMay combine two claims.
FindingcitationHidden influentials are mostly humans, but augmented humans also fall in this category (e.g. User01).
“Hidden influentials, known to be efficient spreaders in viral phenomena [44], are mostly humans but augmented humans also falls in this category (e.g. User01).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [44] RA Baños, 2013, The role of hidden influentials in the diffusion of online information cascades, EPJ Data Science source
Not checked in the free run.
May combine two claims.
PremisecitationMany large-scale studies have reported evidence of online ecosystems altering decision-making of crowds and influencing real-world voting of millions of people.
“Many large-scale studies have reported evidence of online ecosystems altering decision-making of crowds [3] and influencing real-world voting of millions of people [4].”
Quote in the paperSubstantial claim0 of 2 sources checkedMay combine two claims.
Study design and conductreasoningIn the case of Italian elections, no warnings were received, therefore 100% of tweets containing the specified keywords were collected.
“In the case of Italian elections, we received no warnings, therefore we have collected 100% of tweets containing the specified keywords.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataStrong interactions lead to more segregated components, with fewer bridges among connected components than expected from the heterogeneity of interactions alone.
“This indicates that strong interactions lead to more segregated components, with fewer bridges among connected components than expected from the heterogeneity of interactions only.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataThe heterogeneity of social interactions can not explain, alone, the observed level of polarization, which has rather to be attributed to other causes such as political parties or opposing political trends.
“This result indicates that the heterogeneity of social interactions can not explain, alone, the observed level of polarization, which has rather to be attributed to other causes such as political parties or opposing political trends.”
Quote in the paperSubstantial claim- The authors state this as an assumption: If two users share similar political ideologies, they can endorse and subsequently share (i.e. retweet) the content of each other.
May combine two claims.
Interpretationown dataBecause ΦL > ΦF, the mesoscale organization of the social bulk is more accurately described when more than two groups are considered, consistent with the expectation that results reflect the real-world socio-political scenario.
“The fact that ΦL > ΦF indicates that a more accurate description of the mesoscale organization of the social bulk is given when more than two groups are considered, in agreement with our hypothesis that results should reflect the real world socio-political scenario.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationRetweets are traditionally considered as a form of social endorsement: Users tend to retweet and thus endorse content they agree with.
“While Mentions and Replies can have both negative and positive connotations, Retweets are traditionally considered as a form of social endorsement [26, 17]: Users tend to retweet and thus endorse content they agree with.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [26] Metaxas PT, Mustafaraj E, Wong K, Zeng L, O’Keefe M, Finn S. What Do Retweets Indicate? Results from User Survey and Meta-Review of Research. In: ICWSM; 2015. p. 658–661. source
Not checked in the free run.
- cites [17] S Aral, 2012, Identifying influential and susceptible members of social networks, Science source
Not checked in the free run.
May combine two claims.
Interpretationown dataBots play a passive role in the network but are rather highly targeted by humans.
“This indicates that bots play a passive role in the network but are rather highly targeted by humans.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningBy construction, information flows among users who share strong social relationships and are characterized by similar ideologies.
“By construction, information flows among users who share strong social relationships and are characterized by similar ideologies: In fact, when a retweet goes from one user to another one, both of them are endorsing the same content, thus making non-directionality a viable approach for representing the endorsement related to content sharing.”
Quote in the paperSubstantial claim- The authors state this as an assumption: If two users share similar political ideologies, they can endorse and subsequently share (i.e. retweet) the content of each other.
May combine two claims.
InterpretationreasoningBots share messages mainly with each other and hence can resonate with the same content, be it news or spam.
“Since interactions encode content spread [16], this result indicates that bots share messages mainly with each other and hence can resonate with the same content, be it news or spam.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [16] S Aral, 2009, Distinguishing influence-based contagion from homophily-driven diffusion in dynamic networks, Proceedings of the National Academy of Sciences source
Not checked in the free run.
May combine two claims.
PremisecitationNews media and public profiles of politicians usually act as broadcasters and drive online information flow.
“which usually act as broadcasters and drive online information flow [17, 15].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [17] S Aral, 2012, Identifying influential and susceptible members of social networks, Science source
Not checked in the free run.
- cites [15] S González-Bailón, 2013, Broadcasters and hidden influentials in online protest diffusion, American Behavioral Scientist source
Not checked in the free run.
May combine two claims.
Study design and conductown dataThe correlation is statistically significant at the 5% level, and the p-value was computed by direct sampling of rankings without relying on assumptions about the large-scale statistical properties of the data.
“This correlation is statistically significant within a 5% significance level and direct sampling of rankings was used in order to compute the p-value without relying on any assumption about the large-scale statistical properties of the data.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataFor this case study, the volume of online interactions closely mirrors the election outcome, though generalizing this result and confirming its predictive power requires further research.
“The strong correlation found indicates that the volume of online interactions closely mirrors the election outcome for this case study, although further research is needed to generalize this result and confirm its predictive power.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataA prominent amount of human-bot interactions does not involve news media accounts and is not influenced by the presence of information hubs.
“These results indicate that a prominent amount of human-bot interactions does not involve news media accounts and it is not influenced by the presence of information hubs.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataDigital augmentation is a new phenomenon, identified and quantified in this work, characterizing individuals that coordinate from hundreds to thousands of social bots to achieve social influence comparable to that of political parties and news media organisations, with serious real-world repercussions.
“In this work we have identified and quantified a new phenomenon, i.e. digital augmentation, to characterize individuals that coordinate from hundreds to thousands of social bots for achieving a social influence comparable to the one of political parties and news media organisations, with serious repercussions in the real-world.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable).
Interpretationnone givenThis work provides a first step towards a more systematic quantification of the impact of digital augmentation in opinion formation and the manipulation of online attention by means of human-bot interactions.
“Our work provides a first step towards a more systematic quantification of the impact of digital augmentation in opinion formation and the manipulation of online attention by means of human-bot interactions.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationOnline ecosystems acted upon society by altering the emotions and beliefs of large populations of individuals.
“This finding is in full agreement with previous works showing how online ecosystems acted upon society by altering the emotions [19] and beliefs [3, 4] of large populations of individuals.”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites [19] M Bruter, 2017, Understanding the emotional act of voting, Nature Human Behaviour
Not checked in the free run.
- cites [3] L Muchnik, 2013, Social influence bias: A randomized experiment, Science source
Not checked in the free run.
- cites [4] RM Bond, 2012, A 61-million-person experiment in social influence and political mobilization, Nature source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationAn individual is much more likely to adopt a content when his/her neighbors in the social network tend to reinforce it.
“The analysis of information and behavioral spreading on social media [17] revealed that an individual is much more likely to adopt a content when his/her neighbors in the social network tend to reinforce it [48].”
Quote in the paperNot a substantial claim0 of 2 sources checked
The authors state this as an assumption.
If two users share similar political ideologies, they can endorse and subsequently share (i.e. retweet) the content of each other.
“We identified relevant interactions by assuming that if two users share similar political ideologies, they can endorse and subsequently share (i.e. retweet) the content of each other.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
It has been proposed that at population level, dynamics of multiple interacting contagions are indistinguishable from social reinforcement.
“Recently, it has been suggested that this challenging phenomenon might be due to the fact that, at population level, the dynamics of multiple interacting contagions are indistinguishable from social reinforcement 38 .”
From Assessing the risks of ‘infodemics’ in response to COVID-19 epidemics (2020) · cites this paper as [38]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Human-user total omits the aboutme dataset
MinorThe human counts in the table sum to 11,238, not 8,775. The reported total omits the 2,463 humans in the aboutme row, which also makes the grand total too low.
Reported 8775 · recomputed 11238
“| aboutme | 0 | 2463 | 2463 | | omnibots | 3530 | 0 | 3530 | | russian-trolls | 389 | 0 | 389 | | TOTAL | 14218 | 8775 | 22993 |”
Figure 1 panel (e) is assigned two different contents
MinorThe results text identifies panel (e) as the time series, while the figure caption identifies panel (e) as the geographic map.
“Fig 1(e) shows the number of social interactions over time.”
“(e): Geographic location of involved users, where the color encodes the number of tweets per country, in logarithmic scale.”
Cites
Fast unfolding of communities in large networks
Blondel, V. D., Guillaume, J.-L., Lambiotte, R., & Lefebvre, E. (2008) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/2008/10/p10008 · Cited by 21,424 · cited 1× · see citing works
“Here, we use the Louvain multilevel approach, known to be very efficient on large-scale networks [32].”
Modularity and community structure in networks
Newman, M. E. J. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0601602103 · Cited by 12,256 · cited 1× · see citing works
“We quantify polarization by computing the modularity [28] of the social bulk with respect to its Fiedler partitioning.”
Fortunato, S. (2010) · Physics Reports · doi:10.1016/j.physrep.2009.11.002 · Cited by 11,293 · see citing works
The spread of true and false news online
Vosoughi, S., Roy, D., & Aral, S. (2018) · Science · doi:10.1126/science.aap9559 · Cited by 8,847 · cited 3× · see citing works
“By promoting online activity, bots can interact with humans and influence their standing against specific topics such as political issues [7, 10, 12, 13].”
BackgroundNewman, M. (2010) · doi:10.1093/acprof:oso/9780199206650.001.0001 · Cited by 6,595 · see citing works
Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabási, A.-L., Brewer, D., Christakis, N., Contractor, N., Fowler, J., Gutmann, M., et al. (2009) · Science · doi:10.1126/science.1167742 · Cited by 3,365 · see citing works
The spread of behavior in an online social network experiment
Centola, D. (2010) · Science · doi:10.1126/science.1185231 · Cited by 2,692 · cited 1× · see citing works
“his/her neighbors in the social network tend to reinforce it [45].”
BackgroundA simple model of global cascades on random networks
Watts, D. J. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.082090499 · Cited by 2,673 · cited 2× · see citing works
“As expected for complex networks with highly heterogeneous connectivity [37], the distribution of observed cascade sizes is heavy-tailed and compatible with a power law characterized by a scaling exponent γ = −2.”
A 61-million-person experiment in social influence and political mobilization
Bond, R. M., Fariss, C. J., Jones, J. J., Kramer, A. D. I., Marlow, C., Settle, J. E., & Fowler, J. H. (2012) · Nature · doi:10.1038/nature11421 · Cited by 2,575 · cited 4× · see citing works
“This finding is in full agreement with previous works showing how online ecosystems acted upon society by altering the emotions [19] and beliefs [3, 4] of large populations of individuals.”
InfluentialModels of core/periphery structures
Borgatti, S. P., & Everett, M. G. (2000) · Social Networks · doi:10.1016/s0378-8733(99)00019-2 · Cited by 2,248 · cited 1× · see citing works
“Note that we are not referring to users in the periphery [33, 34] of the system, where information can slowly but flawlessly flow among all nodes in the network.”
Critical phenomena in complex networks
Dorogovtsev, S. N., Goltsev, A. V., & Mendes, J. F. F. (2008) · Reviews of Modern Physics · doi:10.1103/revmodphys.80.1275 · Cited by 2,220 · cited 1× · see citing works
“04, similarly to size distribution in percolation theory or avalanches in self-organized criticality [42].”
Community detection in networks: a user guide
Fortunato, S., & Hric, D. (2016) · Physics Reports · doi:10.1016/j.physrep.2016.09.002 · Cited by 2,086 · see citing works
Ferrara, E., Varol, O., Davis, C., Menczer, F., & Flammini, A. (2016) · Communications of the ACM · doi:10.1145/2818717 · Cited by 1,973 · cited 2× · see citing works
“By promoting online activity, bots can interact with humans and influence their standing against specific topics such as political issues [7, 10, 12, 13].”
Everyone's an influencer: quantifying influence on twitter
Bakshy, E., Hofman, J. M., Mason, W. A., & Watts, D. J. (2011) · Proceedings of the fourth ACM international conference on Web search and data mining · doi:10.1145/1935826.1935845 · Cited by 1,689 · see citing works
Political Polarization on Twitter
Conover, M., Ratkiewicz, J., Francisco, M., Goncalves, B., Flammini, A., & Menczer, F. (2011) · Zenodo · doi:10.5281/zenodo.4589064 · Cited by 1,254 · see citing works
Distinguishing influence-based contagion from homophily-driven diffusion in dynamic networks
Aral, S., Muchnik, L., & Sundararajan, A. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0908800106 · Cited by 1,207 · see citing works
Identifying influential and susceptible members of social networks
Aral, S., & Walker, D. (2012) · Science · doi:10.1126/science.1215842 · Cited by 1,047 · cited 4× · see citing works
“The analysis of information and behavioral spreading on social media [17] revealed that an individual is much more likely to adopt a content when”
BackgroundInfluentialOnline Human-Bot Interactions: Detection, Estimation, and Characterization
Varol, O., Ferrara, E., Davis, C., Menczer, F., & Flammini, A. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14871 · Cited by 928 · cited 1× · see citing works
“Nowadays, bots can produce credible content with human-like temporal patterns [10–12].”
Social media competitive analysis and text mining: A case study in the pizza industry
He, W., Zha, S., & Li, L. (2013) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2013.01.001 · Cited by 920 · see citing works
Social Influence Bias: A Randomized Experiment
Muchnik, L., Aral, S., & Taylor, S. J. (2013) · Science · doi:10.1126/science.1240466 · Cited by 871 · cited 1× · see citing works
“Many large-scale studies have reported evidence of online ecosystems altering decision-making of crowds [3] and influencing real-world voting of millions of people [4].”
A min-max cut algorithm for graph partitioning and data clustering
Ding, C. H. Q., He, X., Hongyuan Zha, B., Gu, M., & Simon, H. D. (2002) · Proceedings · doi:10.1109/icdm.2001.989507 · Cited by 850 · cited 2× · see citing works
“In presence of two groups, they can be identified by calculating, for instance, the Fiedler partitioning [12, 33], which is related to the min-max cut problem for finding optimal flows in networks [33].”
Communities, modules and large-scale structure in networks
Newman, M. E. J. (2012) · Nature Physics · doi:10.1038/nphys2162 · Cited by 787 · see citing works
Social media for large studies of behavior
Ruths, D., & Pfeffer, J. (2014) · Science · doi:10.1126/science.346.6213.1063 · Cited by 648 · cited 1× · see citing works
“Online social systems and the information they continuously generate provide an invaluable resource for computational social scientists and their large-scale analysis of human behavior [32, 42, 43] and the emergence of collective attention [2, 44].”
Rapid assessment of disaster damage using social media activity
Kryvasheyeu, Y., Chen, H., Obradovich, N., Moro, E., Hentenryck, P. V., Fowler, J. H., & Cebrián, M. (2016) · Science Advances · doi:10.1126/sciadv.1500779 · Cited by 642 · cited 1× · see citing works
“On the one hand, this allows online media to facilitate, for instance, the dissemination of emergency information and help coordinate relief efforts [46].”
The dynamics of protest recruitment through an online network
González-Bailón, S., Borge-Holthoefer, J., Rivero, A., & Moreno, Y. (2011) · Scientific Reports · doi:10.1038/srep00197 · Cited by 571 · cited 1× · see citing works
“Since ma-noeuvring social platforms can deeply affect real-world dynamics [14, 15], understanding if and how computer-generated activities can alter the behavioral responses of humans to achieve online social manipulation is of ut-most importance [5, 16, 17].”
Disinformation and social bot operations in the run up to the 2017 French presidential election
Ferrara, E. (2017) · First Monday · doi:10.5210/fm.v22i8.8005 · Cited by 524 · cited 1× · see citing works
“…of users in our data set as “humans” or “bots” is based on features providing the best classification accuracy according to recent studies [11] Searching for better performance we tested different machine learning techniques on an independent dataset created ad-hoc (see Supplementary…”
Chu, Z., Gianvecchio, S., Wang, H., & Jajodia, S. (2010) · Annual Computer Security Applications Conference · doi:10.1145/1920261.1920265 · Cited by 498 · cited 1× · see citing works
“The term cyborg has been used in this context to identify, indistinctly, bot-assisted human or human-assisted bot accounts generating spam content over social platforms such as Twitter [5, 20].”
Bots increase exposure to negative and inflammatory content in online social systems
Stella, M., Ferrara, E., & De Domenico, M. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1803470115 · Cited by 492 · see citing works
Self-citationThe Dynamics of Protest Recruitment through an Online Network
González‐Bailón, S., Borge‐Holthoefer, J., Rivero, A., & Moreno, Y. (2011) · ScholarlyCommons (University of Pennsylvania) · Cited by 466 · see citing works
Fame for sale: Efficient detection of fake Twitter followers
Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., & Tesconi, M. (2015) · Decision Support Systems · doi:10.1016/j.dss.2015.09.003 · Cited by 461 · cited 1× · see citing works
“[4, 6, 16, 27] and during voting events [19].”
The structure of online diffusion networks
Goel, S., Watts, D., & Goldstein, D. G. (2012) · ACM Conference on Economics and Computation · doi:10.1145/2229012.2229058 · Cited by 412 · cited 1× · see citing works
“In information cascades, a single piece of information is originated by a seed user, it is endorsed by other users in his/her neighborhood and consequently re-shared across the network [39].”
The Paradigm-Shift of Social Spambots: Evidence, Theories, and Tools for the Arms Race
Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., & Tesconi, M. (2017) · Proceedings of the 26th International Conference on World Wide Web Companion - WWW '17 Companion · doi:10.1145/3041021.3055135 · Cited by 381 · cited 2× · see citing works
“By promoting online activity, bots can interact with humans and influence their standing against specific topics such as political issues [7, 10, 12, 13].”
BackgroundBroadcasters and hidden influentials in online protest diffusion
González-Bailón, S., Borge-Holthoefer, J., & Moreno, Y. (2013) · American Behavioral Scientist · doi:10.1177/0002764213479371 · Cited by 327 · cited 4× · see citing works
“Recently, dynamical activity-connectivity maps based on network and temporal activity patterns – or their variation – have been used to identify influential individuals or broadcasters during online protest diffusion [15] and contagion dynamics of extremist propaganda [29].”
InfluentialThe Anatomy of a Scientific Rumor
Domenico, M. D., Lima, A., Mougel, P.-N., & Musolesi, M. (2013) · Scientific Reports · doi:10.1038/srep02980 · Cited by 285 · cited 1× · see citing works
“Online social actions drive collective attention and dynamics [1, 2], having a deep impact on the construction and perception of social reality.”
Self-citationThe paradigm-shift of social spambots: Evidence, theories, and tools for the arms race
Cresci, S., Pietro, R. D., Petrocchi, M., Spognardi, A., & Tesconi, M. (2017) · Technical University of Denmark, DTU Orbit (Technical University of Denmark, DTU) · doi:10.48550/arxiv.1701.03017 · Cited by 237 · cited 2× · see citing works
“By promoting online activity, bots can interact with humans and influence their standing against specific topics such as political issues [7, 10, 12, 13].”
BackgroundThe Anatomy of a Scientific Rumor
Domenico, M. D., Lima, A., Mougel, P.-N., & Musolesi, M. (2013) · UCL Discovery (University College London) · Cited by 205 · see citing works
Self-citationLearning About Voter Rationality
Ashworth, S., de Mesquita, E. B., & Friedenberg, A. (2018) · American Journal of Political Science · doi:10.1111/ajps.12334 · Cited by 189 · cited 1× · see citing works
“[18] show that even rational voters can take electoral decisions based on events external to the voting event, such as natural disasters or economic shocks.”
What Do Retweets Indicate? Results from User Survey and Meta-Review of Research
Metaxas, P., Mustafaraj, E., Wong, K., Zeng, L., O'Keefe, M., & Finn, S. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v9i1.14661 · Cited by 162 · cited 3× · see citing works
“[22] Panagiotis Takis Metaxas, Eni Mustafaraj, Kily Wong, Laura Zeng, Megan O’Keefe, and Samantha Finn.”
BackgroundMethodInfluentialThe emergence of consensus: a primer
Baronchelli, A. (2018) · Royal Society Open Science · doi:10.1098/rsos.172189 · Cited by 161 · cited 1× · see citing works
“Online social systems and the information they continuously generate provide an invaluable resource for computational social scientists and their large-scale analysis of human behavior [32, 42, 43] and the emergence of collective attention [2, 44].”
Core-Periphery Structure in Networks (Revisited)
Rombach, P., Porter, M. A., Fowler, J. H., & Mucha, P. J. (2017) · SIAM Review · doi:10.1137/17m1130046 · Cited by 145 · cited 1× · see citing works
“Note that we are not referring to users in the periphery [33, 34] of the system, where information can slowly but flawlessly flow among all nodes in the network.”
Exploring Limits to Prediction in Complex Social Systems
Martin, T., Hofman, J. M., Sharma, A. K., Anderson, A., & Watts, D. J. (2016) · International World Wide Web Conference (WWW) · doi:10.1145/2872427.2883001 · Cited by 142 · cited 2× · see citing works
“Such systems are well known for being susceptible to cascading events [23, 24] and, in the case of online social networks, the phenomenon might manifest as collective action and faster diffusion of specific information [1, 2, 25, 26].”
When social bots attack: Modeling susceptibility of users in online social networks
Wagner, C., Mitter, S., Körner, C., & Strohmaier, M. (2012) · MADOC (University of Mannheim) · Cited by 137 · see citing works
Effects of network structure, competition and memory time on social spreading phenomena
Gleeson, J. P., O’Sullivan, K. P., Banos, R., & Moreno, Y. (2016) · Arrow@dit (Dublin Institute of Technology) · Cited by 130 · see citing works
The dynamics of information-driven coordination phenomena: A transfer entropy analysis
Borge‐Holthoefer, J., Perra, N., Gonçalves, B., González-Bailón, S., Arenas, À., Moreno, Y., & Vespignani, A. (2016) · Science Advances · doi:10.1126/sciadv.1501158 · Cited by 110 · see citing works
Classification of Twitter Accounts into Automated Agents and Human Users
Gilani, Z., Kochmar, E., & Crowcroft, J. (2017) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1145/3110025.3110091 · Cited by 102 · see citing works
Contagion dynamics of extremist propaganda in social networks
Ferrara, E. (2017) · Information Sciences · doi:10.1016/j.ins.2017.07.030 · Cited by 96 · cited 1× · see citing works
“Recently, dynamical activity-connectivity maps based on network and temporal activity patterns – or their variation – have been used to identify influential individuals or broadcasters during online protest diffusion [15] and contagion dynamics of extremist propaganda [29].”
Serial activists: Political Twitter beyond influentials and the twittertariat
Bastos, M., & Mercea, D. (2015) · New Media & Society · doi:10.1177/1461444815584764 · Cited by 91 · see citing works
Effects of Network Structure, Competition and Memory Time on Social Spreading Phenomena
Gleeson, J. P., O’Sullivan, K. P., Baños, R. A., & Moreno, Y. (2016) · Physical Review X · doi:10.1103/physrevx.6.021019 · Cited by 91 · see citing works
The role of hidden influentials in the diffusion of online information cascades
Baños, R. A., Borge-Holthoefer, J., & Moreno, Y. (2013) · EPJ Data Science · doi:10.1140/epjds18 · Cited by 79 · cited 1× · see citing works
“Hidden influentials, known to be efficient spreaders in viral phenomena [41], are mostly humans but augmented humans also falls in this category (e.g. User01).”
Temporal profiles of avalanches on networks
Gleeson, J. P., & Durrett, R. (2017) · Nature Communications · doi:10.1038/s41467-017-01212-0 · Cited by 50 · cited 1× · see citing works
“Such systems are well known for being susceptible to cascading events [37, 38] and, in the case of online social networks, the phenomenon might manifest as collection action and faster diffusion of specific information.”
Understanding the emotional act of voting
Bruter, M., & Harrison, S. J. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0024 · Cited by 26 · cited 2× · see citing works
“Bruter and Harrison [19] shift the focus on the psychological influence that electoral arrangements exert on voters by altering their emotions and behavior.”
Data Analytics Applications for Streaming Data From Social Media: What to Predict?
Emmert‐Streib, F., Yli-Harja, O. P., & Dehmer, M. (2018) · Frontiers in Big Data · doi:10.3389/fdata.2018.00002 · Cited by 13 · see citing works
Flagged first, then by citations
14 of the 66 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Assessing the risks of ‘infodemics’ in response to COVID-19 epidemics
Gallotti, R., Valle, F., Castaldo, N., Sacco, P., & De Domenico, M. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-00994-6 · Cited by 591 · see citing works
Self-citationA new sociology of humans and machines
Tsvetkova, M., Yasseri, T., Pescetelli, N., & Werner, T. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02001-8 · Cited by 98 · see citing works
“Bots’ large numbers enhance their visibility and influence to trigger deep information cascades 217 .”
Bots are Less Central than Verified Accounts during Contentious Political Events
González‐Bailón, S., & Domenico, M. D. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2013443118 · Cited by 69 · see citing works
“, to trigger collective attention (9, 10), gain status (11, 12), and monetize public attention (13)] or with malicious intent [e.”
BackgroundSelf-citationSpotting Political Social Bots in Twitter: A Use Case of the 2019 Spanish General Election
Pastor-Galindo, J., Zago, M., Nespoli, P., Bernal, S. L., Celdrán, A., Pérez, M., Ruipérez-Valiente, J. A., Pérez, G., & Mármol, F. G. (2020) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2020.3031573 · Cited by 48 · see citing works
“Slowly, year by year, bots can build more realistic social media behaviors and produce credible content with human-like temporal patterns [9], while creating a coordinated network to spread the forged”
BackgroundFlow of online misinformation during the peak of the COVID-19 pandemic in Italy
Caldarelli, G., Nicola, R. D., Petrocchi, M., Pratelli, M., & Saracco, F. (2021) · EPJ Data Science · doi:10.1140/epjds/s13688-021-00289-4 · Cited by 54 · see citing works
Evolution of bot and human behavior during elections
Luceri, L., Deb, A., Giordano, S., & Ferrara, E. (2019) · First Monday · doi:10.5210/fm.v24i9.10213 · Cited by 56 · see citing works
“Researchers brought to the table a variety of solutions for the detection of bots (Chavoshi et al., 2016; Subrahmanian et al., 2016; Chen & Subramanian, 2018).”
BackgroundEphemeral Astroturfing Attacks: The Case of Fake Twitter Trends
Elmas, T., Overdorf, R., Özkalay, A. F., & Aberer, K. (2021) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp51992.2021.00035 · Cited by 50 · see citing works
Castioni, P., Andrighetto, G., Gallotti, R., di Sorrentino, E. P., & Domenico, M. D. (2022) · Royal Society Open Science · doi:10.1098/rsos.220716 · Cited by 43 · see citing works
“We also restricted our analysis to those users classified as real people, and not bots, using the machine learning methodology described in [25].”
MethodSelf-citationDetecting bots in social-networks using node and structural embeddings
Dehghan, A., Siuta, K., Skorupka, A., Dubey, A., Betlen, A., Miller, D., Xu, W., Kamiński, B., & Prałat, P. (2023) · Journal Of Big Data · doi:10.1186/s40537-023-00796-3 · Cited by 42 · see citing works
Studying the COVID-19 infodemic at scale
Gruzd, A. A., Domenico, M. D., Sacco, P. L., & Briand, S. C. (2021) · Big Data & Society · doi:10.1177/20539517211021115 · Cited by 36 · see citing works
“…the use of automated or semi-automated social media accounts to spread misinfor-mation, also known as social bots (Bessi and Ferrara, 2016; Gonza´lez-Bail (cid:1) on and De Domenico, 2021; Shao et al., 2018; Stella et al., 2018, 2019) has substantially contributed to the scale of this problem.”
BackgroundSelf-citationItalian Twitter semantic network during the Covid-19 epidemic
Mattei, M., Caldarelli, G., Squartini, T., & Saracco, F. (2021) · EPJ Data Science · doi:10.1140/epjds/s13688-021-00301-x · Cited by 28 · see citing works
Understanding anti-immigration sentiment spreading on Twitter
Nasuto, A., & Rowe, F. (2024) · PLoS ONE · doi:10.1371/journal.pone.0307917 · Cited by 30 · see citing works
Sustainable development goals as unifying narratives in large UK firms’ Twitter discussions
Patuelli, A., & Saracco, F. (2023) · Scientific Reports · doi:10.1038/s41598-023-34024-y · Cited by 26 · see citing works
Unraveling the hidden organisation of urban systems and their mobility flows
Gallotti, R., Bertagnolli, G., & Domenico, M. D. (2021) · EPJ Data Science · doi:10.1140/epjds/s13688-020-00258-3 · Cited by 19 · see citing works
Self-citationShould we agree to disagree about Twitter’s bot problem?
Varol, O. (2023) · Online Social Networks and Media · doi:10.1016/j.osnem.2023.100263 · Cited by 20 · see citing works
Quantifying the vulnerabilities of the online public square to adversarial manipulation tactics
Truong, B. T., Lou, X., Flammini, A., & Menczer, F. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae258 · Cited by 18 · see citing works
Monakhov, S. (2020) · PLoS ONE · doi:10.1371/journal.pone.0236832 · Cited by 16 · see citing works
Schuchard, R., & Crooks, A. T. (2021) · PLoS ONE · doi:10.1371/journal.pone.0244309 · Cited by 14 · see citing works
Bonacchi, C. (2022) · UCL Press eBooks · doi:10.2307/j.ctv1wdvx2p · Cited by 14 · see citing works
Bonacchi, C. (2022) · UCL Press eBooks · doi:10.14324/111.9781787358010 · Cited by 27 · see citing works
Artificial intelligence in hybrid and information warfare
Moy, W. R., & Gradoń, K. (2023) · Routledge eBooks · doi:10.4324/9781003284093-4 · Cited by 12 · see citing works
The Diffusion of Mainstream and Disinformation News on Twitter: The Case of Italy and France
Pierri, F. (2020) · International World Wide Web Conference (WWW) · doi:10.1145/3366424.3385776 · Cited by 12 · see citing works
“trolls and bots)–has been reported in several political circumstances including 2016 Brexit referendum [4] and 2017 Catalan referendum [24], but also 2017 French Presidential elections [8, 11], 2018 Italian General elections [6, 9, 23] and 2019 European Parliament elections [16].”
BackgroundLiars and Trolls and Bots Online: The Problem of Fake Persons
Harris, K. R. (2023) · Philosophy & Technology · doi:10.1007/s13347-023-00640-9 · Cited by 11 · see citing works
“In other cases, individual actors may employ bots to amplify their online presences (Stella et al., 2019).”
BackgroundBots are Less Central than Verified Accounts during Contentious Political EventsPreprint
González‐Bailón, S., & Domenico, M. D. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3637121 · Cited by 11 · see citing works
Self-citationSocial bots spoil activist sentiment without eroding engagement
Li, L., Vásárhelyi, O., & Vedres, B. (2024) · Scientific Reports · doi:10.1038/s41598-024-74032-0 · Cited by 16 · see citing works
Av, D., Dmitriev, V., & Balybin, S. N. (2019) · Complexity · doi:10.1155/2019/8750643 · Cited by 10 · see citing works
“Stella and co-authors [69] detect power-law relationships between cascade rate and size on Twitter during a voting event and they show how social bots used by human users were capable of creating avalanches of microposts.”
BackgroundHow do Twitter users react to TV broadcasts dedicated to vaccines in Italy?
Gesualdo, F., D’Ambrosio, A., Agricola, E., et al. (2020) · European Journal of Public Health · doi:10.1093/eurpub/ckaa022 · Cited by 9 · see citing works
Self-citationEmpowering NGOs in countering online hate messages
Chung, Y.-L., Tekiroğlu, S. S., Tonelli, S., & Guerini, M. (2021) · Online Social Networks and Media · doi:10.1016/j.osnem.2021.100150 · Cited by 11 · see citing works
“We therefore propose in this work a novel online platform that enhances NGO operators’ online activities (similar to the concept of ‘digital augmenta-tion’ presented in [49]), by automatizing many tasks in monitoring and counter-narratives writing.”
BackgroundBi-layer voter model: modeling intolerant/tolerant positions and bots in opinion dynamics
Vega‐Oliveros, D. A., Grande, H. L. C., Iannelli, F., et al. (2021) · The European Physical Journal Special Topics · doi:10.1140/epjs/s11734-021-00151-8 · Cited by 8 · see citing works
Cervera, R. B., Pérez‐Solà, C., & Batlle, A. (2024) · Social Network Analysis and Mining · doi:10.1007/s13278-024-01251-8 · Cited by 5 · see citing works
“This type of graph is widely used on the literature (Stella et al. 2019; Pastor-Galindo et al. 2020), and Table 4 shows a summary of its general metrics, where “average degree” represents the average number of edges that go in/out on graph nodes (considering the weight of edges): After the graph…”
Ability, Knowledge, and Non-paradigmatic Testimony
Harris, K. R. (2023) · Episteme · doi:10.1017/epi.2023.2 · Cited by 3 · see citing works
“This latter strategy may involve the establishment of networks of (partially) automated accounts (Stella et al .”
Manipulating the Online Marketplace of IdeasPreprint
Lou, X., Flammini, A., & Menczer, F. (2019) · arXiv (Cornell University) · Cited by 3 · see citing works
Down the bot hole: actionable insights from a 1-year analysis of bots activity on TwitterPreprint
Luceri, L., Cardoso, F. M., & Giordano, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.15820 · Cited by 2 · see citing works
“the abuse of social platforms has put under threat the effective fulfillment of the democratic process, other than creating worldwide concerns for the integrity of voting events [47, 7, 49, 3, 40, 19, 28, 46, 54, 48, 6, 27, 43, 41].”
BackgroundThe Advantage of the Right in Social Media News SharingPreprint
González‐Bailón, S., d’Andrea, V., Freelon, D., & Domenico, M. D. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3895410 · Cited by 3 · see citing works
Self-citationEminente, C., Artime, O., & Domenico, M. D. (2022) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2022.112759 · Cited by 2 · see citing works
“We do not get into the details of how the users were flagged in our dataset but for more information the authors of the dataset explain the procedure in the Supplementary Materials of [52] and to [53, 54, 55, 56].”
BackgroundSelf-citationMining Public Opinion on Aadhaar Card Linking in India
Tomar, N., Srivastava, R., & Verma, J. K. (2018) · International Conference on Computing, Power and Communication Technologies (GUCON) · doi:10.1109/gucon.2018.8674918 · Cited by 2 · see citing works
“Sentiment analysis (SA) is the process of determining and measuring the mood and emotional state of response made by opinion holder[4-10].”
BackgroundCervera, R. B., Batlle, A., & Pérez‐Solà, C. (2025) · Journal of Information Technology & Politics · doi:10.1080/19331681.2025.2574053 · Cited by 1 · see citing works
Efficient quantification on large-scale networks
Micheli, A., Moreo, A., Podda, M., Sebastiani, F., Simoni, W., & Tortorella, D. (2025) · Machine Learning · doi:10.1007/s10994-025-06915-w · Cited by 1 · see citing works
“However, they also present significant challenges, includ - ing the spread of misinformation (Del Vicario et al., 2016), the formation of echo chambers (Cinelli et al., 2021), and the manipulation of online narratives through automated bots (Stella et al., 2019).”
Adversarial Perturbations Maximization in Online Social Networks
Mao, D., Wu, W., & Du, D. (2025) · IEEE Transactions on Networking · doi:10.1109/ton.2025.3646453 · Cited by 1 · see citing works
“…that groups of inauthentic accounts/social bots impersonating users with malicious intent can work collaboratively to disseminate disinformation and fraud [7], [8], sow discord to destabilize communities [9], deteriorate public opinions [10], and impede productive and democratic discourse [9].”
Large-scale digital signatures of emotional response to the COVID-19 vaccination campaign
Bertani, A., Gallotti, R., Menini, S., Sacco, P. L., & Domenico, M. D. (2024) · EPJ Data Science · doi:10.1140/epjds/s13688-024-00452-7 · Cited by 1 · see citing works
Self-citationCervera, R. B., Pérez‐Solà, C., & Batlle, A. (2024) · Connections · doi:10.21307/connections-2019.033 · Cited by 1 · see citing works
The Power of App-Based ExperimentsPreprint
Pin, P., & Rotesi, T. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3865723 · Cited by 1 · see citing works
Interactive Content as a Mean of Attracting an Audience on TV Sites
Kitsa, M., & Mudra, I. (2022) · Postmodern Openings · doi:10.18662/po/13.4/503
“declare that online social interactions during a massive voting event can be used to build an accurate map of real-world political parties and electoral ranks (Stella et al., 2019).”
BackgroundQuantifying the Vulnerabilities of the Online Public Square to Manipulation TacticsPreprint
Truong, B. T., Lou, X., Flammini, A., & Menczer, F. (2023) · Research Square · doi:10.21203/rs.3.rs-3011774/v1
Cervera, R. B., Pérez‐Solà, C., & Batlle, A. (2023) · Research Square · doi:10.21203/rs.3.rs-3204096/v1
How many bots are on Twitter? The question is difficult to answer and misses the point
Yang, K.-C., & Menczer, F. (2022) · doi:10.64628/aai.yrgfgcwxc
AI Bots and the Distortion of Online Social Opinions
Meng, F., Yang, J., Du, W., & Wei, J. (2025) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2025.3599531
“Bots were found to strategically amplify fringe narratives and shift the salience of topics in public discourse through high-frequency engagement and structural embedding [19].”
YSocial: An artificial intelligence powered social media virtual twin
Rossetti, G., Stella, M., Cazabet, R., Abramski, K., Citraro, S., Cau, E., Failla, A., Morini, V., & Pansanella, V. (2026) · Big Data & Society · doi:10.1177/20539517261431576
Self-citation
- Authors
- Stella, M., Cristoforetti, M., & De Domenico, M.
- Affiliations
- Fondazione Bruno Kessler (IT)
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2019-05-16
- Volume
- 14
- Issue
- 5
- Pages
- e0214210
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC6521989
- OpenAlex
- W2789973986
- Semantic Scholar
- b4c0b98c98cdc11ab223e9d9ad2a2f8b4835f389
- Cited by
- 51
- References
- 66
- Citations by year
- 2026: 1 · 2025: 4 · 2024: 8 · 2023: 8 · 2022: 7 · 2021: 12 · 2020: 7 · 2019: 3 · 2018: 1
- FWCI
- 15.33
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Institutional Research Information System (Università degli Studi di Trento) (submittedVersion, cc-by)
- Padua Research Archive (University of Padova) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Social Sciences › Communication › Social Media and Politics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Complex Network Analysis Techniques
- Keywords
- augmented human, political bots, online social interaction, information cascades, collective attention, information flow, digital augmentation, internet influence
- MeSH terms
- Humans, Interpersonal Relations, Internet, Social Networking, Social Media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review