CoChat
Sign Up
PreprintCited by 0Misinformation and Its ImpactsSocial Media and PoliticsMedia Influence and Politics

How Partisan Crowds Affect News Evaluation

Open access
Jakesch, Maurice, Koren, Moran, Evtushenko, Anna, Naaman, Mor·2022
DOI 10.31234/osf.io/baxts
Open this paper in CoChat — free accountOpen PDF

Abstract

Social influence is ubiquitous in politics and online social media. Here we explore how social signals from partisan crowds influence people's evaluations of political news. For example, are liberals easily persuaded by a liberal crowd, while resisting the influence of conservative crowds? We designed a large-scale online experiment (N=1,000) to test how politically-annotated social signals affect participants' opinions. In times rife with misinformation and polarization, our findings are optimistic: the mechanism of social influence works across political lines, that is, liberals are reliably influenced by majority-Republican crowds and vice versa. At the same time, we replicate findings showing that people are inclined to discard news claims that are inconsistent with their political views. Considering that people show negative reactions to politically dissonant news but not to social signals that oppose their views, we point to the possibility of depolarizing social rating systems.

Integrity signals

Registry

Crossref · 10.31234/osf.io/baxts

Cites

54 of 54 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 31 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Conservatives did not change how they evaluated news claims based on crowd partisanship.

    “Conservatives, in particular, did not change how they evaluated news claims based on crowd partisanship.”
    Quote in the paperSubstantial claim
  • Findingown data

    The strength of social influence was mostly independent of the political makeup of the crowd.

    “While evaluations were significantly affected by claim politics and the social signal, the strength of social influence was mostly independent of the political makeup of the crowd.”
    Quote in the paperSubstantial claim
  • Findingown data

    People are inclined to discard news claims that are inconsistent with their political views.

    “people are inclined to discard news claims that are inconsistent with their political views.”
    • Also stated in: § 1 Introduction · plain
      “people -as expectedtended to discard claims that did not align with their political views.”
    Quote in the paperSubstantial claim
  • Findingown data

    Liberal participants rated Democrat-consistent claims as true 66% of the time when they saw no social signal.

    “liberal participants, shown in blue, said that the Democrat-consistent claims they saw were true in 66% of cases when they saw no social signal.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Conservative participants did not evaluate headlines differently from liberal participants on average.

    “We see that conservative participants did not evaluate headlines differently from liberal participants on average (line 1 in Table 1).”
    Quote in the paperNot a substantial claim
  • May combine two claims. 20
  • Classified as probably not a substantial claim. 21

  • May combine two claims.

    Findingown data

    We found no support for selective influence (H3) and saw no evidence of backfire effects.

    “We found no support for selective influence (H3) and saw no evidence of backfire effects.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    There is partial support for influence based on signal informativeness among liberals only; liberals were more likely to change their evaluation when they saw an informative signal from a majority-Democrat crowd.

    “partial support influence based on signal informativeness among liberals only, who were more likely to change their evaluation when they saw an informative signal from a majority-Democrat crowd.”
    • Also stated in: § 4 Results · hedged
      “The results also provide partial evidence for social influence based on informativeness (H4).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Participants were reliably influenced by social signals independent of crowd partisanship, including liberals being influenced by majority-Republican crowds and vice versa.

    “the mechanism of social influence works across political lines, that is, liberals are reliably influenced by majority-Republican crowds and vice versa.”
    • Also stated in: § 1 Introduction · plain
      “However, they were reliably influenced by social signals independent of crowd partisanship.”
    • Also stated in: § 4 Results · plain
      “The results show that social influence is effective and primarily non-political (H1).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For liberals, the model finds a significant reaction to the politics of the crowd, captured as combination influence based on identification (H2) and informativeness (H4), corresponding to the observation that the social influence was about 8% stronger when liberals saw an informative signal from a majority-Democrat crowd.

    “For liberals, the model finds a significant reaction to the politics of the crowd, which it captures as combination influence based on identification (H2, line 5 in Table 1) and informativeness (H4, line 7), corresponding to the observation that the social influence was about 8% stronger when liberals saw an informative signal from a majority-Democrat crowd.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    Participants believed a claim was true in only 49% of cases, showing that people had little prior knowledge of the claims.

    “We also found that our manipulations worked as intended: Participants believed a claim was true in only 49% of cases, showing that people had little prior knowledge of the claims.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The study, analysis, and hypotheses (except H3) were preregistered.

    “We preregistered the study, the analysis and hypotheses (except for H3), available at http: //aspredicted.org/blind.php?x=hx82az.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The final sample was N=971 participants after exclusions.

    “leaving us with a final sample of N=971 participants.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The N=100 pilot provided an initial indication that the manipulation had an effect on the dependent variable.

    “The N = 100 pilot provided an initial indication that the manipulation had an effect on the dependent variable and helped us plan the size of the experiment.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    Participants who failed the attentiveness check, indicated searching for headlines, or rated all headlines as either true or false were excluded.

    “We excluded participants who had failed an attentiveness check, participants who indicated they had searched for headlines on the web, and participants who rated all headlines as either true or false.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    Claims selected as Democrat-consistent were more likely to be believed by Democrats and vice versa.

    “Claims we selected as Democrat-consistent were more likely to be believed by Democrats and vice versa.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Participants were randomly assigned to control, majority-Democrat, or majority-Republican conditions.

    “We randomly assigned respondents to either a control group where they saw no social signal or one of two treatment groups where they saw social signals from majority-Democrat or majority-Republican crowds.”
    • Also stated in: § 3.2 Experimental design · plain
      “Participants were randomly assigned to one of three conditions: the control, the majority-Democrat, or the majority-Republican group.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    People are more influenced by politically like-minded others.

    “which is surprising given research showing that people are more influenced by politically like-minded others (e.g., (Marks et al., 2019)).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Marks et al., 2019

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Social endorsements are more predictive of news selection than source indicators, and their effect is particularly strong for partisans selecting news from ideologically misaligned sources.

    “They found that social endorsements are more predictive of news selection than source indicators and that their effect is particularly strong for partisans selecting news from ideologically misaligned sources.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Messing and Westwood, 2014

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    People's opinions are affected by those of their peers, as groups exert normative pressure and are a source of information.

    “People's opinions are affected by those of their peers, as groups exert normative pressure and are a source of information (Deutsch and Gerard, 1955).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Deutsch and Gerard, 1955

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The regression model estimating how much different hypothesized types of social influence contributed to the result was not included in the preregistration, so its result should be considered exploratory.

    “Note that since we did not include this model in the preregistration, the result should be considered exploratory.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    The social influence manipulation had a significant effect across all groups, and the crowd partisanship indicator changed the evaluation of liberals, showing that all manipulations worked as intended.

    “The social influence manipulation had a significant effect across all groups, and the crowd partisanship indicator changed the evaluation of liberals, showing that all manipulations worked as intended.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    It is hard to separate social influence from homophily in real-world settings.

    “However, it is hard to separate social influence from homophily in real-world settings (Shalizi and Thomas, 2011).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Shalizi and Thomas, 2011

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Previous studies have observed strong adverse reactions to politically misaligned content.

    “Previous studies have observed such strong adverse reactions to politically misaligned content (Hart et al., 2009;Stroud, 2010)”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Hart et al., 2009

      Not checked in the free run.

    • cites Stroud, 2010

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    Headline assignments and order were randomized for each participant.

    “We randomized headline assignments and order for each participant.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Biased cognitive processes affect the way people process information through motivated reasoning; individuals have strategies to be skeptical of information that may falsify their firmly held beliefs.

    “We know that biased cognitive processes affect the way people process information through motivated reasoning (Kunda, 1990): individuals have various strategies to be skeptical of information that may falsify their firmly held beliefs.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Kunda, 1990

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    Influence depends on informativeness (H4), but only for liberals in a majority-Democrat crowd (H2).

    “This result partially supports the idea that influence depends on informativeness (H4), but only for liberals in a majority-Democrat crowd (H2).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The study protocols were approved by the Institutional Review Board.

    “The study protocols were approved by the Institutional Review Board at the primary investigator's institution.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    An attentiveness check and a post-completion search check were conducted.

    “we conducted an attentiveness check at the beginning of the study and asked participants whether they had searched for headlines online after completion.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Repeated variations of the analysis excluding respondents who identified as independents or only mildly liberal or conservative produced similar results.

    “We have repeated variations of the analysis while excluding respondents who identified as independents or only mildly liberal or conservative, with similar results.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Aggregated crowd opinions displayed on social media platforms exert both normative and informative influence over individuals.

    “Research has shown that the aggregated crowd opinions displayed on social media platforms exert both normative and informative influence over individuals (Asch, 1955;Deutsch and Gerard, 1955;Cosley et al., 2003).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Asch, 1955

      Not checked in the free run.

    • cites Deutsch and Gerard, 1955

      Not checked in the free run.

    • cites Cosley et al., 2003

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    People attend more to unusual information.

    “Research showing that people attend more to unusual information (Hope and Wright, 2007) motivates this hypothesis”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Hope and Wright, 2007

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Study design and conductnone given

    The design has 90% statistical power for small effect sizes (Cohen's f2 = 0.25).

    “The design has 90% statistical power for small effect sizes (Cohen's f 2 = 0.25).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Individuals may avoid incongruent information due to the discomfort of cognitive dissonance.

    “Individuals may avoid incongruent information due to the discomfort of cognitive dissonance (Festinger, 1957) it evokes.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Festinger, 1957

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Representative polls show that Americans are divided on their support for a border barrier, with recent polls finding a majority of 60% opposing it.

    “Representative polls show that Americans are divided on their support for a border barrier, with recent polls finding a majority of 60% opposing it (Gallup, 2019).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Gallup, 2019

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Social signals with population corrections would remain informative even in politically homogeneous settings and would avoid creating false impressions of unanimity.

    “Social signals with population corrections would remain informative even in politically homogeneous settings and would avoid creating false impressions of unanimity.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    Our design models the experience of users exposed to political claims on social media sites and our findings cannot be extended to crowd situations in the offline world.

    “Our design models the experience of users exposed to political claims on social media sites and our findings can not be extended to crowd situations in the offline world.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The social signal from the border barrier video creates a false impression of unanimity and influences subsequent (mostly conservative) viewers to evaluate the video more favorably.

    “This social signal not only creates a false impression of unanimity, but it influences subsequent (and mostly conservative) viewers to evaluate the video more favorably.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Expressive responding or demand characteristics could be affecting our results, with participants providing answers that are not their true beliefs.

    “First, expressive responding (Jakesch et al., 2019) or demand characteristics (Nichols and Maner, 2008) could be affecting our results, with participants providing answers that are not their true beliefs.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Jakesch et al., 2019

      Not checked in the free run.

    • cites Nichols and Maner, 2008

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    Individuals tend to discard news claims that do not align with their political views, but they are influenced by social signals independently of the crowd's political makeup.

    “We find that while individuals tend to discard news claims that do not align with their political views, they are influenced by social signals independently of the crowd's political makeup.”
    Quote in the paperSubstantial claim
34 of 34

  • The paper relies on this without support.

    The effect of social influence is stronger and more robust for claims of which participants had little prior knowledge.

    “As we expect the effect of social influence to be stronger and more robust for claims of which participants had little prior knowledge, we evaluated the headlines in a preliminary survey with N=85 workers on Amazon Mechanical Turk.”

  • The paper relies on this without support.

    Viewers may infer crowd partisanship based on platform and content cues in many cases.

    “While the political makeup of the crowd is typically not made explicit, viewers may infer crowd partisanship based on platform and content cues in many cases.”

  • The paper relies on this without support.

    Most current popular social media sites show only aggregated social ratings.

    “We chose to show an aggregate social signal, not separating the opinions of Democrats and Republicans, in order to reflect the setup of most current popular social media sites where only see aggregated social ratings.”

  • The paper relies on this without support.

    The experimental setting (rating systems on websites) adequately models real-world social media rating systems.

    “Our design models the experience of users exposed to political claims on social media sites”
    “our results are limited to the context of rating systems on websites”
4 of 4

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Rejected headline is reported as both gaining and losing support

    Major

    For liberals evaluating a Democrat-consistent claim rejected by a majority-Republican crowd, one passage reports 73.4% calling it true, while another reports 51.8%. The two values describe the same condition and imply opposite directions of change from the control group.

  • Figure caption names the wrong crowd comparison

    Minor

    The caption compares majority-Republican and majority-Conservative crowds, but the experiment's two crowd treatments were majority-Democrat and majority-Republican.

  • Figure reports the recruited count instead of the final sample

    Minor

    The Figure 3 caption gives N = 1000, but the methods say exclusions left 971 participants in the final sample. The caption does not distinguish its count from the sample used for the reported results.

3 of 3

Cites

Flagged first, then by citations

Research works printed but not linked: 5

  1. [7]7. Larry M Bartels. 2002. Beyond the running tally: Parti- san bias in political perceptions. Political behavior, 24(2):117-150.
  2. [18]18. Dan Cosley, Shyong K Lam, Istvan Albert, Joseph A Konstan, and John Riedl. 2003. Is seeing believing?: how recommender system interfaces affect users' opinions. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 585- 592. ACM.
  3. [26]26. Naeemul Hassan, Fatma Arslan, Chengkai Li, and Mark Tremayne. 2017. Toward automated fact- checking: Detecting check-worthy factual claims by claimbuster. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1803-1812. ACM.
  4. [41]41. Brendan Nyhan and Jason Reifler. 2010. When cor- rections fail: The persistence of political mispercep- tions. Political Behavior, 32(2):303-330.
  5. [45]45. Daniel M Romero, Brendan Meeder, and Jon Klein- berg. 2011. Differences in the mechanics of informa- tion diffusion across topics: idioms, political hash- tags, and complex contagion on twitter. In Proceed- ings of the 20th international conference on World wide web, pages 695-704. ACM.

Cited by

Still being checked.

Related

Authors
Jakesch, M., Koren, M., Evtushenko, A., & Naaman, M.
Affiliations
  • Cornell University (US)
  • Stanford University (US)
Corresponding author
Koren, Moran
ORCID
Publisher
Center for Open Science
Published
2022-12-12
DOI
10.31234/osf.io/baxts
OpenAlex
W3108203576
Cited by
0
References
52
Type
posted content
Language
en
Open access
Locations
PsyArXiv (OSF Preprints) (submittedVersion)
Topics
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Social Sciences › Communication › Social Media and Politics
  • Social Sciences › Sociology and Political Science › Media Influence and Politics
Keywords
social influence, social signals, political polarization, misinformation, political dissonance, news credibility, depolarization, social media, partisanship

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-11.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-11.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review