CoChat
Sign Up
Cited by 89Impact of Technology on AdolescentsSocial Media and PoliticsDigital Media and Philosophy

The Welfare Effects of Social Media

Open access
Allcott, Hunt, Braghieri, Luca, Eichmeyer, Sarah, Gentzkow, Matthew·2019·National Bureau of Economic Research
DOI 10.3386/w25514
Open this paper in CoChat — free account

Abstract

The rise of social media has provoked both optimism about potential societal benefits and concern about harms such as addiction, depression, and political polarization.In a randomized experiment, we find that deactivating Facebook for the four weeks before the 2018 US midterm election (i) reduced online activity, while increasing offline activities such as watching TV alone and socializing with family and friends; (ii) reduced both factual news knowledge and political polarization; (iii) increased subjective well-being; and (iv) caused a large persistent reduction in post-experiment Facebook use.Deactivation reduced post-experiment valuations of Facebook, suggesting that traditional metrics may overstate consumer surplus.

AI snapshot

Written from the paper’s full text.

TL;DR

A large randomized experiment deactivating Facebook for four weeks before the 2018 US midterm election found reduced news knowledge and political polarization, improved subjective well-being, and substantially lower post-experiment Facebook use. The authors argue traditional consumer-surplus metrics may overstate Facebook’s welfare gains because deactivation reduced later willingness-to-accept for Facebook.

Key findings

  • Deactivation reduced Facebook use by 59.58 minutes per day (1.59 SD) and shifted time toward offline activities: TV alone +0.17 SD, non-screen alone +0.25 SD, friends/family +0.16 SD, while reducing other online activities.
  • It reduced the news knowledge index by 0.19 SD and news minutes by 8 minutes/day (15% of the control mean), and reduced the political polarization index by 0.16 SD; there was no detectable effect on political engagement or voter turnout.
  • It improved the subjective well-being index by 0.09 SD, with significant effects on life satisfaction (0.12 SD), anxiety (0.10 SD), depression (0.09 SD), and happiness (0.08 SD); text-message well-being effects were positive but not significant.
  • Post-experiment Facebook use fell by 0.61 SD overall; mobile app use was 12 minutes/day (23%) lower, and willingness-to-accept for weeks 5–8 dropped by $14–$18, about 14%, suggesting standard consumer surplus estimates may overstate gains.

Methods

  • Recruited 2,743 US Facebook users via Facebook ads; the impact evaluation sample was 1,661 participants with willingness-to-accept below $102, randomized to a $102 Treatment group paid to deactivate for four weeks before the 2018 midterm or a $0 Control group.
  • Measured outcomes using surveys, daily text messages, direct Facebook/Twitter activity checks, and L2 administrative voting records; compliance with deactivation exceeded 90% in Treatment.
  • Estimated local average treatment effects via IV, instrumenting observed deactivation with treatment assignment, with a pre-analysis plan and False Discovery Rate adjustments; elicited willingness-to-accept using Becker-DeGroot-Marschak mechanisms at midline and endline.

Limitations

  • Stated: effects may differ with deactivation duration, time period, or scale; network effects mean estimates are most relevant for individuals independently deciding their own Facebook use; the sample is not fully representative (younger, well-educated, left-leaning, and only users reporting >15 minutes/day); many outcomes are self-reported, though the paper finds no evidence of demand effects and non-self-reported outcomes paint a similar picture.
  • Evident from the text: the four-week pre-election design limits generalization to other periods or longer deactivations; willingness-to-accept estimates may be affected by anchoring and are not closely held; the iPhone-only measure of post-experiment mobile app use was smaller and not statistically significant.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3386/w25514

Cites

119 of 119 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 28 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Deactivating Facebook freed up 60 minutes per day for the average person in the Treatment group.

    “Deactivating Facebook freed up 60 minutes per day for the average person in our Treatment group.”
    Quote in the paperSubstantial claim
  • Findingown data

    Participants were unwilling to give up Facebook unless offered fairly large amounts of money, even after they had deactivated for four weeks.

    “participants were unwilling to give up Facebook unless offered fairly large amounts of money-even after they had deactivated for four weeks”
    Quote in the paperSubstantial claim
  • Findingown data

    Deactivating Facebook for the four weeks before the 2018 US midterm election increased subjective well-being.

    “increased subjective well-being”
    • Also stated in: § 1 Introduction · plain
      “Deactivation caused small but significant improvements in well-being, and in particular in self-reported happiness, life satisfaction, depression, and anxiety.”
    • Also stated in: § 1 Introduction · plain
      “Our overall index of subjective well-being improved by 0.09 standard deviations.”
    Quote in the paperSubstantial claim
  • Findingown data

    Deactivating Facebook for the four weeks before the 2018 US midterm election increased offline activities such as watching TV alone and socializing with family and friends.

    “while increasing offline activities such as watching TV alone and socializing with family and friends”
    • Also stated in: § 1 Introduction · plain
      “while devoting more time to a range of offline activities such as watching television alone and spending time with friends and family”
    Quote in the paperSubstantial claim
  • Findingown data

    The Treatment group was more likely to agree that Facebook helps people to follow the news.

    “Consistent with our results on news knowledge, the Treatment group was more likely to agree that Facebook helps people to follow the news.”
    Quote in the paperNot a substantial claim
  • May combine two claims. 75
  • Classified as probably not a substantial claim. 61
  • The quote only partly states this claim. 17

  • May combine two claims.

    Findingown data

    The opposing effects on specific metrics cancel out, so the overall index of opinions about Facebook is unaffected.

    “The opposing effects on these specific metrics cancel out, so our overall index of opinions about Facebook is unaffected.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    After weighting the sample to match the average US Facebook user on observables, the median and mean willingness-to-accept to deactivate Facebook for weeks 1-4 were $100 and $180, respectively.

    “After weighting our sample to match the average US Facebook user on observables, the median and mean willingness-to-accept to deactivate Facebook for weeks 1-4 were $100 and $180, respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In response to open-answer questions several weeks after the experiment ended, the Treatment group was more likely to report that they were using Facebook less, had uninstalled the Facebook app from their phones, and were using the platform more judiciously.

    “In response to open-answer questions several weeks after the experiment ended, the Treatment group was more likely to report that they were using Facebook less, had uninstalled the Facebook app from their phones, and were using the platform more judiciously.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Deactivation caused small but significant improvements in self-reported happiness, life satisfaction, depression, and anxiety.

    “Deactivation caused small but significant improvements in well-being, and in particular in self-reported happiness, life satisfaction, depression, and anxiety.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Effects on subjective well-being as measured by responses to brief daily text messages are positive but not significant.

    “Effects on subjective well-being as measured by responses to brief daily text messages are positive but not significant.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    An additional 154 participants had dropped out before the midline survey price offer, leaving 2,743 who received their price offer.

    “An additional 154 participants had dropped out before this point of the midline survey, leaving 2,743 who received their price offer.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    1,892,191 unique users were shown the ad, of whom 32,201 clicked on it.

    “1,892,191 unique users were shown the ad, of whom 32,201 clicked on it.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    1,013 participants were dropped from the experiment because of invalid data or low-quality baseline responses.

    “Of those, 1,013 were dropped from the experiment because of invalid data (for example, invalid Facebook profile URLs) or low-quality baseline responses”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    7,455 people consented to participate in the study.

    “Of those people, 7,455 consented to participate in the study.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    The ad's click-through rate was 1.7 percent, about twice the average click-through rate on Facebook ads across all industries (Irvine 2018).

    “This 1.7 percent click-through rate is about twice the average click-through rate on Facebook ads across all industries (Irvine 2018).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Irvine 2018

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    17,335 people passed the pre-screen.

    “17,335 people passed the pre-screen”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingreasoning

    The remaining 2,897 participants had valid baseline data, were included in stratified randomization, and were invited to take the midline survey.

    “The remaining 2,897 participants had valid baseline data, were included in our stratified randomization, and were invited to take the midline survey.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The sample is relatively high-income, well-educated, female, young, and Democratic, and uses Facebook relatively heavily.

    “Comparing column 1 to columns 2 and 3, we see that our sample is relatively high-income, well-educated, female, young, and Democratic, and uses Facebook relatively heavily.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The point estimates for the other offline activities measured (going to the cinema, talking to friends on the phone, going to a party, going shopping, and spending time with your kids) are all very close to zero.

    “The point estimates for the other offline activities we measure (going to the cinema, talking to friends on the phone, going to a party, going shopping, and spending time with your kids) are all very close to zero.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    There are no effects on the social interaction index, although the point estimate is positive.

    “We find no effects on the social interaction index, although the point estimate is positive.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The point estimates for print, radio, and TV news are all positive but statistically insignificant.

    “The point estimates for print, radio, and TV news are all positive but statistically insignificant.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    At endline, the Control group reported using Facebook for an average of 59.53 minutes per day over the past four weeks, and the local average treatment effect of deactivation is 59.58 minutes per day.

    “At endline, the Control group reported that they had used Facebook for an average of 59.53 minutes per day over the past four weeks, and the local average treatment effect of deactivation is 59.58 minutes per day.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The substitute time uses index, which does not include Facebook minutes, shows an increase in overall non-Facebook activities.

    “The substitute time uses index, which does not include Facebook minutes, shows an increase in overall non-Facebook activities.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    There are no statistically significant effects on the number of friends that participants listed as having met in person last week, or on diverse interactions.

    “Notwithstanding the positive effects on offline activities, there are no statistically significant effects on the number of friends that participants listed as having met in person last week, or on diverse interactions (whether or not they interacted with someone who voted differently in the last presidential election or interacted with someone from another country).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Deactivation caused substantial reductions in both self-reported attention to news and directly measured news knowledge.

    “Deactivation caused substantial reductions in both self-reported attention to news and directly measured news knowledge.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The magnitudes of SWB effects are relatively small overall, with the largest and most significant effects on life satisfaction (0.12 SD), anxiety (0.10 SD), depression (0.09 SD), and happiness (0.08 SD).

    “The magnitudes are relatively small overall, with the largest and most significant effects on life satisfaction (0.12 SD), anxiety (0.10 SD), depression (0.09 SD), and happiness (0.08 SD).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The text message based measures of happiness are not significantly different from zero, with positive point estimates ranging from 0.01 SD to 0.06 SD.

    “The text message based measures of happiness are not significantly different from zero, with positive point estimates ranging from 0.01 SD to 0.06 SD.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Neither the administrative nor the self-reported turnout estimate is statistically different from zero.

    “As reported in Appendix Tables A10 and A11, the point estimates suggest that deactivation increased turnout by three percentage points according to the administrative data and decreased turnout by three percentage points according to the self-reported data, and neither estimate is statistically different from zero.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Subjects' own descriptions in follow-up interviews and free-response questions are consistent with the quantitative findings, while also highlighting substantial heterogeneity in the effects.

    “Subjects' own descriptions in follow-up interviews and free-response questions are consistent with these quantitative findings, while also highlighting substantial heterogeneity in the effects.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The point estimates for text message outcomes do not systematically increase or decrease over time, and if anything, the point estimates are largest in the first week.

    “The point estimates do not systematically increase or decrease over time, and if anything, the point estimates are largest in the first week.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    This is a 23 percent reduction relative to the Control group mean of 53 minutes per day, lining up almost exactly with the planned reductions reported at endline.

    “This is a 23 percent reduction relative to the Control group mean of 53 minutes per day, lining up almost exactly with the planned reductions reported at endline.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The other measures with the largest point estimates are party anger and party affective polarization, although these individual effects are not statistically significant.

    “The other measures with the largest point estimates are party anger and party affective polarization, although these individual effects are not statistically significant.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The baseline correlation between the SWB index and Facebook use is about three times larger than the experimental estimate of the treatment effect of deactivation (about 0.23 SD compared to 0.09 SD), and the point estimates are highly statistically significantly different.

    “The baseline correlation between our SWB index and Facebook use is about three times larger than the experimental estimate of the treatment effect of deactivation (about 0.23 SD compared to 0.09 SD), and the point estimates are highly statistically significantly different.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The effect on reported intentions to use Facebook as of the endline survey is a reduction of 0.78 standard deviations: while the average Control group participant planned to reduce future Facebook use by 22 percent, deactivation caused the Treatment group to plan to reduce Facebook use by an additional 21 percent relative to Control.

    “The effect on reported intentions to use Facebook as of the endline survey is a reduction of 0.78 standard deviations: while the average Control group participant planned to reduce future Facebook use by 22 percent, deactivation caused the Treatment group to plan to reduce Facebook use by an additional 21 percent relative to Control.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The reduction is less than half as large (8 percent of the Control group mean) and not statistically significant (with a t-statistic of -1.16) if the sample is limited to iPhone users who reported their usage as recorded by their Settings app.

    “However, Appendix Table A13 shows that the reduction is less than half as large (8 percent of the Control group mean) and not statistically significant (with a t-statistic of -1.16) if we limit the sample to iPhone users who reported their usage as recorded by their Settings app, thereby excluding participants who were reporting personal estimates of their usage.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Deactivation may reduce polarization more (i.e., Facebook use may increase polarization more) for older people, white people, and men.

    “First, deactivation may reduce polarization more (i.e., Facebook use may increase polarization more) for older people, white people, and men.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Deactivation increases both the positive impacts and negative impacts variables, i.e. it makes people write more about both positive and negative aspects of Facebook.

    “Deactivation increases both the positive impacts and negative impacts variables, i.e. it makes people write more about both positive and negative aspects of Facebook.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The Treatment group's average response on the scale from -5 to +5 is -2.3, while the Control group's average response is -1.9.

    “On this figure, the Treatment group's average response on the scale from -5 to +5 is -2.3, while the Control group's average response is -1.9.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The estimated effects for news knowledge are larger for people who get more news from Facebook, but this difference is not statistically significant.

    “As one might expect, the estimated effects for news knowledge are larger for people who get more news from Facebook, but this difference is not statistically significant.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Overall, deactivation clearly decreased post-experiment use, reducing the index by 0.61 standard deviations.

    “Overall, deactivation clearly decreased post-experiment use, reducing the index by 0.61 standard deviations.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The median WTA to deactivate Facebook for the four weeks after midline instead of only the 24 hours after midline is $100, and almost 20 percent had valuations greater than $500.

    “The median is $100, and almost 20 percent had valuations greater than $500.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The Treatment group's average response on the scale from -5 to +5 was -1.8, while the Control group's average response is -2.0.

    “On this figure, the Treatment group's average response on the scale from -5 to +5 was -1.8, while the Control group's average response is -2.0.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In both panels, the Treatment group has a wider dispersion of responses, with more people strongly agreeing and more people strongly disagreeing.

    “In both panels, the Treatment group has a wider dispersion of responses, with more people strongly agreeing and more people strongly disagreeing.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    After re-weighting the sample to match the observable characteristics of Facebook users in Table 2, the median is still $100, and the winsorized mean is $180.

    “After re-weighting the sample to match the observable characteristics of Facebook users in Table 2, the median is still $100, and the winsorized mean is $180.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    Our sample's WTA for Facebook abstention is larger than in most other studies, but not all.

    “Our sample's WTA for Facebook abstention is larger than in most other studies, but not all.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    About 80 percent of the Treatment group thought that deactivation was at least somewhat good for them, and the modal response was the strongest possible agreement that deactivation was good.

    “Remarkably, about 80 percent of the Treatment group thought that deactivation was at least somewhat good for them, and the modal response was the strongest possible agreement that deactivation was good (the left-most bar on the histogram).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Some people wrote in open-answer questions that they were less willing to deactivate during the Thanksgiving holiday, and they may not have foreseen this as of the midline survey on October 11th.

    “In open-answer questions, some people wrote that they were less willing to deactivate during the Thanksgiving holiday, and they may not have foreseen this as of the midline survey on October 11th.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The Treatment group's valuation for weeks 5-8 jumps substantially relative to its valuation for weeks 1-4, while the Control group's valuation for weeks 5-8 does not.

    “The Treatment group's valuation for weeks 5-8 jumps substantially relative to its valuation for weeks 1-4, while the Control group's valuation for weeks 5-8 does not.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The above-median WTA point estimate is larger and statistically indistinguishable for two indices, smaller and statistically indistinguishable for four indices, and opposite-signed for the final index.

    “The above-median WTA point estimate is larger and statistically indistinguishable for two indices, smaller and statistically indistinguishable for four indices, and opposite-signed for the final index.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Deactivation increases the extent to which participants think Facebook helps them follow the news better, and it also makes participants agree more that people would miss Facebook if they stopped using it.

    “Deactivation increases the extent to which participants think Facebook helps them follow the news better, and it also makes participants agree more that people would miss Facebook if they stopped using it.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Both Treatment and Control tended to think that deactivation was good for them, but the Treatment group is more likely to think that their (longer) deactivation was good for them.

    “The bottom right panel shows that both Treatment and Control tended to think that deactivation was good for them, but the Treatment group is more likely to think that their (longer) deactivation was good for them.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    People with more positive baseline opinions about Facebook are less likely to decrease their post-experiment use and less likely to think that deactivation was good for them.

    “Third, people may have some intuition about whether they will like deactivation: people with more positive baseline opinions about Facebook are less likely to decrease their post-experiment use and less likely to think that deactivation was good for them.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Facebook deactivation increased TV alone time by 0.17 units (SD = 0.17).

    “TV alone time 0.17 0.05 0.17 0.05 0.00 0.00”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Facebook deactivation increased non-screen alone time by 0.23 units (SD = 0.25).

    “Non-screen alone time 0.23 0.05 0.25 0.05 0.00 0.00”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Facebook deactivation reduced following politics by 0.14 units (SD = -0.14).

    “Follow politics -0.14 0.04 -0.14 0.04 0.00 0.00”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Facebook deactivation increased happiness by 0.12 units (SD = 0.08).

    “Happiness 0.12 0.06 0.08 0.04 0.04 0.06”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    Facebook deactivation reduced Facebook minutes by 59.58 minutes per day (SD = -1.59).

    “Facebook minutes -59.58 1.43 -1.59 0.04 0.00 0.00”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    Facebook deactivation reduced issue polarization by 0.29 units (SD = -0.10).

    “Issue polarization -0.29 0.11 -0.10 0.04 0.01 0.02”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    Facebook deactivation reduced news knowledge by 0.14 units (SD = -0.12).

    “News knowledge -0.14 0.06 -0.12 0.05 0.02 0.04”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Facebook deactivation reduced congenial news exposure by 0.31 units (SD = -0.20).

    “Congenial news exposure -0.31 0.08 -0.20 0.05 0.00 0.00”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Facebook deactivation increased life satisfaction by 0.56 units (SD = 0.12).

    “Life satisfaction 0.56 0.20 0.12 0.04 0.00 0.01”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The remaining seven percent of the matches between experiment participants and the L2 database are one-to-many.

    “The remaining seven percent of the matches are one-to-many: a participant in our experiment was matched to more than one individual in the L2 database.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Facebook deactivation increased friends and family time by 0.14 units (SD = 0.16).

    “Friends and family time 0.14 0.05 0.16 0.06 0.00 0.01”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    Facebook deactivation increased offline activities by 0.18 units (SD = 0.12).

    “Offline activities 0.18 0.08 0.12 0.05 0.02 0.03”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Facebook deactivation caused an average increase of 0.56 in life satisfaction (on a 0-21 scale), with the Control group endline average at 12.26.

    “The Control group endline average is 12.26 out of a possible 21, and deactivation caused an average increase of 0.56.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Facebook deactivation caused an average increase of 0.12 in happiness (on a 1-7 scale), with the Control group endline average at 4.47.

    “The Control group endline average is 4.47 out of a possible 7, and deactivation caused an average increase of 0.12.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    Facebook deactivation increased belief that Facebook helps follow news by 0.31 units (SD = 0.13).

    “Helps follow news 0.31 0.11 0.13 0.05 0.01 0.02”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Facebook deactivation reduced negative impacts by 0.21 units (SD = -0.23).

    “Negative impacts × (-1) -0.21 0.05 -0.23 0.05 0.00 0.00”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Omitting congenial news exposure from the political polarization index, the local average treatment effect of Facebook deactivation on the political polarization index is -0.07 (SE 0.04, p=0.09).

    “Congenial news exposure -0.07 0.04 0.09”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Facebook deactivation reduced anxiety by 0.09 units (SD = 0.10).

    “Anxious × (-1) 0.09 0.04 0.10 0.05 0.03 0.05”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    Facebook deactivation increased belief that people would miss Facebook by 0.26 units (SD = 0.13).

    “People would miss Facebook 0.26 0.12 0.13 0.06 0.03 0.05”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Facebook deactivation reduced depression by 0.08 units (SD = 0.09).

    “Depressed × (-1) 0.08 0.04 0.09 0.04 0.03 0.05”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    Facebook deactivation increased positive impacts by 0.21 units (SD = 0.28).

    “Positive impacts 0.21 0.04 0.28 0.05 0.00 0.00”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    Reduced post-experiment use is consistent with the hypothesis that Facebook is habit forming in the sense of Becker and Murphy (1988) or that people learned that they enjoy life without Facebook more than they had anticipated.

    “and it is also consistent with the hypotheses that Facebook is habit forming in the sense of Becker and Murphy (1988) or that people learned that they enjoy life without Facebook more than they had anticipated.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Becker and Murphy (1988)

      Not checked in the free run.

  • May combine two claims.

    Study design and conductnone given

    The paper's participants are relatively young, well-educated, and left-leaning compared to the average Facebook user.

    “Our participants are relatively young, well-educated, and left-leaning compared to the average Facebook user”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premiseown data

    The paper's deactivation period is substantially longer and its sample size an order of magnitude larger than most prior experimental work, including Mosquera et al. (2018).

    “Our deactivation period is substantially longer and our sample size an order of magnitude larger than most prior experimental work, including Mosquera et al. (2018).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Mosquera et al. (2018)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The paper is the only one of the randomized trials to have submitted a pre-analysis plan.

    “we are the only one of these randomized trials to have submitted a pre-analysis plan”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Study design and conductnone given

    People willing to participate in the experiment may differ in unobservable ways.

    “and people willing to participate in our experiment may also differ in unobservable ways”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The paper included only people who reported using Facebook more than 15 minutes per day.

    “we included only people who reported using Facebook more than 15 minutes per day”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The ads were targeted by demographic cells in an attempt to gather an initial sample that was approximately representative of Facebook users on gender, age, college completion, and political ideology.

    “We targeted the ads by demographic cells in an attempt to gather an initial sample that was approximately representative of Facebook users on gender, age, college completion, and political ideology.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Mosquera et al. (2018) also use Facebook deactivation to study news knowledge and well-being, finding results broadly consistent with those reported here.

    “They also use Facebook deactivation to study news knowledge and well-being, finding results broadly consistent with those reported here.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Mosquera et al. (2018)

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    We verified deactivation approximately once per day for the seven days before midline and all days between endline and the end of January 2019.

    “we verified deactivation approximately once per day for the seven days before midline and all days between endline and the end of January 2019”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Participants with valid baseline data were randomized into three groups that determined the BDM offer price p for deactivation in weeks 1-4.

    “Participants with valid baseline data were randomized into three groups that determined the BDM offer price p for deactivation in weeks 1-4”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    The ad did not hint at the research questions or suggest that the study was related to social media or Facebook deactivation, to minimize sample selection bias.

    “To minimize sample selection bias, the ad did not hint at our research questions or suggest that the study was related to social media or Facebook deactivation.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    We did not tell participants the distribution or support of the offer prices.

    “We did not tell participants the distribution or support of the offer prices”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Between midline and endline, we verified deactivation approximately four times per day for the Treatment group and once every four days for everyone else.

    “Between midline and endline, we verified deactivation approximately four times per day for people who were supposed to be deactivated (i.e. the Treatment group) and once every four days for everyone else.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The paper's work relates to quasi-experimental estimates of social media effects by Muller and Schwarz (2018) and Enikolopov, Makarin, and Petrova (2018).

    “Our work also relates to quasi-experimental estimates of social media effects by Muller and Schwarz (2018) and Enikolopov, Makarin, and Petrova (2018).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Muller and Schwarz (2018)

      Not checked in the free run.

    • cites Enikolopov, Makarin, and Petrova (2018)

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    We verified deactivation by checking each participant's Facebook profile page URL regularly at random times.

    “We verified deactivation by checking each participant's Facebook profile page URL regularly at random times.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Prior work has found that a different index of political polarization rose by 0.38 standard deviations between 1996 and 2018 (Boxell 2018).

    “As a point of comparison, prior work has found that a different index of political polarization rose by 0.38 standard deviations between 1996 and 2018 (Boxell 2018).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Boxell 2018

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationown data

    These results are consistent with prior studies suggesting that Facebook may have adverse effects on mental health.

    “These results are consistent with prior studies suggesting that Facebook may have adverse effects on mental health.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The paper measures impacts on a relatively comprehensive range of outcomes.

    “We measure impacts on a relatively comprehensive range of outcomes”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    We asked several comprehension questions to make sure that participants understood the mechanism.

    “We then asked several comprehension questions to make sure that participants understood the mechanism.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Less than two percent of the sample failed to complete the endline survey.

    “Less than two percent of the sample failed to complete the endline survey”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The average participant responded to 92 percent of daily text messages, well above the 75 percent required in order to receive the completion payment.

    “The average participant responded to 92 percent of daily text messages, well above the 75 percent required in order to receive the completion payment.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    We chose $102 because pilot data suggested a point mass of WTAs at $100 and it would maximize statistical power per dollar of cost to set an offer price just high enough to induce those participants to deactivate.

    “We chose $102 because our pilot data correctly suggested that there would be a point mass of WTAs at $100 and that it would maximize statistical power per dollar of cost to set an offer price just high enough to induce those participants to deactivate.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The daily text messages allowed measurement of aspects of subjective well-being that are most important to record in the moment instead of retrospectively.

    “The daily text messages allowed us to measure the aspects of subjective well-being that are most important to record in the moment instead of retrospectively.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The pre-analysis plan was submitted on October 12, the final day before the Treatment and Control groups could have begun to differ.

    “We submitted our pre-analysis plan on October 12, as this was the final day before the Treatment and Control groups could have begun to differ.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The daily text message approach builds on the Experience Sampling Method of Csikszentmihalyi and Larson (2014) and Stone and Shiffman (1994).

    “This approach builds on the Experience Sampling Method of Csikszentmihalyi and Larson (2014) and Stone and Shiffman (1994).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Csikszentmihalyi and Larson (2014)

      Not checked in the free run.

    • cites Stone and Shiffman (1994)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The order of the 15 news knowledge statements was randomized.

    “The order of the 15 statements was randomized.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The pre-analysis plan specified the two regression specifications and the estimation sample.

    “Third, it specified the two regression specifications and the estimation sample as described below.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Study design and conductnone given

    This approach allows us to maintain incentive compatibility in the BDM mechanism, have balance between Treatment and Control groups, and use a straightforward regression to estimate treatment effects of post-midline deactivation.

    “This approach allows us to maintain incentive compatibility in the BDM mechanism, have balance between Treatment and Control groups, and use a straightforward regression to estimate treatment effects of post-midline deactivation.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    For deactivation in weeks 5-8, 0.2 percent of participants were randomly selected to a BDM offer price drawn randomly from p ∈ [0, 170], while the remaining 99.8 percent received offer p = 0.

    “For deactivation in weeks 5-8 (i.e., the four weeks after endline), 0.2 percent of participants were randomly selected to a BDM offer price drawn randomly from p ∈ [0, 170], while the remaining 99.8 percent received offer p = 0.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    Treatment and Control have statistically equal response rates to the endline survey and subjective well-being text messages.

    “Treatment and Control have statistically equal response rates to the endline survey and subjective well-being text messages.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    We balanced the weeks 5-8 offer price p between the weeks 1-4 offer price groups.

    “We balanced this weeks 5-8 offer price p between the weeks 1-4 offer price groups”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Robust standard errors were used in all regressions.

    “We use robust standard errors in all regressions.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisenone given

    The sharpened FDR-adjusted p-values are less conservative than the unadjusted p-values for p-values greater than about 0.15, and more conservative for unadjusted p-values less than that.

    “The sharpened FDR-adjusted p-values are less conservative than the unadjusted p-values for p-values greater than about 0.15, and more conservative for unadjusted p-values less than that.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The subjective well-being measures used are standard measures from the literature.

    “There is a vast literature on measuring subjective well-being (see, for example, Kahneman et al. 2006), and we use standard measures from the literature.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Kahneman et al. 2006

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    We phrased the questions in this way in order to more precisely detect changes in self-reported time use caused by the deactivation.

    “We phrased the questions in this way in order to more precisely detect changes in self-reported time use caused by the deactivation.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data

    A slightly updated pre-analysis plan was submitted on November 7, the day before endline, with only one substantive change: primary specifications would use IV estimates instead of intent-to-treat estimates.

    “We submitted a slightly updated pre-analysis-plan on November 7, the day before endline, with only one substantive change: on the basis of data on reasons for non-compliance described above, we specified that our primary specifications would use IV estimates instead of intent-to-treat estimates.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    We construct indices that combine the outcome variables within each family, weighting by the inverse of the covariance between variables at endline, as described in Anderson (2008).

    “We also construct indices that combine the outcome variables within each family, weighting by the inverse of the covariance between variables at endline, as described in Anderson (2008).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Anderson (2008)

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    In constructing these indices, we orient the variables so that more positive values have the same meaning.

    “In constructing these indices, we orient the variables so that more positive values have the same meaning”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The Treatment group may have avoided logging into other apps such as Spotify and Tinder because participants were informed that using Facebook to actively log into other apps would reactivate Facebook.

    “Furthermore, the Treatment group may have avoided logging into other apps such as Spotify and Tinder because we had informed participants that using Facebook to actively log into other apps would reactivate Facebook.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning

    Deactivating Facebook makes people less likely to be using their phones or computers in the first place, and less likely to follow Facebook links that direct to non-Facebook sites.

    “This makes sense to the extent that deactivating Facebook makes people less likely to be using their phones or computers in the first place, and less likely to follow Facebook links that direct to non-Facebook sites (e.g., a news website or Twitter post).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Facebook is a substitute for offline activities but a complement to other online activities.

    “Overall, these results suggest that Facebook is a substitute for offline activities but a complement to other online activities.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Political polarization may make democratic decision making less efficient, and may lead citizens to perceive democratic outcomes as less legitimate.

    “Political polarization is of interest because it is may make democratic decision making less efficient, and may lead citizens to perceive democratic outcomes as less legitimate (Iyengar, Sood, and Lelkes 2012; Iyengar and Westwood 2015).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Iyengar, Sood, and Lelkes 2012

      Not checked in the free run.

    • cites Iyengar and Westwood 2015

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    Facebook could reduce subjective well-being by reducing in-person interactions, but also impose positive political externalities by increasing news knowledge.

    “This suggests the possibility that Facebook could reduce subjective well-being by reducing in-person interactions, but also impose positive political externalities by increasing news knowledge.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    Both the congenial news exposure and issue polarization effects are highly significant after adjusting for multiple hypothesis testing.

    “Appendix Table A10 shows that both of these effects are highly significant after adjusting for multiple hypothesis testing.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Facebook plays a role in helping people stay informed about current events, but also increases polarization, particularly of views on political issues.

    “Overall, these results suggest that Facebook plays a role in helping people stay informed about current events, but also increases polarization, particularly of views on political issues.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The findings could also reflect a difference between the relatively short-term effects measured in the experiment and the longer-term effects picked up in the cross-section.

    “They could also reflect a difference between the relatively short-term effects measured in our experiment and the longer-term effects picked up in the cross-section.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationown data

    These findings are consistent with reverse causality, for example if people who are lonely or depressed spend more time on Facebook, or with omitted variables, for example if lower socio-economic status is associated with both heavy use and lower well-being.

    “These findings are consistent with reverse causality, for example if people who are lonely or depressed spending more time on Facebook, or with omitted variables, for example if lower socio-economic status is associated with both heavy use and lower well-being.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The magnitude of the SWB effects is moderate and may be smaller than correlation studies would suggest, and the average effect likely masks substantial heterogeneity.

    “However, the magnitude of these effects is moderate and may be smaller than correlation studies would suggest, and our qualitative interviews suggest that the average effect likely masks substantial heterogeneity.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Deactivation eliminated projection bias or facilitated learning that reduced demand for Facebook by 14 percent.

    “This suggests that deactivation eliminated projection bias or facilitated learning that reduced demand for Facebook by 14 percent.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In an online panel weighted for national representativeness, Brynjolfsson, Eggers, and Gannamaneni (2018) estimate that the mean WTA to not use Facebook for one month is $48, and that the median WTA to hypothetically stop using social media for one year was $205 in 2016 and $322 in 2017.

    “In an online panel weighted for national representativeness, Brynjolfsson, Eggers, and Gannamaneni (2018) estimate that the mean WTA to not use Facebook for one month is $48, and that the median WTA to hypothetically stop using social media for one year was $205 in 2016 and $322 in 2017.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Brynjolfsson, Eggers, and Gannamaneni (2018)

      Not checked in the free run.

  • May combine two claims.

    Study design and conductreasoning

    If the anchoring effects decay between midline and endline, this would bias γ away from zero, meaning that the true γ would be less than our estimate.

    “If the anchoring effects decay between midline and endline, this would bias γ away from zero, meaning that the true γ would be less than our estimate”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The estimated magnitudes imply that these negative effects are large enough to be real concerns, but also smaller in many cases than what one might have expected given prior research and popular discussion.

    “The estimated magnitudes imply that these negative effects are large enough to be real concerns, but also smaller in many cases than what one might have expected given prior research and popular discussion.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    If anchoring has the same effects on w2,1 and w2,2 in the Treatment group, then ∆w2 is unaffected, and our estimate of γ is unbiased.

    “If anchoring has the same effects on w 2,1 and w 2,2 in the Treatment group, then ∆w 2 is unaffected, and our estimate of γ is unbiased.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The anchoring effect described above could affect our estimate of γ.

    “One caveat is that the anchoring effect described above could affect our estimate of γ.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Prior literature shows that around 25-50 percent of non-voters tend to incorrectly report on surveys that they voted.

    “The results are in line with prior literature showing that around 25-50 percent of non-voters tend to incorrectly report on surveys that they voted (Belli et al. 1999; DellaVigna et al. 2017; Duff et al. 2007; Silver, Anderson, and Abramson 1986).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Belli et al. 1999

      Not checked in the free run.

    • cites DellaVigna et al. 2017

      Not checked in the free run.

    • cites Duff et al. 2007

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    Facebook can improve people's lives in diverse ways, whether as a source of entertainment, a means to organize a charity or an activist group, or a vital social lifeline for those who are otherwise isolated.

    “Our participants' answers in free response questions and follow-up interviews make clear the diverse ways in which Facebook can improve people's lives, whether as a source of entertainment, a means to organize a charity or an activist group, or a vital social lifeline for those who are otherwise isolated.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Endline survey completion rates were 99% in Treatment and 98% in Control, with no significant difference (p=0.54).

    “Completed endline survey | 0.99 | 0.98 | 0.54”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    The Treatment group deactivated Facebook on 90% of days, while the Control group did so on 2% of days, a significant difference (p=0.00).

    “Share days deactivated | 0.90 | 0.02 | 0.00***”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Age and political party were the 'secondary' moderators in the pre-analysis plan.

    “Age and political party were the “secondary” moderators in our pre-analysis plan.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    This RCT was registered in the American Economic Association Registry for randomized control trials under trial number AEARCTR-0003409.

    “This RCT was registered in the American Economic Association Registry for randomized control trials under trial number AEARCTR-0003409.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Share of text messages completed was 92% in Treatment and 93% in Control, with no significant difference (p=0.45).

    “Share of text messages completed | 0.92 | 0.93 | 0.45”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The study was approved by Institutional Review Boards at Stanford (eProtocol #45403) and NYU (IRB-FY2018-2139).

    “The study was approved by Institutional Review Boards at Stanford (eProtocol #45403) and NYU (IRB-FY2018-2139).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Willingness-to-accept and sample weight were not defined as moderators of interest in the pre-analysis plan.

    “Willingness-to-accept and sample weight were not defined as moderators of interest in our pre-analysis plan.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Post-endline survey completion rates were 95% in Treatment and 92% in Control, with a marginally significant difference (p=0.07).

    “Completed post-endline survey | 0.95 | 0.92 | 0.07*”
    Quote in the paperNot a substantial claim
132 of 132

  • The paper relies on this without support.

    Deactivating Facebook for four weeks is a valid way to measure the welfare effects of Facebook use.

    “In a randomized experiment, we find that deactivating Facebook for the four weeks before the 2018 US midterm election”

  • The paper relies on this without support.

    Self-reported time use and well-being measures accurately reflect the underlying constructs.

    “We measured a suite of outcomes using text messages, surveys, emails, direct measurement of Facebook and Twitter activity, and administrative voting records.”

  • The paper relies on this without support.

    The public profile URL returns a valid response if and only if the account is active.

    “the public profile URL returns a valid response if and only if their account is active”

  • The paper relies on this without support.

    Participants' self-reported Facebook usage is accurate.

    “Several weeks later, the Treatment group's reported usage of the Facebook mobile app was about 11 minutes (22 percent) lower than in Control.”

  • The paper relies on this without support.

    Well-functioning democratic societies fundamentally rely on well-informed voters who actually show up to the polls to vote.

    “News knowledge and political engagement are of interest because well-functioning democratic societies fundamentally rely on well-informed voters who actually show up to the polls to vote.”

  • The paper relies on this without support.

    Very few Americans contribute to political campaigns.

    “In our pre-analysis plan, we labeled this as secondary because very few Americans contribute to political campaigns, and we did not expect to be able to detect effects from four weeks of deactivation.”

  • The paper relies on this without support.

    The L2 voting database accurately matches participants and records voting.

    “we measure whether participants voted in the 2018 midterm election, by matching participants on name, birth year, and zip code to a voting database supplied to Stanford by L2, a voting data provider.”

  • The paper relies on this without support.

    The 15-question news knowledge quiz accurately measures news knowledge.

    “In order to measure specific news knowledge, we included a 15-question news knowledge quiz.”

  • The authors state this as an assumption.

    Willingness-to-accept to abstain from Facebook equals consumer surplus.

    “In a standard model, willingness-to-accept to abstain from Facebook equals consumer surplus.”

  • The authors state this as an assumption.

    Marginal treatment effects are monotonic in WTA.

    “Under the assumption that marginal treatment effects are monotonic in WTA, treatment effect heterogeneity within our impact evaluation sample would be informative about treatment effects for the full population.”

  • The authors state this as an assumption.

    The time effect is the same in both Treatment and Control groups.

    “If the time effect is the same in both groups, then the difference in differences measures the effect of deactivation on valuations due to projection bias, learning, and similar mechanisms.”

  • The authors state this as an assumption.

    The Control group's change in WTA for weeks 5-8 captures any unpredicted time effect.

    “The Control group's change in WTA for weeks 5-8, ∆w 2 := w 2,2 -w 2,1 , captures any unpredicted time effect.”

  • The authors state this as an assumption.

    The Treatment group's WTA change reflects both the time effect and the unexpected change in valuation caused by deactivation.

    “The Treatment group's WTA change ∆w 2 reflects both the time effect and the unexpected change in valuation caused by deactivation.”

  • The paper relies on this without support.

    The L2 voting database accurately records whether individuals voted.

    “we matched the participants in our experiment to a voting database supplied to Stanford by L2, a voting data provider.”

  • The authors state this as an assumption.

    Participants who could not be matched to entries in the L2 database did not vote.

    “Column 2 includes the entire impact evaluation sample, assuming that participants who could not be matched to entries in the L2 database did not vote.”

  • The paper relies on this without support.

    The L2 database only contains records of registered voters.

    “Since the L2 database only contains records of registered voters”
16 of 16

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Experimental evidence on the link between social media use and well-being is mixed.

    “while experimental evidence is mixed8,9.”

    From Windows of developmental sensitivity to social media (2022) · cites this paper as [9]

    The Introduction and §5.3 report experimental evidence that Facebook deactivation improved several well-being measures, while effects measured by daily text messages were not statistically significant. This supports variation across measures in this experiment, but the paper does not establish that experimental evidence across studies is mixed. (LLM, conf=0.80)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Polarization benchmark ends in two different years

    Minor

    The introduction says the comparison index rose by 0.38 standard deviations between 1996 and 2018, while the detailed discussion attributes that increase to 1996-2016.

  • Reported increases in depression and anxiety conflict with the claimed improvements

    Minor

    The figure text defines higher raw responses as feeling depressed or anxious more often, then says deactivation increased those responses. That conflicts with the paper's conclusion that deactivation improved depression and anxiety; the text does not clarify that the stated increases refer to reverse-coded scores.

2 of 2

Cites

Flagged first, then by citations

2 of the 121 references couldn’t be matched to a known paper.

Research works printed but not linked: 15

  1. Argyle, Michael. 2001. The Psychology of Happiness. Routledge.
  2. Burke, Moira and Robert E. Kraut. 2014. "Growing Closer on Facebook: Changes in Tie Strength Through Social Network Site Use." In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 4187-4196.
  3. Burke, Moira, Robert E. Kraut, and Cameron Marlow. 2011. "Social Capital on Facebook: Dif- ferentiating Uses and Users." In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 571-580.
  4. DellaVigna, Stefano, John A. List, Ulrike Malmendier, and Gautam Rao. 2017. "Voting to Tell Others." Review of Economic Studies 84.
  5. Enikolopov, Ruben and Maria Petrova. 2015. "Media Capture: Empirical Evidence." In Handbook of Media Economics, vol. 1. Elsevier, 687-700.
  6. ---. 2018. "Facebook Reports Third Quarter 2018 Results." Press Release. https://investor.fb.com/investor-news/press-release-details/2018/Facebook-Reports-Third- Quarter-2018-Results/default.aspx.
  7. Hughes, Mary Elizabeth, Linda J. Waite, Louise C. Hawkley, and John T. Cacioppo. 2004. "A Short Scale for Measuring Loneliness in Large Surveys: Results From Two Population-Based Studies." Research on Aging 26 (6):655-672.
  8. Iyengar, Shanto, Gaurav Sood, and Yphtach Lelkes. 2012. "Affect, Not Ideology: Social Identity Perspective on Polarization." Public Opinion Quarterly 76 (3):405-431.
  9. Newport, Cal. 2019. Digital Minimalism: Choosing a Focused Life in a Noisy World. Penguin Random House.
  10. ---. 2018c. "Public Is Skeptical of the Iran Agreement -and Trump's Handling of the Issue." http://www.people-press.org/2018/05/08/public-is-skeptical-of-the-iran-agreement-and- trumps-handling-of-the-issue/.
  11. ---. 2018e. "Shifting Public Views on Legal Immigration Into the U.S." http://www.people- press.org/2018/06/28/shifting-public-views-on-legal-immigration-into-the-u-s/.
  12. Stone, Arthur A. and Saul Shiffman. 1994. "Ecological Momentary Assessment (EMA) in Behavorial Medicine." Annals of Behavioral Medicine 16 (3):199-202.
  13. Strömberg, David. 2015. "Media and Politics." Annual Review of Economics 7 (1):173-205.
  14. Sunstein, Cass. 2001. Republic.com. Princeton University Press.
  15. Verduyn, Phuluppe, David Seungjae Lee, Jiyoung Park, Holly Shablack, Ariana Orvell, Joseph Bayer, Oscar Ybarra, John Jonides, and Ethan Kross. 2015. "Passive Facebook Usage Under- mines Affective Well-Being: Experimental and Longitudinal Evidence." Journal of Experimental Psychology 144 (2):480-488.

Cited by

Sorted by citations

Related

Authors
Allcott, H., Braghieri, L., Eichmeyer, S., & Gentzkow, M.
Affiliations
National Bureau of Economic Research (US)
ORCID
Venue
National Bureau of Economic Research
Publisher
National Bureau of Economic Research
Published
2019-01
DOI
10.3386/w25514
OpenAlex
W3123246648
Cited by
81
References
121
Citations by year
2026: 1 · 2025: 3 · 2024: 15 · 2023: 11 · 2022: 14 · 2021: 14 · 2020: 10 · 2019: 10 · 2018: 1 · 2017: 2
Type
report
Language
en
Open access
Locations
National Bureau of Economic Research (publishedVersion)
Funders
  • Alfred P. Sloan Foundation
  • John S. and James L. Knight Foundation
  • Microsoft Research
Topics
  • Social Sciences › Sociology and Political Science › Impact of Technology on Adolescents
  • Social Sciences › Communication › Social Media and Politics
  • Physical Sciences › Computer Science › Computer Science Applications › Digital Media and Philosophy
Keywords
social media, subjective well-being, political polarization, consumer surplus, social media addiction, 2018 midterm elections

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review