Cited by 30

The DOI matches its Crossref record.

Content Moderation Remedies

Goldman, Eric·2021·Michigan Technology Law Review

Abstract

This Article addresses a critical but underexplored aspect of content moderation: if a user’s online content or actions violate an Internet service’s rules, what should happen next? The longstanding expectation is that Internet services should remove violative content or accounts from their services as quickly as possible, and many laws mandate that result. However, Internet services have a wide range of other options—what I call “remedies”—they can use to redress content or accounts that violate the applicable rules. This Article describes dozens of remedies that Internet services have actually imposed. It then provides a normative framework to help Internet services and regulators navigate these remedial options to address the many difficult tradeoffs involved in content moderation. By moving past the binary remove-or-not remedy framework that dominates the current discourse about content moderation, this Article helps to improve the efficacy of content moderation, promote free expression, promote competition among Internet services, and improve Internet services’ community-building functions.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.36645/mtlr.28.1.content

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 16

  1. Audra Schroder, People Risking Concussions for New TikTok Challenge, DAILY DOT (May 19, 2021, 10:32 PM), https://www.dailydot.com/unclick/tiktok-basketball- throwing-meme. 137. In a non-remedial context, Google News once allowed people quoted in news stories to provide additional context for their quotes. Perspectives About the News from People in the News, GOOGLE NEWS BLOG (Aug. 7, 2007), https://news.googleblog.com/2007/08/perspectives-about- news-from-people-in.html. 138. Fact-Checking on Facebook: What Publishers Should Know, FACEBOOK FOR BUS., https://www.facebook.com/help/publisher/182222309230722 (last visited Oct. 23, 2021);
  2. Catherine Shu, Reddit Replaces Its Confusing Shadowban System with Account Suspensions, TECHCRUNCH (Nov. 11, 2015, 11:11 PM), https://techcrunch.com/2015/11/11 /reddit-account-suspensions. 149. Help with Locked or Limited Account, TWITTER HELP CTR., https://help.twitter.com/en/managing-your-account/locked-and-limited-accounts (last visited Oct. 23, 2021). 150. Guy Rosen, Protecting Facebook Live from Abuse and Investing in Manipulated Media Research, FACEBOOK NEWSROOM (May 14, 2019), https://about.fb.com/news/2019/05 /protecting-live-from-abuse ("
  3. nyone who violates our most serious policies will be restricted from using Live for set periods of time -for example 30 days -starting on their first offense."). 151. Community Guidelines Strike Basics, YOUTUBE HELP, https://support.google.com /youtube/answer/2802032?hl=en (last visited Oct. 23, 2021). 152. Verification FAQ, TWITTER HELP CTR., https://help.twitter.com/en/managing-your- account/twitter-verified-accounts (last visited Oct. 23, 2021). The stated reasons are: "Impersonation or intentionally misleading people on Twitter by changing your display name or bio; Violations that result in immediate account suspension; [orRepeat violations in Tweets, including but not limited to: hateful conduct policy, abusive behavior, glorification of violence policy, civic integrity policy, private information policy, or platform manipulation and spam policy." Id. See generally Thomas Kadri, Speech vs. Speakers, SLATE (Jan. 18, 2018, 12:56 PM), https://slate.com/technology/2018/01/twitters-new-rules-blur-the-line- between-extremists-speakers-and-their-speech.html (explaining why de-verification might be counterproductive). Twitter has since updated its "blue-check" policy. Our Plans to Relaunch Verification and What's Next, TWITTER BLOG (Dec. 17, 2020), https://blog.twitter.com/en_us/topics /company/2020/our-plans-to-relaunch-verification-and-whats-next.html. 158. See Abuse and Site Rules, EPIFAQ, http://epifaq.pbworks.com/w/page/11116355 /Abuse_and_Site_Rules#ticketsdo (Jan. 4, 2007). 159. Account Status, RUNESCAPE WIKI, https://runescape.fandom.com/wiki/Account_Status (last visited Oct. 23, 2021). 160. Mnookin, supra note 35, at n.44. 161. Services can quantify an accountholder's reputation and reduce the visibility of low-reputation accountholders. "Karma" sometimes describes quantified reputations, and rule violations can be incorporated into a karma score to influence future visibility. F. RANDALL FARMER & BRYCE GLASS, BUILDING WEB REPUTATION SYSTEMS 72-73 (2010). 162. E.g., G.F., What is "Shadowbanning"?, ECONOMIST (Aug. 1, 2018), https://www.economist.com/the-economist-explains/2018/08/01/what-is-shadowbanning ("Shadowbanned users are not told that they have been affected. They can continue to post messages, add new followers and comment on or reply to other posts. But their messages may not appear in the feed, their replies may be suppressed and they may not show up in searches for their usernames.");
  4. D.N.H. Jan. 28, 2021) ("Shadow banning is the act of blocking or partially blocking a user or their content from an online audience in a manner that is not readily apparent to the user. The user believes they are posting content normally, when in reality other people cannot see the posted content."). For a history of shadowbanning and related remedies, see Samantha Cole, Where Did the Concept of 'Shadow Banning' Come From?, MOTHERBOARD: TECH BY VICE (July 31, 2018, 9:00 AM), https://www.vice.com/en_us/article/a3q744/where-did-shadow-banning-come- from-trump-republicans-shadowbanned. 163.
  5. See Nicolas P. Suzor et al., What Do We Mean When We Talk About Transparency? Towards Meaningful Transparency in Commercial Content Moderation, 13 INT'L J. COMMC'N. 1526, 1531 (2019). 164. E.g., FLA. STAT. ANN. § 501.2041(1)(f) (West 2021) (defining "shadow ban" as "action by a social media platform, through any means, whether the action is determined by a natural person or an algorithm, to limit or eliminate the exposure of a user or content or material posted by a user to other users of the social media platform. This term includes acts of shadow banning by a social media platform which are not readily apparent to a user.");
  6. Chanté Joseph, Instagram's Murky 'Shadow Bans' Just Serve to Censor Marginalised Communities, GUARDIAN (Nov. 8, 2019, 11:03 AM), https://www.theguardian.com • When content violates Blogger's policies, Blogger may unpublish the content so that it is visible only to its author. 171
  7. userpage."). Reddit abandoned shadowbanning in 2015. Shu, supra note 148. 167. Alex Hern, Revealed: How TikTok Censors Videos That Do Not Please Beijing, GUARDIAN (Sept. 25, 2019, 12:00 AM), https://www.theguardian.com/technology/2019/sep /25/revealed-how-tiktok-censors-videos-that-do-not-please-beijing. 168.
  8. WAGNER ET AL., supra note 105, at 17. 169. KRAUT & RESNICK, supra note 118, at 137 ("[I]n a chat room, the gagged person may see an echo of everything he or she types, but his or her comments may not be displayed to others in the room. The gagged person may think that everyone is just ignoring her."). 170. E.g., Chipteck, Chat Restrictions, RIOT GAMES (Mar. 18, 2013, 4:40 PM), https://support.riotgames.com/hc/en-us/articles/201752984-Chat-Restrictions; u/Rocket_Sciencetist, Muting Policy, REDDIT (June 28, 2017), https://www.reddit.com/r/TagPro/comments/6k6k3o/muting _policy; see also Mute, FANDOM OLD SCHOOL RUNESCAPE WIKI, https://oldschoolrunescape.fandom.com/wiki/Mute (last visited Oct. 23, 2021). 171. Blogger Content Policy, BLOGGER, https://www.blogger.com/content.g?hl=en (last visited Oct. 23, 2021). 172. Block Search Indexing with 'Noindex', GOOGLE SEARCH CENT., https:// developers.google.com/search/docs/advanced/crawling/block-indexing (last updated Nov. 22, 2021). 173. Case C-131/12, Google Spain SL v. Agencia Española de Protección de Datos, ECLI:EU:C:2014:317, ¶ ¶ 89-99 (May 13, 2014); see Requests to Delist Content Under European Privacy Law, GOOGLE TRANSPARENCY REP., https://transparencyreport.google.com/eu-privacy /overview?hl=en (last visited Oct. 2, 2021). 174. See Zoe Greenberg, Boston Globe Launches 'Fresh Start' Initiative: People Can Apply to Have Past Coverage About Them Reviewed, MSN (Jan. 22, 2021), https://www.msn.com/en-us/news/us/boston-globe-launches-fresh-start-initiative-people-can- apply-to-have-archive-stories-about-them-reviewed/ar-BB1cYPrp. 175.
  9. See, e.g., Eugene Volokh, Can You Get a Court to Take an Opinion That Mentions You Off Its Google-Searchable Website?, WASH. POST (May 3, 2017), https://www.washingtonpost.com/news/volokh-conspiracy/wp/2017/05/03/can-you-get-a- court-to-take-an-opinion-that-mentions-you-off-its-google-searchable-website. 176. KRAUT & RESNICK, supra note 118, at 154-55. 177. Id.; Qualify Your Outbound Links to Google, GOOGLE SEARCH CENT., https://support.google.com/webmasters/answer/96569?hl=en (last updated Aug. 26, 2021). 178. Bobbie Johnson, Wikipedia Adopts 'Nofollow', GUARDIAN (Jan. 22, 2007, 7:19 AM), https://www.theguardian.com/technology/blog/2007/jan/22/wikipediaadopt. See generally Eric Goldman, Wikipedia's Labor Squeeze and Its Consequences, 8 J. TELECOMM. & HIGH TECH. L. 157, 163 (2010) (discussing Wikipedia's efforts to suppress spam). 192. Reviews, EPIFAQ, http://epifaq.pbworks.com/w/page/11116374/Reviews#nomoney (last visited Oct. 2, 2021). 193. Ratings and the Web of Trust, EPIFAQ, http://epifaq.pbworks.com/w/page /11116373/Ratings_and_Web_of_Trust#communitystandards (last visited Oct. 2, 2021). Slashdot uses a similar scoring system to hide lowly-rated user comments from public view.
  10. See WAGNER ET AL., supra note 105, at 20. 194. Why Your AdSense Account Has a Payment Hold, GOOGLE ADSENSE HELP, https://support.google.com/adsense/answer/1714364?hl=en (last visited Oct. 24, 2021). 195. E.g., Julia Alexander, YouTube Looks to Demonetization as Punishment for Major Creators, But It Doesn't Work, VERGE (June 25, 2019, 12:27 PM), https://www.theverge.com /2019/6/25/18744246/youtube-demonetization-steven-crowder-patreon-advertising-merch. 196.
  11. Benjamin Goggin & Kat Tenbarge, 'Like You've Been Fired from Your Job': YouTubers Have Lost Thousands of Dollars After Their Channels Were Mistakenly Demonetized for Months, BUS. INSIDER (Aug. 24, 2019, 9:31 AM), https://www.businessinsider.com/youtubers-entire- channels-can-get-mistakenly-demonetized-for-months-2019-8. YouTube also demonetizes extremist religious or supremacist content. Walker, supra note 134. 197. What is a Content ID Claim?, YOUTUBE HELP, https://support.google.com/youtube /answer/6013276?hl=en (last visited Oct. 2, 2021). 198. Quarantined Subreddits, supra note 133. 215 207. Brendan Maher, Good Gaming, 531 NATURE 568, 570 (Mar. 30, 2016), https://www.nature.com/articles/531568a. 208. See supra Part II(A)(7). 209. See supra Part II(A)(1). 210. Community Guidelines Strike Basics, supra note 151; see also YouTube Report, supra note 22, at 16. 211. See infra Part IV(A)(7) (revisiting the implications of extra-judicial consequences from outing). 212.
  12. Charlie Warzel, Could Restorative Justice Fix the Internet?, N.Y. TIMES (Aug. 20, 2019), https://www.nytimes.com/2019/08/20/opinion/internet-harassment-restorative-justice.html. 221. Bettina Makalintal, How YouTubers Turned the Apology Video Into a Genre, VICE (June 18, 2019, 1:53 PM), https://www.vice.com/en_us/article/ywykzb/how-youtubers-james- charles-jaclyn-hill-pewdiepie-turned-the-apology-video-into-a-genre. 226. This list is meant to be illustrative, not exhaustive. For example, Google/YouTube says they "value openness and accessibility," "respect user choice," and "build for everyone." YouTube Report, supra note 22, at 3. Elsewhere, it notes the importance of worker wellness. Id. at 16-18. 227. Land & Hamilton, supra note 108, at 148 ("Proportional responses are required both by international human rights law as well as the law of remedies in international law more generally"). Marique & Marique summarize the principles: [T]he proportionality test implies that decision-makers should only follow a course of action if: 1) their objective is legitimate; 2) their means is necessary to achieve the objective; 3) no means would entail a lighter encroachment of the right at stake;
  13. Mark Spottswood, Continuous Burdens of Proof, 21 NEV. L.J. 779, 829 (2021) (discussing how burdens of proof could be a continuum rather than binary options). 237. Douek, supra note 33, at 816. 238. For example, the Wikimedia community has repeatedly resolved copyright disputes through its own independent research without intervention by Wikimedia employees or the courts. See Stories, WIKIMEDIA FOUND. TRANSPARENCY REP., https://transparency-archive.wikimedia.org/stories.html (last visited Oct. 25, 2021). 239. Internet services' willingness to over-remove has been weaponized in the past. See
  14. T.C., What is the Streisand Effect?, ECONOMIST (Apr. 16, 2013), https://www.economist.com/the-economist-explains/2013/04/15 /what-is-the-streisand-effect. 261. KRAUT & RESNICK, supra note 118, at 152-53. 262. IAN AYRES & JOHN BRAITHWAITE, RESPONSIVE REGULATION: TRANSCENDING THE DEREGULATION DEBATE 35-38 (Oxford Univ. Press 1992). 263. John Braithwaite, The Essence of Responsive Regulation, 44 U.B.C. L. REV. 475, 482 (2011).
  15. Howard Rheingold, Bozo Filters, WIRED (Jan. 1, 1993, 12:00 PM), https://www.wired.com/1993/01/bozo-filters. 297.
  16. Jack Dorsey (@Jack), TWITTER (Jan. 13, 2021, 7:16 PM), https://twitter.com/jack /status/1349510769268850690. Services deploying related concepts include diaspora* and Mastadon. WAGNER ET AL., supra note 105, at 16, 18. 304.

Cited by

Sorted by citations

Metadata

Authors
Goldman, E.
Affiliations
Santa Clara University School of Law
Venue
Michigan Technology Law Review
Published
2021
Issue
28.1
Pages
1
ISSN
  • 2688-5484
  • 2688-4941
Cited by
30
References
0
Language
en

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review