Studying Reddit: A Systematic Overview of Disciplines, Approaches, Methods, and Ethics

“Studies such as these suggest that researchers may need to consider the generalizability and representativeness of models built using Reddit data, particularly in the context of ongoing conversations regarding language modeling (see Bender et al., 2021).”

“The Pushshift dataset contains submissions and comments posted on Reddit since June 2005, and has been popular for researchers due to its ease of use and larger querying limits (Baumgartner et al., 2020).”

“Tufekci (2014) once called Twitter the “model organism” for academic study because tweets are considered to be “public,” because Twitter has open APIs which fosters easy data collection, and because Twitter users often respond to world events as they unfold, making it a useful location to gather…”

“For example, personally identifiable information from more than 87 million Facebook users was collected in an academic study, but then the data were used by Cambridge Analytica to micro-target political advertisements (Isaak & Hanna, 2018).”

“Moreover, the smaller the community being studied in any context, the more difficult it is to maintain the anonymity of research subjects (Saunders et al., 2015), even without the additional complication of direct quotes being discoverable by search engines (Bruckman, 2002; Markham, 2012).”

“We selected Reddit as our study site due to the availability of public data on millions of unique online spaces called subreddits (or “subs”), the diversity of types of subs that exist, the increasing presence of governance bots present in subs, and growing research interest in Reddit [75] and human-bot interaction, m…”

“We collected a dataset of 13,138 Reddit posts on relationship issues spanning from 2015 to 2023 with Reddit APIs, the official study tools for researchers (Proferes et al. 2021).”

“Big data often comprises countable metrics, such as the number of likes or replies or automated analysis of content (often based on word counts) as measures of interaction behaviour (see, for example, a recent review of Reddit studies: Proferes et al., 2021).”

“Based on the fact that the data we are utilizing is publicly available and does not include any sensitive information, such as physiological signals, the committee deemed it was exempted from human subjects review, according to the interpretation shared by other IRBs [59].”

“These alternative sources include public forums, news aggregators or archives (Boumans & Trilling, 2016), specialised online communities like Reddit (Proferes et al., 2021), and other digital platforms like Google Trends (Prado-Román et al., 2021) that offer insights into public discourse and social trends.”

“That is, just because the data are public, it does not mean that the use of data may not cause harm (e.g. Proferes et al., 2021).”

“Reddit is a frequent research site in HCI [115], where researchers often explore questions related to moderation and governance [128, 162], rules and norms [27, 45], and computational social science [12].”

“Note that we included an additional platform, Reddit, owing to its popularity within the U.S. social media ecosystem [39] even though it was not classified as a VLOP in the DSA.”

“Our Institutional Review Board (IRB) deemed our study exempt from human subject review, recognizing our use of non-sensitive, publicly accessible data, in line with standard IRB protocols [97].”

“Off of Reddit, rules have been studied on peer production communities such as Wikipedia & Wikia (Hwang and Shaw 2022; Keegan and Fiesler 2017; Kittur and Kraut 2010), the fediverse (Nicholson, Keegan, and Fiesler 2023), and gaming communities (Frey and Sumner 2018; Frey et al. 2022).”

“Reddit users may post links, videos, news, pictures, or texts in subreddits (forums ordered by topic) where other users can respond via comments and interact with upvotes or downvotes to the post, indicating popularity of the content (e.g., Proferes et al., 2021).”

“In this research, we have considered social media posts extracted from Reddit [3-5].”

“Data provided by the Pushshift project has been extensively used in prior research 6 .”

“Reddit uses a source of textual data for different types of academic analyses and has been growing over the last two decades (Proferes et al., 2021).”

“We heed the advice from social media ethicists (Proferes et al. 2021) and model papers in ICWSM and complementary venues (Chandrasekharan et al. 2017; Li, Hecht, and Chancellor 2022) and adopt their recommendations.”

“Proferes et al. (2021) and Amaya et al. (2021) note that Reddit offers many advantages for researchers.”

“But are such paraphrases effective? Reagle (2021b) interrogated 22 Reddit research reports: three of light disguise, using verbatim quotes, and 19 of heavier disguise, claiming to use reworded phrases. They concluded that disguising sources is effective only if done and tested rigorously because they located all of th…”

“Our work builds off of consent design by suggesting a meta-theoretical framework to evaluate and compare prevalent consent procedures, such as heavily scrutinized Terms of Service agreements [26, 49, 64, 79].”

“Following the interpretation common among various IRBs [42], the IRB concluded that our study did not require human subjects review, as our research utilized publicly available data and they not contain sensitive information like physiological signals.”

“This study uses big data analysis of Reddit posts, a platform for diverse, anonymous consumer views (Proferes et al. 2021; Khuman 2024).”

“In addition, users of Reddit on average tend to be younger and more male, which may also impact our findings [81].”

“This is due largely to it’s sub-reddits that support communities focused on particular topics of shared interest or concern [2, 62, 71].”

“Reddit is one of the most popular websites and has also become one of the leading data sources in recent HCI research [56].”

“A sizable portion of social science research involving big data has used datasets obtained from publicly accessible platforms such as social media websites, including Twitter and Reddit (Golder and Macy, 2014; Proferes et al., 2021).”

“survey paper that looked at the types of ethical considerations that researchers made regarding their Reddit data use [36].”

“…et al. 2024; Mehrotra, Degachi, et al. 2024; Vereschak et al. 2021) and best research practices to study a particular topic (Ajmani et al. 2023; Proferes et al. 2021), we define our inclusion criteria as: (I1) One of the contributions of the article is linked to understanding trustworthiness…”

“As one of the most popular social media platforms, Reddit has been the main venue for sharing and discussing many controversial or high-profile events worldwide (Proferes et al. 2021).”

“To take steps to protect the anonymity of the Redditors we quote, the message excerpts are edited and paraphrased, and at times we resort to only describing the messages: both approaches are common and preferred ways to protect users in Reddit research (Fiesler et al., 2024; Proferes et al., 2021).”

“…NLP techniques have been successfully used in previous studies to identify patterns and themes in social media data related to mental health and substance use [28], demonstrating their effectiveness in uncovering subtle linguistic and thematic patterns that might be overlooked in manual analysis.”

“Given recent discussions on data collection involving social media groups such as Reddit [44], we opted to paraphrase all quotes that we include below in support of our themes.”

“The wide thematic spectrum of Reddit conversations enables us to deepen our comprehension of human communication within communities (31): for instance it was shown how users become more intuitive and express their sadness during COVID-19 warning and lockdown phase (32) or how users try to shift the…”

“Additionally, a recent review of existing empirical studies utilizing Reddit data, warn that much of the knowledge generated about Reddit content is gained via computational textual analysis and encourage further contextualization of this data (Proferes et al. 2021).”

“Reddit has been the focal site of numerous studies interested in politics, political identity formation, and extremism (Proferes et al. 2021).”

“The use of Reddit as a data source for academic research has risen over the past decade, with a recent review showing the number of such annual publications ballooned from two in 2010 to 230 in 2019 (Proferes et al. 2021).”

“Because of these characteristics, increasingly more research studies are based on Reddit data, including health-related applications [16].”

“The PCOS subreddit [16], hosted on Reddit, was chosen due to its large number of participants (approximately 30,000), public availability of posts and comments, and encouragement from the platform curators to access and use the data.”

“This function supports sharing research outcomes back to the community, which by definition involves the community, but the onus is on the research to demonstrate its value to online communities [80].”

“A range of reviews have also identified the methods (e.g., Borah, 2017; Neumayer & Rossi, 2016; Özkula et al., 2022; Proferes et al., 2021) and platforms (Pearce et al., 2019; Puschmann & Pentzold, 2020) commonly used in this field.”

“Namely, we collected data from a large, public community, assured no personal identifying information was associated with the data, and we plan to share our results back to Reddit upon completion (Proferes et al. 2021).”

“Reddit is a popular social media platform that is frequently studied in the computational social sciences (Pro-feres et al. 2021).”

“This issue of youth on Reddit has yet to be addressed by ethical reviews of such methods (Proferes et al. 2021), but the IRB for this project included removing posts by anyone known to be under age 18.”

“The Reddit sample, in other words, has benefit of circumscribing attention to a specific audience (Proferes et al., 2021) that tends to reflect the most devoted or “hardcore” gamers; however, this also means, conversely, that there will be less reflection of causal gamer communities reflected in the dialogue (Bergstro…”

“In line with HCI Reddit-research ethics [33, 87], we analyzed only publicly visible posts and reviewed the community rules of each subreddit included in our dataset.”

“We seek to follow the suggestions provided by Proferes et al. [55], with regards to pro-social engagement with a Reddit community, including by sharing our research back with the community, post-publication.”

“Due to its user-driven nature, Reddit has become a popular site for the study of online communities and understanding how users interact and share information in digital spaces (Proferes et al., 2021).”

“Given the sensitivity of the topic, we prioritized user privacy by omitting user - names and subreddit names and avoiding verbatim quotations in our reporting (Hintz & Betts, 2022; Proferes et al., 2021 ).”

“…important platform for studying modern digital economics and politics and a valuable research ground for interdisciplinary approaches to studying societal trends, cultural phenomena, as well as political and regulatory contentions (cp. Logie, 2014; Proferes et al., 2021; Waller & Anderson, 2021).”

“This structure, combined with Reddit’s scale and cultural impact, supports a wide range of qualitative, quantitative, and computational research on content and user behavior [59].”

“Reddit data is publicly available and frequently used in research [26, 27, 32].”

“To this end, we have undertaken several measures to ensure the respect, dignity, and anonymity of the individuals whose experiences are central to our study (see, for example, Proferes et al., 2021).”

“As Proferes et al. (2021) warn in their meta-study on Reddit research, “each subreddit has their own individual norms and cultures, as well as moderation practices, meaning insights from social phenomena in one subreddit may not translate across contexts” (p. 1).”

“Moreover, users benefit from a level of anonymity on Reddit not typically accomplished on other social media platforms [4],”

“Further, while we followed Reddit’s terms of service in the collection and use of this data, important questions around informed consent in the use of the resulting insights and models of this data need to be discussed [63].”

“Reddit has increasingly been recognized as an important resource for researchers [21,22], including research into childhood and parenting [14,23].”

“Those demographics are representative of the Reddit userbase [15].”

“, 2019b) and first systematic considerations of SM and ethics have been made, for example, by observing the presence of ethical approvals (Fiesler and Proferes, 2018; Proferes et al., 2021), there is no systematic overview on ethical considerations in CMH studies.”

“Even then, it has a lot of societal importance since, as prior works show, such debate forums in themselves are extremely important to shape the Internet and public opinion (Proferes et al. 2021).”

“There are also issues of privacy; while Reddit is anonymous, there are risks of revealing sensitive information or the identities of the accounts used in the study (Proferes et al., 2021).”

“Reddit is a particularly popular source for research due to its community structure and the relative accessibility of data [55].”

“Ethical approval was not required as the study did not involve interaction with human subjects and used publicly accessible data [36].”

“…often raises reporting of stigmatized conditions but creates trade-offs for follow-up and data utility; in social media, real-name policies suppress certain disclosures among vulnerable groups while pseudonymous spaces (e.g., Reddit) enable higher rates of stigmatized disclosure [3, 86].”

“There is also significant variation across subreddits which have their own distinctive ‘norms and cultures, as well as moderation practices’ (Proferes et al., 2021: 1).”

“However, social media scholars have argued that the publicness of online posts should not be taken as implicit consent to use such material in research (Fiesler and Proferes, 2018; Proferes et al., 2021).”