CoChat
Sign Up
PreprintCited by 1

The DOI matches its DataCite record.

Uncovering the Sociodemographic Fabric of Reddit

Cinus, Federico, Monti, Corrado, Bajardi, Paolo, Morales, Gianmarco De Francisci·2025·arXiv
DOI 10.48550/arxiv.2502.05049arXiv 2502.05049
Open this paper in CoChat — free account

Abstract

Understanding the sociodemographic composition of online platforms is essential for accurately interpreting digital behavior and its societal implications. Yet, current methods often lack the transparency and reliability required, risking misrepresenting social identities and distorting our understanding of digital society. Here, we introduce a principled framework for sociodemographic inference on Reddit that leverages over 850,000 user self-declarations of age, gender, and partisan affiliation. By training models on sparse user activity signals from this extensive, self-disclosed dataset, we demonstrate that simple probabilistic models, such as Naive Bayes, outperform more complex embedding-based alternatives. Our approach improves classification performance over the state of the art by up to 19% in ROC AUC and maintains quantification error below 15%. The models produce well-calibrated and interpretable outputs, enabling uncertainty estimation and subreddit-level feature importance analysis. More broadly, this work advocates for a shift toward more ethical and transparent computational social science by grounding sociodemographic analysis in user-provided data rather than researcher assumptions.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2502.05049

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Our dataset consists of voluntarily posted Reddit content, which reduces social desirability and recall bias compared to surveys or interviews and can lead to more candid reflections.

    “Our dataset consists of voluntarily posted Reddit content, which offers advantages because posts are shared anonymously and at the time the user wanted to share. This reduces social desirability and recall bias compared to surveys or interviews and can lead to more candid reflections [37, 134].”

    From Understanding Teen Overreliance on AI Companion Chatbots Through Self-Reported Reddit Narratives (2026) · cites this paper as [37]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 42

  1. Ruben L Bach, Christoph Kern, Ashley Amaya, Florian Keusch, Frauke Kreuter, Jan Hecht, and Jonathan Heinemann. Predicting voting behavior using digital trace data. Social Science Computer Review, 39(5):862-883, 2021.
  2. Duilio Balsamo, Paolo Bajardi, and André Panisson. Firsthand opiates abuse on social media: monitoring geospatial patterns of interest through a digital cohort. In The World Wide Web Conference, pages 2572-2579, 2019.
  3. Duilio Balsamo, Paolo Bajardi, Gianmarco De Francisci Morales, Corrado Monti, and Rossano Schifanella. The Pursuit of Peer Support for Opioid Use Recovery on Reddit. International AAAI Conference on Web and Social Media, 17:12-23, June 2023. ISSN 2334-0770, 2162-3449. doi: 10.1609/icwsm.v17i1.22122. URL https://ojs.aaai.org/index.php/ICWSM/article/ view/22122. · DOI
  4. Natã M Barbosa and Monchu Chen. Rehumanized crowdsourcing: A labeling framework addressing bias and ethics in machine learning. In 2019 CHI Conference on Human Factors in Computing Systems, pages 1-12, 2019.
  5. Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. The pushshift reddit dataset. In international AAAI conference on web and social media, volume 14, pages 830-839, 2020.
  6. Lorenzo Betti, Paolo Bajardi, and Gianmarco De Francisci Morales. Moral judgments in online discourse are not biased by gender. Scientific Reports, 15(1):21555, 2025.
  7. Arthur Capozzi, Gianmarco De Francisci Morales, Yelena Mejova, Corrado Monti, André Panisson, and Daniela Paolotti. Clandestino or rifugiato? anti-immigration facebook ad targeting in italy. In 2021 CHI Conference on Human Factors in Computing Systems, pages 1-15, 2021.
  8. Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16: 321-357, 2002.
  9. Robert Chew, Caroline Kery, Laura Baum, Thomas Bukowski, Annice Kim, Mario Navarro, et al. Predicting age groups of reddit users based on posting behavior and metadata: classification model development and validation. JMIR Public Health and Surveillance, 7(3):e25807, 2021.
  10. Matteo Cinelli, Gianmarco De Francisci Morales, Alessandro Galeazzi, Walter Quattrociocchi, and Michele Starnini. The Echo Chamber Effect on Social Media. Proceedings of the National Academy of Sciences, 118(9):e2023301118, 2021.
  11. Ernesto Colacrai, Federico Cinus, Gianmarco De Francisci Morales, and Michele Starnini. Navigating multidimensional ideologies with reddit's political compass: Economic conflict and social affinity. In ACM on Web Conference 2024, pages 2582-2593, 2024.
  12. Sara De Candia, Gianmarco De Francisci Morales, Corrado Monti, and Francesco Bonchi. Social norms on reddit: A demographic analysis. In 14th ACM Web Science Conference 2022, pages 139-147, 2022.
  13. R Maria del Rio-Chanona, Alejandro Hermida-Carrillo, Melody Sepahpour-Fard, Luning Sun, Renata Topinkova, and Ljubica Nedelkoska. Mental health concerns precede quits: shifts in the work discourse during the covid-19 pandemic and great resignation. EPJ Data Science, 12 (1):49, 2023.
  14. Carsten Eickhoff. Cognitive biases in crowdsourcing. In eleventh ACM international conference on web search and data mining, pages 162-170, 2018.
  15. Salvatore Giorgi, Veronica E Lynn, Keshav Gupta, Farhan Ahmed, Sandra Matz, Lyle H Ungar, and H Andrew Schwartz. Correcting sociodemographic selection biases for population prediction from social media. In International AAAI Conference on Web and Social Media, volume 16, pages 228-240, 2022.
  16. Matej Gjurković, Mladen Karan, Iva Vukojević, Mihaela Bošnjak, and Jan Šnajder. Pandora talks: Personality and demographics on reddit. In Ninth International Workshop on Natural Language Processing for Social Media, pages 138-152, 2021.
  17. Pablo González, Alberto Castaño, Nitesh V Chawla, and Juan José Del Coz. A review on quantification learning. ACM Computing Surveys (CSUR), 50(5):1-40, 2017.
  18. James T Hamilton. All the news that's fit to sell. In All the News That's Fit to Sell. Princeton University Press, 2011.
  19. Haibo He, Yang Bai, Edwardo A Garcia, and Shutao Li. Adasyn: Adaptive synthetic sampling approach for imbalanced learning. In 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence), pages 1322-1328. Ieee, 2008.
  20. Danula Hettiachchi, Mark Sanderson, Jorge Goncalves, Simo Hosio, Gabriella Kazai, Matthew Lease, Mike Schaekermann, and Emine Yilmaz. Investigating and mitigating biases in crowd- sourced data. In Companion Publication of the 2021 Conference on Computer Supported Cooperative Work and Social Computing, pages 331-334, 2021.
  21. Rachel Minyoung Kim, Veniamin Veselovsky, and Ashton Anderson. Capturing dynamics in online public discourse: A case study of universal basic income discussions on reddit. In Proceedings of the International AAAI Conference on Web and Social Media, volume 19, pages 1021-1037, 2025.
  22. Aniket Kittur, Ed H Chi, and Bongwon Suh. Crowdsourcing user studies with mechanical turk. In SIGCHI conference on human factors in computing systems, pages 453-456, 2008.
  23. Colin Klein, Peter Clutton, and Adam G Dunn. Pathways to conspiracy: The social and linguistic precursors of involvement in reddit's conspiracy theory forum. PloS one, 14(11): e0225098, 2019.
  24. Angela M Lee and Hsiang Iris Chyi. When newsworthy is not noteworthy: Examining the value of news from the audience's perspective. Journalism studies, 15(6):807-820, 2014.
  25. Usha Lokala, Aseem Srivastava, Triyasha Ghosh Dastidar, Tanmoy Chakraborty, Md Shad Akhtar, Maryam Panahiazar, and Amit Sheth. A computational approach to understand mental health from reddit: knowledge-aware multitask learning framework. In International AAAI Conference on Web and Social Media, volume 16, pages 640-650, 2022.
  26. Alexey N Medvedev, Renaud Lambiotte, and Jean-Charles Delvenne. The anatomy of reddit: An overview of academic research. Dynamics on and of Complex Networks III: Machine Learning and Statistical Physics Approaches 10, pages 183-204, 2019.
  27. Lillio Mok, Michael Inzlicht, and Ashton Anderson. Echo tunnels: Polarized news sharing online runs narrow but deep. In International AAAI Conference on Web and Social Media, volume 17, pages 662-673, 2023.
  28. Corrado Monti, Luca Maria Aiello, Gianmarco De Francisci Morales, and Francesco Bonchi. The language of opinion change on social media under the lens of communicative action. Scientific Reports, 12(1):17920, 2022.
  29. Corrado Monti, Jacopo D'Ignazi, Michele Starnini, and Gianmarco De Francisci Morales. Evidence of demographic rather than ideological segregation in news discussion on reddit. In ACM Web Conference 2023, pages 2777-2786, 2023.
  30. Alejandro Moreo, Andrea Esuli, and Fabrizio Sebastiani. Quapy: a python-based framework for quantification. In 30th ACM International Conference on Information & Knowledge Management, pages 4534-4543, 2021.
  31. Nature Editorial. The powers and perils of using digital data to understand human behaviour. Nature, 595:149-150, 2021. doi: 10.1038/d41586-021-01736-y. URL https://www.nature. com/articles/d41586-021-01736-y. Editorial. · DOI
  32. Ryan Rivas, Shouq A Sadah, Yuhang Guo, Vagelis Hristidis, et al. Classification of health- related social media posts: Evaluation of post content-classifier models and analysis of user demographics. JMIR Public Health and Surveillance, 6(2):e14952, 2020.
  33. Cesare Rollo, Gianmarco De Francisci Morales, Corrado Monti, and André Panisson. Com- munities, Gateways, and Bridges: Measuring Attention Flow in the Reddit Political Sphere. In Frank Hopfgartner, Kokil Jaidka, Philipp Mayr, Joemon Jose, and Jan Breitsohl, editors, Social Informatics, volume 13618 of SocInfo, pages 3-19. Springer, [best paper award], 2022. ISBN 978-3-031-19096-4 978-3-031-19097-1.
  34. Kuleen Sasse, Ron Mahabir, Olga Gkountouna, Andrew Crooks, and Arie Croitoru. Under- standing the determinants of vaccine hesitancy in the united states: A comparison of social surveys and social media. Plos one, 19(6):e0301488, 2024.
  35. Mohammad Shahrokh Esfahani and Edward R Dougherty. Effect of separate sampling on classification accuracy. Bioinformatics, 30(2):242-250, 2014.
  36. Connor Shorten and Taghi M Khoshgoftaar. A survey on image data augmentation for deep learning. Journal of big data, 6(1):1-48, 2019.
  37. Nagesh Bhattu Sristy and DVLN Somayajulu. Semi-supervised learning of naive bayes classifier with feature constraints. In First International Workshop on Optimization Techniques for Human Language Technology, pages 65-78, 2012.
  38. Justin M Stritch, Mogens Jin Pedersen, and Gabel Taggart. The opportunities and limitations of using mechanical turk (mturk) in public administration and management scholarship. International Public Management Journal, 20(3):489-511, 2017.
  39. Evgenii Vasilev. Inferring gender of reddit users. PhD thesis, Universität Koblenz, 2018.
  40. Isaac Waller and Ashton Anderson. Quantifying social organization and political polarization in online platforms. Nature, 600(7888):264-268, 2021.
  41. Zijian Wang and David Jurgens. It's going to be okay: Measuring access to support in online communities. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 33-45, 2018.
  42. Jing Zhang, Xindong Wu, and Victor S Sheng. Learning from crowdsourced labeled data: a survey. Artificial Intelligence Review, 46:543-576, 2016.

Cited by

Sorted by citations

Metadata

Authors
Cinus, F., Monti, C., Bajardi, P., & Morales, G. D. F.
Venue
arXiv
Publisher
arXiv
Published
2025
DOI
10.48550/arxiv.2502.05049
arXiv
2502.05049
References
0
Type
Preprint
Licence
creativecommons.org/licenses/by-sa/4.0/legalcode

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review