The DOI matches its Crossref record; 79 of the 79 references it cites were found in a registry or an index.
Large language models, social demography, and hegemony: comparing authorship in human and synthetic text
Open accessPublished version of a preprint
Abstract
Large language models have become popular over a short period of time because they can generate text that resembles human writing across various domains and tasks. The popularity and breadth of use also put this technology in the position to fundamentally reshape how written language is perceived and evaluated. It is also the case that spoken language has long played a role in maintaining power and hegemony in society, especially through ideas of social identity and “correct” forms of language. But as human communication becomes even more reliant on text and writing, it is important to understand how these processes might shift and who is more likely to see their writing styles reflected back at them through modern AI. We therefore ask the following question: who does generative AI write like? To answer this, we compare writing style features in over 150,000 college admissions essays submitted to a large public university system and an engineering program at an elite private university with a corpus of over 25,000 essays generated with GPT-3.5 and GPT-4 to the same writing prompts. We find that human-authored essays exhibit more variability across various individual writing style features (e.g., verb usage) than AI-generated essays. Overall, we find that the AI-generated essays are most similar to essays authored by students who are males with higher levels of social privilege. These findings demonstrate critical misalignments between human and AI authorship characteristics, which may affect the evaluation of writing and calls for research on control strategies to improve alignment.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures








Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This makes LLM outputs less diverse and novel than human-written text (43, 44).”
From Delving into LLM-assisted writing in biomedical publications through excess vocabulary (2025) · cites this paper as [44]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“Recent advances in transformer-based architectures have enabled the development of LLMs that can generate impressively human-like text and respond to complex questions and prompts [19].”
BackgroundAdvances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1×
“A better understanding of LLMs will therefore require consideration of the bias literature alongside studies of sociolinguistic profiles of LLMs. Extensive research has uncovered notable biases within LLMs [13, 25, 32, 47, 50, 63, 65, 80].”
BackgroundThe Development and Psychometric Properties of LIWC2015
Pennebaker, J. W., Boyd, R. L., Jordan, K., & Blackburn, K. (2015) · University of Texas at Austin · doi:10.15781/t25p41 · Cited by 1,674
Artificial hallucinations in ChatGPT: implications in scientific writing
Alkaissi, H., & McFarlane, S. I. (2023) · Cureus · doi:10.7759/cureus.35179 · Cited by 1,540 · cited 1×
“However, the self-evident potential of LLMs has raised critical questions about their biases, tendency to emulate certain political and moral stances, and ability to fabricate references when used as a research assistant [1, 73].”
BackgroundLaMDA: Language Models for Dialog Applications
Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos, T., Baker, L., Du, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2201.08239 · Cited by 778 · cited 1×
“For example, Wordcraft [88] employs a LLM [81] to assist writers with tasks such as proofreading, scene interpolation, and content generation.”
MethodSelf-citationRacial disparities in automated speech recognition
Koenecke, A., Nam, A., Lake, E., Nudell, J., Quartey, M., Mengesha, Z., Toups, C., Rickford, J. R., Jurafsky, D., & Goel, S. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1915768117 · Cited by 765
Artificial intelligence and illusions of understanding in scientific research
Messeri, L., & Crockett, M. J. (2024) · Nature · doi:10.1038/s41586-024-07146-0 · Cited by 707 · cited 1×
“The reduced variation could potentially narrow the scope of what is considered an appropriate way to present oneself in writing if the treatment of AI as an “oracle” that is always correct persists [59].”
BackgroundGPT detectors are biased against non-native English writers
Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023) · Patterns · doi:10.1016/j.patter.2023.100779 · Cited by 673 · cited 1×
“Conversely, students who do not speak English as a first language might find that their writing is more often labeled as “AI-generated” [52], pointing to additional ways that the current AI eco-system could reinforce linguistic hegemony (in this case, English language hegemony).”
Background
Research works printed but not linked: 20
- Bellemare MG, Dabney W, Munos R. A distributional perspective on reinforcement learning. In: International conference on machine learning, PMLR, pp 449–458. 2017.
- Bohacek M, Farid H. Nepotistically trained generative-AI models collapse. arXiv preprint. 2023. arXiv:2311.12202. · arXiv
- Dhingra H, Jayashanker P, Moghe S, et al. Queer people are people first: deconstructing sexual identity stereotypes in large language models. http://arxiv.org/abs/2307.00101, [cs] 2023. · arXiv
- Dominguez-Olmedo R, Hardt M, Mendler-Dünner C. Questioning the survey responses of large language models. arXiv preprint arXiv:2306.07951. 2023. · arXiv
- Gallegos IO, Rossi RA, Barrow J, et al. Bias and fairness in large language models: a survey. http://arxiv.org/abs/2309.00770, [cs] 2023. · arXiv
- Gramsci A. Prison notebooks, vol. 1. New York: Columbia University Press; 1992.
- Kim J, Lee B. Ai-augmented surveys: leveraging large language models for opinion prediction in nationally representative surveys. arXiv preprint arXiv:2305.09620. 2023. · arXiv
- Kirk HR, Whitefield A, Röttger P, et al. The PRISM alignment project: what participatory, representative and individualised human feedback reveals about the subjective and multicultural alignment of large language models. arXiv preprint. 2024. arXiv:2404.16019. · arXiv
- Lee J, Thymes B, Zhou J, et al. Augmenting holistic review in university admission using natural language processing for essays and recommendation letters. arXiv preprint arXiv:2306.17575. 2023. · arXiv
- Lee J, Hicke Y, Yu R, et al (in press) The life cycle of large language models in education: a framework for understanding sources of bias. Br J Educ Technol.
- Liang W, Izzo Z, Zhang Y, et al. Monitoring AI-modified content at scale: a case study on the impact of chatgpt on ai conference peer reviews. arXiv preprint. 2024. arXiv:2403.07183. · arXiv
- Lucy L, Blodgett SL, Shokouhi M, et al. “One-size-fits-all”? Observations and expectations of NLG systems across identity-related language features. http://arxiv.org/abs/2310.15398, [cs] 2023. · arXiv
- Matelsky JK, Parodi F, Liu T, et al. A large language model-assisted education tool to provide feedback on open-ended responses. http://arxiv.org/abs/2308.02439, [cs] 2023. · arXiv
- Padmakumar V, He H. Does writing with language models reduce content diversity? In: The Twelfth International Conference on Learning Representations. 2023.
- Shen H, Wu T. Parachute: evaluating interactive human-LM co-writing systems. arXiv:2303.06333. [cs] 2023. · arXiv
- Stevens ML. Creating a class. Cambridge: Harvard University Press; 2009.
- Tao Y, Viberg O, Baker RS, et al. Auditing and mitigating cultural bias in llms. arXiv preprint arXiv:2311.14096. 2023. · arXiv
- Wang A, Morgenstern J, Dickerson JP. Large language models cannot replace human participants because they cannot portray identity groups. arXiv preprint arXiv:2402.01908. 2024. · arXiv
- Workshop B, Scao TL, Fan A, et al (2022) Bloom: a 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100. · arXiv
- Wu S, Irsoy O, Lu S, et al. BloombergGPT: a large language model for finance. http://arxiv.org/abs/2303.17564, [cs, q-fin] 2023. · arXiv
Cited by
Bias in Large Language Models: Origin, Evaluation, and Mitigation
Guo, Y., Guo, M., Su, J., Yang, Z., Zhu, M., Li, H., Qiu, M., & Liu, S. S. (2026) · Electronics · doi:10.3390/electronics15091824 · Cited by 247
“Another example is that the Latinx population, especially Mexican-American students, remains underrepresented in U.S. higher education (Alvero et al., 2024).”
Delving into LLM-assisted writing in biomedical publications through excess vocabulary
Kobak, D., González-Márquez, R., Horvát, E.-Á., & Lause, J. (2024) · arXiv · doi:10.48550/arxiv.2406.07016 · Cited by 214
“This makes LLM outputs less diverse and novel than human-written text (Padmakumar and He, 2023; Alvero et al., 2024).”
Delving into LLM-assisted writing in biomedical publications through excess vocabulary
Kobak, D., González-Márquez, R., Horvát, E.-Á., & Lause, J. (2025) · Science Advances · doi:10.1126/sciadv.adt3813 · Cited by 213
“This makes LLM outputs less diverse and novel than human-written text (Padmakumar and He, 2023; Alvero et al., 2024).”
Bias in Large Language Models: Origin, Evaluation, and Mitigation
Guo, Y., Guo, M., Su, J., Yang, Z., Zhu, M., Li, H., Qiu, M., & Liu, S. S. (2024) · arXiv · doi:10.48550/arxiv.2411.10915 · Cited by 208
“Another example is that the Latinx population, especially Mexican-American students, remains underrepresented in U.S. higher education (Alvero et al., 2024).”
Tejeda, A. S., Oller, J. C. D., Escandell, J. M. B., Gómez-Díaz, R., & Rodríguez, A. G. (2026) · Frontiers in Education · doi:10.3389/feduc.2025.1711718 · Cited by 130
“There is also concern about the homogenization of language, as GAI could impose uniformity in expression (Alvero et al., 2024).”
Influential
Metadata
- Authors
- Alvero, A. J., Lee, J., Regla-Vargas, A., Kizilcec, R. F., Joachims, T., & Antonio, A. L.
- Affiliations
- Cornell University (US)
- University of Pennsylvania (US)
- Stanford University (US)
- Corresponding author
- Alvero, A. J.
- Antonio, Anthony Lising
- ORCID
- Venue
- Journal of Big Data
- Published
- 2024-09-27
- Volume
- 11
- Issue
- 1
- ISSN
- 2196-1115 (ISSN-L)
- OpenAlex
- W4402923080
- Cited by
- 48
- References
- 72
- Citations by year
- 2026: 17 · 2025: 26 · 2024: 4
- FWCI
- 6.94
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc-nd
- creativecommons.org/licenses/by-nc-nd/4.0 (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0 (vor)
- Locations
- Journal Of Big Data (publishedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- SocArXiv (OSF Preprints) (submittedVersion, cc-by-nd)
- Funders
- Yale University
- University of Warwick
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Linguistics and Language › Discourse Analysis in Language Studies
- Keywords
- large language models, generative AI, authorship, writing style, social demography, hegemony, social privilege, gender differences, college admissions essays, GPT-3.5, GPT-4, writing assessment, human-AI comparison, language and power
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Large language models, social demography, and hegemony: comparing authorship in human and synthetic text. https://app.cochat.ai/papers/10.1186/s40537-024-00986-7
- The paper
- Alvero, A. J., Lee, J., Regla-Vargas, A., Kizilcec, R. F., Joachims, T., & Antonio, A. L. (2024). Large language models, social demography, and hegemony: comparing authorship in human and synthetic text. Journal of Big Data. https://doi.org/10.1186/s40537-024-00986-7
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.