Demographic Dialectal Variation in Social Media: A Case Study of African-American English

“This model has broadly similar goals as nonlatent, log-linear generative models of text that condition on document-level covariates (Monroe et al., 2008; Eisenstein et al., 2011a; Taddy, 2013).”

“, 2016) and sociolinguistic (Stewart, 2014; Eisenstein, 2015; Jones, 2015) research.”

“Studies on code-switching (Khanuja et al. 2020, Piergallini et al. 2016), dialectal NLP (Blodgett et al. 2016, Faisal et al. 2024), and low-resource benchmarks (Tan and Celis 2019), argue for treating variation as a meaningful signal.”

“Our findings corroborate past work that shows degradation in performance for language varieties other than standard American English for other tasks (Blodgett et al., 2016; Ziems et al., 2023; Joshi et al., 2025; Srirag et al., 2025b).”

“Following Ravfogel et al. (2020) and Han et al. (2022a) we also evaluate on the Emoji (Elazar and Goldberg, 2018) sentiment analysis task of Twitter data (Blodgett et al., 2016).”

“For the AAE task, we use a lexicon-based scoring from a paper by Blodgett et al. (2016), which defines African-American English (AAE/AAVE) as a dialect of Standard American English with specific linguistic features and uses a distantly supervised model to identify AAE-like language on Twitter.”

“TwitterAAE [21] is a dataset of over one million tweets that were automatically found to have a high probability of being authored by a speaker of AAE.”

“It is possible that our use of (Blodgett, Green, and O’Connor 2016); as such, we manually evaluate it.”

“I manually checked these, as language detection is unreliable with AAE (Blodgett et al., 2016, 2020).”

“[2] created a corpus of 60 million geolocated tweets with dialectal language based on the population of non-Hispanic whites, non-Hispanic blacks, Hispanics, and Asians.”

“Relevant to this decision are factors 5For an English analogue, consider splitting the lexicalized verb enforce into a prefix en- and root force.”

“In addition, unsupervised approaches could be explored, which would lessen the impact of any imbalances in the training data set, such as emoji analysis [84] or lexical dialect detection [85], especially for improving model performance on the in-group variable.”

“…has begun to shift from relating language choices and macrosocial categories like age, gender, and race (Burger et al., 2011; Nguyen et al., 2013; Blodgett et al., 2016), to a more nuanced approach to identity, adopting the framework of communities of practice (Eckert and McConnell-Ginet, 1992;…”

“As a starting point for measuring disparities between dialects, we include T WITTER AAE [Blodgett et al., 2016], two corpora representing African-American and White-aligned English, automatically classified via geolocation information and demographic census statistics.”

“…historically marginalized communities that speak “non-standard” language varieties vulnerable to further discrimination by modern AI systems trained on historical data (Blodgett et al., 2016; Jurgens et al., 2017; Kantharuban et al., 2023; Dro˙zd˙zowicz and Peled, 2024; Nayeem and Rafiei, 2026).”

“…strong performance across a range of NLP tasks (Wang et al., 2024; Touvron et al., 2023), but often falter on dialectal and non-standard varieties—regional and social forms that diverge from standardised norms (Joshi et al., 2025; Blodgett et al., 2016; Hofmann et al., 2024; Mire et al., 2025).”

“Finally, P LU R ULE is multilingual and multimodal, capturing the visual (Gomez et al., 2020) and linguistic (Blodgett et al., 2016) diversity of online communities often overlooked by text-only benchmarks.”

“Studies on English, Hindi, and Chinese di-alects show that dialect choice signals social identity, politeness, norms, power relations, and pragmatic conventions (Hovy, 2015; Blodgett et al., 2016; Jurgens et al., 2017; Hershcovich et al., 2022).”

“These communities represent systematically underrepresented varieties with well-documented linguistic structures distinct from Standard American English [5, 14, 17, 23, 25].”

“…highlight key differences between BAE as spoken between native speakers and BAE found in the many corpora on which many popular LLMs are trained [3, 17], finding that the publicly accessible written BAE in popular media and on social media is insufficient to enable LLMs to acceptably imitate BAE.”

“Dialect variation further amplifies this issue: demographic-aligned language features (e.g., African-American English) differ from mainstream norms (Blodgett et al., 2016), and insensitivity to dialect has been shown to induce racial bias in hate-speech annotation and down-stream classifiers (Sap…”

“Recent studies indicate that language models exhibit dialect prejudice, assigning lower employability scores to AAE speakers, which underscores the potential of AI systems to perpetuate linguistic biases (Hofmann et al., 2024; Blodgett et al., 2016; Davidson et al., 2019).”

“To choose white and African-American target words, we ran the mixed-membership demographic-language model developed by Blodgett et al. (2016).”

“MAE : ‘I am usually in my office by 7:30’ Although relatively infrequent, habitual be is employed regularly by AAE speakers (Blodgett et al., 2016) as well as speakers of 90 other Englishes (Kortmann, 2020).”

“While not strictly comparable, our 0.85 threshold is more conservative than the 0.80 used in Blodgett et al. (2016).”

“For instance, Blodgett et al. (2016) found that existing language identification and dependency parsing tools on African-American Vernacular English text demonstrated very poor performances compared to on Standard English text.”

“Subsequently, many studies (e.g., [41, 42]) have explored potential risks of algorithmic decision-making that are potentially biased and discriminatory to a certain group of people such as underrepresented groups of gender, race, disability.”

“, 2015), African American English (Jones, 2015), and global Spanish (Gonçalves and Sánchez, 2014), and to study variation associated with factors such as race/ethnicity (Jones, 2015; Blodgett et al., 2016; Jørgensen et al., 2015) and gender (Bamman et al.”

“Research in computational sociolinguistics emphasizes that such variation reflects underlying social structures and subcultures (Nguyen et al. 2016; Blodgett et al. 2016; Kellert and Matlis 2022).”

“African American Vernacular English is often the vehicle chosen to explore the topic of racial bias in NLP due to its prevalence on social media, its linguistic diversity, and its ties to a historically marginalized community [1], [27].”

“Detecting explicit offensive language often relies on key-word detection (Wiegand et al., 2019), but keyword-driven systems can lead to messages being falsely flagged, causing unchecked or unnoticed racial biases to propagate in the system’s decisions (Sap et al., 2021; Blodgett et al., 2016).”

“Computational efforts to discover these relationships include demographic dialectal variation (Blodgett et al., 2016), geographical variation (Eisenstein et al., 2010; Nguyen and Eisenstein, 2017), syntactic or stylistic variation over age and gender (Johannsen et al., 2015), socio-economic status…”

“The demographic variables are based on the dialectal corpus from Blodgett et al. (2016), where race was assigned based on geolocation and words used in the tweet, obtaining a binary AAE (African-American English) and SAE (Standard American English) which we use as proxies for non-Hispanic…”

“We, therefore, isolated from the test set the comments highly characterized by this slang, using a classifier 5 specifically trained to recognize texts belonging to the African-American English dialect [10].”

“MOJI is a tweet-sentiment task with happy/sad labels from emoji annotations and African-American English (AAE) versus Standard American English (SAE) group annotations (Blodgett et al., 2016).”

“The effect of increasing label imbalance is illustrated on Dial Mention (Fig.”

“…rich, multidimensional identities to binary contrasts (left vs. right political lean and AAL vs. WME) and rely on imperfect proxies for each: AllSides’ editorial lean labels, an LLM classifier for Reddit nat ideology, and the demographic-alignment classifier of Blodgett et al. (2016) for dialect.”

“…of the model against expert annotations (§B) and that dense AAL features are not common in the corpus: the demographic alignment classifier introduced in Blodgett et al. (2016) predicts AAL as the most likely label for ∼ 6% of texts, and a probability exceeding .8 for less than 1% of texts.”

“Dialect NLP has long modelled variation directly, from demographic dialect variation in social media [3] to dialect behaviour in generation [12] and the dialect discrimination reproduced by current LLMs [10].”

“Research on dialectal variation, including African American English and other sociolects, reveals that models often misclas-sify identity-linked linguistic features as toxicity indicators due to spurious correlations embedded within training data (Sap et al., 2019; Blodgett et al., 2016).”

“Second, real-AAE stud-a”

“Many of these methods attempt to detect overt racial bias, e.g., the use of racial slurs and stereotypes, while others assess differential behavior of NLP tools based on racialized dialects such as African-American English (Sap et al. 2019; Blodgett, Green, and O’Connor 2016).”

“Part-of-speech taggers trained on standard varieties degrade when applied to dialect text [Hovy and Søgaard, 2015, Blodgett et al., 2016].”

“Su Lin Blodgett (2016) [6] studied how Twitter users speak African-American English and Standard American English by analyzing dialectal variations between these two languages.”

“For each training sample, the sensitive attribute label a (e.g., gender in Bias in Bios or race/dialect group in Twitter Sentiment) is provided by the dataset annotations and is only used to supervise the sensitive-attribute discriminator D .”

“For example, AAVE samples emphasize authentic syntactic and lexical constructions, while Singlish prompts incorporate pragmatic particles and col-loquial phrasing.”

“As this work takes place in the early stage of text dataset distillation, its studies cover the use of a multi-layer perceptron on binary sentiment analysis and occupation classification tasks over datasets with bias (Blodgett et al., 2016; De-Arteaga et al., 2019).”

“In practice, it has been proven that models trained with less diverse data suffer from representation biases and problems in performance (Blod-gett et al., 2016).”

“Similar disparities were observed in sentiment analysis of African-American English tweets [16] and species classification on the iNaturalist dataset [17].”

“More details on the pre-finetuning and loss curves can be found in Appendix C. 1 .”

“In the experiment, we fixed the training settings and dataset of the model and ran the training 10 times to ensure the reliability of the results and reduce the randomness.”

“, 2021; Eisenstein, 2013), stylistic and lexical variation on writing (Blodgett et al., 2016; Nguyen, 2017; Shoemark, 2020; Wurschinger, 2021; Pavalanathan and Eisenstein, 2015), dialectal studies (Eisenstein, 2017; Jorgensen et al.”

“Dataset There are several datasets provided already, such as the Twitter dataset [4], preprocessed data [5], and blend datasets [6-8].”

“The demographic variables are based on the dialectal corpus from Blodgett et al. (2016), where race was assigned based on geolocation and words used in the tweet, obtaining a binary AAE (African-American English) and SAE (Standard American English) which we use as proxies for non-Hispanic…”

“λ is selected from the list [2, 3, 5] for all methods that performed constant upweighting.”

“To compare the performance of our classifier with different forms of English, we used an additional corpus of tweets from [23] that have been machine labeled with a probability of being written by an African American, Hispanic, or White person, or none of the listed 3.”

“More worrying, however, is that training or finetuning on datasets that contain biases and artefacts may result in models implementing undesired, biased behavior (e.g. Rudinger et al., 2018; Blodgett et al., 2016).”

“(1) Twitter [24] (2016): A binary-label text corpus released by Twitter, Inc. containing 2 sentiment labels (positive vs. negative) and 2 race labels (African-American English vs. Standard American English).”

“Thus, understanding the underlying causes of this unbalanced performance between the majority and minority groups is crucial to their reliable and safe deployment in various real-world scenarios Blodgett et al. [2016], Buolamwini and Gebru [2018], Hashimoto et al. [2018].”

“Finally, we show that by promoting task related lexicon entries, we are able to demote spurious correlations with AAE annotations on toxicity datasets.”

“In selecting tweets from Twitter AAE Corpus to use in DivSumm, there were some requirements created to ensure their quality.”

“In this preliminary analysis we use the TwitterAAE corpus (Blodgett et al., 2016), which contains geographic and census information.”

“…should a dataset embody the language production in multiple sociolects, a pre-trained language model (Devlin et al. 2019) 6 or mixed-member language models (Blodgett, Green, and O’Connor 2016) may provide deeper information about the sociolects in question by examining the sentential context.”