The DOI matches its Crossref record; 36 of the 37 references it cites were found in a registry or an index; none of the 9 sources its claims cite is checked for support yet.
Racial disparities in automated speech recognition
Open accessAbstract
Automated speech recognition (ASR) systems, which use sophisticated machine-learning algorithms to convert spoken language to text, have become increasingly widespread, powering popular virtual assistants, facilitating automated closed captioning, and enabling digital dictation platforms for health care. Over the last several years, the quality of these systems has dramatically improved, due both to advances in deep learning and to the collection of large-scale datasets used to train the systems. There is concern, however, that these tools do not work equally well for all subgroups of the population. Here, we examine the ability of five state-of-the-art ASR systems—developed by Amazon, Apple, Google, IBM, and Microsoft—to transcribe structured interviews conducted with 42 white speakers and 73 black speakers. In total, this corpus spans five US cities and consists of 19.8 h of audio matched on the age and gender of the speaker. We found that all five ASR systems exhibited substantial racial disparities, with an average word error rate (WER) of 0.35 for black speakers compared with 0.19 for white speakers. We trace these disparities to the underlying acoustic models used by the ASR systems as the race gap was equally large on a subset of identical phrases spoken by black and white individuals in our corpus. We conclude by proposing strategies—such as using more diverse training datasets that include African American Vernacular English—to reduce these performance differences and ensure speech recognition technology is inclusive.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This paper audits five commercial automated speech recognition (ASR) systems and finds large racial disparities in word error rates for African American speakers compared with white speakers. The authors trace the gap primarily to acoustic models rather than language or lexical models, suggesting insufficient audio data from black speakers in training.
Key findings
- Across five ASR systems (Amazon, Apple, Google, IBM, Microsoft), the aggregate word error rate (WER) was 0.35 for black speakers versus 0.19 for white speakers; for Microsoft, the best overall system, WER was 0.27 for black speakers versus 0.15 for white speakers, and for Apple, the worst overall, 0.45 versus 0.23.
- Disparities were larger for black men: averaging across the five ASR systems, error rate was 0.41 for black men versus 0.30 for black women, compared with 0.21 for white men and 0.17 for white women.
- More than 20% of snippets of black speakers had an average WER of at least 0.5, versus fewer than 2% of white speakers; if WER >0.5 implies an unusable transcript, 23% of black-speaker snippets versus 1.6% of white-speaker snippets were unusable.
- Lexical and language-model gaps did not explain the disparity: 98–99% of spoken words were in reconstructed ASR vocabularies for both groups, and language-model perplexity was lower (better) for black speakers (e.g., GPT-2: 45 vs 55). On 206 matched identical short phrases, WERs were about twice as large for black speakers (e.g., Microsoft: 0.13 vs 0.07), pointing to acoustic-model differences.
Methods
- The study used sociolinguistic interviews from the Corpus of Regional African American Language (CORAAL) and Voices of California (VOC), restricting to adult interviewee-only snippets of 5–50 s. The main matched sample contained 2,141 snippets each from black and white speakers, averaging 17 s per snippet and totaling 19.8 h of audio.
- Propensity-score matching selected black and white snippets with similar age, gender, and snippet duration; WER was computed between cleaned human transcripts and cleaned ASR transcripts.
- Dialect density was hand-coded in a random sample of 150 snippets of black speakers (50 per AAVE site), counting AAVE phonological and grammatical features normalized by word count; interrater Pearson correlations were 0.92 and 0.74. Language-model perplexity was also examined using Transformer-XL, GPT, and GPT-2.
Limitations
- Stated: the white and black audio samples came from different U.S. regions (white speakers in California; black speakers in the Eastern United States), so some differences could reflect regional rather than ethnic linguistic variation; the authors argue AAVE dialect density and similar error rates across two California white sites support an AAVE-driven interpretation.
- Stated: data were collected across different years and recording equipment (Princeville in 2004 on cassette, others digitally in 2014–2017), and the small number of speakers per location limits conclusions about broader geographic dialect patterns; the authors say these issues are unlikely to affect main results.
- Evident: the analysis covers five commercial ASR systems and specific corpora, so findings may not generalize to all ASR systems, speakers, or nonstandard English varieties beyond those studied.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
28 Own result · 18 Own interpretation · 9 Own method · 4 Not checked yet · 1 No basis given
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The pairwise differences in DDM by location are statistically significant, with P < 0.05 in all cases.”
§ Racial disparities in automated speech recognition > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Aggregated across the three AAVE sites, the DDM for male speakers is 0.13 (n=52; SE: 0.02), compared with 0.096 for female speakers (n=98; SE: 0.01).”
§ Racial disparities in automated speech recognition > Results
plaincorrelationaln = 52- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Under all three language models, we find the average perplexity of snippets by black speakers is lower—meaning better performance—than the average perplexity of snippets by white speakers in our sample.”
§ Racial disparities in automated speech recognition > Results
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In comparison, the average error rates for white men and women are more similar at 0.21 (SE: 0.004) and 0.17 (SE: 0.003), respectively.”
§ Racial disparities in automated speech recognition > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, the error rate in the third AAVE site, Rochester (0.20), is comparable to the error rates in the two California locations with white speakers.”
§ Racial disparities in automated speech recognition > Results
plainn = 13- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In Fig. 4, we directly examine the relationship between DDM (on the horizontal axis) and WER (on the vertical axis), which illustrates the positive correlation between DDM and error rates.”
§ Racial disparities in automated speech recognition > Results
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“By analyzing a large corpus of sociolinguistic interviews with white and African American speakers, we demonstrate large racial disparities in the performance of five popular commercial ASR systems.”
- Also stated in: § Results · plain
“For each of the five commercial ASR systems we examine, Fig. 1 shows that the average WER for black speakers is substantially larger than the average WER for white speakers.”
§ Racial disparities in automated speech recognition > Abstract
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Results · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“word error rate is strongly associated with AAVE dialect density, as seen in Fig. 4.”
§ Racial disparities in automated speech recognition > Discussion
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The dialect density measure (DDM) used in this study provides insight into drivers of the ASR performance gaps.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, if one considers a WER of 0.5 to be the bar for a useful transcription, more than 10 times as many snippets of black speakers fail to meet that standard.”
§ Racial disparities in automated speech recognition > Results
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For both white and black speakers, and across the five ASR systems, 98 to 99% of the words spoken are in the reconstructed vocabularies.”
§ Racial disparities in automated speech recognition > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 10
- It may state two separate claims as one. 8
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, “he a pastor” has perplexity of 305, compared with 67 for “he’s a pastor.””
§ Racial disparities in automated speech recognition > Results
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, of the 104,486 words uttered by black speakers in our sample, Google’s ASR had at least 103,142 (98.7%) of them in its vocabulary; in comparison, of the 98,653 words spoken by white individuals in our sample, at least 97,260 (98.6%) were in the vocabulary.”
§ Racial disparities in automated speech recognition > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We find that average DDM is lowest in Rochester (0.047)—and also relatively small on an absolute scale—followed by Washington, DC (0.088), and Princeville (0.19), mirroring the ordering of word error rates by location seen in Fig. 3.”
§ Racial disparities in automated speech recognition > Results
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, more than 20% of snippets of black speakers have an error rate of at least 0.5; in contrast, fewer than 2% of snippets of white speakers are above that threshold.”
§ Racial disparities in automated speech recognition > Results
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As we discuss next, we find evidence of a gap in the acoustic models but not in the language models.”
§ Racial disparities in automated speech recognition > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although the total duration and number of words spoken by black and white speakers in our sample were similar, black speakers uttered fewer unique words (5,651) than white speakers (6,280).”
§ Racial disparities in automated speech recognition > Results
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“whose speakers may similarly be burdened by poor ASR performance—including those with regional and nonnative-English accents.”
§ Racial disparities in automated speech recognition > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Most previous studies using DDMs have focused on the syntactic complexity of AAVE (34, 35).”
§ Racial disparities in automated speech recognition > Materials and Methods > Measuring Dialect Density.
What the sources say
- Not checked yetcites [34] Craig H. K., Washington J. A., An assessment battery for identifying language impairments in African American children. J. Speech Lang. Hear. Res. 43, 366– 379 (2000). 10757690 source
- Not checked yetcites [35] Oetting J. B., McDonald J. L., Methods for characterizing participants’ nonmainstream dialect use in child language research. J. Speech Lang. Hear. Res. (2002). PMC3390149 12069003 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Given the relatively small number of speakers in each location, we cannot determine whether these patterns are representative of more general geographic differences in dialect or are simply idiosyncratic trends in our particular sample of speakers.”
§ Racial disparities in automated speech recognition > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We compared our results with those where all snippets containing a list of hard-to-spell city names uttered in the audio snippets were removed. Again, our results did not meaningfully change”
§ Racial disparities in automated speech recognition > Materials and Methods > Data.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We match these snippets across the two datasets based on the age and gender of the speaker and the duration of the snippet.”
- Also stated in: § Abstract · plain
“After matching, we are left with 2,141 snippets from each dataset, with an average length of 17 s per snippet, amounting to 19.8 total hours of audio.”
§ Racial disparities in automated speech recognition > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
- Also stated in: § Abstract · plain
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We confirmed that our results were nearly identical if, instead of performing the above operations, snippets with questionable content were removed entirely.”
§ Racial disparities in automated speech recognition > Materials and Methods > Data.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“The complete list of AAVE features that we tagged is based on past work (23–25)”
§ Racial disparities in automated speech recognition > Materials and Methods > Measuring Dialect Density.
What the sources say
- Not checked yetcites [23] Rickford J. R., Phonological and Grammatical Features of African American Vernacular (AAVE) (Blackwell Publishers, 1999).
- Not checked yetcites [24] Thomas E. R., Phonological and phonetic characteristics of African American Vernacular English. Lang. Linguist. Compass 1, 450– 475 (2007). source
- Not checked yetcites [25] Bailey G., Thomas E., “ Some aspects of African-American Vernacular English phonology” in African-American English: Structure, History and Use, Mufwene S., Rickford J. R., Bailey G., Baugh J., Eds. (Routledge, New York, NY, 1998), pp. 85– 109.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Past work has also found that ASRs perform somewhat worse on conversational speech from male speakers than female speakers, likely due to male speakers using more informal style with shorter, more reduced pronunciations and more disfluencies (28, 29).”
§ Racial disparities in automated speech recognition > Results
What the sources say
- Not checked yetcites [28] Goldwater S., Jurafsky D., Manning C. D., Which words are hard to recognize? Prosodic, lexical, and disfluency factors that increase speech recognition error rates. Speech Commun. 52, 181– 200 (2010). source
- Not checked yetcites [29] Adda-Decker M., Lamel L., “ Do speech recognizers prefer female speakers?” in Proceedings of INTERSPEECH-2005 (International Speech Communication Association, 2005), pp. 2205– 2208. source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“In both datasets, the interviews were transcribed by human experts, which we use as the ground truth when evaluating the performance of machine transcriptions.”
§ Racial disparities in automated speech recognition > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These disparities may also actively harm African American communities when, for example, speech recognition software is used by employers to automatically evaluate candidate interviews or by criminal justice agencies to automatically transcribe courtroom proceedings.”
§ Racial disparities in automated speech recognition > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
The paper relies on this without support.
Commercial ASR systems use language models that have similar statistical properties to publicly available state-of-the-art models like Transformer-XL, GPT, and GPT-2.
“However, it is likely that these systems use language models that have similar statistical properties to state-of-the-art models that are publicly available, like Transformer-XL (30), GPT (31), and GPT-2 (32).”
The paper relies on this without support.
The study's limitations (different years and recording equipment) are unlikely to impact the main results.
“While it is important to recognize these limitations of our study design, we believe they are unlikely to impact our main results.”
The paper relies on this without support.
The time gap between the 2004 Princeville data and the 2014–2017 data does not represent a significant time gap for sociolinguistic analysis.
“The majority of our data come from 2014 to 2017—a span that does not represent a significant time gap for sociolinguistic analysis”
The paper relies on this without support.
Combining audio collected across different years and recorded with different equipment is acceptable because it is common in dialectology studies.
“Given the obstacles in assembling data from a large number of speakers across multiple field sites, it is not uncommon in dialectology studies to combine audio collected across different years and recorded with different equipment.”
The paper relies on this without support.
The dialect density measure (DDM) used in this study provides insight into drivers of the ASR performance gaps.
“in our setting, the measure we use provides insight into drivers of the ASR performance gaps we see.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“but also pose usability barriers for novice users such as older adults learning how to engage with them [59, 93].”
From "It's Kind of Like Code-Switching": Black Older Adults' Experiences with a Voice Assistant for Health Information Seeking (2022) · cites this paper as [59]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Many sources show ASR's volatility with diferent datasets: ASR performs diferently for male or female speakers, shows racial biases or varying accuracy rates for speakers with diferent accents [Cumbal et al. 2021;Koenecke et al. 2020;Speechmatics 2023;Tadimeti et al. 2022;Tatman and Kasten 2017].”
From Measuring the Accuracy of Automatic Speech Recognition Solutions (2023) · cites this paper as Koenecke et al. 2020
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The studies aim to show that ASR has speciic biases that often afect speakers of underrepresented groups, but are not concerned with the overall accuracy of the engines.”
From Measuring the Accuracy of Automatic Speech Recognition Solutions (2023) · cites this paper as Koenecke et al. 2020
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Unsurprisingly, gendered and racial disparities have been documented in a number of language technologies [37, 79, 141]”
From Data Governance in the Age of Large-Scale Data-Driven Language Technology (2022) · cites this paper as [79]
The cited source says what the claim attributes to it.
In the citing paper
“it is well-documented that standard models often perform poorly on under-represented demographics (Buolamwini and Gebru, 2018; Koenecke et al., 2020)”
From WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020) · cites this paper as Koenecke et al., 2020
The cited source says what the claim attributes to it.
In the citing paper
“Performance disparities across individuals have been observed in a wide range of tasks and applications, including in natural language processing (Geva et al., 2019), automatic speech recognition (Koenecke et al., 2020; Tatman, 2017), federated learning (Li et al., 2019b; Caldas et al., 2018), and medical imaging (Badgeley et al., 2019).”
From WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020) · cites this paper as Koenecke et al., 2020
In the citing paper
“This means that AIaaS might not only contribute to replicating fairness issues at scale by propagating model's intrinsic biases (or other issues) to the vast number of user applications powered by a given service (see previous work including [22,81,133,139]).”
From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [81]
In the citing paper
“These issues are particularly related to biases in datasets and algorithms, such as unfair representation.”
From Artificial intelligence in assessment and intervention of speech and language disorders: A literature review (2025) · cites this paper as [47]
In the citing paper
“In addition, Koenecke et al . [47] emphasized that ethical issues need to be systematically addressed in the clinical use of AI.”
From Artificial intelligence in assessment and intervention of speech and language disorders: A literature review (2025) · cites this paper as [47]
In the citing paper
“For instance, automatic speech recognition systems demonstrate lower accuracy for African American speakers of English compared to speakers of General American English, and this difference is particularly pronounced in children [47].”
From Artificial intelligence in assessment and intervention of speech and language disorders: A literature review (2025) · cites this paper as [47]
Citations
Cites
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137
MatchIt : Nonparametric Preprocessing for Parametric Causal Inference
Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2011) · Journal of Statistical Software · doi:10.18637/jss.v042.i08 · Cited by 4,385
(2018) · Cited by 3,510
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159
MatchIt: Nonparametric Preprocessing for Parametric Causal Inference
Ho, D. E., Imai, K., King, G., Stuart, E. A., & Greifer, N. (2005) · doi:10.32614/cran.package.matchit · Cited by 1,210
Algorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149
Self-citationInherent trade-offs in the fair determination of risk scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.itcs.2017.43 · Cited by 943
The Risk of Racial Bias in Hate Speech Detection
Sap, M., Card, D., Gabriel, S., Choi, Y., & Smith, N. A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1163 · Cited by 817
Research works printed but not linked: 2
- I. D. Raji, J. Buolamwini, “Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial ai products” in AAAI/ACM Conference on AI Ethics and Society (Association for Computing Machinery, 2019), Vol. 1, pp. 429–435.
- J. R. Rickford, Phonological and Grammatical Features of African American Vernacular (AAVE) (Blackwell Publishers, 1999).
Cited by
Sorted by citations
Items 201–300 of 769
Who is more trustworthy, Alexa or mom?: Children’s selective trust in a digital age.
Murray, G. W. (2021) · Technology Mind and Behavior · doi:10.1037/tmb0000050 · Cited by 13
GailBot: An automatic transcription system for Conversation Analysis
Umair, M., Mertens, J. B., Albert, S., & de Ruiter, J. P. (2022) · Dialogue & Discourse · doi:10.5210/dad.2022.103 · Cited by 12
Miner, A. S., Fleming, S. L., Haque, A., Fries, J., Althoff, T., Wilfley, D. E., Agras, W. S., Milstein, A. S., Hancock, J. T., Asch, S. M., et al. (2022) · npj Mental Health Research · doi:10.1038/s44184-022-00020-9 · Cited by 18
Biased sampling of early users and the direction of startup innovationPreprint
Cao, R., Koning, R. M., & Nanda, R. (2020) · doi:10.31235/osf.io/g6wjn · Cited by 13
Using Digital Data to Protect and Promote the Most Vulnerable in the Fight Against COVID-19
Chunara, R., & Cook, S. H. (2020) · Frontiers in Public Health · doi:10.3389/fpubh.2020.00296 · Cited by 10
Bias and stereotyping: Human and artificial intelligence (AI)
Kang, O., & Hirschi, K. (2025) · Annual Review of Applied Linguistics · doi:10.1017/s026719052500008x · Cited by 15
Fulton, R., Fulton, D., & Kaplan, S. (2022) · International Conference on Embedded Systems and Applications (EMSA) · doi:10.5121/csit.2022.120613 · Cited by 12
Newman-Griffis, D. R., Hurwitz, M. B., McKernan, G., Houtrow, A. J., & Dicianno, B. E. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000135 · Cited by 11
Performance Disparities between Accents in Automatic Speech Recognition (Student Abstract)
DiChristofano, A., Shuster, H., Chandra, S., & Patwari, N. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i13.26960 · Cited by 11
Evaluating and mitigating unfairness in multimodal remote mental health assessments
Jiang, Z., Seyedi, S. S., Griner, E. L., Abbasi, A., Rad, A. B., Kwon, H., Cotes, R. O., & Clifford, G. D. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000413 · Cited by 13
Why Can't Black Women Just Be?: Black Femme Content Creators Navigating Algorithmic Monoliths
Williams, G., Chen, N., DeVito, M. A., & To, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713842 · Cited by 12
Nandkumar, C., & Peternel, L. (2025) · Frontiers in Robotics and AI · doi:10.3389/frobt.2025.1576348 · Cited by 20
Personal Digital Assistants: The Need for Governance
Almeida, V. A. F., Furtado, E. S., & Furtado, V. (2020) · IEEE Internet Computing · doi:10.1109/mic.2020.3009897 · Cited by 11
Preliminary Evaluation of Automated Speech Recognition Apps for the Hearing Impaired and Deaf
Pragt, L., van Hengel, P., Grob, D., & Wasmann, J.-W. A. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.806076 · Cited by 11
Johnson, A., Fan, R., Morris, R., & Alwan, A. A. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746281 · Cited by 11
Dialect-robust Evaluation of Generated Text
Sun, J., Sellam, T., Clark, E., Vu, T., Dozat, T., Garrette, D., Siddhant, A., Eisenstein, J., & Gehrmann, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.331 · Cited by 18
Seamless equal accuracy ratio for inclusive CTC speech recognition
Gao, H., Wang, X., Kang, S., Mina, R. J. L., Issa, D., Harvill, J. B., Sarı, L., Hasegawa‐Johnson, M., & Yoo, C. D. (2021) · Speech Communication · doi:10.1016/j.specom.2021.11.004 · Cited by 11
Auditing the Audits: Lessons for Algorithmic Accountability from Local Law 144's Bias Audits
Gerchick, M. K., Encarnación, R., Tanigawa-Lau, C., Armstrong, L., Gutiérrez, A., & Metaxa, D. (2025) · Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3715275.3732004 · Cited by 10
Considerations for Ethical Speech Recognition Datasets
Papakyriakopoulos, O., & Xiang, A. (2023) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3539597.3575793 · Cited by 10
Exploring Subgroup Performance in End-to-End Speech Models
Koudounas, A., Pastor, E., Attanasio, G., Mazzia, V., Giollo, M., Gueudré, T., Cagliero, L., de Alfaro, L., Baralis, E., & Amberti, D. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095284 · Cited by 10
Harrington, L. (2023) · Frontiers in Communication · doi:10.3389/fcomm.2023.1165233 · Cited by 12
Zellou, G., & Lahrouchi, M. (2024) · Scientific Reports · doi:10.1038/s41598-023-50516-3 · Cited by 10
Exploring Data Augmentation in Bias Mitigation Against Non-Native-Accented Speech
Zhang, Y., Herygers, A., Patel, T. B., Yue, Z., & Scharenborg, O. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389756 · Cited by 10
Enhancing Communication Equity: Evaluation of an Automated Speech Recognition Application in Ghana
Ayoka, G., Barbareschi, G., Cave, R., & Holloway, C. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641903 · Cited by 12
AI-Driven Support for People with Speech & Language Difficulties
Dangol, A., Huang, Y., Setlur, S., Smolansky, A., Subramonyam, H., Suh, H., Xiong, J., & Kientz, J. A. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3643984 · Cited by 10
A robust accent classification system based on variational mode decomposition
Subhash, D., Lal, G. J., Premjith, B., & Ravi, V. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109512 · Cited by 10
Jiang, L. G., & Gu, M. M. (2025) · Journal of Multilingual and Multicultural Development · doi:10.1080/01434632.2025.2526103 · Cited by 12
Wei, X., Wang, H., Wang, Y., Zhang, W., Chen, C., Li, K., Han, L., Rufo, J., Xu, J., Yao, Y., et al. (2025) · Nature Biomedical Engineering · doi:10.1038/s41551-025-01505-6 · Cited by 11
Wong, J. M. S. (2026) · Early Childhood Education Journal · doi:10.1007/s10643-026-02138-3 · Cited by 11
Borgese, M., Joyce, C., Anderson, E. E., Churpek, M. M., & Afshar, M. (2021) · PubMed Central · Cited by 10
Machine Learning practices and infrastructures
Berman, G. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604689 · Cited by 10
van Osta, J. M., Dreis, B., Meyer, E. A., Grogan, L. F., & Castley, J. G. (2023) · Ecological Informatics · doi:10.1016/j.ecoinf.2023.102233 · Cited by 12
Large Language Models are Catalyzing Chemistry EducationPreprint
Du, Y., Duan, C., Bran, A. M., Sotnikova, A., Qu, Y., Kulik, H. J., Bosselut, A., Xu, J., & Schwaller, P. (2024) · ChemRxiv · doi:10.26434/chemrxiv-2024-h722v · Cited by 10
Understanding How to Administer Voice Surveys through Smart Speakers
Wei, J., Jiang, W., Wang, C., Yu, D., Goncalves, J., Dingler, T., & Kostakos, V. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555606 · Cited by 9
Four Surveillance Technologies Creating Challenges for Education
Pea, R. D., Biernacki, P., Bigman, M., Boles, K. L., Coelho, R., Docherty, V., Garcia, J. E., Lin, V. J., Nguyen, J., Pimentel, D. R., et al. (2022) · Springer eBooks · doi:10.1007/978-3-031-09687-7_19 · Cited by 9
Applying Artificial Intelligence for Phenotyping of Inherited Arrhythmia Syndromes
Sigfstead, S., Jiang, R., Avram, R., Davies, B., Krahn, A. D., & Cheung, C. C. (2024) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2024.04.014 · Cited by 9
Fatehi, K., Torres, M. T., & Küçükyılmaz, A. (2024) · Speech Communication · doi:10.1016/j.specom.2024.103151 · Cited by 12
Nabende, J. N., Kagumire, S., Kantono, C., & Nabende, P. (2025) · ACM Computing Surveys · doi:10.1145/3769089 · Cited by 9
Revisiting Parity of Human vs. Machine Conversational Speech Transcription
Mansfield, C., Ng, S., Levow, G., WRIGHT, R. A., & Mari Ostendorf, E. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1908 · Cited by 9
Cognitive Hexagon-Controlled Intelligent Speech Interaction System
Chaurasiya, H. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2022.3168807 · Cited by 9
Koenecke, A., Giannella, E., Willer, R., & Goel, S. (2023) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v17i1.22163 · Cited by 11
Self-citationAn Acoustical and Lexical Machine-Learning Pipeline to Identify Connectional Silences
Matt, J. E., Rizzo, D. M., Javed, A., Eppstein, M. J., Manukyan, V., Gramling, C. J., Dewoolkar, A., & Gramling, R. E. (2023) · Journal of Palliative Medicine · doi:10.1089/jpm.2023.0087 · Cited by 10
Cuadra, A., Tin, A. L., Moffat, G. T., Alexander, K., Downey, R. J., Korc‐Grodzicki, B., Vickers, A. J., & Shahrokni, A. (2022) · European Journal of Surgical Oncology · doi:10.1016/j.ejso.2022.11.011 · Cited by 9
A study of bias mitigation strategies for speaker recognition
Peri, R., Somandepalli, K., & Narayanan, S. S. (2022) · Computer Speech & Language · doi:10.1016/j.csl.2022.101481 · Cited by 17
Language Across the Disciplines
Hudley, A. H. C., Clemons, A. M., & Villarreal, D. (2023) · Annual Review of Linguistics · doi:10.1146/annurev-linguistics-022421-070340 · Cited by 9
The Sins of the Parents Are to Be Laid Upon the Children: Biased Humans, Biased Data, Biased Models
Osborne, M. R., Omrani, A., & Dehghani, M. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916231180099 · Cited by 9
Rating Sentiment Analysis Systems for Bias Through a Causal Lens
Lakkaraju, K., Srivastava, B., & Valtorta, M. (2024) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2024.3375519 · Cited by 9
Black-Box Adaptation of ASR for Accented Speech
Khandelwal, K., Jyothi, P., Awasthi, A., & Sarawagi, S. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-3162 · Cited by 9
Hauglid, M. K. (2022) · Schizophrenia Bulletin · doi:10.1093/schbul/sbac008 · Cited by 9
ASR4REAL: An extended benchmark for speech modelsPreprint
Rivière, M., Copet, J., & Synnaeve, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.08583 · Cited by 15
Bakir, V., Laffer, A., & McStay, A. (2023) · Surveillance & Society · doi:10.24908/ss.v21i2.16015 · Cited by 8
Accents in Speech Recognition through the Lens of a World Englishes Evaluation Set
Río, M. D., Miller, C., Profant, J., Drexler, J., McNamara, Q., Bhandari, N., Delworth, N., Pirkin, I., Jetté, M., Chandra, S., et al. (2023) · Research in Language · doi:10.18778/1731-7533.21.3.02 · Cited by 8
Bear, E., Chen, X., Souto, D. V., Ribeiro-Flucht, L., Rudzewitz, B., & Meurers, D. (2024) · System · doi:10.1016/j.system.2024.103460 · Cited by 8
Ezema, K., Chandler, C., Southwell, R. V., Cholendiran, N., & D’Mello, S. K. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714059 · Cited by 10
Artificial intelligence and physician burnout: A productivity paradox
Goodson, D. A., Garcia, B., Hogarth, M. A., & Tu, S.-P. (2025) · Learning Health Systems · doi:10.1002/lrh2.70013 · Cited by 11
Vindigni, G. (2025) · European Journal of Applied Science Engineering and Technology · doi:10.59324/ejaset.2025.3(3).06 · Cited by 11
Advancements in Multilingual Speech Technologies for Human-Computer Interaction
Ravikumar, R. N., Aarthi, S., Usmani, M. A., Rai, P. K., Xusinovna, J. M., & Narboshova, M. (2025) · IGI Global eBooks · doi:10.4018/979-8-3373-3048-8.ch004 · Cited by 8
Ethical considerations for clinical adoption of ambient digital scribe technology
Anderson, T. N., Mohan, V., & Gold, J. A. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf227 · Cited by 14
Richer Countries and Richer Representations
Zhou, K., Ethayarajh, K., & Jurafsky, D. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.164 · Cited by 10
Self-citationMind the data gap(s): Investigating power in speech and language datasets
Markl, N. (2022) · Workshop on Language Technology for Equality, Diversity, Inclusion (LT-EDI) · doi:10.18653/v1/2022.ltedi-1.1 · Cited by 11
DADA: Dialect Adaptation via Dynamic Aggregation of Linguistic Rules
Liu, Y., Held, W., & Yang, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.850 · Cited by 10
Leblebici, D. (2024) · Multilingua · doi:10.1515/multi-2023-0072 · Cited by 10
Putting algorithmic bias on top of the agenda in the discussions on autonomous weapons systems
Bhila, I. (2024) · Digital War · doi:10.1057/s42984-024-00094-z · Cited by 8
Characterising the Role of Pre-Processing Parameters in Audio-based Embedded Machine Learning
Toussaint, W., Mathur, A., Ding, A. Y., & Kawsar, F. (2021) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3485730.3493448 · Cited by 8
Lialin, V., Rawls, S., Chan, D., Ghosh, S., Rumshisky, A., & Hamza, W. (2023) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw58289.2023.00043 · Cited by 8
Multilingualism and mismatching: Spanish language usage in college admissions essays
Alvero, A., & Pattichis, R. (2024) · Poetics · doi:10.1016/j.poetic.2024.101903 · Cited by 8
SVEva Fair: A Framework for Evaluating Fairness in Speaker VerificationPreprint
Toussaint, W., & Ding, A. Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.12049 · Cited by 9
Towards temporal sentence grounding in videos
Zhang, H. (2022) · doi:10.32657/10356/163788 · Cited by 8
Ethnography and Machine Learning
Li, Z., & Abramson, C. M. (2023) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780197653609.013.36 · Cited by 11
Analysis of forced aligner performance on L2 English speech
Williams, S., Foulkes, P., & Hughes, V. (2024) · Speech Communication · doi:10.1016/j.specom.2024.103042 · Cited by 7
The Social Consequences of Language Technologies and Their Underlying Language Ideologies
Goldshtein, M., Ocumpaugh, J. L., Potter, A., & Roscoe, R. D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60875-9_18 · Cited by 9
Monitoring student behavior in autonomous automatic speech recognition-based pronunciation practice
Inceoglu, S., Chen, W.-H., & Lim, H. (2024) · System · doi:10.1016/j.system.2024.103387 · Cited by 14
Fair XIDS: Ensuring fairness and transparency in intrusion detection models
Chinu, & Bansal, U. (2024) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.8268 · Cited by 7
Stoykova, R., Porter, K., & Beka, T. (2024) · Forensic Science International Synergy · doi:10.1016/j.fsisyn.2024.100563 · Cited by 14
“I Felt Listened to”: Evaluating an AI-Powered Reflection Tool for Care Partners
Johnson, J., Jamshed, H., Zuppke, R., Leggett, A. N., Provost, E. M., & Brewer, R. N. (2025) · ACM Transactions on Accessible Computing · doi:10.1145/3731562 · Cited by 7
Mobile medical systems for equitable healthcare
Chan, J., Goel, M., Gollakota, S., & Nandakumar, R. (2025) · Nature Reviews Bioengineering · doi:10.1038/s44222-025-00330-5 · Cited by 8
de Barbaro, K., Madden‐Rusnak, A., & Timmons, A. C. (2026) · Developmental Science · doi:10.1111/desc.70144 · Cited by 7
Rewriting Algorithms for Just Recognition: From Digital Aural Redlining to Accent Activism
Eidsheim, N. S. (2023) · University of California Press eBooks · doi:10.1525/luminos.148.h · Cited by 7
Johnson, B., Bartola, J., Angell, R., Witty, S., Giguere, S., & Brun, Y. (2023) · EURO Journal on Decision Processes · doi:10.1016/j.ejdp.2023.100031 · Cited by 18
Szymański, P., Augustyniak, Ł., Morzy, M., Szymczak, A., Surdyk, K., & Żelasko, P. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.98 · Cited by 7
TADA : Task Agnostic Dialect Adapters for English
Held, W., Ziems, C., & Yang, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.51 · Cited by 8
Slaughter, I., Greenberg, C. S., Schwartz, R., & Caliskan, A. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.602 · Cited by 12
Automatic Speech Recognition in Diverse English Accents
Mohyuddin, H., & Kwak, D. (2023) · International Conference on Computational Science and Computational Intelligence (CSCI) · doi:10.1109/csci62032.2023.00122 · Cited by 7
Computational sociophonetics using automatic speech recognition
Coto‐Solano, R. (2022) · Language and Linguistics Compass · doi:10.1111/lnc3.12474 · Cited by 8
Adoption of Socio-Cultural Aspects in PSS Design for Smart Home Products: An Integrative Review
Ganvir, L., & Kalita, P. C. (2022) · Archives of Design Research · doi:10.15187/adr.2022.11.35.4.7 · Cited by 7
Study on the Fairness of Speaker Verification Systems Across Accent and Gender Groups
Estévez, M., & Ferrer, L. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095150 · Cited by 7
Markl, N., & Lai, C. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-1847 · Cited by 7
Decolonizing digital accessibility within land/water realities using minimal computing
Rice, M. F., & de Jesús, J. T. A. (2024) · Learning Media and Technology · doi:10.1080/17439884.2024.2394471 · Cited by 10
Towards Making Virtual Human-Robot Interaction a Reality
Higgins, P., Kebe, G. Y., Berlier, A. J., Darvish, K., Engel, D., Ferraro, F., & Matuszek, C. (2021) · Maryland Shared Open Access Repository (USMAI Consortium) · doi:10.13016/m2lhch-cuzp · Cited by 7
Johnson, B., Bartola, J., Angell, R., Keith, K. A., Witty, S., Giguere, S., & Brun, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.09951 · Cited by 7
Law and the Emerging Political Economy of Algorithmic AuditsPreprint
Terzis, P., Veale, M., & Gaumann, N. (2024) · doi:10.31219/osf.io/xvqz7 · Cited by 7
Gautam, S., Chandra, M., De, A., Chakravorti, T., Malik, G., & De Choudhury, M. (2025) · arXiv · doi:10.48550/arxiv.2508.06849 · Cited by 7
The Limits of Calibration and the Possibility of Roles for Trustworthy AI
Franke, U. (2024) · Philosophy & Technology · doi:10.1007/s13347-024-00771-7 · Cited by 7
Camp, N. P., & Voigt, R. (2024) · Behavioral Science & Policy · doi:10.1177/23794607241308636 · Cited by 6
Gender and racial bias issues in a commercial “tone of voice” analysis system
Holliday, N. R., & Reed, P. E. (2025) · PLoS ONE · doi:10.1371/journal.pone.0314470 · Cited by 8
Ensuring Equity and Access for Students With Disabilities
Sarkar, R., & Madhu, R. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-5097-4.ch004 · Cited by 8
MultiMediate '25: Cross-cultural Multi-domain Engagement Estimation
Don, D. S. W., Funk, M., Balážia, M., Qiu, H., Okada, S., Brémond, F., Alexandersson, J., Bulling, A., André, E., & Müller, P. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3762076 · Cited by 10
Schneider, B., & de Oliveira, M. M. (2025) · AI & Society · doi:10.1007/s00146-025-02727-7 · Cited by 12
Hirai, A., & Kovalyova, A. (2023) · English language education · doi:10.1007/978-3-031-27825-9_17 · Cited by 6
Li, T., & Iacobelli, F. (2023) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3584931.3606954 · Cited by 6
Metadata
- Authors
- Koenecke, A., Nam, A., Lake, E., Nudell, J., Quartey, M., Mengesha, Z., Toups, C., Rickford, J. R., Jurafsky, D., & Goel, S.
- Affiliations
- Stanford University (US)
- Georgetown University (US)
- Corresponding author
- Goel, Sharad
- ORCID
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2020-03-23
- Volume
- 117
- Issue
- 14
- Pages
- 7684-7689
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC7149386
- OpenAlex
- W3012624518
- Semantic Scholar
- 219b7266ae848937da170c5510b2bfc66d17859a
- Cited by
- 765
- References
- 37
- Citations by year
- 2026: 169 · 2025: 110 · 2024: 112 · 2023: 171 · 2022: 112 · 2021: 68 · 2020: 23
- FWCI
- 36.76
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- speech recognition, racial disparities, algorithmic bias, word error rate, acoustic modeling, African American English, training data diversity, inclusive technology
- MeSH terms
- Adult, Automation, Black or African American, Humans, Language, Speech Perception, White People, Speech Recognition Software, Racism
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
