The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Scaling Down Inequality: Rating Scales, Gender Bias, and the Architecture of Evaluation
Abstract
Quantitative performance ratings are ubiquitous in modern organizations—from businesses to universities—yet there is substantial evidence of bias against women in such ratings. This study examines how gender inequalities in evaluations depend on the design of the tools used to judge merit. Exploiting a quasi-natural experiment at a large North American university, we found that the number of scale points used in faculty teaching evaluations—whether instructors were rated on a scale of 6 versus a scale of 10—significantly affected the size of the gender gap in evaluations in the most male-dominated fields. A survey experiment, which presented all participants with an identical lecture transcript but randomly varied instructor gender and the number of scale points, replicated this finding and suggested that the number of scale points affects the extent to which gender stereotypes of brilliance are expressed in quantitative ratings. These results highlight how seemingly minor technical aspects of performance ratings can have a major effect on the evaluation of men and women. Our findings thus contribute to a growing body of work on organizational practices that reduce workplace inequalities and the sociological literature on how rating systems—rather than being neutral instruments—shape the distribution of rewards in organizations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Role congruity theory of prejudice toward female leaders.
Eagly, A. H., & Karau, S. J. (2002) · Psychological Review · doi:10.1037//0033-295x.109.3.573 · Cited by 7,300
Getting a Job: Is There a Motherhood Penalty?
Correll, S. J., Benard, S., & Paik, I. (2007) · American Journal of Sociology · doi:10.1086/511799 · Cited by 2,525 · cited 1×
“…the perception of male and female instructors, we included a set of simple items based on prior research on gender stereotypes of performance (e.g., Correll, Benard, and Paik 2007; Rivera and Tilcsik 2016; Rudman and Glick 2001; Storage et al. 2016), asking respondents to indicate the extent to…”
MethodHeilman, M. E. (2001) · Journal of Social Issues · doi:10.1111/0022-4537.00234 · Cited by 2,482 · cited 2×
“On average, people rate male workers as significantly more able, likable, and worthy than female workers, even when their qualifications, performance, and behaviors are identical (for reviews, see Foschi, Lai, and Sigerson 1994; Heilman 2001; Quadlin 2018; Ridgeway 2011).”
BackgroundKalev, A., Dobbin, F., & Kelly, E. (2006) · American Sociological Review · doi:10.1177/000312240607100404 · Cited by 2,052 · cited 1×
“…gaps in sociological knowledge by illuminating concrete organizational practices that can reduce workplace inequalities (see Dobbin, Schrage, and Kalev 2015; Kalev, Dobbin, and Kelly 2006; Williams 2014) and by showing how the design of evaluation tools affects gender dynamics in organizations.”
BackgroundThe Gender Revolution: Uneven and Stalled
England, P. (2010) · Gender & Society · doi:10.1177/0891243210361475 · Cited by 1,687
Toward a Comparative Sociology of Valuation and Evaluation
Lamont, M. (2012) · Annual Review of Sociology · doi:10.1146/annurev-soc-070308-120022 · Cited by 1,492 · cited 1×
“They have far-reaching effects on how people rank individuals, objects, and organizations, and how they distribute valued social and material rewards (for reviews, see Brandtner 2017; Lamont 2012; Sauder 2006; Sauder, Lynn, and Podolny 2012).”
BackgroundEspeland, W. N., & Stevens, M. L. (2008) · European Journal of Sociology · doi:10.1017/s0003975609000150 · Cited by 1,265
Gender stereotypes about intellectual ability emerge early and influence children's interests
Bian, L., Leslie, S.-J., & Cimpian, A. (2017) · Science · doi:10.1126/science.aah6524 · Cited by 1,206
Interaction and the Conservation of Gender Inequality: Considering Employment
Ridgeway, C. L. (1997) · American Sociological Review · doi:10.2307/2657301 · Cited by 939 · cited 2×
“Just as the way people evaluate men and women varies based on the structure of the task at hand (see Ridgeway 1997), we argue that the design of tools used to assess merit—or what we term the architecture of evaluation—can influence the degree to which gender stereotypes are expressed in…”
BackgroundMethodWhat's in a name: Exposing gender bias in student ratings of teaching
MacNell, L., Driscoll, A., & Hunt, A. N. (2015) · Innovative Higher Education · doi:10.1007/s10755-014-9313-4 · Cited by 762 · cited 2×
“Likewise, students rate female instructors as taking significantly longer to return feedback than male instructors, even when feedback is provided at the exact same time (MacNell et al. 2015).”
Background
Cited by
The Invisible Cage: Workers’ Reactivity to Opaque Algorithmic Evaluations
Rahman, H. A. (2021) · Administrative Science Quarterly · doi:10.1177/00018392211010118 · Cited by 375
“…context, informal, opaque inputs such as biases and discrimination exist, but they are viewed as problematic (Castilla, 2008; Castilla and Benard, 2010; Rivera and Tilcsik, 2019) and are subject to established internal (e.g., human resources) and external (e.g., litigation) pathways for recourse.”
BackgroundGender Biases in Estimation of Others’ Pain
Zhang, L., Losin, E. A. R., Ashar, Y. K., Koban, L., & Wager, T. D. (2021) · The Journal of Pain · doi:10.1016/j.jpain.2021.03.001 · Cited by 312
Socioeconomic roots of academic faculty
Morgan, A. C., LaBerge, N., Larremore, D. B., Galesic, M., Brand, J. E., & Clauset, A. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01425-4 · Cited by 269
Kreitzer, R. J., & Sweet-Cushman, J. (2022) · Journal of Academic Ethics · doi:10.1007/s10805-021-09400-w · Cited by 254
On students’ (mis)judgments of learning and teaching effectiveness.
Carpenter, S. K., Witherby, A. E., & Tauber, S. K. (2020) · Journal of Applied Research in Memory and Cognition · doi:10.1016/j.jarmac.2019.12.009 · Cited by 183
Metadata
- Authors
- Rivera, L. A., & Tilcsik, A.
- Venue
- American Sociological Review
- Publisher
- SAGE Publications
- Published
- 2019-03-12
- Volume
- 84
- Issue
- 2
- Pages
- 248-274
- ISSN
- 0003-1224 (ISSN-L)
- 1939-8271
- OpenAlex
- W2921085129
- Cited by
- 201
- References
- 16
- FWCI
- 35.51
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Scaling Down Inequality: Rating Scales, Gender Bias, and the Architecture of Evaluation. https://app.cochat.ai/papers/10.1177/0003122419833601
- The paper
- Rivera, L. A., & Tilcsik, A. (2019). Scaling Down Inequality: Rating Scales, Gender Bias, and the Architecture of Evaluation. American Sociological Review. https://doi.org/10.1177/0003122419833601
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.