Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agency
Open accessThe DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Abstract
Twitter uses machine learning to crop images, where crops are centered around the part predicted to be the most salient. In fall 2020, Twitter users raised concerns that the automated image cropping system on Twitter favored light-skinned over dark-skinned individuals, as well as concerns that the system favored cropping woman's bodies instead of their heads. In order to address these concerns, we conduct an extensive analysis using formalized group fairness metrics. We find systematic disparities in cropping and identify contributing factors, including the fact that the cropping based on the single most salient point can amplify the disparities because of an effect we term argmax bias. However, we demonstrate that formalized fairness metrics and quantitative analysis on their own are insufficient for capturing the risk of representational harm in automatic cropping. We suggest the removal of saliency-based cropping in favor of a solution that better preserves user agency. For developing a new solution that sufficiently address concerns related to representational harm, our critique motivates a combination of quantitative and qualitative methods that include human-centered design.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures















![Fig. 6 An image with multiple AI researchers (source: [78]) with multiple almost equally likely salient points. Top: the heat map and the crops based on the first and second most salient points. Bottom: saliency scores of all salient points in the images (top left to bottom right, each row is demarcated by vertical slices). Always cropping based on the top salient point amplifies this crop on the social system compared to the next best salient point.](https://figures.cochat.ai/paper-figures/f1508679af2b4e2286eebee49f4c3c91/image-cropping-on-twitter-fairness-metrics-their-limitations-and-the-importance-figure-6.png)

Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the algorithmic audit has emerged as one of the most popular approaches to algorithmic accountability [10,66].”
From Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem (2022) · cites this paper as [66]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“All individuals mentioned by multiple respondents have published at least one audit: Cathy O'Neil (mentioned by 9 respondents); Dr. Joy Buolamwini, Dr. Timnit Gebru, and Deborah Raji (6 respondents); Christo Wilson (3 respondents); and Dr. Rumman Chowdhury (2 respondents) [10,28,50,66,67].”
From Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem (2022) · cites this paper as [66]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Black Feminist Thought: Knowledge, Consciousness, and the Politics of Empowerment.
Brewer, R. M., & Collins, P. H. (1992) · Contemporary Sociology · doi:10.2307/2074808 · Cited by 19,445
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“In order to reduce the computational cost to be able to use the model in production, Twitter leverages a combination of fisher pruning [72] and knowledge distillation [43].”
BackgroundCrenshaw, K. (2018) · Feminist Legal Theory · doi:10.4324/9780429500480-5 · Cited by 9,510
Algorithms of Oppression: How Search Engines Reinforce Racism
Noble, S. U. (2020) · doi:10.2307/j.ctt1pwt9w5 · Cited by 2,709
Learning to predict where humans look
Judd, T., Ehinger, K., Durand, F., & Torralba, A. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459462 · Cited by 2,016 · cited 1×
“The model is trained on three publicly available external datasets: Borji and Itti [11], Jiang et al. [46], Judd et al. [47].”
MethodCertifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827 · cited 1×
“where R = 1 denotes that the most salient point (the focal point for the crop) is on the person, A represents group status [5, 33], ε is a slack value, and P (R = 1|A = a) is the probability that when two persons, one from a and one from b, are given, the auto-cropping model has the most salience on the person from a.”
BackgroundThe Chicago face database: a free stimulus set of faces and norming data
Ma, D. S., Correll, J., & Wittenbrink, B. (2015) · Behavior Research Methods · doi:10.3758/s13428-014-0532-5 · Cited by 1,788
Fairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631 · cited 2×
“Posing concerns related to image cropping purely in terms of demographic parity fails to question the "closure of predetermined,technical problems" [38] and fails to consider sociotechnical considerations, an example of the "framing trap" [66].”
BackgroundGender and Race: (What) Are They? (What) Do We Want Them To Be?
Haslanger, S. (2000) · Noûs · doi:10.1111/0029-4624.00201 · Cited by 1,564 · cited 1×
“Many critical race scholars also define race in terms of marked or perceived difference, rather than traceable ancestry in itself [42, 44, 65].”
BackgroundJiang, M., Huang, S., Duan, J., & Zhao, Q. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298710 · Cited by 729
Research works printed but not linked: 22
- Ruha Benjamin. 2019. Race After Technology: Abolitionist Tools for the New Jim Code. John Wiley & Sons.
- J.D. Biersdorfer. 2017. Fitting Your Photos on Instagram. The New York Times (2017).
- James E Bollman, Ramana L Rao, Dennis L Venable, and Reiner Eschbach. 1999. Automatic image cropping. US Patent 5,978,519.
- Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. 77-91.
- Judith Butler, Fabiana AA Jardim, Jacqueline Moraes Teixeira, and Sebastião Rinaldi. 2020. Endangered/endangering: Schematic racism and white paranoia. Educação e Pesquisa 46 (2020).
- Kate Crawford. 2017. The trouble with bias. In Conference on Neural Information Processing Systems, invited speaker.
- Terrance de Vries, Ishan Misra, Changhan Wang, and Laurens van der Maaten. 2019. Does object recognition work for everyone?. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 52-59.
- Malkia Devich-Cyril. 2020. Defund Facial Recognition. The Atlantic (2020).
- Ravit Dotan and Smitha Milli. 2019. Value-laden disciplinary shifts in machine learning. arXiv preprint arXiv:1912.01172 (2019). · arXiv
- Arturo Escobar. 2018. Designs for the Pluriverse. Duke University Press.
- Karen E Fields and Barbara Jeanne Fields. 2014. Racecraft: The soul of inequality in American life. Verso Trade.
- Barbara L Fredrickson and Tomi-Ann Roberts. 1997. Objectification theory: Toward understanding women's lived experiences and mental health risks. Psychology of women quarterly 21, 2 (1997), 173-206.
- Karamjit S Gill. 1990. Summary of human-centered systems research in Europe. University of Brighton, SEAKE Centre.
- Charles Hirschman. 2004. The origins and demise of the concept of race. Population and development review 30, 3 (2004), 385-415.
- Xun Huang, Chengyao Shen, Xavier Boix, and Qi Zhao. 2015. Salicon: Reducing the semantic gap in saliency prediction by adapting deep neural networks. In Proceedings of the IEEE International Conference on Computer Vision. 262-270. · doi:10.5555/2919332.2919806
- Catharine A MacKinnon. 1987. Feminism unmodified: Discourses on life and law. Harvard university press.
- Paul Mozur. 2019. One month, 500,000 face scans: How China is using AI to profile a minority. The New York Times 4 (2019).
- Laura Mulvey. 1989. Visual pleasure and narrative cinema. In Visual and other pleasures. Springer, 14-26.
- Arvind Narayanan. 2018. Translation tutorial: 21 fairness definitions and their politics. In Proc. Conf. Fairness Account- ability Transp., New York, USA, Vol. 1170.
- Ben Shneiderman and Pattie Maes. 1997. Direct manipulation vs. interface agents. interactions 4, 6 (1997), 42-61.
- Nancy A Van House. 2011. Feminist HCI meets Facebook: Performativity and social networking sites. Interacting with computers 23, 5 (2011), 422-429.
- Qian Yang. 2017. The role of design in creating machine-learning-enhanced user experience. In 2017 AAAI spring symposium series.
Cited by
Ethical and social risks of harm from Language Models
Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., Cheng, M., Glaese, M., Balle, B., Kasirzadeh, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.04359 · Cited by 477
“The authors described this emphasis on the female body as perpetuating the ‘male gaze, a term used for the pervasive depiction of women as sexual objects for the pleasure of and from the perspective heterosexual men’ (Yee et al., 2021), emphasis added.”
BackgroundInfluentialCan Generative AI improve social science?
Bail, C. A. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2314021121 · Cited by 369
Can Generative AI Improve Social Science?
Bail, C. A. (2023) · doi:10.31235/osf.io/rwtzs · Cited by 369
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction
Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., et al. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604673 · Cited by 279
“Representational harms include instances where certain social groups experience both over- and under-exposure [27], leading to unequal visibility [242].”
BackgroundWho Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem
Costanza-Chock, S., Raji, I. D., & Buolamwini, J. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533213 · Cited by 277
Metadata
- Authors
- Yee, K., Tantipongpipat, U., & Mishra, S.
- Affiliations
- Twitter (United States) (US)
- Twitter Inc., NYC, NY, USA
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2021-10-13
- Volume
- 5
- Issue
- CSCW2
- Pages
- 1-24
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W3163680969
- Cited by
- 59
- References
- 26
- Citations by year
- 2026: 5 · 2025: 6 · 2024: 10 · 2023: 24 · 2022: 11 · 2021: 3
- FWCI
- 15.02
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion, cc-by-nd)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Keywords
- Twitter, skin tone bias, representational harm, user agency, human-centered design, algorithmic fairness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.