CoChat

Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agency

Open access
Yee, Kyra, Tantipongpipat, Uthaipon, Mishra, Shubhanshu · 2021 · Proceedings of the ACM on Human-Computer Interaction

The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.

Abstract

Twitter uses machine learning to crop images, where crops are centered around the part predicted to be the most salient. In fall 2020, Twitter users raised concerns that the automated image cropping system on Twitter favored light-skinned over dark-skinned individuals, as well as concerns that the system favored cropping woman's bodies instead of their heads. In order to address these concerns, we conduct an extensive analysis using formalized group fairness metrics. We find systematic disparities in cropping and identify contributing factors, including the fact that the cropping based on the single most salient point can amplify the disparities because of an effect we term argmax bias. However, we demonstrate that formalized fairness metrics and quantitative analysis on their own are insufficient for capturing the risk of representational harm in automatic cropping. We suggest the removal of saliency-based cropping in favor of a solution that better preserves user agency. For developing a new solution that sufficiently address concerns related to representational harm, our critique motivates a combination of quantitative and qualitative methods that include human-centered design.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3479594

Cites

42 of 42 references verified0 citation errors · 0 not checkable

Statistics

Figures

Saliency Map
Saliency MapCC-BY-ND · From the PDF
Image 2
CC-BY-ND · From the PDF
Image 3
CC-BY-ND · From the PDF
Saliency Rank: 1 | ar=1.14
Saliency Rank: 1 | ar=1.14CC-BY-ND · From the PDF
Image 5
CC-BY-ND · From the PDF
Image 6
CC-BY-ND · From the PDF
Image 7
CC-BY-ND · From the PDF
Saliency Rank: 1 | ar=0.40
Saliency Rank: 1 | ar=0.40CC-BY-ND · From the PDF
Image 9
CC-BY-ND · From the PDF
Image 10
CC-BY-ND · From the PDF
Fig. 1 Automated image crops (yellow rectangles) on the image of two individuals separated by the white background in the middle. Crops are produced by Twitter saliency-based model for different aspect ratios. Top left: the heat map of saliency scores produced by Twitter image cropping model on the image. Locations with high predicted saliency could be humans, objects, texts, and high-contrast backgrounds. Others: the original image with its maximum saliency point (as the yellow dot) and the automatic crop (as the yellow rectangle) with different aspect ratios (the 'ar' values). Images from Wikidata.
Fig. 1 Automated image crops (yellow rectangles) on the image of two individuals separated by the white background in the middle. Crops are produced by Twitter saliency-based model for different aspect ratios. Top left: the heat map of saliency scores produced by Twitter image cropping model on the image. Locations with high predicted saliency could be humans, objects, texts, and high-contrast backgrounds. Others: the original image with its maximum saliency point (as the yellow dot) and the automatic crop (as the yellow rectangle) with different aspect ratios (the 'ar' values). Images from Wikidata.CC-BY-ND · From the PDF
Fig. 2 Left : The proportion of times (on y-axis) a group was selected as the crop focus when comparing each pair of groups (on x-axis). The right most panel aggregates subgroups over race or gender.
Fig. 2 Left : The proportion of times (on y-axis) a group was selected as the crop focus when comparing each pair of groups (on x-axis). The right most panel aggregates subgroups over race or gender.CC-BY-ND · From the PDF
Fig. 3 Wescale the images to a fixed height before performing the experiment. The sample size is 5000, and each probability has ± 1 . 4% (after rounding) as its 95% confidence interval. We see no significant difference in results from the original in Figure 2 ( Right ).
Fig. 3 Wescale the images to a fixed height before performing the experiment. The sample size is 5000, and each probability has ± 1 . 4% (after rounding) as its 95% confidence interval. We see no significant difference in results from the original in Figure 2 ( Right ).CC-BY-ND · From the PDF
Fig. 4 We find maximum of saliency score of each image individually before comparing them, instead of attaching images first then find the max saliency point. The probability is computed over all possible image pairs on each subgroup comparison. Model disparity on race, but not on gender, has significantly changed compared to Figure 2 ( Right ).
Fig. 4 We find maximum of saliency score of each image individually before comparing them, instead of attaching images first then find the max saliency point. The probability is computed over all possible image pairs on each subgroup comparison. Model disparity on race, but not on gender, has significantly changed compared to Figure 2 ( Right ).CC-BY-ND · From the PDF
Fig. 5 Body areas can be salient because of certain artifacts in images. Top: Images where the crop is on the body because of jersey numbers. Bottom: similar images where the crop is on the head but jersey number is among a major salient point. Images used in this figure are of public figures available on Wikidata.
Fig. 5 Body areas can be salient because of certain artifacts in images. Top: Images where the crop is on the body because of jersey numbers. Bottom: similar images where the crop is on the head but jersey number is among a major salient point. Images used in this figure are of public figures available on Wikidata.CC-BY-ND · From the PDF
Fig. 6 An image with multiple AI researchers (source: [78]) with multiple almost equally likely salient points. Top: the heat map and the crops based on the first and second most salient points. Bottom: saliency scores of all salient points in the images (top left to bottom right, each row is demarcated by vertical slices). Always cropping based on the top salient point amplifies this crop on the social system compared to the next best salient point.
Fig. 6 An image with multiple AI researchers (source: [78]) with multiple almost equally likely salient points. Top: the heat map and the crops based on the first and second most salient points. Bottom: saliency scores of all salient points in the images (top left to bottom right, each row is demarcated by vertical slices). Always cropping based on the top salient point amplifies this crop on the social system compared to the next best salient point.CC-BY-ND · From the PDF
Fig. 7 Empirical cumulative distribution functions (ECDFs) of maximum (left) and median (right) saliency scores across all images of Black (top) and White (bottom) individuals, separated by gender. Curves of ECDFs that are on the left are the distributions that on overall take smaller values than ones on the right. The wider the gap, the larger the differences of values.
Fig. 7 Empirical cumulative distribution functions (ECDFs) of maximum (left) and median (right) saliency scores across all images of Black (top) and White (bottom) individuals, separated by gender. Curves of ECDFs that are on the left are the distributions that on overall take smaller values than ones on the right. The wider the gap, the larger the differences of values.CC-BY-ND · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 42 references

Research works printed but not linked: 22

  1. Ruha Benjamin. 2019. Race After Technology: Abolitionist Tools for the New Jim Code. John Wiley & Sons.
  2. J.D. Biersdorfer. 2017. Fitting Your Photos on Instagram. The New York Times (2017).
  3. James E Bollman, Ramana L Rao, Dennis L Venable, and Reiner Eschbach. 1999. Automatic image cropping. US Patent 5,978,519.
  4. Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. 77-91.
  5. Judith Butler, Fabiana AA Jardim, Jacqueline Moraes Teixeira, and Sebastião Rinaldi. 2020. Endangered/endangering: Schematic racism and white paranoia. Educação e Pesquisa 46 (2020).
  6. Kate Crawford. 2017. The trouble with bias. In Conference on Neural Information Processing Systems, invited speaker.
  7. Terrance de Vries, Ishan Misra, Changhan Wang, and Laurens van der Maaten. 2019. Does object recognition work for everyone?. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 52-59.
  8. Malkia Devich-Cyril. 2020. Defund Facial Recognition. The Atlantic (2020).
  9. Ravit Dotan and Smitha Milli. 2019. Value-laden disciplinary shifts in machine learning. arXiv preprint arXiv:1912.01172 (2019). · arXiv
  10. Arturo Escobar. 2018. Designs for the Pluriverse. Duke University Press.
  11. Karen E Fields and Barbara Jeanne Fields. 2014. Racecraft: The soul of inequality in American life. Verso Trade.
  12. Barbara L Fredrickson and Tomi-Ann Roberts. 1997. Objectification theory: Toward understanding women's lived experiences and mental health risks. Psychology of women quarterly 21, 2 (1997), 173-206.
  13. Karamjit S Gill. 1990. Summary of human-centered systems research in Europe. University of Brighton, SEAKE Centre.
  14. Charles Hirschman. 2004. The origins and demise of the concept of race. Population and development review 30, 3 (2004), 385-415.
  15. Xun Huang, Chengyao Shen, Xavier Boix, and Qi Zhao. 2015. Salicon: Reducing the semantic gap in saliency prediction by adapting deep neural networks. In Proceedings of the IEEE International Conference on Computer Vision. 262-270. · doi:10.5555/2919332.2919806
  16. Catharine A MacKinnon. 1987. Feminism unmodified: Discourses on life and law. Harvard university press.
  17. Paul Mozur. 2019. One month, 500,000 face scans: How China is using AI to profile a minority. The New York Times 4 (2019).
  18. Laura Mulvey. 1989. Visual pleasure and narrative cinema. In Visual and other pleasures. Springer, 14-26.
  19. Arvind Narayanan. 2018. Translation tutorial: 21 fairness definitions and their politics. In Proc. Conf. Fairness Account- ability Transp., New York, USA, Vol. 1170.
  20. Ben Shneiderman and Pattie Maes. 1997. Direct manipulation vs. interface agents. interactions 4, 6 (1997), 42-61.
  21. Nancy A Van House. 2011. Feminist HCI meets Facebook: Performativity and social networking sites. Interacting with computers 23, 5 (2011), 422-429.
  22. Qian Yang. 2017. The role of design in creating machine-learning-enhanced user experience. In 2017 AAAI spring symposium series.

Cited by

See all 94

Metadata

Authors
Yee, K., Tantipongpipat, U., & Mishra, S.
Affiliations
  • Twitter (United States) (US)
  • Twitter Inc., NYC, NY, USA
Venue
Proceedings of the ACM on Human-Computer Interaction
Published
2021-10-13
Volume
5
Issue
CSCW2
Pages
1-24
ISSN
2573-0142 (ISSN-L)
OpenAlex
W3163680969
Cited by
59
References
26
Citations by year
2026: 5 · 2025: 6 · 2024: 10 · 2023: 24 · 2022: 11 · 2021: 3
FWCI
15.02
Type
journal article
Language
en
Open access
Locations
  • Proceedings of the ACM on Human-Computer Interaction (publishedVersion, cc-by-nd)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
Keywords
Twitter, skin tone bias, representational harm, user agency, human-centered design, algorithmic fairness

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agency. https://app.cochat.ai/papers/10.1145/3479594
The paper
Yee, K., Tantipongpipat, U., & Mishra, S. (2021). Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agency. Proceedings of the ACM on Human-Computer Interaction. https://doi.org/10.1145/3479594

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.