The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Looking for a Handsome Carpenter! Debiasing GPT-3 Job Advertisements
Open accessAbstract
The growing capability and availability of generative language models has enabled a wide range of new downstream tasks.Academic research has identified, quantified and mitigated biases present in language models but is rarely tailored to downstream tasks where wider impact on individuals and society can be felt.In this work, we leverage one popular generative language model, GPT-3, with the goal of writing unbiased and realistic job advertisements.We first assess the bias and realism of zero-shot generated advertisements and compare them to real-world advertisements.We then evaluate prompt-engineering and fine-tuning as debiasing methods.We find that prompt-engineering with diversity-encouraging prompts gives no significant improvement to bias, nor realism.Conversely, fine-tuning, especially on unbiased real advertisements, can improve realism and reduce bias.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Modified prompting language techniques have limited effectiveness for bias removal.
“Modified prompting language techniques have been shown to have limited effectiveness. Borchers et al. (2022), for example, find instructions that prompt diversity or gender equality to be unsuccessful for bias removal in outputs. Similarly, Li and Zhang (2023) find similar generated outputs when using biased and unbiased prompts.”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Borchers et al. 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Evaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436
Language (Technology) is Power: A Critical Survey of “Bias” in NLP
Blodgett, S. L., Barocas, S., Daumé III, H., & Wallach, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.485 · Cited by 1,050
The Woman Worked as a Babysitter: On Biases in Language Generation
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1339 · Cited by 537
Ethical and social risks of harm from Language Models
Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., Cheng, M., Glaese, M., Balle, B., Kasirzadeh, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.04359 · Cited by 477
Measuring Bias in Contextualized Word Representations
Kurita, K., Vyas, N., Pareek, A., Black, A. W., & Tsvetkov, Y. (2019) · Proceedings of the First Workshop on Gender Bias in Natural Language Processing · doi:10.18653/v1/w19-3823 · Cited by 348
Smith, S., Patwary, M., Norick, B., LeGresley, P., Rajbhandari, S., Casper, J., Liu, Z., Prabhumoye, S., Zerveas, G., Korthikanti, V., et al. (2022) · arXiv · doi:10.48550/arxiv.2201.11990 · Cited by 286
Flagged first, then by citations
Research works printed but not linked: 29
- Hugo Berg, Siobhan Mackenzie Hall, Yash Bhalgat, Wonsuk Yang, Hannah Rose Kirk, Aleksandar Shtedritski, and Max Bain. 2022. A prompt array keeps the bias away: Debiasing vision-language models with adversarial learning. arXiv preprint arXiv:2203.11933. · arXiv
- Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS'16, page 4356-4364, Red Hook, NY, USA.
- Sunipa Dev and Jeff Phillips. 2019. Attenuating bias in word vectors. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 879-887.
- PMLR.
- Elena Vladimirovna Fell and Maria Dyban. 2017. Against discrimination: equality act 2010 (UK). The European Proceedings of Social & Behavioural Sciences (EpSBS). Vol. 19: Lifelong Wellbeing in the World (WELLSO 2016).-Nicosia, 2017., 192016:188-194.
- Oluwaseyi Feyisetan, Sepideh Ghanavati, and Patricia Thaine. 2020. Workshop on Privacy in NLP (PrivateNLP 2020), pages 903-904. Association for Computing Machinery, New York, NY, USA. · DOI
- Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5):378.
- Albert Gatt and Emiel Krahmer. 2018. Survey of the state of the art in natural language generation: Core tasks, applica- tions and evaluation. J. Artif. Int. Res., 61(1):65-170. · DOI
- Danielle Gaucher, Justin Friesen, and Aaron C. Kay. 2011. Evidence That Gendered Wording in Job Advertisements Exists and Sustains Gender Inequality. Journal of Person- ality and Social Psychology, 101(1):109-128. · DOI
- Hila Gonen and Yoav Goldberg. 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 609-614, Minneapolis, Minnesota. Association for Computational Linguistics. · DOI · Paper page
- Vidur Joshi, Matthew Peters, and Mark Hopkins. 2018. Ex- tending a parser to distant domains using a few dozen partially annotated examples. In Proceedings of the 56th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 1190-1199, Mel- bourne, Australia. Association for Computational Linguis- tics. · DOI
- Hannah Kirk, Yennie Jun, Haider Iqbal, Elias Benussi, Fil- ippo Volpin, Frederic A. Dreyer, Aleksandar Shtedritski, and Yuki M. Asano. 2021. Bias Out-of-the-Box: An Em- pirical Analysis of Intersectional Occupational Biases in Popular Generative Language Models. Advances in Neural Information Processing Systems, 34.
- Liuqing Li, Jack Geissinger, William A. Ingram, and Ed- ward A. Fox. 2020. Teaching natural language processing through big data text summarization with problem-based learning. Data and Information Management, 4(1):18-43. · DOI
- Liting Liu, Jie Liu, Wenzheng Zhang, Ziming Chi, Wenxuan Shi, and Yalou Huang. 2020. Hiring now: A skill-aware multi-attention model for job posting generation. In Pro- ceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3096-3104, Online. Association for Computational Linguistics. · DOI
- Ruibo Liu, Chenyan Jia, Jason Wei, Guangxuan Xu, Lili Wang, and Soroush Vosoughi. 2021b. Mitigating political bias in language models through reinforced calibration. In Pro- ceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 14857-14866.
- Kaiji Lu, Piotr Mardziel, Fangjing Wu, Preetam Amancharla, and Anupam Datta. 2020. Gender bias in neural natural language processing. In Logic, Language, and Security, pages 189-202. Springer.
- Thomas Margoni. 2019. Artificial Intelligence, Machine Learning and EU Copyright Law: Who Owns AI? SSRN Electronic Journal, 12(December). · DOI
- Saif Mohammad. 2018. Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 english words. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 174-184.
- Maarten Sap, Marcella Cindy Prasettio, Ari Holtzman, Han- nah Rashkin, and Yejin Choi. 2017. Connotation frames of power and agency in modern films. In Proceedings of the 2017 Conference on Empirical Methods in Natu- ral Language Processing, pages 2329-2334, Copenhagen, Denmark. Association for Computational Linguistics. · DOI
- Toni Schmader, Jessica Whitehead, and Vicki H Wysocki. 2007. A Linguistic Comparison of Letters of Recommen- dation for Male and Female Chemistry and Biochemistry Job Applicants. Sex Roles, pages 509-514. · DOI
- Indira Sen, Mattia Samory, Fabian Floeck, Claudia Wagner, and Isabelle Augenstein. 2021. How does counterfactu- ally augmented data impact models for social computing constructs? arXiv preprint arXiv:2109.07022. · arXiv
- Eric Michael Smith and Adina Williams. 2021. Hi, my name is Martha: Using names to measure and miti- gate bias in generative dialogue models. arXiv preprint arXiv:2109.03300. · arXiv
- Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps, et al. 2019. Re- lease strategies and the social impacts of language models. arXiv preprint arXiv:1908.09203. · arXiv
- Irene Solaiman and Christy Dennison. 2021. Process for adapting language models to society (palms) with values- targeted datasets. Advances in Neural Information Process- ing Systems, 34.
- Harold Somers, Bill Black, Joakim Nivre, Torbjörn Lager, Annarosa Multari, Luca Gilardoni, Jeremy Ellman, and Alex Rogers. 1997. Multilingual generation and summa- rization of job adverts: The tree project. In Proceedings of the Fifth Conference on Applied Natural Language Pro- cessing, ANLC '97, page 269-276, USA. Association for Computational Linguistics. · DOI
- Karolina Stanczak and Isabelle Augenstein. 2021. A survey on gender bias in natural language processing. arXiv preprint arXiv:2112.14168. · arXiv
- Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai- Wei Chang, and William Yang Wang. 2019. Mitigating gender bias in natural language processing: Literature re- view. arXiv preprint arXiv:1906.08976. · arXiv
- Tony Veale. 2016. Round Up The Usual Suspects: Knowledge-Based Metaphor Generation. Proceedings of The Fourth Workshop on Metaphor in NLP, San Diego, USA, pages 34-41. · DOI
- Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. 2018. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 335-340. · DOI · Paper page
Cited by
Sorted by citations
Bias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245
Can Large Language Models Transform Computational Social Science?
Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z., & Yang, D. (2024) · Computational Linguistics · doi:10.1162/coli_a_00502 · Cited by 585
DeAR: Debiasing Vision-Language Models with Additive Residuals
Seth, A., Hemani, M., & Agarwal, C. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00659 · Cited by 34
Metadata
- Authors
- Borchers, C., Gala, D., Gilburt, B., Oravkin, E., Bounsi, W., Asano, Y. M., & Kirk, H.
- Affiliations
- University of Oxford (GB)
- Corresponding author
- Borchers, Conrad
- Venue
- Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP)
- Published
- 2022
- Pages
- 212-224
- OpenAlex
- W4287889445
- Cited by
- 29
- References
- 12
- Citations by year
- 2026: 5 · 2025: 8 · 2024: 8 · 2023: 8
- FWCI
- 5.49
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Workshop on Gender Bias in Natural Language Processing (GeBNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Economic and Social Research Council (ES/P000649/, P000649, ES/P000649/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- job advertisements, GPT-3, bias mitigation, generative language models, bias in language models, prompt engineering, fine-tuning, zero-shot generation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
