CoChat
Sign Up
Cited by 53Ethics and Social Impacts of AIHate Speech and Cyberbullying DetectionTopic Modeling

Are AI systems biased against the poor? A machine learning analysis using Word2Vec and GloVe embeddings

Open access
Curto, Georgina, Jojoa Acosta, Mario Fernando, Comim, Flavio, Garcia-Zapirain, Begoña·2024·AI & SOCIETY
DOI 10.1007/s00146-022-01494-zPMC PMC9243923
Open this paper in CoChat — free accountOpen PDF

Abstract

Among the myriad of technical approaches and abstract guidelines proposed to the topic of AI bias, there has been an urgent call to translate the principle of fairness into the operational AI reality with the involvement of social sciences specialists to analyse the context of specific types of bias, since there is not a generalizable solution. This article offers an interdisciplinary contribution to the topic of AI and societal bias, in particular against the poor, providing a conceptual framework of the issue and a tailor-made model from which meaningful data are obtained using Natural Language Processing word vectors in pretrained Google Word2Vec, Twitter and Wikipedia GloVe word embeddings. The results of the study offer the first set of data that evidences the existence of bias against the poor and suggest that Google Word2vec shows a higher degree of bias when the terms are related to beliefs, whereas bias is higher in Twitter GloVe when the terms express behaviour. This article contributes to the body of work on bias, both from and AI and a social sciences perspective, by providing evidence of a transversal aggravating factor for historical types of discrimination. The evidence of bias against the poor also has important consequences in terms of human development, since it often leads to discrimination, which constitutes an obstacle for the effectiveness of poverty reduction policies.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s00146-022-01494-z

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • GloVe utilizes word co-occurrence data or global statistics to infer semantic relationships between words in the corpus, as opposed to word2vec's dependence on local context windows.

    “The GloVe algorithm utilizes word co-occurrence data or global statistics to infer semantic relationships between words in the corpus, as opposed to word2vec's dependence on local context windows for deriving these associations (Badri et al. 2022; Gan et al. 2022; Curto et al. 2022).”

    From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Curto et al. 2022

    The Materials section describes GloVe as based on a global word co-occurrence matrix and says embeddings capture semantic relationships. It also describes context-based embeddings in terms of words neighbouring a core word, but does not specifically state that Word2Vec depends on local context windows or draw the claimed contrast between the algorithms. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 80 · Cited by 53

Research works printed but not linked: 72

  1. Adamuthe AC Improved text classification using long short-term memory and word embedding technique Int J Hybrid Inf Technol 2020 10.21742/IJHIT.2020.13.1.03 · DOI
  2. Aggarwal N The norms of algorithmic credit scoring SSRN Electron J 2020 10.2139/SSRN.3569083 · DOI
  3. Alesina A Stantcheva S Teso E Intergenerational mobility and preferences for redistribution Am Econ Rev 2018 108 521 554 10.1257/AER.20162015 · DOI
  4. Alessina A Glaeser EL Fighting poverty in the US and Europe 2013 Oxford Oxford University Press
  5. Almeida F, Xexéo G (2019) Word Embeddings: a survey. https://arxiv.org/abs/1901.09069v1. Accessed 16 Jan 2022 · arXiv
  6. Anderson ES What is the point of equality? Ethics 1999 109 287 337 10.1086/233897/0 · DOI
  7. Anshari M Almunawar MN Masri M Hrdy M Financial technology with AI-enabled and ethical challenges Society 2021 58 189 195 10.1007/S12115-021-00592-W · DOI
  8. Applebaum LD The influence of perceived deservingness on policy decisions regarding aid to the poor Polit Psychol 2001 10.1111/0162-895X.00248 · DOI
  9. Arneson RJ Egalitarianism and the undeserving poor J Polit Philos 1997 5 327 350 10.1111/1467-9760.00037 · DOI
  10. Beukeboom CJ Burgers C How stereotypes are shared through language: a review and introduction of the Social Categories and Stereotypes Communication (SCSC) framework Rev Commun Res 2019 7 1 37 10.12840/ISSN.2255-4165.017 · DOI · Paper page
  11. Bolukbasi T, Chang K-W, Saligrama V, et al (2016) Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. arXiv:1607.06520v1 · arXiv
  12. Bourdieu P Distinction: a social critique of the judgement of taste/Pierre Bourdieu; translated by Richard Nice, with a new introduction by Tony Bennett 2010 Routledge Classics
  13. Caliskan A Bryson JJ Narayanan A Semantics derived automatically from language corpora contain human-like biases Science 2017 356 6334 183 186 10.1126/science.aal4230 28408601 · DOI · Paper page
  14. Camacho-Collados J, Pilehvar MT (2020) Embeddings in natural language processing. ACL anthology. In: Proceedings of the 28th international conference on computational linguistics, pp 10–15. 10.18653/V1/2020.COLING-TUTORIALS.2 · DOI
  15. Card D Smith NA On consequentialism and fairness Front Artif Intell 2020 3 34 10.3389/FRAI.2020.00034/BIBTEX 33733152 PMC7861221 · DOI
  16. Chetty R Hendren N Kline P Where is the land of opportunity? The geography of intergenerational mobility in the United States Q J Econ 2014 129 1553 1623 10.1093/QJE/QJU022 · DOI · Paper page
  17. Dwork C, Hardt M, Pitassi T, et al (2011) Fairness Through Awareness. arXiv:1104.393
  18. Ess C Digital media ethics 2020 Wiley
  19. Eubanks V Automating inequality. How high-tech tools profile police and punish the poor 2018 St. Martin’s Press
  20. Everatt D The undeserving poor: poverty and the politics of service delivery in the poorest nodes of South Africa Politikon 2009 35 293 319 10.1080/02589340903017932 · DOI
  21. Fishkin J Bottlenecks Bottlenecks 2014 10.1093/ACPROF:OSO/9780199812141.001.0001 · DOI
  22. Floridi L The onlife manifesto: being human in a hyperconnected era Onlife Manif Being Hum a Hyperconnected Era 2015 10.1007/978-3-319-04093-6 · DOI
  23. Floridi L Translating principles into practices of digital ethics: five risks of being unethical Philos Technol 2019 322 32 185 193 10.1007/S13347-019-00354-X · DOI · Paper page
  24. Freeman JB Ambady N A dynamic interactive theory of person construal Psychol Rev 2011 118 247 279 10.1037/A0022327 21355661 · DOI
  25. Fry H Hello world: being human in the age of algorithms 2018 Penguin
  26. Green B Algorithmic realism: expanding the boundaries of algorithmic thought Proceedings of the 2020 Conference on Fairness Accountability and Transparency 2020 10.1145/3351095 · DOI
  27. Green B, Chen Y (2019) Disparate interactions: An algorithm-in-the-loop analysis of fairness in risk assessments. FAT* 2019 - Proc 2019 Conf Fairness, Accountability, Transpar 90–99. 10.1145/3287560.3287563 · DOI · Paper page
  28. Gutiérrez L Keith B A systematic literature review on word embeddings Adv Intell Syst Comput 2019 865 132 141 10.1007/978-3-030-01171-0_12 · DOI
  29. Habermas J Moral consciousness and communicative action 1990 London Polity Press
  30. Hacker P Teaching fairness to artificial intelligence: existing and novel strategies against algorithmic discrimination under EU law Common Mark Law Rev 2018 55 1143 1185 10.54648/COLA2018095 · DOI · Paper page
  31. Hardt M, Price E, Srebro N (2016) Equality of opportunity in supervised learning. ArXiv ID: 1610.02413v1
  32. Hauge MV Stevenson MD Rossmo DK Le Comber SC Tagging Banksy: using geographic profiling to investigate a modern art mystery J Spacial Sci 2016 61 185 190 10.1080/14498596.2016.1138246 · DOI
  33. Hegel GWF Elements of the philosophy of right 1991 Oxford’s Worlds Classics
  34. Honneth A The struggle for recognition 1996 Polity Press
  35. Imbrie A Kania E Laskai L The question of comparative advantage in artificial intelligence: enduring strengths and emerging challenges for the United States Cent Secur Emerg Technol 2020 10.51593/20190047 · DOI
  36. Joseph K, Morgan JH (2020) When do word embeddings accurately reflect surveys on our beliefs about people? ACL Anthology. In: Proceedings of the 58th annual meeting of the association of computational linguistics, pp 4392–4415. 10.18653/v1/2020.acl-main.405 · DOI
  37. Kroll J Huey J Barocas S Accountable algorithms 2017 University of Pennsylvania Law Review 165
  38. Kusner MJ Loftus JR The long road to fairer algorithms Nature 2020 578 34 36 10.1038/D41586-020-00274-3 32020122 · DOI
  39. Lee MSA Floridi L Algorithmic fairness in mortgage lending: from absolute conditions to relational trade-offs SSRN Electron J 2020 10.2139/SSRN.3559407 · DOI
  40. Maass A Linguistic intergroup bias: stereotype perpetuation through language Adv Exp Soc Psychol 1999 31 79 121 10.1016/S0065-2601(08)60272-5 · DOI
  41. Manzini T, Chong LY, Black AW, Tsvetkov Y (2019) Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddings. NAACL HLT 2019 - 2019 Conf North Am Chapter Assoc Comput Linguist Hum Lang Technol - Proc Conf 1:615–621. 10.18653/V1/N19-1062 · DOI
  42. Merler M, Ratha N, Feris RS, Smith JR (2019) Diversity in faces. arXiv:1901.10436. Accessed 12 June 2022 · arXiv
  43. Mikolov T, Chen K, Corrado G, Dean J (2013a) Efficient estimation of word representations in vector space. 10.48550/arXiv.1301.3781. Accessed 10 Dec 2021 · DOI · Paper page
  44. Mikolov T, Sutskever I, Chen K, et al (2013b) Distributed Representations of Words and Phrases and their Compositionality. arXiv:1310.4546 · arXiv
  45. Morley J Elhalal A Garcia F Ethics as a service: a pragmatic operationalisation of AI ethics Minds Mach 2021 31 239 256 10.1007/S11023-021-09563-W PMC8550007 34720418 · DOI
  46. Mounk Y The age of responsibility: luck, choice and the welfare state 2017 Cambridge University Press
  47. Nunn H Biressi A The undeserving poor Soundings 2009 41 107 116 10.3898/136266209787778920 · DOI
  48. O’Neal C Weapons of math destruction 2016 Penguin Random House
  49. Paolini S White F Tropp L Intergroup contact research in the 21st century. Lessons learned and forward progress if we remain open J Soc Issues 2021 77 11 37 10.1111/josi.12427 · DOI
  50. Pennington J, Socher R, Manning CD (2014) GloVe: global vectors for word representation. In: Empirical Methods in Natural Language Processing (EMNLP). ACL Anthology. In: Proceedings of the 2014 conference on empirical methods in natural language processing, pp 1532–1543
  51. Piketty T Saez E Zucman G Distributional national accounts: methods and estimates for the United States Q J Econ 2018 133 553 609 10.1093/QJE/QJX043 · DOI · Paper page
  52. Reicher S Rethinking the paradigm of prejudice South African J Psychol 2007 37 820 834 10.1177/008124630703700410 · DOI
  53. Ridgeway CL Smith-Lovin L The gender system and interaction Annu Rev Sociol 1999 25 191 216 10.1146/ANNUREV.SOC.25.1.191 · DOI · Paper page
  54. Roberts H Cowls J Hine E Achieving a ‘Good AI Society’: comparing the aims and progress of the EU and the US SSRN Electron J 2021 10.2139/SSRN.3851523 PMC8587491 34767085 · DOI
  55. Rudinger R, Naradowsky J, Leonard B, Durme B Van (2018) Gender Bias in Coreference Resolution. NAACL HLT 2018 - 2018 Conf North Am Chapter Assoc Comput Linguist Hum Lang Technol - Proc Conf 2:8–14. 10.18653/V1/N18-2002 · DOI · Paper page
  56. Sandel MJ The tyranny of merit 2020 Penguin Random House
  57. Sap M, Gabriel S, Qin L et al (2020) Social Bias Frames: Reasoning about Social and Power Implications of Language. Association of Computational Linguistics. In: Proceedings of the 58th Annual Meeting of the Association of Computational Linguistics, pp 5477–5490. 10.18653/V1/2020.ACL-MAIN.486 · DOI
  58. SCMP Research (2020) China AI Report. In: World Sci. https://www.worldscientific.com/page/china-ai-report. Accessed 5 Feb 2022
  59. Sen A Development as freedom 2001 Oxford University Press
  60. Talmor A, Yoran O, Le Bras R, et al (2021) CommonsenseQA 2.0: Exposing the Limits of AI through Gamification. Thirty-fifth Conf Neural Inf Process Syst Datasets Benchmarks Track (Round 1), 2021
  61. Taylor C Multiculturalism and “the politics of recognition” 1931 Princeton University Press
  62. Tsamados A Aggarwal N Cowls J The ethics of algorithms: key problems and solutions AI Soc 2021 1 1 16 10.1007/S00146-021-01154-8 · DOI · Paper page
  63. Vakkuri V, Kemell KK, Jantunen M, Abrahamsson P (2020) “This is Just a Prototype”: How Ethics Are Ignored in Software Startup-Like Environments. Lect Notes Bus Inf Process 383 LNBIP:195–210. 10.1007/978-3-030-49392-9_13 · DOI
  64. Vallès-Peris N Domènech M Caring in the in-between: a proposal to introduce responsible AI and robotics to healthcare AI Soc 2021 10.1007/S00146-021-01330-W · DOI
  65. Vinuesa R Azizpour H Leite I The role of artificial intelligence in achieving the sustainable development goals Nat Commun 2020 111 11 1 10 10.1038/s41467-019-14108-y PMC6957485 31932590 · DOI · Paper page
  66. Watson DS Krutzinna J Bruce IN Clinical applications of machine learning algorithms: beyond the black box BMJ 2019 10.1136/BMJ.L886 30862612 · DOI · Paper page
  67. Xu K Nosek B Greenwald AG Data from the race implicit association test on the project implicit demo website J Open Psychol Data 2014 2 e3 10.5334/JOPD.AC · DOI
  68. Yapa L How the discipline of geography exacerbates poverty in the Third World Futures 2002 34 33 46 10.1016/S0016-3287(01)00033-7 · DOI
  69. Young M The rise of the meritocracy (classics in organization and management series) 1964 Routledge
  70. Zajko M Conservative AI and social inequality: conceptualizing alternatives to bias through social theory AI Soc 2021 36 1047 1056 10.1007/S00146-021-01153-9 · DOI
  71. Zetterholm MV Lin Y Jokela P Digital contact tracing applications during COVID-19: a scoping review about public acceptance Informatics 2021 8 48 10.3390/INFORMATICS8030048 · DOI
  72. Zhao J Khashabi D Khot T Ethical-advice taker: do language models understand natural language interventions arXiv 2021 10.18653/v1/2021.findings-acl.364 · DOI

Authors
Curto, G., Jojoa Acosta, M. F., Comim, F., & Garcia-Zapirain, B.
Affiliations
Universitat Autònoma de Barcelona (ES)
Corresponding author
Curto, Georgina
ORCID
Venue
AI & SOCIETY
Publisher
Springer Science and Business Media LLC
Published
2022-06-28
Volume
39
Issue
2
Pages
617-632
ISSN
  • 0951-5666 (ISSN-L)
  • 1435-5655
DOI
10.1007/s00146-022-01494-z
PMC
PMC9243923
OpenAlex
W4283654020
Cited by
53
References
80
Citations by year
2026: 9 · 2025: 16 · 2024: 13 · 2023: 14 · 2022: 1
FWCI
12.76
Type
journal article
Language
en
Open access
Licence
Locations
  • AI & Society (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DeustoTeka (submittedVersion, cc-by-nc-nd)
  • Dipòsit Digital de Documents de la UAB (Universitat Autònoma de Barcelona) (submittedVersion, cc-by)
  • Figshare (submittedVersion)
  • University of Notre Dame
Funders
Universitat Ramon Llull
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
algorithmic bias, poverty, word embeddings, Word2Vec, gloves, natural language processing, discrimination, algorithmic fairness, poverty reduction, social bias, human development

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review