CoChat

Challenges of responsible AI in practice: scoping review and recommended actions

Open access
Sadek, Malak, Kallina, Emma, Bohné, Thomas, Mougenot, Céline, Calvo, Rafael A., Cave, Stephen · 2025 · AI & SOCIETY
Cited by 79

The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.

Abstract

Responsible AI (RAI) guidelines aim to ensure that AI systems respect democratic values. While a step in the right direction, they currently fail to impact practice. Our work discusses reasons for this lack of impact and clusters them into five areas: (1) the abstract nature of RAI guidelines, (2) the problem of selecting and reconciling values, (3) the difficulty of operationalising RAI success metrics, (4) the fragmentation of the AI pipeline, and (5) the lack of internal advocacy and accountability. Afterwards, we introduce a number of approaches to RAI from a range of disciplines, exploring their potential as solutions to the identified challenges. We anchor these solutions in practice through concrete examples, bridging the gap between the theoretical considerations of RAI and on-the-ground processes that currently shape how AI systems are built. Our work considers the socio-technical nature of RAI limitations and the resulting necessity of producing socio-technical solutions.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s00146-024-01880-9

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 57

  1. Abuhusain M (2020) The role of artificial intelligence and big data on loan decisions. Accounting 6(7):1291–1296 · doi:10.5267/j.ac.2020.8.022
  2. Barker M, Kallina E, Ashok D, Collins K, Casovan A, Weller A, Talwalkar A, Chen V, Bhatt U (2023) Feedbacklogs: Recording and incorporating stakeholder feedback into machine learning pipelines. In: Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, pp 1–15 · DOI
  3. Chazette L, Schneider K (2020) Explainability as a non-functional requirement: challenges and recommendations. Requir Eng 25(4):493–514 · DOI · Paper page
  4. Chen J, Storchan V, Kurshan E (2021) Beyond fairness metrics: roadblocks and challenges for ethical ai in practice. arXiv preprint arXiv:2108.06217 · arXiv
  5. Chouldechova A, Benavides-Prado D, Fialko O, Vaithianathan R (2018) A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions. In: Conference on Fairness, Accountability and Transparency, pp 134–148
  6. Christian B (2020) The alignment problem - machine learning and human values. W. W. Norton & Company
  7. Delgado F, Yang S, Madaio M, Yang Q (2021) Stakeholder participation in ai: Beyond" add diverse stakeholders and stir". arXiv preprint arXiv:2111.01122 · arXiv
  8. Delgado F, Yang S, Madaio M, Yang Q (2023) The participatory turn in ai design: theoretical foundations and the current state of practice. In: Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, pp 1–23 · DOI · Paper page
  9. Edwards L, McAuley D, Diver L (2016) From privacy impact assessment to social impact assessment. In: Proceedings of the IEEE Computer Society Conference · DOI
  10. Elshan E, Zierau N, Engel C, Janson A, Leimeister JM (2022) Understanding the design elements affecting user acceptance of intelligent agents: past, present and future. Inf Syst Front 24(3):699–730 · DOI · Paper page
  11. Friedman B, Hendry D (2019) Value sensitive design: shaping technology with moral imagination. MIT Press · DOI · Paper page
  12. Fuchsberger V, Moser C, Tscheligi M (2012) Values in action (via): Combining usability, user experience and user acceptance. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI) · DOI
  13. Gaggioli A, Riva G, Peters D, Calvo RA (2017) Positive technology, computing, and design: shaping a future in which technology promotes psychological well-being. Emotions and affect in human factors and human-computer interaction. Elsevier, pp 477–502 · DOI
  14. Garst J, Blok V, Jansen L, Omta O (2022) From value sensitive design to values absorption - building an instrument to analyze organizational capabilities for value-sensitive innovation. J Responsible Innov 9(2):196–223 · DOI
  15. Gupta A, Wright C, Ganapini MB, Sweidan M, Butalid R (2022) State of AI ethics report (volume 6, february 2022). arXiv preprint arXiv:2202.07435 · arXiv
  16. Gutierrez CI, Marchant GE (2021) A global perspective of soft law programs for the governance of artificial intelligence. Available at SSRN 3855171 · DOI
  17. Harbers M, Overdiek A (2022) Towards a living lab for responsible applied ai. In: Proceedings of the DRS 2022. Retrieved from https://doi.org/10.21606/drs.2022.422 · DOI
  18. Hemment D, Aylett R, Belle V, Murray-Rust D, Luger E, Hillston J, Rovatsos M, Broz F (2019) Experiential AI. Computing Research Repository (CoRR). Retrieved from arXiv:1908.02619 · DOI
  19. Ho CW, Ali J, Caals K (2020) Ensuring trustworthy use of artificial intelligence and big data analytics in health insurance. Bull World Health Org 98(4):263 · DOI
  20. Holden J (2018) Improving nursing student empathy with experiential learning. In: Proceedings of the Nursing Education Research Conference
  21. Ibáñez JC, Olmeda MV (2021) Operationalising ai ethics: how are companies bridging the gap between practice and principles? an exploratory study. AI Soc 37(4):1663–87 · DOI
  22. IEEE standard model process for addressing ethical concerns during system design (Standard) (2021) The Institute of Electrical and Electronics Engineers
  23. Jacobs AZ (2021) Measurement as governance in and for responsible AI. arXiv preprint arXiv:2109.05658 · arXiv
  24. John-Mathews J-M, Cardon D, Balagué C (2022) From reality to world. A critical perspective on ai fairness. J Bus Ethics 178(4):945–59 · DOI · Paper page
  25. Keyes O, Hutson J, Durbin M (2019) A mulching proposal: Analysing and improving an algorithmic system for turning the elderly into high-nutrient slurry. In: Extended abstracts of the 2019 CHI conference on human factors in computing systems, pp 1–11 · DOI · Paper page
  26. Koesten L, Gregory K, Groth P, Simperl E (2021) Talking datasets-understanding data sensemaking behaviours. Int J Hum-Comput Stud 146:102562 · DOI
  27. Krijger J (2021) Enter the metrics: critical theory and organizational operationalization of ai ethics. AI Soc 37(4):1427–37 · DOI
  28. Lee MK, Kusbit D, Kahng A, Kim JT, Yuan X, Chan A, See D, Noothigattu R, Lee S, Psomas A, et al (2019) Webuildai: Participatory framework for algorithmic governance. In: Proceedings of the ACM on Human-Computer Interaction 3(CSCW):1–35 · DOI · Paper page
  29. Lohr S (2018) Facial recognition is accurate, if you’re a white guy. Ethics data and analytics. Auerbach Publications, NY, pp 143–147
  30. Manders-Huits N, Zimmer M (2009) Values and pragmatic action: the challenges of introducing ethical intelligence in technical design communities. Int Rev Inf Ethics 10:37–44
  31. McCradden M, Mazwi M, Joshi S, Anderson JA (2020) When your only tool is a hammer: ethical limitations of algorithmic fairness solutions in healthcare machine learning. In: Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, pp 109–109 · DOI
  32. Modha S, Majumder P, Mandl T, Mandalia C (2020) Detecting and visualizing hate speech in social media: a cyber watchdog for surveillance. Expert Syst Appl 161:113725 · DOI · Paper page
  33. Morley J, Elhalal A, Garcia F, Kinsey L, Mökander J, Floridi L (2021) Ethics as a service: a pragmatic operationalisation of ai ethics. Minds Mach 31(2):239–256 · DOI · Paper page
  34. Moss E, Watkins E, Metcalf J, Singh R, Elish M (2021) Governing with algorithmic impact assessments: six observations. In: Proceedings of the ACM Conference on Artificial Intelligence, Ethics and Society (AIES) · DOI
  35. Mujtaba DF, Mahapatra NR (2019) Ethical considerations in ai-based recruitment. In: 2019 IEEE International Symposium on Technology and Society (ISTAS), pp 1–7 · DOI · Paper page
  36. Munn L (2023) The uselessness of AI ethics. AI Ethics 3(3):869–877 · DOI · Paper page
  37. Narayanan A (2018) Translation tutorial: 21 fairness definitions and their politics. In: Proc. conf. fairness accountability transp., New York, USA, vol. 1170,p 3
  38. Office ATP (2022) Statement on prinicples for responsible algorithmic systems. https://www.acm.org/binaries/content/assets/public-policy/final-joint-ai-statement-update.pdf. Accessed 23 Aug 2023
  39. Ozkaramanli D, Desmet P, Özcan E (2016) Beyond resolving dilemmas: three design directions for addressing intrapersonal concern conflicts. Des Issues 32:78–91. https://doi.org/10.1162/DESI_a_00401 · DOI
  40. Pasquinelli M, Joler V (2021) The nooscope manifested: AI as instrument of knowledge extractivism. AI Soc 36:1263–1280 · DOI · Paper page
  41. Rakova B, Yang J, Cramer H, Chowdhury R (2021) Where responsible ai meets reality: Practitioner perspectives on enablers for shifting organizational practices. In: Proceedings of the ACM on Human-Computer Interaction 5(CSCW1):1–23 · DOI · Paper page
  42. Sadek M, Calvo R, Mougenot C (2023) Co-designing conversational agents: a comprehensive review and recommendations for best practices. Des Stud 89:101230 · DOI
  43. Sadek M, Calvo R, Mougenot C (2023) Designing value-sensitive ai: a critical review and recommendations for socio-technical design processes. AI Ethics. https://doi.org/10.1007/s43681-023-00373-7 · DOI
  44. Sadek M, Calvo R, Mougenot C (2023c) Why codesigning ai is different and difficult. ACM Interactions. Retrieved from https://interactions.acm.org/blog/view/why-codesigning-ai-is-differentand-difficult
  45. Scantamburlo T, Cortés A, Schacht M (2020) Progressing towards responsible ai. arXiv preprint arXiv:2008.07326 · arXiv
  46. Schiff D, Rakova B, Ayesh A, Fanti A, Lennon M (2020) Principles to practices for responsible AI: closing the gap. arXiv preprint arXiv:2006.04707 · arXiv
  47. Spiekermann S (2021) From value-lists to value-based engineering with IEEE 7000$$^{{\rm TM}}$$. In: IEEE International Symposium on Technology and Society (ISTAS) · DOI
  48. Spiekermann S, Winkler T (2020) Value-based engineering for ethics by design. Computing Research Repository (CoRR) · DOI
  49. Stray J (2020) Aligning ai optimization to community well-being. Int J Commun Well-Being 3(4):443–463 · DOI
  50. Stray J, Vendrov I, Nixon J, Adler S, Hadfield-Menell D (2021) What are you optimizing for? aligning recommender systems with human values. arXiv preprint arXiv:2107.10939 · arXiv
  51. Tasioulas J (2022) Artificial intelligence, humanistic ethics. Daedalus 151(2):232–243. https://doi.org/10.1162/daed_a_01912 · DOI · Paper page
  52. Thomas R, Uminsky D (2020) The problem with metrics is a fundamental problem for ai. arXiv preprint arXiv:2002.08512 · arXiv
  53. Thomas RL, Uminsky D (2022) Reliance on metrics is a fundamental challenge for ai. Patterns 3(5):100476 · DOI
  54. Varanasi R, Goyal N (2023) It is currently hodgepodge”: Examining ai/ml practitioners’ challenges during co-production of responsible ai values. In: Proceedings of the CHI Conference on Human Factors in Computing Systems · DOI · Paper page
  55. Widder DG, Nafus D, Dabbish L, Herbsleb J (2022) Limits and possibilities for “ethical ai” in open source: A study of deepfakes. In: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22, New York, NY, USA. Association for Computing Machinery. pp 2035–2046. Retrieved from https://doi.org/10.1145/3531146.3533779 · DOI · Paper page
  56. Yildirim N, Pushkarna M, Goyal N, Wattenberg M, Viégas F (2023) Investigating how practitioners use human-ai guidelines: a case study on the people + ai guidebook. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI ’23, New York, NY, USA. Association for Computing Machinery. Retrieved from https://doi.org/10.1145/3544548.3580900 · DOI · Paper page
  57. Young M, Magassa L, Friedman B (2019) Toward inclusive tech policy design: a method for underrepresented voices to strengthen tech policy documents. Ethics Inf Technol 12:89–103 · DOI · Paper page

Cited by

See all 79

Metadata

Authors
Sadek, M., Kallina, E., Bohné, T., Mougenot, C., Calvo, R. A., & Cave, S.
Venue
AI & SOCIETY
Published
2024-02-19
Volume
40
Issue
1
Pages
199-215
ISSN
  • 0951-5666 (ISSN-L)
  • 1435-5655
OpenAlex
W4391924277
Cited by
79
References
33
FWCI
32.61
Type
journal article
Language
en
Open access
hybrid
Licence
Funders
Leverhulme Trust (RC-2015-067)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.