Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 207

The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.

Ethics-Based Auditing of Automated Decision-Making Systems: Nature, Scope, and Limitations

Open access
Mökander, Jakob, Morley, Jessica, Taddeo, Mariarosaria, Floridi, Luciano · 2021 · Science and Engineering Ethics

Abstract

Important decisions that impact humans lives, livelihoods, and the natural environment are increasingly being automated. Delegating tasks to so-called automated decision-making systems (ADMS) can improve efficiency and enable new solutions. However, these benefits are coupled with ethical challenges. For example, ADMS may produce discriminatory outcomes, violate individual privacy, and undermine human self-determination. New governance mechanisms are thus needed that help organisations design and deploy ADMS in ways that are ethical, while enabling society to reap the full economic and social benefits of automation. In this article, we consider the feasibility and efficacy of ethics-based auditing (EBA) as a governance mechanism that allows organisations to validate claims made about their ADMS. Building on previous work, we define EBA as a structured process whereby an entity’s present or past behaviour is assessed for consistency with relevant principles or norms. We then offer three contributions to the existing literature. First, we provide a theoretical explanation of how EBA can contribute to good governance by promoting procedural regularity and transparency. Second, we propose seven criteria for how to design and implement EBA procedures successfully. Third, we identify and discuss the conceptual, technical, social, economic, organisational, and institutional constraints associated with EBA. We conclude that EBA should be considered an integral component of multifaced approaches to managing the ethical risks posed by ADMS.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s11948-021-00319-4

Cites

57 of 57 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 57 references

Research works printed but not linked: 60

  1. Aggarwal, N., Eidenmüller, H., Enriques, L., Payne, J., & Zwieten, K. (2019). Autonomous systems and the law. München: Baden-Baden.
  2. AIEIG. 2020. From principles to practice — An interdisciplinary framework to operationalise AI ethics. AI Ethics Impact Group, VDE Association for Electrical Electronic & Information Technologies e.V., Bertelsmann Stiftung, 1–56. https://doi.org/10.11586/2020013. · DOI
  3. Aizenberg, E., & van den Hoven, J. (2020). Designing for human rights in AI. Big Data and Society. https://doi.org/10.1177/2053951720949566 · DOI · Paper page
  4. Auer, F., & Felderer, M. (2018). Shifting quality assurance of machine learning algorithms to live systems. In M. Tichy, E. Bodden, M. Kuhrmann, S. Wagner, & J.-P. Steghöfer (Eds.), Software Engineering und Software Management 2018 (S. 211–212). Bonn: Gesellschaft für Informatik.
  5. Binns, R. (2018). What can political philosophy teach us about algorithmic fairness? IEEE Security & Privacy, 16(3), 73–80. · DOI
  6. Boddington, P., Millican, P., & Wooldridge, M. (2017). Minds and machines special issue: Ethics and artificial intelligence. Minds and Machines, 27(4), 569–574. https://doi.org/10.1007/s11023-017-9449-y · doi:10.1007/s11023-017-9449-y
  7. Brundage, M., Avin, S., Wang, J., Belfield, H., Krueger, G., Hadfield, G., Khlaaf, H., et al. 2020. Toward trustworthy AI development: Mechanisms for supporting verifiable claims. ArXiv, no. 2004.07213[cs.CY]. http://arxiv.org/abs/2004.07213. · arXiv
  8. Bryson, J., & Winfield, A. (2017). Standardizing ethical design for artificial intelligence and autonomous systems. Computer, 50(5), 116–19. · DOI · Paper page
  9. Cabrera, Á. A., Epperson, W., Hohman, F., Kahng, M., Morgenstern, J., Chau, D. H. 2019. FairVis: Visual analytics for discovering intersectional bias in machine learning. http://arxiv.org/abs/1904.05419. · DOI · Paper page
  10. Chopra, A. K., Singh, M. P. 2018. Sociotechnical systems and ethics in the large. In AIES 2018—Proceedings of the 2018 AAAI/ACM conference on AI, ethics, and society (pp. 48–53). https://doi.org/10.1145/3278721.3278740. · DOI
  11. Christian, B. (2020). The alignment problem: Machine learning and human values. W.W. Norton & Company Ltd.
  12. Citron, D. K., & Pasquale, F. (2014). The scored society: Due process for automated predictions. HeinOnline, 1, 1–34.
  13. Conrad, C. A. (2018). Business ethics—A philosophical and behavioral approach. Springer. https://doi.org/10.1007/978-3-319-91575-3 · DOI
  14. Dafoe, A. (2017). AI governance: A research agenda. American Journal of Psychiatry. https://doi.org/10.1176/ajp.134.8.aj1348938 · DOI
  15. D’Agostino, M., & Durante, M. (2018). Introduction: The governance of algorithms. Philosophy and Technology, 31(4), 499–505. https://doi.org/10.1007/s13347-018-0337-z · DOI
  16. Deloitte. 2020. Deloitte introduces trustworthy AI framework to guide organizations in ethical application of technology. Press Release. 2020. https://www2.deloitte.com/us/en/pages/about-deloitte/articles/press-releases/deloitte-introduces-trustworthy-ai-framework.html.
  17. Di Maio, P. (2014). Towards a metamodel to support the joint optimization of socio technical systems. Systems, 2(3), 273–296. https://doi.org/10.3390/systems2030273 · DOI
  18. Diakopoulos, N. (2015). Algorithmic accountability: Journalistic investigation of computational power structures. Digital Journalism, 3(3), 398–415. https://doi.org/10.1080/21670811.2014.976411 · DOI · Paper page
  19. Ellemers, N., van der Toorn, J., Paunov, Y., & van Leeuwen, T. (2019). The psychology of morality: A review and analysis of empirical studies published From 1940 Through 2017. Personality and Social Psychology Review, 23(4), 332–366. https://doi.org/10.1177/1088868318811759 · DOI · Paper page
  20. Epstein, Z., Payne, B. H., Shen, J. H., Hong, C. J., Felbo, B., Dubey, A., Groh, M., Obradovich, N., Cebrian, M., Rahwan, I. 2018. Turingbox: An experimental platform for the evaluation of AI systems. In IJCAI international joint conference on artificial intelligence 2018-July (pp. 5826–28). https://doi.org/10.24963/ijcai.2018/851. · DOI
  21. Evans, K., de Moura, N., Chauvier, S., Chatila, R., & Dogan, E. (2020). Ethical decision making in autonomous vehicles: The AV ethics project. Science and Engineering Ethics, 26(6), 3285–3312. https://doi.org/10.1007/s11948-020-00272-8 · DOI · Paper page
  22. Fagerholm, F., Guinea, A. S., Mäenpää, H., Münch, J. 2014. Building blocks for continuous experimentation. In Proceedings of the 1st international workshop on rapid continuous software engineering (pp. 26–35). RCoSE 2014. ACM. https://doi.org/10.1145/2593812.2593816. · DOI
  23. Falkenberg, L., & Herremans, I. (1995). Ethical behaviours in organizations: Directed by the formal or informal systems? Journal of Business Ethics, 14(2), 133–143. https://doi.org/10.1007/BF00872018 · DOI · Paper page
  24. Floridi, L. (2016b). Tolerant paternalism: Pro-ethical design as a resolution of the dilemma of toleration. Science and Engineering Ethics, 22(6), 1669–1688. https://doi.org/10.1007/s11948-015-9733-2. · DOI
  25. Friedler, S. A., Scheidegger, C., Venkatasubramanian, Suresh. 2016. On the (im)possibility of fairness, no. im: 1–16. http://arxiv.org/abs/1609.07236. · arXiv
  26. Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, Hal., Crawford, K. 2018. Datasheets for datasets. http://arxiv.org/abs/1803.09010. · arXiv
  27. Goodman, B. 2016. A step towards accountable algorithms? : Algorithmic discrimination and the European Union general data protection. In:29th conference on neural information processing systems (NIPS 2016), Barcelona, Spain., no. Nips (pp. 1–7).
  28. ICO. 2018. Guide to the general data protection regulation (GDPR). Guide to the general data protection regulation, n/a. https://doi.org/10.1111/j.1751-1097.1994.tb09662.x. · DOI
  29. Jotterand, F., & Bosco, C. (2020). Keeping the ‘human in the loop’ in the age of artificial intelligence: Accompanying commentary for ‘correcting the brain?’ By Rainey and Erden. Science and Engineering Ethics, 26(5), 2455–2460. https://doi.org/10.1007/s11948-020-00241-1 · DOI
  30. Karanasiou, A. P., & Pinotsis, D. A. (2017). A Study into the layers of automated decision-making: emergent normative and legal aspects of deep learning. International Review of Law, Computers & Technology, 31(2), 170–187. https://doi.org/10.1080/13600869.2017.1298499 · DOI
  31. Kazim, E., Denny, D. M. T., & Koshiyama, A. (2021). AI auditing and impact assessment: According to the UK information commissioner’s office. AI and Ethics, no. 0123456789. https://doi.org/10.1007/s43681-021-00039-2. · DOI
  32. Keyes, O., Hutson, J., Durbin, M. 2019. A mulching proposal no. May 2019 (pp. 1–11). https://doi.org/10.1145/3290607.3310433. · DOI
  33. Kim, P. 2017. Auditing algorithms for discrimination. University of Pennsylvania Law Review, 166, 189–203.
  34. Koene, A., Clifton, C., Hatada, Y., Webb, H., Richardson, R. 2019. A governance framework for algorithmic accountability and transparency. https://doi.org/10.2861/59990. · DOI
  35. Kolhar, M., Abu-Alhaj, M. M., & El-Atty, S. M. A. (2017). Cloud data auditing techniques with a focus on privacy and security. IEEE Security and Privacy, 15(1), 42–51. https://doi.org/10.1109/MSP.2017.16 · DOI
  36. Krafft, T. D., Zweig, K. A., & König, P. D. (2020). How to regulate algorithmic decision-making: A framework of regulatory requirements for different applications. Regulation and Governance. https://doi.org/10.1111/rego.12369 · DOI · Paper page
  37. Kroll, J. A., Huey, J., Barocas, S., Felten, E. W., Reidenberg, J. R., Robinson, D. G., Yu, H. 2016. Accountable algorithms. University of Pennsylvania Law Review, no. 633: 66. https://doi.org/10.1002/ejoc.201200111. · DOI · Paper page
  38. LaBrie, R. C., Steinke, G. H. 2019. Towards a framework for ethical audits of AI algorithms. In 25th americas conference on information systems, AMCIS 2019 (pp 1–5).
  39. Lauer, D. (2020). You cannot have AI ethics without ethics. AI and Ethics, 0123456789, 1–5. https://doi.org/10.1007/s43681-020-00013-4 · DOI
  40. Lee, M., Floridi, L., & Denev, A. (2020). Innovating with confidence: Embedding governance and fairness in a financial services risk management framework. Berkeley Technology Law Journal, 34(2), 1–19.
  41. Mahajan, V., Venugopal, V. K., Murugavel, M., & Mahajan, H. (2020). The algorithmic audit: Working with vendors to validate radiology-AI algorithms—How we do it. Academic Radiology, 27(1), 132–135. https://doi.org/10.1016/j.acra.2019.09.009. · DOI
  42. Mau, S., & Howe, S. (2019). The metric society: On the quantification of the social. Ebook Central.
  43. Microsoft. 2020. Fairlearn: A toolkit for assessing and improving fairness in AI (pp. 1–6).
  44. Mittelstadt, B. (2016). Auditing for transparency in content personalization systems. International Journal of Communication, 10(June), 4991–5002.
  45. PDPC. 2020. Model artificial intelligence governance framework second edition. Personal data protection commission of Singapore.
  46. Rahwan, I. (2018). Society-in-the-loop: Programming the algorithmic social contract. Ethics and Information Technology, 20(1), 5–14. https://doi.org/10.1007/s10676-017-9430-8 · DOI · Paper page
  47. Raji, I. D., & Buolamwini, J. (2019). Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial AI products. In AIES 2019—Proceedings of the 2019 AAAI/ACM conference on AI, ethics, and society (pp. 429–435). https://doi.org/10.1145/3306618.3314244. · DOI · Paper page
  48. Sandvig, C., Hamilton, K., Karahalios, K., Langbort, C. 2014. Auditing algorithms. In ICA 2014 data and discrimination preconference. (pp. 1–23). https://doi.org/10.1109/DEXA.2009.55. · DOI
  49. Schulam, P., Saria, S. 2019. Can you trust this prediction? Auditing pointwise reliability after learning 89. http://arxiv.org/abs/1901.00403. · arXiv
  50. Springer, A., Whittaker, S. 2019. Making transparency clear.
  51. Steghöfer, J. P., Knauss, E., Horkoff, J., Wohlrab, R. 2019. Challenges of scaled agile for safety-critical systems. Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) 11915 LNCS (pp. 350–66). https://doi.org/10.1007/978-3-030-35333-9_26. · DOI
  52. Strenge, B., & Schack, T. (2020). AWOSE—A process model for incorporating ethical analyses in agile systems engineering. Science and Engineering Ethics, 26(2), 851–870. https://doi.org/10.1007/s11948-019-00133-z · DOI
  53. Tasioulas, J. (2018). First steps towards an ethics of robots and artificial intelligence. SSRN Electronic Journal, 7(1), 61–95. https://doi.org/10.2139/ssrn.3172840 · DOI
  54. Thaler, R., & Sunstein, C. (2008). Nudge: Improving decisions about health, wealth, and happiness. New Haven, Conn.: Yale University Press.
  55. Tsamados, A., Aggarwal, N., Cowls, J., Morley, J., Roberts, H., Taddeo, M., et al. (2020). The ethics of algorithms: Key problems and solutions. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3662302. · DOI
  56. Turner Lee, N. (2018). Detecting racial bias in algorithms and machine learning. Journal of Information, Communication and Ethics in Society, 16(3), 252–260. https://doi.org/10.1108/JICES-06-2018-0056 · DOI · Paper page
  57. Tutt, A. (2017). An FDA for algorithms. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.2747994 · DOI
  58. Vakkuri, V., Kemell, K. K., Kultanen, J., Siponen, M., Abrahamsson, P. 2019. Ethically aligned design of autonomous systems: Industry viewpoint and an empirical study. ArXiv.
  59. Wachter, S., Mittelstadt, B., Russell, C. 2017. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. · DOI · Paper page
  60. Wiener, N. (1988). The human use of human beings: Cybernetics and society. Da Capo Series in Science.

Cited by

See all 207

Metadata

Authors
Mökander, J., Morley, J., Taddeo, M., & Floridi, L.
Venue
Science and Engineering Ethics
Published
2021-07-06
Volume
27
Issue
4
Pages
44-44
ISSN
  • 1353-3452 (ISSN-L)
  • 1471-5546
OpenAlex
W3178786669
Cited by
207
References
57
FWCI
34.60
Type
journal article
Language
en
Open access
hybrid
Licence
Funders
  • The Welcome Trust
  • The Society of Swedish Engineers in Great Britain (2021)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Ethics-Based Auditing of Automated Decision-Making Systems: Nature, Scope, and Limitations. https://app.cochat.ai/papers/10.1007/s11948-021-00319-4
The paper
Mökander, J., Morley, J., Taddeo, M., & Floridi, L. (2021). Ethics-Based Auditing of Automated Decision-Making Systems: Nature, Scope, and Limitations. Science and Engineering Ethics. https://doi.org/10.1007/s11948-021-00319-4

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.