The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Ethics-Based Auditing of Automated Decision-Making Systems: Nature, Scope, and Limitations
Open accessAbstract
Important decisions that impact humans lives, livelihoods, and the natural environment are increasingly being automated. Delegating tasks to so-called automated decision-making systems (ADMS) can improve efficiency and enable new solutions. However, these benefits are coupled with ethical challenges. For example, ADMS may produce discriminatory outcomes, violate individual privacy, and undermine human self-determination. New governance mechanisms are thus needed that help organisations design and deploy ADMS in ways that are ethical, while enabling society to reap the full economic and social benefits of automation. In this article, we consider the feasibility and efficacy of ethics-based auditing (EBA) as a governance mechanism that allows organisations to validate claims made about their ADMS. Building on previous work, we define EBA as a structured process whereby an entity’s present or past behaviour is assessed for consistency with relevant principles or norms. We then offer three contributions to the existing literature. First, we provide a theoretical explanation of how EBA can contribute to good governance by promoting procedural regularity and transparency. Second, we propose seven criteria for how to design and implement EBA procedures successfully. Third, we identify and discuss the conceptual, technical, social, economic, organisational, and institutional constraints associated with EBA. We conclude that EBA should be considered an integral component of multifaced approaches to managing the ethical risks posed by ADMS.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220
Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R., Pagallo, U., Rossi, F., et al. (2018) · Minds and Machines · doi:10.1007/s11023-018-9482-5 · Cited by 4,774
Self-citationHow the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 1×
“While it should be noted that opacity can also be a result of intentional corporate or state secrecy (Burrell 2016), our main concern here relates to inherent technical complexity.”
BackgroundModel Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 1×
“This pragmatic insight implies that audits need to monitor and evaluate system outputs continuously , i.e. through ‘oversight programs’ (Etzioni and Etzioni 2016), and document performance characteristics in a comprehensible way (Mitchell et al. 2019).”
BackgroundRaji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245 · cited 4×
“Such checklists are simple tools that help designers get a more informed view of edge cases and system failures (Raji et al. 2020).”
BackgroundInfluentialThe Ethics of AI Ethics: An Evaluation of Guidelines
Hagendorff, T. (2020) · Minds and Machines · doi:10.1007/s11023-020-09517-8 · Cited by 1,911 · cited 6×
“EBA can, for example, take place within one of the ethical frameworks already mentioned, especially the Ethics Guidelines for Trustworthy AI (AI HLEG 2019) for countries belonging to the European Union and the Recommendation of the Council on Artificial Intelligence (OECD 2019) for countries that officially adopted th…”
BackgroundInfluentialThe Ethics of AI Ethics -- An Evaluation of Guidelines
Hagendorff, T. (2019) · arXiv · doi:10.48550/arxiv.1903.03425 · Cited by 1,907 · cited 6×
“EBA can, for example, take place within one of the ethical frameworks already mentioned, especially the Ethics Guidelines for Trustworthy AI (AI HLEG 2019) for countries belonging to the European Union and the Recommendation of the Council on Artificial Intelligence (OECD 2019) for countries that officially adopted th…”
BackgroundInfluentialAnanny, M., & Crawford, K. (2018) · New Media & Society · doi:10.1177/1461444816676645 · Cited by 1,836
Research works printed but not linked: 60
- Aggarwal, N., Eidenmüller, H., Enriques, L., Payne, J., & Zwieten, K. (2019). Autonomous systems and the law. München: Baden-Baden.
- AIEIG. 2020. From principles to practice — An interdisciplinary framework to operationalise AI ethics. AI Ethics Impact Group, VDE Association for Electrical Electronic & Information Technologies e.V., Bertelsmann Stiftung, 1–56. https://doi.org/10.11586/2020013. · DOI
- Aizenberg, E., & van den Hoven, J. (2020). Designing for human rights in AI. Big Data and Society. https://doi.org/10.1177/2053951720949566 · DOI · Paper page
- Auer, F., & Felderer, M. (2018). Shifting quality assurance of machine learning algorithms to live systems. In M. Tichy, E. Bodden, M. Kuhrmann, S. Wagner, & J.-P. Steghöfer (Eds.), Software Engineering und Software Management 2018 (S. 211–212). Bonn: Gesellschaft für Informatik.
- Binns, R. (2018). What can political philosophy teach us about algorithmic fairness? IEEE Security & Privacy, 16(3), 73–80. · DOI
- Boddington, P., Millican, P., & Wooldridge, M. (2017). Minds and machines special issue: Ethics and artificial intelligence. Minds and Machines, 27(4), 569–574. https://doi.org/10.1007/s11023-017-9449-y · doi:10.1007/s11023-017-9449-y
- Brundage, M., Avin, S., Wang, J., Belfield, H., Krueger, G., Hadfield, G., Khlaaf, H., et al. 2020. Toward trustworthy AI development: Mechanisms for supporting verifiable claims. ArXiv, no. 2004.07213[cs.CY]. http://arxiv.org/abs/2004.07213. · arXiv
- Bryson, J., & Winfield, A. (2017). Standardizing ethical design for artificial intelligence and autonomous systems. Computer, 50(5), 116–19. · DOI · Paper page
- Cabrera, Á. A., Epperson, W., Hohman, F., Kahng, M., Morgenstern, J., Chau, D. H. 2019. FairVis: Visual analytics for discovering intersectional bias in machine learning. http://arxiv.org/abs/1904.05419. · DOI · Paper page
- Chopra, A. K., Singh, M. P. 2018. Sociotechnical systems and ethics in the large. In AIES 2018—Proceedings of the 2018 AAAI/ACM conference on AI, ethics, and society (pp. 48–53). https://doi.org/10.1145/3278721.3278740. · DOI
- Christian, B. (2020). The alignment problem: Machine learning and human values. W.W. Norton & Company Ltd.
- Citron, D. K., & Pasquale, F. (2014). The scored society: Due process for automated predictions. HeinOnline, 1, 1–34.
- Conrad, C. A. (2018). Business ethics—A philosophical and behavioral approach. Springer. https://doi.org/10.1007/978-3-319-91575-3 · DOI
- Dafoe, A. (2017). AI governance: A research agenda. American Journal of Psychiatry. https://doi.org/10.1176/ajp.134.8.aj1348938 · DOI
- D’Agostino, M., & Durante, M. (2018). Introduction: The governance of algorithms. Philosophy and Technology, 31(4), 499–505. https://doi.org/10.1007/s13347-018-0337-z · DOI
- Deloitte. 2020. Deloitte introduces trustworthy AI framework to guide organizations in ethical application of technology. Press Release. 2020. https://www2.deloitte.com/us/en/pages/about-deloitte/articles/press-releases/deloitte-introduces-trustworthy-ai-framework.html.
- Di Maio, P. (2014). Towards a metamodel to support the joint optimization of socio technical systems. Systems, 2(3), 273–296. https://doi.org/10.3390/systems2030273 · DOI
- Diakopoulos, N. (2015). Algorithmic accountability: Journalistic investigation of computational power structures. Digital Journalism, 3(3), 398–415. https://doi.org/10.1080/21670811.2014.976411 · DOI · Paper page
- Ellemers, N., van der Toorn, J., Paunov, Y., & van Leeuwen, T. (2019). The psychology of morality: A review and analysis of empirical studies published From 1940 Through 2017. Personality and Social Psychology Review, 23(4), 332–366. https://doi.org/10.1177/1088868318811759 · DOI · Paper page
- Epstein, Z., Payne, B. H., Shen, J. H., Hong, C. J., Felbo, B., Dubey, A., Groh, M., Obradovich, N., Cebrian, M., Rahwan, I. 2018. Turingbox: An experimental platform for the evaluation of AI systems. In IJCAI international joint conference on artificial intelligence 2018-July (pp. 5826–28). https://doi.org/10.24963/ijcai.2018/851. · DOI
- Evans, K., de Moura, N., Chauvier, S., Chatila, R., & Dogan, E. (2020). Ethical decision making in autonomous vehicles: The AV ethics project. Science and Engineering Ethics, 26(6), 3285–3312. https://doi.org/10.1007/s11948-020-00272-8 · DOI · Paper page
- Fagerholm, F., Guinea, A. S., Mäenpää, H., Münch, J. 2014. Building blocks for continuous experimentation. In Proceedings of the 1st international workshop on rapid continuous software engineering (pp. 26–35). RCoSE 2014. ACM. https://doi.org/10.1145/2593812.2593816. · DOI
- Falkenberg, L., & Herremans, I. (1995). Ethical behaviours in organizations: Directed by the formal or informal systems? Journal of Business Ethics, 14(2), 133–143. https://doi.org/10.1007/BF00872018 · DOI · Paper page
- Floridi, L. (2016b). Tolerant paternalism: Pro-ethical design as a resolution of the dilemma of toleration. Science and Engineering Ethics, 22(6), 1669–1688. https://doi.org/10.1007/s11948-015-9733-2. · DOI
- Friedler, S. A., Scheidegger, C., Venkatasubramanian, Suresh. 2016. On the (im)possibility of fairness, no. im: 1–16. http://arxiv.org/abs/1609.07236. · arXiv
- Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, Hal., Crawford, K. 2018. Datasheets for datasets. http://arxiv.org/abs/1803.09010. · arXiv
- Goodman, B. 2016. A step towards accountable algorithms? : Algorithmic discrimination and the European Union general data protection. In:29th conference on neural information processing systems (NIPS 2016), Barcelona, Spain., no. Nips (pp. 1–7).
- ICO. 2018. Guide to the general data protection regulation (GDPR). Guide to the general data protection regulation, n/a. https://doi.org/10.1111/j.1751-1097.1994.tb09662.x. · DOI
- Jotterand, F., & Bosco, C. (2020). Keeping the ‘human in the loop’ in the age of artificial intelligence: Accompanying commentary for ‘correcting the brain?’ By Rainey and Erden. Science and Engineering Ethics, 26(5), 2455–2460. https://doi.org/10.1007/s11948-020-00241-1 · DOI
- Karanasiou, A. P., & Pinotsis, D. A. (2017). A Study into the layers of automated decision-making: emergent normative and legal aspects of deep learning. International Review of Law, Computers & Technology, 31(2), 170–187. https://doi.org/10.1080/13600869.2017.1298499 · DOI
- Kazim, E., Denny, D. M. T., & Koshiyama, A. (2021). AI auditing and impact assessment: According to the UK information commissioner’s office. AI and Ethics, no. 0123456789. https://doi.org/10.1007/s43681-021-00039-2. · DOI
- Keyes, O., Hutson, J., Durbin, M. 2019. A mulching proposal no. May 2019 (pp. 1–11). https://doi.org/10.1145/3290607.3310433. · DOI
- Kim, P. 2017. Auditing algorithms for discrimination. University of Pennsylvania Law Review, 166, 189–203.
- Koene, A., Clifton, C., Hatada, Y., Webb, H., Richardson, R. 2019. A governance framework for algorithmic accountability and transparency. https://doi.org/10.2861/59990. · DOI
- Kolhar, M., Abu-Alhaj, M. M., & El-Atty, S. M. A. (2017). Cloud data auditing techniques with a focus on privacy and security. IEEE Security and Privacy, 15(1), 42–51. https://doi.org/10.1109/MSP.2017.16 · DOI
- Krafft, T. D., Zweig, K. A., & König, P. D. (2020). How to regulate algorithmic decision-making: A framework of regulatory requirements for different applications. Regulation and Governance. https://doi.org/10.1111/rego.12369 · DOI · Paper page
- Kroll, J. A., Huey, J., Barocas, S., Felten, E. W., Reidenberg, J. R., Robinson, D. G., Yu, H. 2016. Accountable algorithms. University of Pennsylvania Law Review, no. 633: 66. https://doi.org/10.1002/ejoc.201200111. · DOI · Paper page
- LaBrie, R. C., Steinke, G. H. 2019. Towards a framework for ethical audits of AI algorithms. In 25th americas conference on information systems, AMCIS 2019 (pp 1–5).
- Lauer, D. (2020). You cannot have AI ethics without ethics. AI and Ethics, 0123456789, 1–5. https://doi.org/10.1007/s43681-020-00013-4 · DOI
- Lee, M., Floridi, L., & Denev, A. (2020). Innovating with confidence: Embedding governance and fairness in a financial services risk management framework. Berkeley Technology Law Journal, 34(2), 1–19.
- Mahajan, V., Venugopal, V. K., Murugavel, M., & Mahajan, H. (2020). The algorithmic audit: Working with vendors to validate radiology-AI algorithms—How we do it. Academic Radiology, 27(1), 132–135. https://doi.org/10.1016/j.acra.2019.09.009. · DOI
- Mau, S., & Howe, S. (2019). The metric society: On the quantification of the social. Ebook Central.
- Microsoft. 2020. Fairlearn: A toolkit for assessing and improving fairness in AI (pp. 1–6).
- Mittelstadt, B. (2016). Auditing for transparency in content personalization systems. International Journal of Communication, 10(June), 4991–5002.
- PDPC. 2020. Model artificial intelligence governance framework second edition. Personal data protection commission of Singapore.
- Rahwan, I. (2018). Society-in-the-loop: Programming the algorithmic social contract. Ethics and Information Technology, 20(1), 5–14. https://doi.org/10.1007/s10676-017-9430-8 · DOI · Paper page
- Raji, I. D., & Buolamwini, J. (2019). Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial AI products. In AIES 2019—Proceedings of the 2019 AAAI/ACM conference on AI, ethics, and society (pp. 429–435). https://doi.org/10.1145/3306618.3314244. · DOI · Paper page
- Sandvig, C., Hamilton, K., Karahalios, K., Langbort, C. 2014. Auditing algorithms. In ICA 2014 data and discrimination preconference. (pp. 1–23). https://doi.org/10.1109/DEXA.2009.55. · DOI
- Schulam, P., Saria, S. 2019. Can you trust this prediction? Auditing pointwise reliability after learning 89. http://arxiv.org/abs/1901.00403. · arXiv
- Springer, A., Whittaker, S. 2019. Making transparency clear.
- Steghöfer, J. P., Knauss, E., Horkoff, J., Wohlrab, R. 2019. Challenges of scaled agile for safety-critical systems. Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) 11915 LNCS (pp. 350–66). https://doi.org/10.1007/978-3-030-35333-9_26. · DOI
- Strenge, B., & Schack, T. (2020). AWOSE—A process model for incorporating ethical analyses in agile systems engineering. Science and Engineering Ethics, 26(2), 851–870. https://doi.org/10.1007/s11948-019-00133-z · DOI
- Tasioulas, J. (2018). First steps towards an ethics of robots and artificial intelligence. SSRN Electronic Journal, 7(1), 61–95. https://doi.org/10.2139/ssrn.3172840 · DOI
- Thaler, R., & Sunstein, C. (2008). Nudge: Improving decisions about health, wealth, and happiness. New Haven, Conn.: Yale University Press.
- Tsamados, A., Aggarwal, N., Cowls, J., Morley, J., Roberts, H., Taddeo, M., et al. (2020). The ethics of algorithms: Key problems and solutions. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3662302. · DOI
- Turner Lee, N. (2018). Detecting racial bias in algorithms and machine learning. Journal of Information, Communication and Ethics in Society, 16(3), 252–260. https://doi.org/10.1108/JICES-06-2018-0056 · DOI · Paper page
- Tutt, A. (2017). An FDA for algorithms. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.2747994 · DOI
- Vakkuri, V., Kemell, K. K., Kultanen, J., Siponen, M., Abrahamsson, P. 2019. Ethically aligned design of autonomous systems: Industry viewpoint and an empirical study. ArXiv.
- Wachter, S., Mittelstadt, B., Russell, C. 2017. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. · DOI · Paper page
- Wiener, N. (1988). The human use of human beings: Cybernetics and society. Da Capo Series in Science.
Cited by
The Ethics of Artificial Intelligence: Principles, Challenges, and Opportunities
Floridi, L. (2023) · doi:10.1093/oso/9780198883098.001.0001 · Cited by 507
Self-citationThe European legislation on AI: A brief analysis of its philosophical approach
Floridi, L. (2021) · Philosophy & Technology · doi:10.1007/s13347-021-00460-9 · Cited by 169
Self-citationMökander, J., Axente, M., Casolari, F., & Floridi, L. (2022) · Minds and Machines · doi:10.1007/s11023-021-09577-4 · Cited by 153
Self-citationAuditing of AI: Legal, Ethical and Technical Approaches
Mökander, J. (2023) · Digital Society · doi:10.1007/s44206-023-00074-y · Cited by 152
“This development is not limited to the private sector (Morley et al., 2021).”
BackgroundSelf-citationRoberts, H., Zhang, J., Bariach, B., Cowls, J., Gilburt, B., Juneja, P., Tsamados, A., Ziosi, M., Taddeo, M., & Floridi, L. (2024) · AI & SOCIETY · doi:10.1007/s00146-022-01596-8 · Cited by 123
Self-citation
Metadata
- Authors
- Mökander, J., Morley, J., Taddeo, M., & Floridi, L.
- Venue
- Science and Engineering Ethics
- Published
- 2021-07-06
- Volume
- 27
- Issue
- 4
- Pages
- 44-44
- ISSN
- 1353-3452 (ISSN-L)
- 1471-5546
- OpenAlex
- W3178786669
- Cited by
- 207
- References
- 57
- FWCI
- 34.60
- Type
- journal article
- Language
- en
- Open access
- hybrid
- Licence
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Funders
- The Welcome Trust
- The Society of Swedish Engineers in Great Britain (2021)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Ethics-Based Auditing of Automated Decision-Making Systems: Nature, Scope, and Limitations. https://app.cochat.ai/papers/10.1007/s11948-021-00319-4
- The paper
- Mökander, J., Morley, J., Taddeo, M., & Floridi, L. (2021). Ethics-Based Auditing of Automated Decision-Making Systems: Nature, Scope, and Limitations. Science and Engineering Ethics. https://doi.org/10.1007/s11948-021-00319-4
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.