The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Artificial intelligence ethics has a black box problem
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Bélisle-Pipon et al. [ 7 ] reviewed 47 major AI ethics guidelines from private and public stakeholders.Their findings show that only 38% reported any kind of stakeholder engagement in their processes; only 9% reported any engagement with citizens at large.”
From Positioning Paradata: A Conceptual Frame for AI Processual Documentation in Archives and Recordkeeping Contexts (2023) · cites this paper as [7]
In the citing paper
“guidelines developed based on stakeholder engagement were widely applicable across different contexts [ 7 ].”
From Positioning Paradata: A Conceptual Frame for AI Processual Documentation in Archives and Recordkeeping Contexts (2023) · cites this paper as [7]
Citations
Cites
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220
Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R., Pagallo, U., Rossi, F., et al. (2018) · Minds and Machines · doi:10.1007/s11023-018-9482-5 · Cited by 4,774 · cited 1×
“…to recommending a set of guidelines for types of actors with a particular role to play in the development, regulation or use of AI (i.e., normative guidance); and to stating general principles to ensure that AI benefits and is not harmful to humanity (Floridi et al. 2018; Greene et al. 2019).”
BackgroundThe Black Box Society: The Secret Algorithms That Control Money and Information
Pasquale, F. (2015) · doi:10.4159/harvard.9780674736061 · Cited by 3,796
The Ethics of AI Ethics: An Evaluation of Guidelines
Hagendorff, T. (2020) · Minds and Machines · doi:10.1007/s11023-020-09517-8 · Cited by 1,911 · cited 7×
“As such, Jobin et al. (2019), the Berkman Klein Center for Internet and Society (BKC) (Fjeld et al. 2020), Ryan and Stahl (2020), Zeng et al. (2018), and Hagendorff (2020) have paved the way for the mapping of the main axiological land‑ marks at the heart of these normative declarations.”
BackgroundMethodInfluentialThe Ethics of AI Ethics -- An Evaluation of Guidelines
Hagendorff, T. (2019) · arXiv · doi:10.48550/arxiv.1903.03425 · Cited by 1,907 · cited 7×
“As such, Jobin et al. (2019), the Berkman Klein Center for Internet and Society (BKC) (Fjeld et al. 2020), Ryan and Stahl (2020), Zeng et al. (2018), and Hagendorff (2020) have paved the way for the mapping of the main axiological land‑ marks at the heart of these normative declarations.”
BackgroundMethodInfluentialPrinciples alone cannot guarantee ethical AI
Mittelstadt, B. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0114-4 · Cited by 1,886 · cited 4×
“(Mittelstadt 2019a).”
BackgroundInfluentialMorley, J., Floridi, L., Kinsey, L., & Elhalal, A. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-019-00165-5 · Cited by 882 · cited 2×
“…2020; Estrada 2020; Mittelstadt 2019b; Hagerty and Rubinov 2019; LaGrandeur 2020; Lauer 2020; Sloane 2018), as well as the contextual applicabil‑ ity in some areas of these ethical guidelines (Ryan and Stahl 2020; Morley et al. 2020; Hickok 2020; Floridi 2019; ÓhÉigeartaigh et al. 2020).”
BackgroundHow AI can be a force for good
Taddeo, M., & Floridi, L. (2018) · Science · doi:10.1126/science.aat5991 · Cited by 760
The Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence
CRAWFORD, K. (2021) · doi:10.2307/j.ctv1ghv45t · Cited by 742
Emerging challenges in AI and the need for AI ethics education
Borenstein, J., & Howard, A. (2021) · AI and Ethics · doi:10.1007/s43681-020-00002-7 · Cited by 626
Research works printed but not linked: 10
- Crawford K, Dobbe R, Dryer T et al (2019) AI now 2019 report. New York University, New York
- Estrada D (2020) Ideal theory in AI ethics. http://arxiv.org/abs/201102279
- Fjeld J, Hilligoss H, Achten N et al (2020) Principled artificial intelligence: a map of ethical and rights-based approaches. Berkman Klein Center for Internet and Society, Cambridge
- Gagnon M-A (2020) Ghost management as a central feature of accumulation in corporate capitalism: the case of the global pharmaceutical sector. In: Benquet M, Bourgeron T (eds) Accumulating capital today: contemporary strategies of profit and dispossessive policies, 1st edn. Routledge, pp 163–177
- Goldsmith J, Burton E (2017) Why Teaching Ethics to AI Practitioners is Important. Proc AAAI Conf Artif Intell. https://doi.org/10.5555/3297863.3297908 · doi:10.5555/3297863.3297908
- Gordon K, Miyake M (1999) Deciphering codes of corporate conduct: a review of their contents. Organisation for Economic Co-operation and Development, Paris
- Government of Canada (2020) Responsible use of artificial intelligence. Canada, Ottawa
- Hagerty A, Rubinov I (2019) Global AI ethics: a review of the social impacts and ethical implications of artificial intelligence. http://arxiv.org/abs/190707892
- Sloane M, Moss E, Awomolo O, Forlano L (2020) Participation is not a design fix for machine learning. http://arxiv.org/abs/200702423
- Zeng Y, Lu E, Huangfu C (2018) Linking artificial intelligence principles. arXiv. https://arxiv.org/abs/1812.04814v1 · arXiv
Cited by
Khawaja, Z., & Bélisle-Pipon, J.-C. (2023) · Frontiers in Digital Health · doi:10.3389/fdgth.2023.1278186 · Cited by 214
Self-citationSaeidnia, H. R., Hashemi Fotami, S. G., Lund, B., & Ghiasi, N. (2024) · Social Sciences · doi:10.3390/socsci13070381 · Cited by 169
Amann, J., Vayena, E., Ormond, K. E., Frey, D., Madai, V. I., & Blasimme, A. (2023) · PLoS ONE · doi:10.1371/journal.pone.0279088 · Cited by 119
Specific challenges posed by artificial intelligence in research ethics
Bouhouita-Guermech, S., Gogognon, P., & Bélisle‐Pipon, J. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1149082 · Cited by 153
“Considering the multidisciplinary aspect of AI, different experts are called to provide their knowledge and expertise on the matter (Bélisle-Pipon et al., 2022).”
BackgroundSelf-citationChow, J. C. L., & Li, K. (2024) · JMIR Bioinformatics and Biotechnology · doi:10.2196/64406 · Cited by 75
Metadata
- Authors
- Bélisle-Pipon, J.-C., Monteferrante, E., Roy, M.-C., & Couture, V.
- Affiliations
- McGill University (CA)
- Université de Montréal (CA)
- Université Laval (CA)
- Venue
- AI & SOCIETY
- Published
- 2022-01-05
- Volume
- 38
- Issue
- 4
- Pages
- 1507-1522
- ISSN
- 0951-5666 (ISSN-L)
- 1435-5655
- OpenAlex
- W4206760316
- Semantic Scholar
- 7e86fad5eca4ae4b5709c6b6c856fff2d5693998
- Cited by
- 85
- References
- 50
- Citations by year
- 2026: 16 · 2025: 28 · 2024: 25 · 2023: 15 · 2022: 1
- FWCI
- 19.99
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- AI & Society (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Ethics in Business and Education
- Keywords
- AI ethics, moral norms, stakeholder engagement, citizen participation, transparency, black box problem, public trust, private sector, AI regulation, normative development
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Artificial intelligence ethics has a black box problem. https://app.cochat.ai/papers/10.1007/s00146-021-01380-0
- The paper
- Bélisle-Pipon, J.-C., Monteferrante, E., Roy, M.-C., & Couture, V. (2023). Artificial intelligence ethics has a black box problem. AI & SOCIETY. https://doi.org/10.1007/s00146-021-01380-0
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.