AI Ethics in the Public, Private, and NGO Sectors: A Review of a Global Document Collection
Open accessPublished version of a preprint
The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Abstract
In recent years, numerous public, private, and non-governmental organizations (NGOs) have produced documents addressing the ethical implications of artificial intelligence (AI). These normative documents include principles, frameworks, and policy strategies that articulate the ethical concerns, priorities, and associated strategies of leading organizations and governments around the world. We examined 112 such documents from 25 countries that were produced between 2016 and the middle of 2019. While other studies identified some degree of consensus in such documents, our work highlights meaningful differences across public, private, and NGO sectors. We analyzed each document in terms of how many of 25 ethical topics were covered and the depth of discussion for those topics. As compared to documents from private entities, NGO and public sector documents reflect more ethical breadth in the number of topics covered, are more engaged with law and regulation, and are generated through processes that are more participatory. These findings may reveal differences in underlying beliefs about an organization's responsibilities, the relative importance of relying on experts versus including representatives from the public, and the tension between prosocial and economic goals.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Though differences exist, it is shown that global convergence on the requirement of AI trustworthiness is recognized as summarized in Section 2.”
From Trustworthy AI: From Principles to Practices (2023) · cites this paper as [72]
In the citing paper
“In the first phase from 2016 to 2019, many high-level ethical principles for AI were published as evidenced by these catalogues of ethical principles and frameworks for ethical, trustworthy responsible AI [4-6, 18, 19].”
From Putting AI ethics to work: are the tools fit for purpose? (2022) · cites this paper as [19]
In the citing paper
“It also serves to illustrate how complex this landscape is, and as others have noted [7,19,98,102,104,105], this provides a barrier to those developing or purchasing AI systems as to which tool is appropriate for their purposes.”
From Putting AI ethics to work: are the tools fit for purpose? (2022) · cites this paper as [19]
Citations
Cites
A General Inductive Approach for Analyzing Qualitative Evaluation Data
Thomas, D. R. (2006) · American Journal of Evaluation · doi:10.1177/1098214005283748 · Cited by 10,858 · cited 1×
“Based on these documents and our own knowledge of AI ethics, we inductively developed an initial list of codes, including a preliminary set of ethics topics [18].”
MethodThe global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · cited 5×
“Though our ethics topics are more finely parsed than in other studies, our results are nevertheless highly consistent with those studies, including [3], [4], and [6].”
InfluentialComputing Inter-Rater Reliability for Observational Data: An Overview and Tutorial
Hallgren, K. A. (2012) · Tutorials in Quantitative Methods for Psychology · doi:10.20982/tqmp.08.1.p023 · Cited by 3,884 · cited 1×
“We then engaged in simple reliability testing, with all coders asynchronously coding two documents and comparing results [19].”
MethodThe Fourth Industrial Revolution: Proceedings of a Workshopâ€"in Brief
Government-University-Industry Research Roundtable, Affairs, P. A. G., & National Academies of Sciences, E. (2017) · National Academies Press eBooks · doi:10.17226/24699 · Cited by 3,027
The Ethics of AI Ethics: An Evaluation of Guidelines
Hagendorff, T. (2020) · Minds and Machines · doi:10.1007/s11023-020-09517-8 · Cited by 1,911 · cited 3×
“Some have used qualitative methods to identify themes across documents [3]–[7] or to support comparative analyses [8]–[10]; others have employed quantitative content analysis for similar reasons [11].”
BackgroundMethodThe Ethics of AI Ethics -- An Evaluation of Guidelines
Hagendorff, T. (2019) · arXiv · doi:10.48550/arxiv.1903.03425 · Cited by 1,907 · cited 3×
“Some have used qualitative methods to identify themes across documents [3]–[7] or to support comparative analyses [8]–[10]; others have employed quantitative content analysis for similar reasons [11].”
BackgroundMethodPrinciples alone cannot guarantee ethical AI
Mittelstadt, B. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0114-4 · Cited by 1,886 · cited 2×
“15 We overwhelmingly found that these discussions were absent, which may itself be a finding.”
BackgroundA Unified Framework of Five Principles for AI in Society
Floridi, L., & Cowls, J. (2019) · Harvard Data Science Review · doi:10.1162/99608f92.8cd550d1 · Cited by 1,631 · cited 4×
“A sizable amount of prior work views socially responsible or beneficent AI as a consensus value [4], [6].”
BackgroundResultInfluentialFjeld, J., Achten, N., Hilligoss, H., Nagy, A., & Srikumar, M. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3518482 · Cited by 998 · cited 3×
“A sizable amount of prior work views socially responsible or beneficent AI as a consensus value [4], [6].”
BackgroundResultMorley, J., Floridi, L., Kinsey, L., & Elhalal, A. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-019-00165-5 · Cited by 882 · cited 1×
“Still others have discussed second-order themes, such as the ethical assumptions underlying such documents [12] and the gap between ethical principles and actual practices [3], [7], [13]–[15].”
BackgroundArtificial Intelligence and the ‘Good Society’: the US, EU, and UK approach
Cath, C., Wachter, S., Mittelstadt, B., Taddeo, M., & Floridi, L. (2017) · Science and Engineering Ethics · doi:10.1007/s11948-017-9901-7 · Cited by 733 · cited 1×
“Some have used qualitative methods to identify themes across documents [3]–[7] or to support comparative analyses [8]–[10]; others have employed quantitative content analysis for similar reasons [11].”
MethodResearch Priorities for Robust and Beneficial Artificial Intelligence
Russell, S., Dewey, D., & Tegmark, M. (2015) · AI Magazine · doi:10.1609/aimag.v36i4.2577 · Cited by 637 · cited 1×
“For those concerned with long-term safety and alignment of AI systems with human values [31], [32], this explicit deprioritization may”
BackgroundThe Ethics of AI Ethics: An Evaluation of Guidelines
Hagendorff, T. (2020) · PhilPapers (PhilPapers Foundation) · Cited by 394
Greene, D., Hoffmann, A. L., & Stark, L. (2019) · Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.24251/hicss.2019.258 · Cited by 386 · cited 7×
“However, such a focus places “design as the locus of ethical scrutiny” [12], potentially at the expense of questioning business values or practices that should be within the bounds of conversation.”
BackgroundInfluentialGlicken, J. (2000) · Environmental Science & Policy · doi:10.1016/s1462-9011(00)00105-2 · Cited by 269 · cited 1×
“Beyond examining which stakeholders are included in document creation as we do here, assessing the impact of participation or the lack thereof requires a fuller understanding of best practices for crafting meaningful participatory processes [38].”
BackgroundParticipation Is not a Design Fix for Machine Learning
Sloane, M., Moss, E., Awomolo, O., & Forlano, L. (2022) · Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3551624.3555285 · Cited by 267 · cited 2×
“Under this frame, participation serves as a mere legitimation exercise, meant to alleviate concerns and constituting a form of participation-washing [22].”
BackgroundFrom ethics washing to ethics bashing: a view on tech ethics from within moral philosophy
Bietti, E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372860 · Cited by 261 · cited 1×
“Yet other scholars and commentators have specifically criticized the private sector and argued that its stated motivations are better understood as signaling strategies than as genuine signs of social responsibility [27], [28].”
BackgroundSocial responsibility, profit maximisation and the small firm owner‐manager
Spence, L. J., & Rutherfoord, R. (2001) · Journal of Small Business and Enterprise Development · doi:10.1108/eum0000000006818 · Cited by 241 · cited 1×
“However, while the tension between economic and social values in the private sector has a long history [39], [40], our results also challenge single-minded attention to the private”
ResultMorley, J., Floridi, L., Kinsey, L., & Elhalal, A. (2021) · Philosophical studies series · doi:10.1007/978-3-030-81907-1_10 · Cited by 224
Roles for computing in social change
Abebe, R., Barocas, S., Kleinberg, J., Levy, K., Raghavan, M., & Robinson, D. G. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372871 · Cited by 219 · cited 2×
“Third, though the private sector is not typically higher than other sectors in absolute terms, it gives higher relative priority to social responsibility and trust, which suggests that the private sector wants to signal that it is taking these issues seriously [26]– [28].”
BackgroundWhat's Next for AI Ethics, Policy, and Governance? A Global Overview
Schiff, D., Biddle, J., Borenstein, J., & Laas, K. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375804 · Cited by 199 · cited 3×
“Organizations might have different perceived scopes of responsibility given their traditional roles and audiences; they might view the role and relative importance of experts and the public in shaping decisions differently, and organizations may have varied motivations [26].”
BackgroundSelf-citationRepertoires: A post-Kuhnian perspective on scientific change and collaborative research
Ankeny, R. A., & Leonelli, S. (2016) · Studies in History and Philosophy of Science Part A · doi:10.1016/j.shpsa.2016.08.003 · Cited by 183
Mixed Motives: Economic, Social, and Normative Motivations in Business Compliance
Nielsen, V. L., & Parker, C. (2012) · Law & Policy · doi:10.1111/j.1467-9930.2012.00369.x · Cited by 182 · cited 1×
“However, while the tension between economic and social values in the private sector has a long history [39], [40], our results also challenge single-minded attention to the private sector’s motivations.”
ResultSocial Responsibility, Profit-Maximisation and the Small Firm Owner-Manager
Spence, L. J., & Rutherfoord, R. (2004) · Responsibility and Social Capital · doi:10.1007/978-1-349-72886-2_4 · Cited by 158
Artificial Intelligence, Governance and Ethics: Global PerspectivesPreprint
Daly, A., Hagendorff, T., Li, H., Mann, M., Marda, V., Wagner, B., Wang, W., & Witteborn, S. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3414805 · Cited by 107
From Ethics Washing to Ethics Bashing: A View on Tech Ethics from Within Moral PhilosophyPreprint
Bietti, E. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3914119 · Cited by 79
Collier, S. J., & Lakoff, A. (2007) · Global Assemblages · doi:10.1002/9780470696569.ch2 · Cited by 75
Linking Artificial Intelligence Principles
Zeng, Y., Lu, E., & Huangfu, C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.04814 · Cited by 71 · cited 1×
“Some have used qualitative methods to identify themes across documents [3]–[7] or to support comparative analyses [8]–[10]; others have employed quantitative content analysis for similar reasons [11].”
MethodPrunkl, C., & Whittlestone, J. (2020) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3375627.3375803 · Cited by 50 · cited 1×
“For those concerned with long-term safety and alignment of AI systems with human values [31], [32], this explicit deprioritization may be a source of serious concern.”
BackgroundArtificial Intelligence Governance and Ethics : Global Perspectives
Daly, A., Hagendorff, T., Hui, L., Mann, M., Marda, V., Wagner, B., Wang, W., & Witteborn, S. (2019) · Strathprints: The University of Strathclyde institutional repository (University of Strathclyde) · Cited by 35
Prediction paradigm: the human price of instrumentalism
Gill, K. S. (2020) · AI & Society · doi:10.1007/s00146-020-01035-6 · Cited by 30 · cited 1×
“Yet a lack of engagement with law could reveal a lack of seriousness about AI ethics, akin to what some have termed “ethics washing” or “ethics shirking” [25].”
BackgroundDifferent "Intelligibility" for Different Folks
Zhou, Y., & Danks, D. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375810 · Cited by 24 · cited 1×
“Organizations can also define topics differently, such as by defining explainability in a way that places more stringent demands on AI developers [35], potentially opening organizations up to increased scrutiny and legal liability.”
BackgroundPrunkl, C., & Whittlestone, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.04335 · Cited by 15 · cited 1×
“For those concerned with long-term safety and alignment of AI systems with human values [31], [32], this explicit deprioritization may be a source of serious concern.”
BackgroundPrinciples to Practices for Responsible AI: Closing the Gap
Schiff, D., Rakova, B., Ayesh, A., Fanti, A., & Lennon, M. (2020) · arXiv · doi:10.48550/arxiv.2006.04707
Self-citation
Research works printed but not linked: 17
- 2018, Shaping the Future of Austria With Robotics and Artificial Intelligence
- 2017, United States Artificial Intelligence Risk to Critical Infrastructure
- rothstein, 2009, Grey literature, The Handbook of Research Synthesis and Meta-Analysis
- perrault, 2020, The AI Index 2019 Annual Report
- 2016, United States Preparing for the Future of Artificial Intelligence
- gibert, 2018, Montréal Declaration of Responsible AI 2018 Overview of International Recommendations for AI Ethics
- dutton, 2018, Building an AI World Report on National and Regional AI Strategies
- 10.1007/s11023-020-09517-8 · DOI · Paper page
- schwab, 2016, The Fourth Industrial Revolution
- bughin, 2018, Modeling the Global Economic Impact of AI
- 2019, Spanish RDI Strategy in Artificial Intelligence
- 2019, Malta Towards an AI Strategy
- 2018, National Strategy for Artificial Intelligence #AIFORALL
- 2018, The Futurice Principles for Ethical AI
- 2017, China’s New Generation of Artificial Intelligence Development Plan
- 2018, Guardians of Trust Who Is Responsible for Trusted Analytics in the Digital Age?
- 2017, Finland’s Age of Artificial Intelligence Turning Finland into a Leading Country in the Application of Artificial Intelligence
Cited by
Trustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612
“Developments in non-technical areas have also been reviewed in a few studies, and involve guidelines [145, 178, 294], standardization [210], and management processes [31, 274, 301].”
BackgroundSchiff, D. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00270-2 · Cited by 489
“…breadth and depth in their reflection on various social and ethical issues (Fjeld et al. 2020; Jobin et al. 2019), and address a sweeping scope of policy sectors (Schiff et al. 2021), traditional AIED is neglected and consideration of AIED’s ethical implications is almost entirely absent.”
BackgroundInfluentialSelf-citationAI governance: themes, knowledge gaps and future agendas
Birkstedt, T., Minkkinen, M., Tandon, A., & Mäntymäki, M. (2023) · Internet Research · doi:10.1108/intr-01-2022-0042 · Cited by 309
Mannuru, N. R., Shahriar, S., Teel, Z. A., Wang, T., Lund, B. D., Tijani, S., Pohboon, C. O., Agbaji, D., Alhassan, J., Galley, J., et al. (2025) · Information Development · doi:10.1177/02666669231200628 · Cited by 314
“Having these policies in place allows employers to take appropriate actions if issues arise concerning employee use of AI technologies (Schiff et al., 2021).”
The impact of automation and artificial intelligence on worker well-being
Nazareno, L., & Schiff, D. S. (2021) · Technology in Society · doi:10.1016/j.techsoc.2021.101679 · Cited by 302
Self-citation
Metadata
- Authors
- Schiff, D., Borenstein, J., Biddle, J., & Laas, K.
- Affiliations
- Georgia Institute of Technology (US)
- Illinois Institute of Technology (US)
- Venue
- IEEE Transactions on Technology and Society
- Published
- 2021-03
- Volume
- 2
- Issue
- 1
- Pages
- 31-42
- ISSN
- 2637-6415 (ISSN-L)
- OpenAlex
- W3126531007
- Semantic Scholar
- 8349034f31db26cd197e07674a4ad136896d42af
- Cited by
- 149
- References
- 33
- Citations by year
- 2026: 30 · 2025: 33 · 2024: 26 · 2023: 22 · 2022: 23 · 2021: 14
- FWCI
- 26.57
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- IEEE Transactions on Technology and Society (publishedVersion)
- Figshare (submittedVersion, cc-by-nc-sa)
- Funders
- Science, Technology, and Innovation Policy Program, Georgia Institute of Technology
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Ethics in Business and Education
- Keywords
- AI ethics, public sector, private sector, non-governmental organizations, cross-sector comparison, AI regulation, participatory processes, organizational responsibility
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.