An Ontology for Standardising Trustworthy AI
Open accessAbstract
Worldwide, there are a multiplicity of parallel activities being undertaken in developing international standards, regulations and individual organisational policies related to AI and its trustworthiness characteristics. The current lack of mappings between these activities presents the danger of a highly fragmented global landscape emerging in AI trustworthiness. This could present society, government and industry with competing standards, regulations and organisational practices that will then serve to undermine rather than build trust in AI. This chapter presents a simple ontology that can be used for checking the consistency and overlap of concepts from different standards, regulations and policies. The concepts in this ontology are grounded in an overview of AI standardisation currently being undertaken in ISO/IEC JTC 1/SC 42 and identifies its project to define an AI management system standard (AIMS or ISO/IEC WD 42001) as the starting point for establishing conceptual mapping between different initiatives. We propose a minimal, high level ontology for the support of conceptual mapping between different documents and show in the first instance how this can help map out the overlaps and gaps between and among SC 42 standards currently under development.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
ISO/IEC SC 42 Working Group 3 (WG3) works on establishing trustworthiness for general ML systems through transparency, robustness, explainability, and so on [46].
“For example, at the holistic level, ISO/IEC SC 42 Working Group 3 (WG3) works on establishing trustworthiness for general ML systems through transparency, robustness, explainability, and so on [46].”
From Trustworthy AI: From Principles to Practices (2023) · cites this paper as [46]
Multiple standardization activities are ongoing parallelly, including IEEE P7000 and ISO/IEC SC 42 [46], which specifically focuses on AI trustworthiness.
“Multiple standardization activities are ongoing parallelly, including IEEE P7000 and ISO/IEC SC 42 [46], which specifically focuses on AI trustworthiness.”
From Trustworthy AI: From Principles to Practices (2023) · cites this paper as [46]
A variety of guidelines and principles have been proposed by governmental organizations in recent years [5, 34, 57, 81, 46].
“To regulate the development of AI, a variety of guidelines and principles have been proposed by governmental organizations in recent years [5, 34, 57, 81, 46].”
From Trustworthy AI: From Principles to Practices (2023) · cites this paper as [46]
Current explanatory models and post-hoc model explanation techniques share issues: explanations are fragile to perturbations, not always consistent with human interpretation, and difficult to judge as correct or faithful.
“Both current explanatory models and post-hoc model explanation techniques share a few common issues, e.g., 1) the explanation is fragile to perturbations [130], 2) the explanation is not always consistent with human interpretation [46], and 3) it is difficult to judge if the explanation is correct or faithful [250].”
From Trustworthy AI: From Principles to Practices (2023) · cites this paper as [46]
Citation [46] is attached to the claim that explanations are not always consistent with human interpretation. Section 2 discusses the opacity of some machine-learning methods and their unsuitability for clear human-readable explanations, but does not address whether explanations from explanatory models or post-hoc techniques conflict with human interpretation. Searches for that phrasing and for post-hoc explanations found no passage addressing the attributed point. (LLM, conf=0.72)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · cited 2× · see citing works
“One extensive survey identifies an apparent consensus on the importance of ethical principles of transparency, justice, non-maleficence, responsibility, and privacy, whereas other issues of sustainability, dignity, and solidarity in relation to labour impact and distribution garner far less attention across works [6].”
BackgroundMethodModel Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · see citing works
The Ethics of AI Ethics: An Evaluation of Guidelines
Hagendorff, T. (2020) · Minds and Machines · doi:10.1007/s11023-020-09517-8 · Cited by 1,911 · see citing works
Bender, E. M., & Friedman, B. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00041 · Cited by 854 · cited 2× · see citing works
“Such AI trustworthiness characteristics can be for example associated with: the datasets used to train data-driven machine learning AI [47][48][49]; trained ML models [50][51] or AI-based product or services [52].”
BackgroundMethodFactSheets: Increasing trust in AI services through supplier's declarations of conformity
Arnold, M., Bellamy, R. K. E., Hind, M., Houde, S., Mehta, S., Mojsilović, A., Nair, R., Ramamurthy, K. N., Olteanu, A., Piorkowski, D., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942288 · Cited by 434 · see citing works
The Role and Limits of Principles in AI Ethics: Towards a Focus on Tensions
Whittlestone, J., Nyrup, R., Alexandrova, A., & Cave, S. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314289 · Cited by 399 · see citing works
ETHICS AS AN ESCAPE FROM REGULATION.: FROM “ETHICS-WASHING” TO ETHICS-SHOPPING?
WAGNER, B. (2018) · BEING PROFILED · doi:10.2307/j.ctvhrd092.18 · Cited by 228 · see citing works
Artificial Intelligence Policy: A Roadmap
Calo, R. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3015350 · Cited by 191 · see citing works
AI Ethics in the Public, Private, and NGO Sectors: A Review of a Global Document Collection
Schiff, D., Borenstein, J., Biddle, J., & Laas, K. (2021) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2021.3052127 · Cited by 149 · see citing works
Flagged first, then by citations
Research works printed but not linked: 14
- Adamson, G., Havens, J. C., & Chatila, R. "Designing a Value-Driven Future for Ethical Autonomous and Intelligent Systems", Proceedings of the IEEE, 107(3), 518-525. 2019, DOI: 10.1109/JPROC.2018.2884923 · DOI
- Ethically Aligned Design -First Edition", The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems, IEEE, 2019. available at: https://standards.ieee.org/industry- connections/ec/ead-v1.html
- Latzer, M., Hollnbuchner, K., Just, N., & Saurwein, F. (2016). The economics of algorithmic selection on the Internet. Handbook on the Economics of the Internet, (October 2014), pp 395-425. Retrieved from https://doi.org/10.4337/ 9780857939852.00028 · DOI
- Tutt, A. (2016). An FDA for Algorithms. SSRN Electronic Journal, 83-123. https://doi.org/10.2139/ ssrn.2747994 · DOI
- Erdelyi, O.J. and Goldsmith, J., Regulating Artificial Intelligence: Proposal for a Global Solution (February 2, 2018). 2018 AAAI/ACM Conference on AI, Ethics, and Society (AIES '18), February 2--3, 2018, New Orleans, LA, USA. DOI: 10.1145/3278721.3278731, Available at SSRN: https://ssrn.com/ abstract=3263992 · DOI · Paper page
- Calo, R. (2013). Consumer Subject Review Boards: A Thought Experiment. Stanford Law Review Online, 66(2002), 97.
- Narayanan, A., Lechevalier, D., Morris, K.C., Rachuri, S. Communicating standards through structured terminology, Computer Standards & Interfaces, Vol 40, 2015, Pages 34-41, ISSN 0920-5489, DOI: 10.1016/j.csi.2015. 02.004 · DOI
- Smith B, Ashburner M, Rosse C, et al. The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration. Nat Biotechnol. 2007; 25(11):1251-1255. DOI:10.1038/nbt1346 · DOI
- Bizer, C., Heath, T., Berners-Lee,T., "Linked Data: The Story so Far", Semantic Services, Interoperability and Web Applications: Emerging Concepts, pp. 205-227, 2011. DOI: 10/dh8v52. [Online]. Available: https://www. igi-global.com/chapter/linkeddata- story-far/55046 (visited on 09/17/2019).
- Fellmann, M., Zasada, A. State-of- the-Art of Business Process Compliance Approaches: A Survey, Proceedings of the 22nd European Conference on Information Systems (ECIS), Tel Aviv, June 2014
- Pandit, H.J., . O'Sullivan, D., and D. Lewis, D., "Queryable Provenance Metadata For GDPR Compliance", in Procedia Computer Science, ser. Proceedings of the 14th International Conference on Semantic Systems 10th -13th of September 2018 Vienna, Austria, vol. 137, Jan. 1, 2018, pp. 262- 268. DOI: 10/gfdc6r. [Online]. Available: http:/ /www.sciencedirect.com/science/ article/pii/S1877050918316314 (visited on 10/18/2018).
- Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J.W., Wallach, H., Daumé, H., Crawford, K. "Data sheets for Datasets, in Proceedings of the 5th Workshop on Fairness, Accountability, and Transparency in Machine Learning", Stockholm, Sweden, 2018.
- Holland, S., Hosny, A., Newman, S., Joseph, J., Kasia Chmielinski, K., (2018) The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards. arXiv:1805.03677 · arXiv
- Lewis, D., Hogan, L., Filip, D., P. J. Wall, PJ., Global Challenges in the Standardization of Ethics for Trustworthy AI, Journal of ICT Standardization, vol 8, iss2, 2020, DOI: 10.13052/jicts2245-800X.823 · DOI
Cited by
Sorted by citations
Trustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612 · see citing works
- Authors
- Lewis, D., Filip, D., & J. Pandit, H.
- Venue
- Factoring Ethics in Technology, Policy Making, Regulation and AI
- Publisher
- IntechOpen
- Published
- 2021-10-27
- OpenAlex
- W3165819116
- Cited by
- 13
- References
- 9
- Citations by year
- 2025: 4 · 2024: 4 · 2023: 3 · 2022: 2
- FWCI
- 9.28
- Type
- book chapter
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- IntechOpen eBooks (publishedVersion, cc-by)
- Arrow@dit (Dublin Institute of Technology) (submittedVersion, cc-by-nc-sa)
- BiblioBoard Library Catalog (Open Research Library) (submittedVersion, cc-by)
- Contact-less Assessment of In-vivo Body Signals Using Microwave Doppler Radar (InTech) (submittedVersion, cc-by-nc)
- Trinity's Access to Research Output (TARA) (Trinity College Dublin) (submittedVersion, cc-by)
- Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by-nc-sa)
- Zenodo (CERN European Organization for Nuclear Research) (cc-by)
- Funders
- European Commission (813497, HORIZON2020)
- Science Foundation Ireland (13/RC/2106, 13/RC/2106_P2)
- Irish Research Council (GOIPD/2020/790)
- European Regional Development Fund (# 13/RC/2106_P2)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Social Sciences › Law › Digitalization, Law, and Regulation
- Keywords
- trustworthy AI, ontology (information science), conceptual mapping, AI management systems, AI regulation, AI policy, consistency checking, conceptual overlap
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
