The Sanction of Authority: Promoting Public Trust in AI
The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
Trusted AI literature to date has focused on the trust needs of users who knowingly interact with discrete AIs. Conspicuously absent from the literature is a rigorous treatment of public trust in AI. We argue that public distrust of AI originates from the underdevelopment of a regulatory ecosystem that would guarantee the trustworthiness of the AIs that pervade society. Drawing from structuration theory and literature on institutional trust, we offer a model of public trust in AI that differs starkly from models driving Trusted AI efforts. This model provides a theoretical scaffolding for Trusted AI research which underscores the need to develop nothing less than a comprehensive and visibly functioning regulatory ecosystem. We elaborate the pivotal role of externally auditable AI documentation within this model and the work to be done to ensure it is effective, and outline a number of actions that would promote public trust in AI. We discuss how existing efforts to develop AI documentation within organizations---both to inform potential adopters of AI components and support the deliberations of risk and ethics review boards---is necessary but insufficient assurance of the trustworthiness of AI. We argue that being accountable to the public in ways that earn their trust, through elaborating rules for AI and developing resources for enforcing these rules, is what will ultimately make AI trustworthy enough to be woven into the fabric of our society.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“AI documentation is advocated to increase the transparency and facilitate understanding of AI [35,36,40,75,76,82], and is embodied in recently proposed features of AI Factsheets [8,81] and Model Cards [64].”
From Investigating Explainability of Generative AI for Code through Scenario-based Design (2022) · cites this paper as [40]
In the citing paper
“When these mechanisms are aligned with efforts needed to establish social, organizational, and regulatory ecosystems for the assurance of trustworthy AI [9, 30, 54], they are likely to satisfy the calibration condition.”
From Designing for Responsible Trust in AI Systems: A Communication Perspective (2022) · cites this paper as [30]
Citations
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · cited 2×
“Legitimation: The avalanche of corporate, non-governmental and (inter-)governmental statements on principles to guide ethical AI [24, 38, 47] are attempts at articulating face by defining a set of positive values claimed by AI-as-an-institution.”
Developing and validating trust measures for e-commerce: an integrative typology
McKnight, D. H., Choudhury, V., & Kacmar, C. (2002) · Information Systems Research · doi:10.1287/isre.13.3.334.81 · Cited by 5,550
Lewis, J. D., & Weigert, A. (1985) · Social Forces · doi:10.2307/2578601 · Cited by 4,163
On Face-Work: An Analysis of Ritual Elements in Social Interaction
Goffman, E. (1955) · Psychiatry · doi:10.1080/00332747.1955.11023008 · Cited by 2,870
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681
Institutionalization and Structuration: Studying the Links between Action and Institution
Barley, S. R., & Tolbert, P. S. (1997) · Organization Studies · doi:10.1177/017084069701800106 · Cited by 2,604
The constitution of society: Outline of the theory of structuration
Palm, R. (1986) · Political Geography Quarterly · doi:10.1016/0260-9827(86)90040-6 · Cited by 1,590
Lee, M. K. (2018) · Big Data & Society · doi:10.1177/2053951718756684 · Cited by 1,338
Initial trust formation in new organizational relationships
McKnight, D. H., Cummings, L. L., & Chervany, N. L. (1998) · The Academy of Management Review · doi:10.2307/259290 · Cited by 1,296
Two Concepts of Accountability: Accountability as a Virtue and as a Mechanism
Bovens, M. (2010) · West European Politics · doi:10.1080/01402382.2010.486119 · Cited by 1,018 · cited 1×
“…that accountability relies on parties being able to act (e.g. sanction untrustworthy actions on the part of the trustee), the irrelevance of action here removes a critical “mechanism to facilitate better behavior” [53] (see also [7, 14]), which would otherwise feed a virtuous cycle for trust.”
BackgroundSocial translucence: an approach to designing systems that support social processes
Erickson, T., & Kellogg, W. A. (2000) · ACM Transactions on Computer-Human Interaction · doi:10.1145/344949.345004 · Cited by 998 · cited 1×
“sanction untrustworthy actions on the part of the trustee), the irrelevance of action here removes a critical “mechanism to facilitate better behavior” [53] (see also [7, 14]), which would typically feed a virtuous cycle for trust.”
BackgroundWynne, B. (2006) · Public Health Genomics · doi:10.1159/000092659 · Cited by 972 · cited 1×
“Toreini et al. [48] attempt to resolve this by noting that there can be dissonance between beliefs, decisions and actions, resulting in “ as if trust” [54], though this leaves unresolved the issue of why, if the final result is the same, trust beliefs or decisions even matter.”
BackgroundBender, E. M., & Friedman, B. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00041 · Cited by 854
Mittelstadt, B., Russell, C., & Wachter, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287574 · Cited by 794 · cited 2×
“Explainable AI (xAI)—within which there are a variety of explanation types [39]—seeks to rectify this information gap by, generally, providing explainees a description of how a model or algorithm inside a “black box” functions [49], or why a particular decision or outcome was reached.”
BackgroundHengstler, M., Enkel, E., & Duelli, S. (2016) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2015.12.014 · Cited by 661
Li, X., Hess, T. J., & Valacich, J. S. (2008) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2008.01.001 · Cited by 635
How Much Information?: Effects of Transparency on Trust in an Algorithmic Interface
Kizilcec, R. F. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858402 · Cited by 615 · cited 1×
“That reason is, in contrast, instrumental: trust is presumed to be necessary for AI adoption (e.g. [4, 18, 21, 26, 30, 41, 46, 48]).”
BackgroundMeasuring trust inside organisations
Dietz, G., & Den Hartog, D. N. (2006) · Personnel Review · doi:10.1108/00483480610682299 · Cited by 596
Giddens's structuration theory and information systems research
Jones, M., & Karsten, H. (2008) · Journal of the Association for Information Systems · doi:10.5555/2017378.2017385 · Cited by 575
The mechanics of trust: A framework for research and design
Riegelsberger, J., Sasse, M. A., & McCarthy, J. D. (2005) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2005.01.001 · Cited by 508
FactSheets: Increasing trust in AI services through supplier's declarations of conformity
Arnold, M., Bellamy, R. K. E., Hind, M., Houde, S., Mehta, S., Mojsilović, A., Nair, R., Ramamurthy, K. N., Olteanu, A., Piorkowski, D., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942288 · Cited by 434
Self-citationSituating methods in the magic of Big Data and AI
Elish, M. C., & boyd, D. (2018) · Communication Monographs · doi:10.1080/03637751.2017.1375130 · Cited by 409 · cited 5×
“Perceptions of AI are “informed by long-standing cultural imaginaries of machines that escape the control of their creators, and the promises and perils of automata and artificial life” [12].”
BackgroundInfluentialThe Role and Limits of Principles in AI Ethics: Towards a Focus on Tensions
Whittlestone, J., Nyrup, R., Alexandrova, A., & Cave, S. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314289 · Cited by 399 · cited 1×
“Where they fall short is in prescribing or even indicating normative implementation of these principles—a short-coming much lamented in the literature [52].”
Background(2018) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 390
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000375277 · Cited by 360
Wieringa, M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372833 · Cited by 321 · cited 2×
“But as noted by Wieringa [53], “The transparent workings of a system do not tell you why it was deemed ‘good enough’ at decision making, or why it was deemed desirable to begin with. . .”
BackgroundThe relationship between trust in AI and trustworthy machine learning technologies
Toreini, E., Aitken, M., Coopamootoo, K., Elliott, K., Zelaya, C. G., & van Moorsel, A. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372834 · Cited by 296
Toward establishing trust in adaptive agents
Glass, A., McGuinness, D. L., & Wolverton, M. (2008) · Proceedings of the 13th international conference on Intelligent user interfaces · doi:10.1145/1378773.1378804 · Cited by 211
Facework: creating trust in systems, institutions and organisations
Kroeger, F. (2016) · Cambridge Journal of Economics · doi:10.1093/cje/bew038 · Cited by 81
Trust models in ubiquitous computing
Krukow, K., Nielsen, M., & Sassone, V. (2008) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2008.0134 · Cited by 46 · cited 1×
“This problem holds for ubiquitous computing generally, and it is surprising that models of trust in ubiquitous computing have focused more on issues to do with people being separated in space [13, 28, 29, 45] than accounting for aspects of “systemness”.”
BackgroundOnward for the freedom of others: marching beyond the AI ethics
Terzis, P. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3373152 · Cited by 36
Raji, I. D., & Yang, J. (2019) · arXiv · doi:10.48550/arxiv.1912.06166 · Cited by 36 · cited 2×
“As an artifact, documentation can improve communication within a team—in particular regarding teammembers’ individual impacts on the resulting product—and can be used to illustrate and argue the need for additional resources to ensure the AI lives up to ethical standards; and as a process, it encourages individuals to…”
BackgroundThinking about Trust: People, Process, and Place
Marsh, S., Atele-Williams, T., Basu, A., Dwyer, N., Lewis, P. R., Miller-Bakewell, H., & Pitt, J. V. (2020) · Patterns · doi:10.1016/j.patter.2020.100039 · Cited by 32
A Methodology for Creating AI FactSheets
Richards, J., Piorkowski, D., Hind, M., Houde, S., & Mojsilović, A. (2020) · arXiv · doi:10.48550/arxiv.2006.13796 · Cited by 28
Self-citationOnward for the freedom of others: Marching beyond the AI EthicsPreprint
Terzis, P. (2020) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/3xzaf · Cited by 6
(1990) · Choice Reviews Online · doi:10.5860/choice.28-1843 · Cited by 3 · cited 9×
“Giddens refers to these as signification, legitimation, and domination [17]: Signification refers to the implicit language of symbols and symbolic actions that derive their meaning in reference to the institution.”
BackgroundMethodInfluential
Research works printed but not linked: 13
- Malta AI. 2019. Malta The Ultimate Launchpad: A Strategy and Vision for Artificial Intelligence in Malta 2030. https://malta.ai/wp-content/uploads/2019/11/Malta_The_Ultimate_AI_Launchpad_vFinal.pdf.
- Guruduth Banavar. 2016. Learning to trust artificial intelligence systems. Report, IBM, Armonk, NY (2016).
- Colin English, Paddy Nixon, Sotirios Terzis, Andrew McGettrick, and Helen Lowe. 2002. Dynamic trust models for ubiquitous computing environments. In First workshop on security in ubiquitous computing at the fourth annual conference on ubiquitous computing (Ubicomp2002).
- Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, and Kate Crawford. 2018. Datasheets for datasets. arXiv preprint arXiv:1803.09010 (2018). · arXiv
- Richard Harper. 2014. Trust, computing, and society. Cambridge University Press.
- Michael Hind, Sameep Mehta, Aleksandra Mojsilovic, Ravi Nair, Karthikeyan Natesan Ramamurthy, Alexandra Olteanu, and Kush R Varshney. 2018. Increasing trust in AI services through supplier's declarations of conformity. arXiv preprint arXiv:1808.07261 (2018). · arXiv
- Sarah Holland, Ahmed Hosny, and Sarah Newman. 2020. The Dataset Nutrition Label. Data Protection and Privacy: Data Protection and Democracy (2020), 1.
- Marc Langheinrich. 2003. When trust does not compute-the role of trust in ubiquitous computing. In Workshop on Privacy at UBICOMP, Vol. 2003.
- Brent Mittelstadt. 2019. AI Ethics-Too principled to fail. arXiv preprint arXiv:1906.06668 (2019). · arXiv
- Narendar Shankar and William Arbaugh. 2002. On trust for ubiquitous computing. In Workshop on Security in Ubiquitous Computing, UBICOMP, Vol. 32. Citeseer.
- Sandra Wachter, Brent Mittelstadt, and Chris Russell. 2017. Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GPDR. Harv. JL & Tech. 31 (2017), 841.
- Sandra Wachter, Brent Mittelstadt, and Chris Russell. 2020. Why fairness cannot be automated: Bridging the gap between EU non-discrimination law and AI. Available at SSRN (2020).
- Darrell M. West and John R. Allen. 2020. Turning Point: Policymaking in the Era of Artificial Intelligence. Brookings Institution Press.
Cited by
Trust in AI: progress, challenges, and future directions
Afroogh, S., Akbari, A., Malone, E., Kargar, M., & Alambeigi, H. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-04044-8 · Cited by 507
“Some believe that the public distrust in AI originates from the under-development of a regulatory ecosystem that would guarantee AI ’ s trustworthiness (Knowles and Richards, 2021).”
Trustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612
“On the whole, transparency serves as a basic requirement to build the public’s trust in AI systems [22, 178, 189].”
BackgroundTowards a standard for identifying and managing bias in artificial intelligence
Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., & Hall, P. (2022) · doi:10.6028/nist.sp.1270 · Cited by 548
Investigating Explainability of Generative AI for Code through Scenario-based Design
Sun, J., Liao, Q. V., Muller, M., Agarwal, M., Houde, S., Talamadupula, K., & Weisz, J. D. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511119 · Cited by 219
“AI documentation is advocated to increase the transparency and facilitate understanding of AI [35, 36, 40, 75, 76, 82], and is embodied in recently proposed features of AI Factsheets [8, 81] and Model Cards [64].”
BackgroundLaine, J., Minkkinen, M., & Mäntymäki, M. (2024) · Information & Management · doi:10.1016/j.im.2024.103969 · Cited by 224
“According to the reviewed literature, a robust regulatory system and concrete methods must be developed because public trust in AI will arise only by guar-anteeing that the public is protected from the harmful consequences of AI [3,66].”
Influential
Metadata
- Authors
- Knowles, B., & Richards, J. T.
- Affiliations
- Lancaster University (GB)
- IBM (United States) (US)
- Venue
- Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2021-03
- Pages
- 262-271
- OpenAlex
- W3134203054
- Cited by
- 86
- References
- 37
- Citations by year
- 2026: 10 · 2025: 32 · 2024: 15 · 2023: 19 · 2022: 7 · 2021: 3
- FWCI
- 9.95
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
- Funders
- ESRC (ES/T012382/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Law › Digitalization, Law, and Regulation
- Keywords
- public trust in AI, trustworthy AI, AI regulation, structuration theory, institutional trust, AI documentation, AI accountability, AI governance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.