Reciprocal trust and distrust in artificial intelligence systems: the hard problem of regulation
Open accessThe DOI matches its Crossref record; 83 of the 83 references it cites were found in a registry or an index.
Abstract
Policymakers, scientists, and the public are increasingly confronted with thorny questions about the regulation of artificial intelligence (AI) systems. A key common thread concerns whether AI can be trusted and the factors that can make it more trustworthy in front of stakeholders and users. This is indeed crucial, as the trustworthiness of AI systems is fundamental for both democratic governance and for the development and deployment of AI. This article advances the discussion by arguing that AI systems should also be recognized, at least to some extent, as artifacts capable of exercising a form of agency, thereby enabling them to engage in relationships of trust or distrust with humans. It further examines the implications of these reciprocal trust dynamics for regulators tasked with overseeing AI systems. The article concludes by identifying key tensions and unresolved dilemmas that these dynamics pose for the future of AI regulation and governance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Importantly, the role of human oversight as a safeguard must itself be treated with caution: rather than a straightforward solution, it represents a complex governance challenge shaped by institutional design, competing logics of trust and distrust, and the distributed nature of control in AI systems (Laux 2024; Maggetti 2026; Foth 2026).”
From Digital civics in action: the promise and emerging practices of generative AI in grassroots community advocacy (2026) · cites this paper as Maggetti 2026
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Can Quantum-Mechanical Description of Physical Reality Be Considered Complete?
Einstein, A., Podolsky, B., & Rosen, N. (1935) · Physical Review · doi:10.1103/physrev.47.777 · Cited by 16,654 · cited 1×
“Thought experiments are famously employed in physics (Einstein et al. 1935; Schrödinger 1935) and in moral philosophy (Singer 1972; Thomson 1984), but they may serve as valuable analytical tools in political science and regulatory governance as well.”
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736 · cited 1×
“Classic models in psychology and organizational studies highlight key attributes that make a trustee trustworthy, typically some combination of predictable competence, benevolence, and integrity, as articulated in the ABI framework (Mayer et al. 1995).”
Not So Different After All: A Cross-Discipline View Of Trust
Rousseau, D. M., Sitkin, S. B., Burt, R. S., & Camerer, C. (1998) · Academy of Management Review · doi:10.5465/amr.1998.926617 · Cited by 10,274 · cited 2×
“…can nevertheless be understood as functionally equivalent to trust behaviors— even if not phenomenologically identical—particularly the subtype Rousseau et al. (1998, p. 399) term “calculus-based trust”: a rational, incentive-driven evaluation without psychological and affective components.”
Trust And Distrust: New Relationships and Realities
Lewicki, R. J., McAllister, D. J., & Bies, R. J. (1998) · The Academy of Management Review · doi:10.5465/amr.1998.926620 · Cited by 3,091 · cited 1×
“Accord-ingly, it has been argued that one can simultaneously trust and distrust the same entity in different respects (Lewicki et al. 1998).”
Capoccia, G., & Kelemen, R. D. (2007) · World Politics · doi:10.1017/s0043887100020852 · Cited by 2,410 · cited 1×
“…has also been adopted by comparative historical institutional scholars to deal with the concepts of path dependence and critical junctures, emphasizing that claims regarding institutional lock-in require the explicit identification of credible alternative trajectories (Capoccia and Kelemen 2007).”
Thomson, J. J. (1985) · The Yale Law Journal · doi:10.2307/796133 · Cited by 1,304 · cited 1×
“Thought experiments are famously employed in physics (Einstein et al. 1935; Schrödinger 1935) and in moral philosophy (Singer 1972; Thomson 1984), but they may serve as valuable analytical tools in political science and regulatory governance as well.”
In AI we trust? Perceptions about automated decision-making by artificial intelligence
Araujo, T., Helberger, N., Kruikemeier, S., & de Vreese, C. H. (2020) · AI & SOCIETY · doi:10.1007/s00146-019-00931-w · Cited by 1,038
Human-in-the-loop machine learning: a state of the art
Mosqueira-Rey, E., Hernández-Pereira, E., Alonso-Ríos, D., Bobes-Bascarán, J., & Fernández-Leal, Á. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10246-w · Cited by 1,025
Managing artificial intelligence
Berente, N., Gu, B., Recker, J., & Santhanam, R. (2021) · MIS Quarterly · doi:10.25300/misq/2021/16274 · Cited by 898 · cited 1×
“Second, the autonomy of the system could be tied to codified invariants (Berente et al. 2021; Wang 2021).”
Algorithmic regulation: A critical interrogation
Yeung, K. (2018) · Regulation & Governance · doi:10.1111/rego.12158 · Cited by 842 · cited 1×
“Yet , precisely because they rely heavily on feed-back, AI systems may also exhibit behaviors analogous to distrust when confronted with conflicting, incomplete, systematically biased, or unreliable inputs (Kordzadeh and Ghasemaghaei 2022; Yeung 2018).”
Research works printed but not linked: 31
- Anisimov AP, Ryzhenkov AJ (2016) Thirty years after the accident at the Chernobyl nuclear power plant: historical causes, lessons and legal effects. J Energy Nat Resour Law 34(3):265–283
- Bengio Y, Hinton G, Yao A, Song D, Abbeel P, Darrell T, Harari YN, Zhang Y-Q, Xue L, Shalev-Shwartz S (2024) Managing extreme AI risks amid rapid progress. Science 384(6698):842–845 · DOI · Paper page
- Bostrom N (2024) Superintelligence. Dunod, Malakoff
- Butlin P, Long R, Elmoznino E, Bengio Y, Birch J, Constant A, Deane G, Fleming SM, Frith C, Ji X (2023) Consciousness in artificial intelligence: insights from the science of consciousness. arXiv preprint arXiv:2308.08708 · arXiv
- Choudhury M, Dutta A, De AK (2023) Data corroboration of the catastrophic chernobyl tragedy using arc-length estimate conjecture. Vertices Duke’s Undergrad Res J 1(2):69–84
- Christian B (2021) The alignment problem: Machine learning and human values. London: WW Norton & Company.
- Downing T (2018) 1983: Reagan, Andropov, and a World on the Brink. Hachette, London
- Greenblatt R, Denison C, Wright B, Roger F, MacDiarmid M, Marks S, Treutlein J, Belonax T, Chen J, Duvenaud D (2024) Alignment faking in large language models. arXiv preprint arXiv:2412.14093 · arXiv
- Guha N, Lawrence CM, Gailmard LA, Rodolfa KT, Surani F, Bommasani R, Raji ID, Cuéllar M-F, Honigsberg C, Liang P (2024) Ai regulation has its own alignment problem: the technical and institutional feasibility of disclosure, registration, licensing, and auditing. Geo Wash L Rev 92:1473
- Hellemans M (2009) The safety relief valve handbook: design and use of process safety valves to ASME and International Codes and Standards. Elsevier, Amsterdam
- Hill CA, O’Connor EO (2006) A cognitive theory of trust. Wash U Law Rev 84(7):1717–1796
- Hoffman D (1999) I had a funny feeling in my gut. Washington Post 10:A19
- Ingram S (2005) The Chernobyl nuclear disaster. Infobase Publishing, New York
- Kaminski ME (2023) Regulating the risks of AI. BUL Rev 103:1347
- Malinauskas AP (1987) The Chernobyl accident: causes and consequences. Oak Ridge National Laboratory (ORNL), Oak Ridge
- Marcus G, Davis E (2019) Rebooting AI: building artificial intelligence we can trust. Vintage, New York
- McDermott D (2007) Artificial intelligence and consciousness. The Cambridge handbook of consciousness. Cambridge University Press, Cambridge, pp 117–150
- Meinke A, Schoen B, Scheurer J, Balesni M, Shah R, Hobbhahn M (2024) Frontier models are capable of in-context scheming. arXiv preprint arXiv:2412.04984 · arXiv
- Morgan CA (1985) The downing of Korean Air Lines Flight 007. Yale J Int Law 11:231
- Ord T (2020) The precipice: existential risk and the future of humanity. Hachette UK, London
- Santos O, Radanliev P (2024) Beyond the algorithm: AI, security, privacy, and ethics. Addison-Wesley Professional, Boston
- Scharowski N, Perrig SAC, Aeschbach LF, von Felten N, Opwis K, Wintersberger P, Brühlmann F (2024) To trust or distrust trust measures: Validating questionnaires for trust in AI. arXiv preprint arXiv:2403.00582 · arXiv
- Singer P (1972) Famine, affluence, and morality. Philos Public Aff 1:229–243
- Six F, Verhoest K (eds) (2017) Trust in regulatory regimes. Edward Elgar Publishing, Cheltenham
- Sunstein CR, Gaffe JH (2024) An anatomy of algorithm aversion. Colum Sci Technol L Rev 26:290
- Sztompka P (1999) Trust: a sociological theory. Cambridge University Press, Cambridge
- Tetlock P, Belkin A (1996) Counterfactual thought experiments in world politics: logical, methodological, and psychological perspectives. Princeton University Press, Princeton
- Wang Y-F, Chen Y-C, Chien S-Y, Wang P-J (2024) Citizens’ trust in AI-enabled government systems. Inf Polity 29(3):293–312
- Yablokov AV, Nesterenko VB, Nesterenko AV, Sherman-Nevinger JD (2010) Chernobyl: consequences of the catastrophe for people and the environment. Wiley, Hoboken
- Bresinsky M, Hager E & Xanthos G (2024). ChatGPT for futures: how large language models can support the development of future scenarios using the Cone of Plausibility. CCEW SYMPOSIUM 2024: Predictive Synergies: Crisis early earning & foresight, 19. und 20. September 2024, München
- Wang L (2021). Autonomy vs. artifi cial intelligence : studies on healthcare work and analytics. Doctoral thesis, Nanyang Technological University, Singapore. https://hdl.handle.net/10356/146910
Cited by
Hostile interaction design: AI, governance, and the quest for human oversight
Foth, M. (2026) · AI & SOCIETY · doi:10.1007/s00146-026-03065-y · Cited by 1
“What matters is how authority, responsibility, and decision-making are calibrated across human and machine actors under conditions of uncertainty and risk (Maggetti 2026).”
Üstündağ, B., Foth, M., & Seevinck, J. (2026) · AI & SOCIETY · doi:10.1007/s00146-026-03108-4
“…a safeguard must itself be treated with caution: rather than a straightforward solu-tion, it represents a complex governance challenge shaped by institutional design, competing logics of trust and distrust, and the distributed nature of control in AI systems (Laux 2024; Maggetti 2026; Foth 2026).”
Metadata
- Authors
- Maggetti, M.
- Affiliations
- University of Lausanne (CH)
- Corresponding author
- Maggetti, Martino
- ORCID
- Maggetti, Martino
- Venue
- AI & SOCIETY
- Published
- 2026-03-17
- Volume
- 41
- Issue
- 5
- Pages
- 5009-5026
- ISSN
- 0951-5666 (ISSN-L)
- 1435-5655
- PMC
- PMC13690886
- OpenAlex
- W7138063264
- Cited by
- 2
- References
- 83
- Citations by year
- 2026: 2
- FWCI
- 11.35
- Type
- journal article
- Language
- en
- Update notices
- correction (2026-05-07)
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- AI & Society (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University)
- Funders
- Open access funding provided by University of Lausanne Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (204521)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Physical Sciences › Computer Science › Computer Networks and Communications › Access Control and Trust
- Keywords
- AI regulation, trust in AI, distrust, mutual trust, AI agency, human-AI relationship, AI governance, democratic governance, trustworthiness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Reciprocal trust and distrust in artificial intelligence systems: the hard problem of regulation. https://app.cochat.ai/papers/10.1007/s00146-026-02966-2
- The paper
- Maggetti, M. (2026). Reciprocal trust and distrust in artificial intelligence systems: the hard problem of regulation. AI & SOCIETY. https://doi.org/10.1007/s00146-026-02966-2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.