CoChat

Trust, regulation, and human-in-the-loop AI: within the European region

Middleton, Stuart E., Letouzé, Emmanuel, Hossaini, Ali, Chapman, Adriane · 2022 · Communications of the ACM

The DOI matches its Crossref record, with a small difference in title or year; 21 of the 21 references it cites were found in a registry or an index.

Abstract

Artificial intelligence (AI) systems employ learning algorithms which adapt to their users and environment, with learning either pre-trained or allowed to adapt during deployment. Because AI can optimize its behaviour, a unit's factory model behaviour can diverge after release, often at the perceived expense of safety, reliability, and human controllability. Since the Industrial Revolution, trust has ultimately resided in regulatory systems set up by governments and standards bodies. Research into human interactions with autonomous machines demonstrates a shift in the locus of trust: we must trust non-deterministic systems such as AI to self-regulate, albeit within boundaries. This radical shift is one of the biggest issues facing the deployment of AI in the European region.Trust has no accepted definition, but [Rousseau 1998] define it as "a psychological state comprising the intention to accept vulnerability based upon positive expectations of the intentions or behaviour of another". Trust is an attitude that an agent will behave as expected and can be relied upon to reach its goal. Trust breaks down after an error or a misunderstanding between the agent and the trusting individual. The psychological state of trust in AI is an emergent property of a complex system, usually involving many cycles of design, training, deployment, measurement of performance, regulation, redesign and retraining.Trust matters, especially in critical sectors such as healthcare, defence & security where duty of care is foremost. Trustworthiness must be planned rather than an afterthought. We can trust in AI, such as when a doctor uses algorithms to screen medical images [NHS-X 2021]. We can also trust with AI, such as when journalists reference a social network algorithm to analyse sources of a news story [WeVerify 2021]. Growing adoption of AI into institutional systems relies on citizens to trust in these systems and have confidence in the way these systems are designed and regulated.Regional approaches for managing trust in AI have recently emerged, leading to different regulatory regimes in the United States, the European region and China. We review these regulatory divergences. Within the European region, research programs are examining how trust impacts user acceptance of AI. Examples include the UKRI Trustworthy Autonomous Systems Hub , the French Confidiance.ai project and the German AI Breakthrough Hub . Europe appears to be developing a "third way" alongside the United States and China [Morton 2021].Healthcare contains many examples of AI applications including online harm risk identification [ProTechThem 2021], mental health behaviour classification [SafeSpacesNLP 2021] and automated blood testing [Pinpoint 2021]. In defence and security, examples include combat management systems [DSTL 2021] and using machine learning to identify chemical and biological contamination [Alan Turing Institute 2021]. There is a growing awareness within critical sectors [Kerasidou 2020] [Taddeo 2019] that AI systems need to address a "public trust deficit" by adding reliability into the perception of AI. In the next two sections we discuss research highlights around the key trends of building safer and more reliable AI systems to engender trust and putting humans in the loop with regards AI systems and teams. We conclude with a discussion about applications and what we consider the future outlook is in this area.

Overview

Integrity signals

Registry

Crossref · 10.1145/3511597

Cites

21 of 21 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 21 references

Research works printed but not linked: 8

  1. 10.1145/3453444 · DOI · Paper page
  2. Barrett , P. , Hendrix , J. , and Sims , G . How tech platforms fuel U.S. political polarization and what government can do about it. The Brookings Institution 27 ( Nov. 2021 ); https://brook.gs/3sK3Cev Barrett, P., Hendrix, J., and Sims, G. How tech platforms fuel U.S. political polarization and what government can do about it. The Brookings Institution 27 (Nov. 2021); https://brook.gs/3sK3Cev
  3. Bhargava , R. , Deahl , E. , Letouzé , E. , Noonan , A. , Sangokoya , D. , and Shoup , N . Beyond Data Literacy: Reinventing Community Engagement and Empowerment in the Age of Data , Data-Pop Alliance White Paper. Sept. 29, 2015 ; https://bit.ly/3qNgBtm Bhargava, R., Deahl, E., Letouzé, E., Noonan, A., Sangokoya, D., and Shoup, N. Beyond Data Literacy: Reinventing Community Engagement and Empowerment in the Age of Data, Data-Pop Alliance White Paper. Sept. 29, 2015; https://bit.ly/3qNgBtm
  4. Ministry of Defense. Commander of Strategic Command RUSI conference speech , May 26, 2021; https://www.gov.uk/government/speeches/commander-of-strategic-command-rusi-conference-speech Ministry of Defense. Commander of Strategic Command RUSI conference speech, May 26, 2021; https://www.gov.uk/government/speeches/commander-of-strategic-command-rusi-conference-speech
  5. Morton , S. and Booth , M . The EU's "third way" to AI regulation. Pillsbury , Sept. 22 , 2021 ; https://www.internetandtechnologylaw.com/eu-third-way-ai-regulation/ Morton, S. and Booth, M. The EU's "third way" to AI regulation. Pillsbury, Sept. 22, 2021; https://www.internetandtechnologylaw.com/eu-third-way-ai-regulation/
  6. Schabus , D. The IEEE CertifAIEd Framework for AI Ethics Applied to the City of Vienna . IEEE Standards Assoc . ( 2021 ); https://bit.ly/3EZWzRp Schabus, D. The IEEE CertifAIEd Framework for AI Ethics Applied to the City of Vienna. IEEE Standards Assoc. (2021); https://bit.ly/3EZWzRp
  7. RUSI-TAS. Trusting Machines ? RUSI-TAS 2021 Conf.; https://www.tas.ac.uk/eventslist/trusting-machines/trust-machines-conference-programme/ RUSI-TAS. Trusting Machines? RUSI-TAS 2021 Conf.; https://www.tas.ac.uk/eventslist/trusting-machines/trust-machines-conference-programme/
  8. van der Valk , H. , Haße , H. , Möller , F. , Arbter , M. , Henning , J. , and Otto , B . A Taxonomy of digital twins . In Proceedings of AMCIS 2020; https://bit.ly/3HE6ZYI van der Valk, H., Haße, H., Möller, F., Arbter, M., Henning, J., and Otto, B. A Taxonomy of digital twins. In Proceedings of AMCIS 2020; https://bit.ly/3HE6ZYI

Cited by

See all 74

Metadata

Authors
Middleton, S. E., Letouzé, E., Hossaini, A., & Chapman, A.
Affiliations
  • University of Southampton (GB)
  • Universitat Pompeu Fabra (ES)
  • King's College London (GB)
Venue
Communications of the ACM
Published
2022-03-19
Volume
65
Issue
4
Pages
64-68
ISSN
  • 0001-0782 (ISSN-L)
  • 1557-7317
OpenAlex
W4220993791
Cited by
73
References
21
Citations by year
2026: 22 · 2025: 25 · 2024: 14 · 2023: 8 · 2022: 4
FWCI
14.78
Type
journal article
Language
en
Open access
closed
Locations
  • Communications of the ACM (publishedVersion)
  • ePrints Soton (University of Southampton) (acceptedVersion)
  • SOAS Research Online (SOAS University of London) (submittedVersion)
Funders
  • Engineering and Physical Sciences Research Council (EP/V00784X/1)
  • Natural Environment Research Council (NE/S015604/1)
  • Economic and Social Research Council (ES/V011278/1,ES/R003254/1)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
trust in AI, AI regulation, human-in-the-loop AI, autonomous systems, trustworthiness, AI safety, AI governance, European AI policy, United States AI policy, artificial intelligence in healthcare, AI acceptance, AI self-regulation, algorithmic transparency, explainability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Trust, regulation, and human-in-the-loop AI: within the European region. https://app.cochat.ai/papers/10.1145/3511597
The paper
Middleton, S. E., Letouzé, E., Hossaini, A., & Chapman, A. (2022). Trust, regulation, and human-in-the-loop AI: within the European region. Communications of the ACM. https://doi.org/10.1145/3511597

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.