Cited by 173Information Systems Theories and ImplementationEthics and Social Impacts of AIImpact of AI and Big Data on Business and Society

The DOI matches its Crossref record; 82 of the 82 references it cites were found in a registry or an index; 20 of the 48 sources its claims cite were checked for support.

Keeping the organization in the loop: a socio-technical extension of human-centered artificial intelligence

Open access
Herrmann, Thomas, Pfeiffer, Sabine·2023·AI & SOCIETY

Abstract

The human-centered AI approach posits a future in which the work done by humans and machines will become ever more interactive and integrated. This article takes human-centered AI one step further. It argues that the integration of human and machine intelligence is achievable only if human organizations—not just individual human workers—are kept “in the loop.” We support this argument with evidence of two case studies in the area of predictive maintenance, by which we show how organizational practices are needed and shape the use of AI/ML. Specifically, organizational processes and outputs such as decision-making workflows, etc. directly influence how AI/ML affects the workplace, and they are crucial for answering our first and second research questions, which address the pre-conditions for keeping humans in the loop and for supporting continuous and reliable functioning of AI-based socio-technical processes. From the empirical cases, we extrapolate a concept of “keeping the organization in the loop” that integrates four different kinds of loops: AI use, AI customization, AI-supported original tasks, and taking contextual changes into account. The analysis culminates in a systematic framework of keeping the organization in the loop look based on interacting organizational practices.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

The paper argues that human-centered AI must be extended to keep the organization—not just individual workers—in the loop, because organizational practices shape how AI/ML affects work. Using two predictive-maintenance case studies, it derives four organizational loops and a framework of six interacting practice categories.

Key findings

  • The authors identify six organizational categories needed for AI/ML embedding in predictive maintenance: leadership and HR as enablers; decision-making about AI outcome; coordination of original tasks supported with AI; coordinating with the external world; dealing with changes; and evolving the AI system.
  • They extrapolate four loops: UA-tasks (using and assessing AI output), CA-tasks (customizing the AI system), O-tasks (original tasks supported with AI), and C-tasks (contextual changes and factors in relation to AI).
  • Case A involved a German car manufacturer’s chassis manufacturing division with 1,500 employees; four interviews in 2017 yielded 77 specific challenges for PM integration. Case B involved an automotive factory with around 15,000 employees; the sociological study included 28 qualitative interviews, several workplace observations, four group discussions, and six feedback workshops.
  • The paper answers RQ1–RQ3 by arguing that keeping humans in the loop depends mainly on organizational practices, that reliable AI functioning requires continuous handling of external world and changes, and that a systematic framework must synchronize the four loops rather than treat organizational measures as isolated.

Methods

  • Interdisciplinary reanalysis of two qualitative case studies of AI/ML-based predictive maintenance in the German automotive industry: Case A was a bottom-up PM implementation studied from business informatics, with four interviews and company documentation; Case B was a top-down Industry 4.0 case studied sociologically, with 28 qualitative interviews, workplace observations, four group discussions, and six feedback workshops.
  • Interviews were transcribed and content-analyzed; the authors jointly re-examined both case materials through a cross-disciplinary re-interpretation combining business informatics and sociology.
  • The analysis grouped challenges into six organizational categories and developed two general models: four intertwined loops of organizational practices and a schematic framework of tasks, actors, and interacting organizational practices.

Limitations

  • The paper does not state a formal limitations section.
  • Evident from the text: the findings are based on two predictive-maintenance cases in the automotive sector, one bottom-up and one top-down, so generalization to other sectors and AI applications is limited; the medical image analysis example is described as deliberately tentative.
  • Evident from the text: the study reanalyzes existing case material originally collected for different research questions and disciplines, and the literature review notes a broader lack of reliable organizational-level data and qualitative case studies on AI use.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s00146-022-01391-5

Cites

82 of 82 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

92 Own interpretation · 35 No basis given · 12 Own result · 10 Partly supported · 10 No full text of the sources · 5 Supported · 4 Sources not found · 2 Supported so far · 1 Not supported · 1 Not checked yet · 1 Own method · accuracy not yet measured

Verify all 1 references — free account

Show all 55 claims

  • Probably not a substantial claim. 96
  • It may state two separate claims as one. 46
118 of 118
  • The authors state this as an assumption.

    For every domain of AI application, activities and the strengths needed to accomplish them have to be analyzed in the context of organizational practices.

    “Consequently, we assume that for every domain of AI application-such as predictive maintenance-activities and the strengths needed to accomplish them have to be analyzed in the context of organizational practices.”
  • The paper relies on this without support.

    Organizations are subject to their own logic, integrated into complex external environments, and divided internally by competition among departments with divergent interests.

    “Organizations-be they commercial enterprises or public institutions-are subject to their own logic, integrated into complex external environments, and divided internally by competition among departments with divergent interests.”
  • The paper relies on this without support.

    Decisions, whether technically or humanly generated, must be negotiated and processed within the organization.

    “No matter whether decisions are technically or humanly generated, they must be negotiated and processed within the organization.”
  • The paper relies on this without support.

    The organizational context surrounds human actors and affects the way they handle their tasks.

    “This organizational context surrounds human actors and affects the way they handle their tasks.”
  • The paper relies on this without support.

    Predictive maintenance can have consequences not only for the organization and its members, but also for production technology, product quality, delivery reliability, etc.

    “Precisely because PM can have consequences not only for the organization and its members, but also for production technology, product quality, delivery reliability, etc., we ask about the organizational conditions for this entire complex--and, also technical--system, not only for the impacts and consequences for the organization.”
  • The paper relies on this without support.

    In socio-technical systems, organizational practices and human-computer interaction are closely intertwined, shaping the roles humans play in their daily work.

    “In socio-technical systems, organizational practices and human-computer interaction are closely intertwined, shaping the roles humans play in their daily work-when interacting with technology as well as within organizational practices and processes.”
  • The paper relies on this without support.

    The two case studies of PM implementation provide a sufficient basis for deriving generic organizational requirements for the design of human-machine systems of AI-based systems.

    “The following analysis systematically traces these organizational categories based on both case studies and then discusses the generic organizational requirements for the design of human-machine systems of AI-based systems.”
  • The paper relies on this without support.

    Four feedback cycles are sufficient to ensure regulation, control, and supervision of AI-based processes.

    “we argue that ensuring regulation, control, and supervision of AI-based processes needs four feedback cycles.”
  • The paper relies on this without support.

    The six organizational categories identified from the PM cases are applicable to AI/ML use in organizations generally.

    “Below, we describe these categories in general terms that could also be valid for any organization.”
  • The paper relies on this without support.

    The two empirical cases in predictive maintenance are representative of AI-using organizations generally.

    “Based on sociological theory and two empirical cases in the field of predictive maintenance, the contribution of our study is to broaden human-centered AIdesign research by keeping the organization in the loop in analyses and design of AI use.”
10 of 10

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 75 references

Research works printed but not linked: 3

  1. Davenport T, Miller S (2021) Working with smart machines: insights on the future of work. Asian Manage Insights 8(1):18–25
  2. Kamar E (2016) Directions in Hybrid Intelligence: Complementing AI Systems with Human Intelligence. IJCAI, 4070–4073
  3. Silverman K (2020) Why your board needs a plan for AI oversight. MIT Sloan Manag Rev 62(1):1–6

Cited by

Sorted by citations

Metadata

Authors
Herrmann, T., & Pfeiffer, S.
Affiliations
  • Ruhr University Bochum (DE)
  • Friedrich-Alexander-Universität Erlangen-Nürnberg (DE)
Corresponding author
  • Herrmann, Thomas
  • Pfeiffer, Sabine
Venue
AI & SOCIETY
Published
2022-02-18
Volume
38
Issue
4
Pages
1523-1542
ISSN
  • 0951-5666 (ISSN-L)
  • 1435-5655
OpenAlex
W4213272379
Cited by
173
References
75
Citations by year
2026: 67 · 2025: 41 · 2024: 43 · 2023: 19 · 2022: 3
FWCI
38.13
Type
journal article
Language
en
Open access
Locations
AI & Society (publishedVersion, cc-by)
Funders
Ruhr-Universität Bochum
Topics
  • Social Sciences › Business, Management and Accounting › Management Information Systems › Information Systems Theories and Implementation
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Impact of AI and Big Data on Business and Society
Keywords
human-centered AI, socio-technical systems, organizational practices, predictive maintenance, human-machine interaction, context change

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review