From Oracles to Productive Friction: Designing for Meaningful Human Oversight in Clinical AI

May combine two claims.

Withholding AI assistance until after the user forms an independent judgment reduces overreliance compared to presenting it upfront, though at the cost of decision time.

“Withholding AI assistance until after the user forms an independent judgment reduces overreliance compared to presenting it upfront, though at the cost of decision time(Swaroop et al. 2024).”

• cites Swaroop et al. 2024 sourceSupported

May combine two claims.

The Oracular prototype retains merit for streamlining routine confirmations and quick verification in straightforward cases, but its prescriptive framing was consistently resisted under uncertainty.

“The Oracular prototype retains merit for streamlining routine confirmations and quick verification in straightforward cases, but its prescriptive framing was consistently resisted under uncertainty.”

May combine two claims.

Radiologists emphasised that outputs remain editable and accountable.

“Although auto-generated text can save time, radiologists emphasised that outputs remain editable and accountable (P8, P2).”

May combine two claims.

Behavioural Intention (r = .42) and Effort Expectancy (r = .34) showed smaller effects, indicating less differentiation between prototypes in adoption intent and perceived ease of use.

“Behavioural Intention (r = .42)and Effort Expectancy (r = .34)showed smaller effects, indicating less differentiation between prototypes in adoption intent and perceived ease of use.”

May combine two claims.

Experts preferred fast confirmation, while trainees stood to benefit from exploratory systems that supported learning.

“Differences across expertise levels reinforced the pattern.Experts preferred fast confirmation (P2), while trainees stood to benefit from exploratory systems that supported learning (P8).”

May combine two claims.

Oracular outputs were valued for speed in routine cases but resisted when they foreclosed deliberation under uncertainty.

“Evaluative, differential-first interfaces activated reflection, preserved professional agency, and aligned with the probabilistic logic of radiological diagnosis, while oracular outputs were valued for speed in routine cases but resisted when they foreclosed deliberation under uncertainty.”

May combine two claims.

The largest effects appeared for Facilitating Conditions (Mdn = 5.0 vs. 3.4, r = .85) and Job Relevance (r = .80).

“The largest effects appeared for Facilitating Conditions (Mdn = 5.0 vs. 3.4, r = .85)and Job Relevance (r = .80),indicating that radiologists perceived the evaluative paradigm as substantially better aligned with their work context and professional needs.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants reported generally low levels of technology-related anxiety (mean 1.37, SD 0.38).

“The mean score was 1.37 (SD = 0.38), indicating that respondents overall did not feel anxious about engaging with new technology, including AI systems.”

May combine two claims.

Context-specific preferences resolved into a stance of selective reliance, with AI as a guide in unfamiliar domains but secondary to one's own judgement in routine cases.

“These contextspecific preferences resolved into a stance of selective reliance, with AI as a guide in unfamiliar domains, but secondary to one's own judgement in routine cases (P6).”

May combine two claims.

Trainees benefited from the exploratory structure, while experienced radiologists sometimes found multiple options redundant for routine tasks.

“Trainees in particular benefited from the exploratory structure in ways that experienced practitioners did not need; experienced radiologists, by contrast, sometimes found multiple options redundant for routine tasks (P2), reinforcing the case for adaptivity rather than uniform depth.”

May combine two claims.

The Evaluative prototype was favoured for complex diagnostic cases involving uncertainty, ambiguous or atypical findings, and learning contexts.

“Instead, the Evaluative prototype was favoured for complex di-agnostic cases involving uncertainty, ambiguous or atypical findings, and learning contexts.”

May combine two claims.

Medium-to-large effects were found for Anxiety (r = .67), Result Demonstrability (r = .63), Performance Expectancy (r = .62), and Attitude toward Use (r = .55).

“Medium-to-large effects were also found for Anxiety (r = .67),Result Demonstrability (r = .63),Performance Expectancy (r = .62),and Attitude toward Use (r = .55).”

May combine two claims.

The Oracular prototype was preferred for simple, binary decisions such as fracture detection, routine straightforward cases, time-critical ER triage, and high-volume repetitive tasks.

“The Oracular prototype was preferred for simple, binary decisions such as fracture detection, routine straightforward cases, time-critical ER triage, and high-volume repetitive tasks.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants proposed the inclusion of an AI triage lane to the worklist.

“To further support their workflow, participants proposed the inclusion of an AI triage lane to the worklist.”

May combine two claims.

Participants asked for tools that fit into their existing environment, expose their reasoning, preserve agency, and support both fast confirmation and deeper learning.

“Participants asked for tools that fit into their existing environment, expose their reasoning, preserve agency, and support both fast confirmation and deeper learning.”

May combine two claims.

The Evaluative prototype demonstrated superior overall alignment with radiological practice, supported by consistent positive quantitative contrasts across Performance Expectancy, Result Demonstrability, and Facilitating Conditions.

“The Evaluative prototype demonstrated superior overall alignment with radiological practice, supported by consistent positive quantitative contrasts across Performance Expectancy (r = .62),Result Demonstrability (r = .63),and Facilitating Conditions (r = .85).”

May combine two claims.

Evaluative, differential-first interfaces activated reflection, preserved professional agency, and aligned with the probabilistic logic of radiological diagnosis.

“Evaluative, differential-first interfaces activated reflection, preserved professional agency, and aligned with the probabilistic logic of radiological diagnosis, while oracular outputs were valued for speed in routine cases but resisted when they foreclosed deliberation under uncertainty.”

May combine two claims.

Quantitative measures revealed a conditional but favourable disposition toward the Evaluative prototype, with effect sizes consistently favouring the Evaluative paradigm.

“Quantitative measures revealed a conditional but favourable disposition toward the Evaluative prototype.9 Effect sizes consistently favoured the Evaluative paradigm.”

Probably not a substantial claim: may not pass S2 (checkable).

The four productive friction dimensions can be read as designable operationalisations of what meaningful oversight demands in practice.

“The four productive friction dimensions we propose (temporal delay, option cardinality, evidence depth, and commitment structure) can be read as designable operationalisations of what meaningful oversight demands in practice, translating a normative aspiration into concrete interface proper-ties that determine whether a clinician's engagement with an AI recommendation is substantive or ceremonial.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Ethical approval was granted and informed consent obtained from all participants.

“Ethical approval was granted by the university's ethics committee, and informed consent, including permission for audio and video recording, was obtained from all participants before the study.”

May combine two claims.

The evaluative prototype fit the multi-step reasoning sequence radiologists already follow, whereas the oracular prototype's single output was experienced as a shortcut that bypassed established verification steps.

“The large Facilitating Conditions effect (r = .85)is consistent with this distribution: the evaluative prototype fit the multi-step reasoning sequence radiologists already follow, whereas the oracular prototype's single output was experienced as a shortcut that bypassed, rather than supported, established verification steps.”

Classified as probably not a substantial claim. May combine two claims.

Cases were assigned to the two prototypes using a fixed, balanced design, in which each prototype was tested on the same 10 cases across all participants, with both sets matched for baseline complexity.

“Cases were assigned to the two prototypes using a fixed, balanced design, in which each prototype was tested on the same 10 cases across all participants, with both sets matched for baseline complexity.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Participants gave their consent to participate voluntarily in the case study that was approved by the SUPSI ethical committee.

“Participants gave their consent to participate voluntarily in the case study that was approved by the SUPSI ethical committee.”

Probably not a substantial claim: may not pass S2 (checkable).

Productive friction is an ethical intervention that operates through usability.

“Productive friction, on this account, is thus an ethical intervention that operates through usability.”

Classified as probably not a substantial claim.

Only the explanation and interaction structure of the cases differed between prototypes.

“Only the explanation and interaction structure of the cases differed.”

May combine two claims.

This fixed assignment may introduce a confound between case content and interaction type; although matching cases by complexity reduces this risk, it does not fully remove it.

“This fixed assignment may introduce a confound between case content and interaction type; although matching cases by complexity reduces this risk, it does not fully remove it.”

Classified as probably not a substantial claim. May combine two claims.

A differential-first interface presenting a small set of likely diagnoses with discriminative cues better mirrors how clinicians reason and report, while encouraging breadth before commitment.

“A differential-first interface presenting a small set of likely diagnoses with discriminative cues better mirrors how clinicians reason and report, while encouraging breadth before commitment.”

May combine two claims.

Participants' rejection of opaque outputs, together with the observation that no participant asked for all evidence simultaneously, supports progressive disclosure.

“Participants' rejection of opaque outputs (P4, P8, P10), together with the observation that no participant asked for all evidence simultaneously, supports progressive disclosure, with a concise default view showing the hypothesis and its confidence, that can be expanded on demand to salient image regions, discriminative cues, and case-linked exemplars.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Frictionless compliance optimises throughput at the cost of reflective engagement, while deliberative friction preserves judgement at the cost of speed.

“where frictionless compliance optimises throughput at the cost of reflective engagement, while deliberative friction preserves judgement at the cost of speed.”

Classified as probably not a substantial claim.

A balanced dataset of 23 TC slices representing varying diagnostic complexity was curated.

“A balanced dataset of 23 TC slices representing varying diagnostic complexity was curated.”

Classified as probably not a substantial claim. May combine two claims.

The design-probe approach captures preferences and reasoning patterns rather than behavioural outcomes, and ecological validity is necessarily limited for this proof-of-concept.

“The design-probe approach captures preferences and reasoning patterns rather than behavioural outcomes, and ecological validity is necessarily limited for this proof-of-concept.”

• AssumptionRests on an assumption: Participants' self-reported preferences and think-aloud accounts reflect their actual clinical reasoning and would generalise to real clinical settings.

Probably not a substantial claim: may not pass S3 (load-bearing).

The order in which participants used the prototypes was counterbalanced.

“The order in which participants used the prototypes was counterbalanced.”

Probably not a substantial claim: may not pass S2 (checkable).

Productive friction reveals epistemic limits of what the AI and the clinician can know about a case, unlike seamful design.

“And unlike seamful design (Chalmers 2003;Inman and Ribes 2019), which reveals infrastructural boundaries, productive friction reveals epistemic ones, the limits of what the AI (and the clinician) can know about a given case.”

• cites Chalmers 2003Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Inman and Ribes 2019 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S2 (checkable).

The question of who controls friction intensity is ethically charged.

“Second, the question of who controls friction intensity is itself ethically charged.”

May combine two claims.

The study used a mixed-methods, within-subject proof-of-concept design with practising radiologists.

“To address our research questions, we conducted a mixedmethods, within-subject proof-of-concept study with practising radiologists.”

Probably not a substantial claim: may not pass S4 (based).

In high-stakes clinical AI, the ease of accepting a recommendation determines whether the human decision-maker has exercised the judgement on which moral and legal accountability depends.

“In high-stakes clinical AI, the ease of accepting a recommendation determines whether the human decision-maker has exercised the judgement on which moral and legal accountability depends.”

Classified as probably not a substantial claim.

Neither the philosophical account nor the regulation specifies how meaningfulness should be evidenced at the interface level.

“However, neither the philosophical account nor the regulation specifies how meaningfulness should be evidenced at the interface level.”

May combine two claims.

A single 'answer' risks recasting the AI as the primary source of truth, diminishing the radiologist's visible contribution and eroding agency, skill display, and professional recognition.

“A single 'answer' risks recasting the AI as the primary source of truth, diminishing the radiologist's visible contribution and eroding agency, skill display, and professional recognition (Natali et al. 2025a).”

• cites Natali et al. 2025aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

The combination of a large Facilitating Conditions effect and a more modest Behavioural Intention effect reflects that the evaluative paradigm was seen as a strong structural fit for clinical reasoning, but adoption remained contingent on conditions neither prototype fully satisfied.

“The combination of a large Facilitating Conditions effect (r = .85)with a more modest Behavioural Intention effect (r = .42)thus reflects a coherent pattern: the evaluative paradigm was seen as a strong structural fit for clinical reasoning, but adoption remained contingent on conditions that neither prototype yet fully satisfied.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Two researchers applied categories independently, comparing interpretations and resolving discrepancies.

“Two researchers applied these categories independently, comparing interpretations and resolving discrepancies.”

Classified as probably not a substantial claim.

The four dimensions of productive friction capture both the design variables the two prototypes manipulate and the themes participants spontaneously raised.

“These dimensions capture both the design variables our two prototypes manipulate and the themes participants spontaneously raised.”

• AssumptionRests on an assumption: The two simulated prototypes isolate interaction style rather than algorithmic performance.

Classified as probably not a substantial claim.

Given the small sample size (N=11), quantitative data were used to corroborate qualitative findings rather than support independent inferential claims.

“Given the small sample size (N = 11), the quantitative data were used to corroborate the findings emerging from the qualitative analysis rather than to support independent inferential claims.”

Classified as probably not a substantial claim.

In both modes, adaptive commitment ensures that finalising a diagnosis feels like a deliberate choice.

“In both modes, adaptive commitment ensures that finalising a diagnosis feels like a deliberate choice.”

Classified as probably not a substantial claim.

The statements from participants constitute phenomenological evidence of the conditions under which clinicians experience themselves as responsible agents versus ratifiers of machine output.

“Read through an ethical lens, these statements constitute phenomenological evidence of the conditions under which clinicians experience themselves as responsible agents versus ratifiers of machine output.”

May combine two claims.

The four dimensions can each be tuned independently and must be tuned, since their optimal settings vary with case complexity, time pressure, and clinician expertise.

“Read this way, the prototype contrast is evidence that the four dimensions can each be tuned independently and must be tuned, since their optimal settings vary with case complexity, time pressure, and clinician expertise.”

Classified as probably not a substantial claim.

Productive friction is graded and context-sensitive, varying with case complexity and user expertise, unlike cognitive forcing functions.

“Unlike cognitive forcing functions (Buc ¸inca, Malaya, and Gajos 2021), which impose a uniform deliberation barrier before any AI suggestion, productive friction is graded and context-sensitive, varying with case complexity and user expertise.”

• cites Buc ¸inca, Malaya, and Gajos 2021Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

Grote and Berens (2020) argued that optimising for compliance efficiency in high-stakes automated systems may systematically undermine the human judgement that regulation assumes is present.

“Grote and Berens ( 2020) have argued that optimising for compliance efficiency in high-stakes automated systems may systematically undermine the human judgement that regulation assumes is present.”

• cites Grote and Berens ( 2020) sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

If agreeing with an AI requires less effort than disagreeing, the decision may be formally human but substantively automated.

“If agreeing with the AI requires less effort than disagreeing, the decision may be formally human but substantively automated.”

May combine two claims.

Phase 0 was conducted remotely and independently two weeks prior to the main study, helping to mitigate anchoring or recall effects.

“This phase was conducted remotely and independently by participants two weeks prior to the main study, helping to mitigate anchoring or recall effects (Cockburn, Quinn, and Gutwin 2017).”

• cites Cockburn, Quinn, and Gutwin 2017 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S2 (checkable).

Efficiency and agency constitute a design tension to be managed through adaptive, context-sensitive support that calibrates deliberation cost to diagnostic uncertainty.

“Efficiency and agency thus constitute a design tension to be managed through adaptive, context-sensitive support that calibrates deliberation cost to diagnostic uncertainty.”

Classified as probably not a substantial claim.

The reflective engagement participants described when forming an impression before AI output translates into a 'think-first' gate that withholds AI output until the clinician has recorded an initial impression.

“The reflective engagement participants described when forming an impression before AI output (P3, P8) translates into a 'think-first' gate that withholds AI output until the clinician has recorded an initial impression.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

System-set thresholds introduce meta-level automation risk, while clinician-controlled settings may incentivise self-serving minimisation under time pressure.

“System-set thresholds introduce meta-level automation risk, while clinician-controlled settings may incentivise self-serving minimisation under time pressure.”

May combine two claims.

The first form of friction obstructs reasoning, whereas the second activates it.

“The first form of friction obstructs reasoning, whereas the second activates it.”

May combine two claims.

The appropriate paradigm is contingent on case complexity and clinician experience, motivating decision support that can adaptively offer either mode.

“Overall, our findings indicate that the appropriate paradigm is contingent on case complexity and clinician experience, motivating decision support that can adaptively offer either mode.”

May combine two claims.

Productive friction's primary goal is in-the-moment decision quality rather than long-term retention, though the data suggest pedagogical benefits as a secondary effect.

“Unlike desirable difficulty (Bjork and Bjork 2011), its primary goal is in-the-moment decision quality rather than long-term retention, though our data suggest pedagogical benefits as a secondary effect.”

• cites Bjork and Bjork 2011Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

Radiologists produce diagnoses while also enacting a role that carries authority and accountability for patient care.

“Radiologists produce diagnoses while also enacting a role that carries authority and accountability for patient care.”

Classified as probably not a substantial claim.

Each participant engaged with both systems in randomised order during think-aloud sessions.

“Each participant engaged with both systems in randomised order during think-aloud sessions, followed by an adapted UTAUT survey and a semistructured interview.”

May combine two claims.

The source of residual unease differed: in the oracular condition it stemmed from prescriptive authority without explanation; in the evaluative condition it stemmed from the cognitive cost of comparing multiple options.

“In the oracular condition it stemmed from prescriptive authority without explanation; in the evaluative condition it stemmed from the cognitive cost of comparing multiple options, a qualitatively different and arguably more productive form of friction, whose implications we return to under RQ3.”

Probably not a substantial claim: may not pass S3 (load-bearing).

There is a counter-tradition in HCI that treats friction as a resource rather than a defect.

“These observations connect to a counter-tradition in HCI that treats friction as a resource rather than a defect, from reflective design (Sengers et al. 2005) to slow technology (Hallnäs and Redström 2001) and seamful design (Chalmers 2003;Inman and Ribes 2019;Ehsan et al. 2024).”

• cites Sengers et al. 2005 sourceNot checked — no full text of the source
• cites Hallnäs and Redström 2001Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Chalmers 2003Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

Qualitative analysis used hybrid inductive-deductive thematic analysis following Braun and Clarke's guidelines.

“We analysed the interview and think-aloud transcripts using a hybrid inductive-deductive thematic analysis, following Braun and Clarke's guidelines (Braun and Clarke 2021).”

• cites Braun and Clarke 2021Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Fixed case-to-prototype assignment introduces a potential confound between case content and interaction paradigm; baseline complexity matching mitigates but does not eliminate this threat.

“Finally, fixed case-to-prototype assignment introduces a potential confound between case content and in-teraction paradigm; baseline complexity matching mitigates but does not eliminate this threat.”

• AssumptionRests on an assumption: The case-complexity matching via baseline classification adequately balances cases across prototypes.