Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 144Human-Automation Interaction and SafetyTeam Dynamics and PerformanceDecision Support System Applications

The DOI matches its Crossref record; 81 of the 82 references it cites were found in a registry or an index, and 1 are not checked yet; none of the 13 sources its claims cite is checked for support yet.

Enhancing AI-Assisted Group Decision Making through LLM-Powered Devil's Advocate

Open access
Chiang, Chun-Wei, Lu, Zhuoran, Li, Zhuoyan, Yin, Ming·2024·Proceedings of the 29th International Conference on Intelligent User Interfaces
DOI 10.1145/3640543.3645199
Open this paper in CoChat — free accountOpen PDF

Abstract

Group decision making plays a crucial role in our complex and interconnected world. The rise of AI technologies has the potential to provide data-driven insights to facilitate group decision making, although it is found that groups do not always utilize AI assistance appropriately. In this paper, we aim to examine whether and how the introduction of a devil’s advocate in the AI-assisted group decision making processes could help groups better utilize AI assistance and change the perceptions of group processes during decision making. Inspired by the exceptional conversational capabilities exhibited by modern large language models (LLMs), we design four different styles of devil’s advocate powered by LLMs, varying their interactivity (i.e., interactive vs. non-interactive) and their target of objection (i.e., challenge the AI recommendation or the majority opinion within the group). Through a randomized human-subject experiment, we find evidence suggesting that LLM-powered devil’s advocates that argue against the AI model’s decision recommendation have the potential to promote groups’ appropriate reliance on AI. Meanwhile, the introduction of LLM-powered devil’s advocate usually does not lead to substantial increases in people’s perceived workload for completing the group decision making tasks, while interactive LLM-powered devil’s advocates are perceived as more collaborating and of higher quality. We conclude by discussing the practical implications of our findings.

Checked 2026-11-14 · 1 reference not checked yet

Overview

CoChat Summary

TL;DR

This paper tests LLM-powered devil’s advocates in AI-assisted group decision making through a randomized Prolific experiment. It finds that an interactive devil’s advocate challenging AI recommendations improves groups’ appropriate reliance and accuracy, while challenging the majority opinion has limited effects and interactive advocates are perceived as more collaborating and higher quality.

Key findings

  • Groups in the Dynamic-AI treatment had significantly higher standardized decision accuracy than Control (β = 0.135, SE = 0.068, 95% CI [0.002, 0.269], p = 0.047); Static-AI marginally increased reliance on correct AI recommendations (β = 0.200, SE = 0.107, 95% CI [−0.010, 0.410], p = 0.062).
  • Target of objection had a marginal main effect on standardized accuracy (F(1, 92) = 3.500, p = 0.064, η² = 0.029), with AI-target higher (M = 0.042, SD = 0.261) than majority-target (M = −0.036, SD = 0.227); interactive devil’s advocates marginally reduced reliance on incorrect AI versus non-interactive (M = −0.013, SD = 0.351 vs. M = 0.029, SD = 0.252; F(1, 91) = 3.554, p = 0.063, η² = 0.038).
  • Dynamic-AI’s accuracy increase mainly occurred on in-distribution tasks (β = 0.122, SE = 0.067, 95% CI [−0.009, 0.254], p = 0.069); Static-AI’s increase in reliance on correct AI mainly came from out-of-distribution tasks (β = 0.619, SE = 0.355, 95% CI [−0.077, 1.315], p = 0.081).
  • Interactive devil’s advocates were perceived as more collaborating (F(1, 268) = 9.757, p = 0.002, η² = 0.035) and of higher quality (F(1, 268) = 11.211, p = 0.001, η² = 0.040); Dynamic-AI participants reported the lowest self-perceived performance (M = 4.107, SD = 0.791) and lower perceived teammate precision/usefulness than Control or Static-Majority.

Methods

  • Randomized human-subject experiment on Prolific: 350 participants formed 120 groups (Control: 24, Static-AI: 28, Static-Majority: 23, Dynamic-AI: 28, Dynamic-Majority: 17), completing 8 formal recidivism risk prediction tasks using COMPAS profiles and AI model RiskComp.
  • Five treatments: Control plus a 2×2 LLM-powered devil’s advocate design varying target of objection (AI recommendation vs. majority opinion) and interactivity (static vs. dynamic), implemented with OpenAI GPT-3.5-turbo.
  • Main analysis used linear regression with Control as reference, two-way ANOVA for design factors, one-way ANOVA for perceptions, and qualitative coding of chat logs (Cohen’s Kappa = 0.75); outcomes included standardized accuracy/reliance z-scores, NASA-TLX workload, teamwork quality, and devil’s advocate perceptions.

Limitations

  • Stated: online setting may not generalize to in-person group decision making; anonymity, one-shot collaborations, no domain expertise, and one recidivism domain limit generalizability; no data on dropouts; the intentionally biased AI model limits generalizability to other biases or performance levels.
  • Stated/evident: the LLM devil’s advocate sometimes repeated arguments, suggested unavailable features, and had time lag/out-of-context responses in multi-person chat; participants sometimes ignored, mocked, or expressed frustration toward it.
  • Evident: groups could end with only two members due to dropout/removal, and effects on out-of-distribution over-reliance were not statistically significant.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3640543.3645199

Cites

81 of 82 references verified0 citation errors · 0 not checkable

Statistics

7 tests recomputed · 0 with the reported and recomputed p on opposite sides of .05

Figures

Figure 1 The task interface used in the formal task interface of our experiment, and (A) an example of the chat log reflecting the discussion in the Static-AI treatment, and (B) an example of the chat log reflecting the discussion in the Dynamic-Majority treatment. (A): In the Static-AI treatment, the LLM-powered devil's advocate (displayed as a red skull) asked three questions to criticize the AI model's decision recommendation at the beginning of the discussion. (B): In the Dynamic-Majority treatment, the LLM-powered devil's advocate actively responds to group members' arguments and challenges the majority opinion within the group.
Figure 1 The task interface used in the formal task interface of our experiment, and (A) an example of the chat log reflecting the discussion in the Static-AI treatment, and (B) an example of the chat log reflecting the discussion in the Dynamic-Majority treatment. (A): In the Static-AI treatment, the LLM-powered devil's advocate (displayed as a red skull) asked three questions to criticize the AI model's decision recommendation at the beginning of the discussion. (B): In the Dynamic-Majority treatment, the LLM-powered devil's advocate actively responds to group members' arguments and challenges the majority opinion within the group.CC-BY · From the PDF
Figure 2 Our experiment had two phases. Phase 1 focused on participant recruiting, while Phase 2 was the actual experiment. During Phase 2, we introduced the LLM-powered devil's advocate in the group decision making processes of the formal tasks for participants in all but the Control treatment.
Figure 2 Our experiment had two phases. Phase 1 focused on participant recruiting, while Phase 2 was the actual experiment. During Phase 2, we introduced the LLM-powered devil's advocate in the group decision making processes of the formal tasks for participants in all but the Control treatment.CC-BY · From the PDF
Figure 3 Estimated coefficients from linear regression models for predicting a group's (a) standardized accuracy, (b) standardized reliance on correct AI recommendations, and (c) standardized reliance on incorrect AI recommendations. The error bars indicate the 95% confidence intervals. For standardized accuracy and standardized reliance on correct AI recommendations, an interval above zero is better; for standardized reliance on incorrect AI recommendations, an interval below zero is better.
Figure 3 Estimated coefficients from linear regression models for predicting a group's (a) standardized accuracy, (b) standardized reliance on correct AI recommendations, and (c) standardized reliance on incorrect AI recommendations. The error bars indicate the 95% confidence intervals. For standardized accuracy and standardized reliance on correct AI recommendations, an interval above zero is better; for standardized reliance on incorrect AI recommendations, an interval below zero is better.CC-BY · From the PDF
Figure 4 Estimated coefficients from linear regression models for predicting a group's (a) standardized accuracy on all indistribution tasks, (b) standardized reliance on in-distribution tasks where the AI recommendation is correct, and (c) standardized reliance on in-distribution tasks where the AI recommendation is wrong. The error bars indicate the 95% confidence intervals. For standardized accuracy and standardized reliance on correct AI recommendations, an interval above zero is better; for standardized reliance on incorrect AI recommendations, an interval below zero is better.
Figure 4 Estimated coefficients from linear regression models for predicting a group's (a) standardized accuracy on all indistribution tasks, (b) standardized reliance on in-distribution tasks where the AI recommendation is correct, and (c) standardized reliance on in-distribution tasks where the AI recommendation is wrong. The error bars indicate the 95% confidence intervals. For standardized accuracy and standardized reliance on correct AI recommendations, an interval above zero is better; for standardized reliance on incorrect AI recommendations, an interval below zero is better.CC-BY · From the PDF
Figure 5 Estimated coefficients from linear regression models for predicting a group's (a) standardized accuracy on all outof-distribution tasks, (b) standardized reliance on out-of-distribution tasks where the AI recommendation is correct, and (c) standardized reliance on out-of-distribution tasks where the AI recommendation is wrong. The error bars indicate the 95% confidence intervals. For standardized accuracy and standardized reliance on correct AI recommendations, an interval above zero is better; for standardized reliance on incorrect AI recommendations, an interval below zero is better.
Figure 5 Estimated coefficients from linear regression models for predicting a group's (a) standardized accuracy on all outof-distribution tasks, (b) standardized reliance on out-of-distribution tasks where the AI recommendation is correct, and (c) standardized reliance on out-of-distribution tasks where the AI recommendation is wrong. The error bars indicate the 95% confidence intervals. For standardized accuracy and standardized reliance on correct AI recommendations, an interval above zero is better; for standardized reliance on incorrect AI recommendations, an interval below zero is better.CC-BY · From the PDF
Figure 6 Participants' responses on their perceived workload in completing the AI-assisted group decision making tasks using the NASA-TLX questionnaire, as well as their perceived teamwork quality within the group. All responses are given on a 5-point Likert scale. Error bars represent the 95% confidence interval of the mean value.
Figure 6 Participants' responses on their perceived workload in completing the AI-assisted group decision making tasks using the NASA-TLX questionnaire, as well as their perceived teamwork quality within the group. All responses are given on a 5-point Likert scale. Error bars represent the 95% confidence interval of the mean value.CC-BY · From the PDF
Figure 7 Participants' responses on their perception of the LLM-powered devil's advocate. All responses are given on a 5-point Likert scale. Error bars represent the 95% confidence interval of the mean value.
Figure 7 Participants' responses on their perception of the LLM-powered devil's advocate. All responses are given on a 5-point Likert scale. Error bars represent the 95% confidence interval of the mean value.CC-BY · From the PDF

Verification

Claims the paper makes

0 Valid · 0 Not valid · 9 Not checked · 80 Own results and interpretations

42 more claims

Create a free account to see all 89 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 29
  • The quote only partly states this claim. 1
  • Probably not a substantial claim. 19

Create a free account to see these claims.

Create a free account to see these assumptions.

Statistics that don’t add up

Each reported p-value recomputed from its test statistic and degrees of freedom.

1 of 1

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Doesn’t add upF(1, 92) = 3.5reported p = .064 · recomputed p = .065

    “F (1, 92) = 3.500, p = .064”

  • Not confirmedF(4, 328) = 2.417reported p = .048 · recomputed p = .049

    “F (4, 328) = 2.417, p = .048”

  • ConsistentF(1, 91) = 3.554reported p = .063 · recomputed p = .063

    “F (1, 91) = 3.554, p = .063”

  • ConsistentF(4, 328) = 3.396reported p = .01 · recomputed p = .010

    “F (4, 328) = 3.396, p = .010”

  • ConsistentF(4, 328) = 4.813reported p < .001 · recomputed p < .001

    “F (4, 328) = 4.813, p < .001”

5 of 7 · the rest aren’t loaded

2 more recomputed tests

Create a free account to see every test the Stats check recomputed.

See all tests — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Self-check results appear here once an independent check has confirmed them.

Citations

Cites

Show all 81 references

Research works printed but not linked: 34

  1. Ramon J Aldag and Sally R Fuller. 1993. Beyond fiasco: A reappraisal of the groupthink phenomenon and a new model of group decision processes.Psychological bulletin 113, 3 (1993), 533.
  2. Kiana Alikhademi, Emma Drobina, Diandra Prioleau, Brianna Richardson, Duncan Purves, and Juan E Gilbert. 2022. A review of predictive policing from the perspective of fairness. Artificial Intelligence and Law (2022), 1–17.
  3. Hussam Alkaissi and Samy I McFarlane. 2023. Artificial hallucinations in ChatGPT: implications in scientific writing. Cureus 15, 2 (2023).
  4. Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine Bias. ProPublica (2016). URL: https://www. propublica. org/article/machine-bias-risk-asses sments-in-criminal-sentencing (2016).
  5. Chun-Wei Chiang and Ming Yin. 2022. Exploring the Effects of Machine Learning Literacy Interventions on Laypeople’s Reliance on Machine Learning Models. In 27th International Conference on Intelligent User Interfaces. 148–161.
  6. Mustafa Demir, Nathan J McNeese, and Nancy J Cooke. 2016. Team communication behaviors of the human-automation teaming. In 2016 IEEE international multi-disciplinary conference on cognitive methods in situation awareness and decision support (CogSIMA). IEEE, 28–34.
  7. Berkeley J Dietvorst, Joseph P Simmons, and Cade Massey. 2015. Algorithm aversion: people erroneously avoid algorithms after seeing them err.Journal of Experimental Psychology: General 144, 1 (2015), 114.
  8. Julia Dressel and Hany Farid. 2018. The accuracy, fairness, and limits of predicting recidivism. Science advances 4, 1 (2018), eaao5580.
  9. Danielle Ensign, Sorelle A Friedler, Scott Neville, Carlos Scheidegger, and Suresh Venkatasubramanian. 2018. Runaway feedback loops in predictive policing. In Conference on fairness, accountability and transparency. PMLR, 160–171.
  10. Nicolas Fay, Simon Garrod, and Jean Carletta. 2000. Group discussion as interactive dialogue or as serial monologue: The influence of group size. Psychological science 11, 6 (2000), 481–486.
  11. Adrian Furnham and Hua Chu Boo. 2011. A literature review of the anchoring effect. The journal of socio-economics 40, 1 (2011), 35–42.
  12. Daniel Gigone and Reid Hastie. 1993. The common knowledge effect: Information sharing and group judgment.Journal of Personality and social Psychology 65, 5 (1993), 959.
  13. Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139–183.
  14. Tatsuya Kameda and Shinkichi Sugimori. 1993. Psychological entrapment in group decision making: An assigned decision rule and a groupthink phenomenon.Journal of personality and social psychology 65, 2 (1993), 282.
  15. Antino Kim, Mochen Yang, and Jingjng Zhang. 2020. When Algorithms Err: Differential Impact of Early vs. Late Errors on Users’ Reliance on Algorithms. Late Errors on Users’ Reliance on Algorithms (July 2020) (2020).
  16. Colin MacDougall and Frances Baum. 1997. The devil’s advocate: A strategy to avoid groupthink and stimulate discussion in focus groups. Qualitative health research 7, 4 (1997), 532–541.
  17. Richard O Mason. 1969. A dialectical approach to strategic planning. Management science 15, 8 (1969), B–403.
  18. David G Myers and George D Bishop. 1971. Enhancement of dominant attitudes in group discussion.Journal of personality and social psychology 20, 3 (1971), 386.
  19. Samir Passi and Mihaela Vorvoreanu. 2022. Overreliance on AI Literature Review. Microsoft Research (2022).
  20. Dean G Pruitt. 1971. Choice shifts in group discussion: An introductory review.Journal of personality and social psychology 20, 3 (1971), 339.
  21. Fabian Reinkemeier, Ulrich Gnewuch, and Waldemar Toporowski. 2022. Can Humanizing Voice Assistants Unleash the Potential of Voice Commerce? (2022).
  22. Julian Sanchez, Wendy A Rogers, Arthur D Fisk, and Ericka Rovira. 2014. Understanding reliance on automation: effects of error type, error distribution, age and experience. Theoretical issues in ergonomics science 15, 2 (2014), 134–160.
  23. Daniel L Schwartz. 1999. The productive agency that drives collaborative learning. Collaborative learning: Cognitive and computational approaches 200 (1999).
  24. David M Schweiger, William R Sandberg, and James W Ragan. 1985. An Empirical Evaluation of Dialectial Inquiry, Devil’s Advocate, and Consensus Approaches to Strategic Decision Making.. In Academy of Management Proceedings, Vol. 1985. Academy of Management Briarcliff Manor, NY 10510, 40–44.
  25. David M Schweiger, Wiliam R Sandberg, and Paula Rechner. 1988. A Longitudinal Comparative Analysis of Dialectical Inquiry, Devil’s Advocacy and Consensus Approaches to Strategic Decision Making.. In Academy of Management Proceedings, Vol. 1988. Academy of Management Briarcliff Manor, NY 10510, 32–36.
  26. Charles Schwenk and Joseph S Valacich. 1994. Effects of devil’s advocacy and dialectical inquiry on individuals versus groups. Organizational behavior and human decision processes 59, 2 (1994), 210–222.
  27. Margaret Chase Smith and US Senator from Maine. 2018. The Naysayer Mindset. Unlocking Creativity: How to Solve Any Problem and Make the Best Decisions by Shifting Creative Mindsets (2018), 137.
  28. Gerry Stahl. 2002. Rediscovering cscl. In Cscl Vol. 2. 169–181.
  29. Garold Stasser and Dennis Stewart. 1992. Discovery of hidden profiles by decision-making groups: Solving a problem versus making a judgment.Journal of personality and social psychology 63, 3 (1992), 426.
  30. Garold Stasser, Laurie A Taylor, and Coleen Hanna. 1989. Information sampling in structured and unstructured discussions of three-and six-person groups.Journal of personality and social psychology 57, 1 (1989), 67.
  31. Harini Suresh, Natalie Lao, and Ilaria Liccardi. 2020. Misplaced Trust: Measuring the Interference of Machine Learning in Human Decision-Making. In 12th ACM Conference on Web Science. 315–324.
  32. John E Tropman. 2013. Effective meetings: Improving group decision making. Vol. 17. Sage Publications.
  33. Daisuke Wakabayashi. 2018. Self-driving Uber car kills pedestrian in Arizona, where robots roam. The New York Times 19, 03 (2018).
  34. Rui Zhang, Nathan J McNeese, Guo Freeman, and Geoff Musick. 2021. " An ideal human" expectations of AI teammates in human-AI teaming. Proceedings of the ACM on Human-Computer Interaction 4, CSCW3 (2021), 1–25.

Cited by

Related

See all

Metadata

Authors
Chiang, C.-W., Lu, Z., Li, Z., & Yin, M.
Affiliations
Purdue University West Lafayette (US)
ORCID
Venue
Proceedings of the 29th International Conference on Intelligent User Interfaces
Publisher
ACM
Published
2024-03-18
Pages
103-119
DOI
10.1145/3640543.3645199
OpenAlex
W4393970695
Semantic Scholar
6e2845e3099e2b300dc085b4d0b9180c2d9d139a
Cited by
142
References
81
Citations by year
2026: 80 · 2025: 55 · 2024: 7
FWCI
51.05
Type
conference paper
Language
en
Open access
Licence
Locations
ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion, cc-by)
Funders
National Science Foundation (IIS-2229876)
Topics
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
  • Social Sciences › Psychology › Social Psychology › Team Dynamics and Performance
  • Social Sciences › Business, Management and Accounting › Management Information Systems › Decision Support System Applications
Keywords
group decision making, AI-assisted decision-making, devil's advocate, large language models, AI reliance, perceived workload

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-14, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-14 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.