CoChat
Sign Up
Cited by 579Human-Automation Interaction and SafetySocial Robot Interaction and HRIArtificial Intelligence Applications

When combinations of humans and AI are useful: A systematic review and meta-analysis

Open access
Vaccaro, Michelle, Almaatouq, Abdullah, Malone, Thomas·2024·Nature Human Behaviour
DOI 10.1038/s41562-024-02024-1PMC PMC11659167

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Inspired by the increasing use of artificial intelligence (AI) to augment humans, researchers have studied human–AI systems involving different tasks, systems and populations. Despite such a large body of work, we lack a broad conceptual understanding of when combinations of humans and AI are better than either alone. Here we addressed this question by conducting a preregistered systematic review and meta-analysis of 106 experimental studies reporting 370 effect sizes. We searched an interdisciplinary set of databases (the Association for Computing Machinery Digital Library, the Web of Science and the Association for Information Systems eLibrary) for studies published between 1 January 2020 and 30 June 2023. Each study was required to include an original human-participants experiment that evaluated the performance of humans alone, AI alone and human–AI combinations. First, we found that, on average, human–AI combinations performed significantly worse than the best of humans or AI alone (Hedges’ g = −0.23; 95% confidence interval, −0.39 to −0.07). Second, we found performance losses in tasks that involved making decisions and significantly greater gains in tasks that involved creating content. Finally, when humans outperformed AI alone, we found performance gains in the combination, but when AI outperformed humans alone, we found losses. Limitations of the evidence assessed here include possible publication bias and variations in the study designs analysed. Overall, these findings highlight the heterogeneity of the effects of human–AI collaboration and point to promising avenues for improving human–AI systems.

AI snapshot

Written from the paper’s full text.

TL;DR

This preregistered systematic review and meta-analysis of 106 experiments (370 effect sizes) finds that, on average, human–AI combinations perform worse than the best of humans or AI alone, though they do outperform humans alone. Effects vary by task type and by whether humans or AI perform better alone.

Key findings

  • Human–AI combinations showed negative overall synergy versus the better of human alone or AI alone: Hedges’ g = −0.23 (95% CI, −0.39 to −0.07; P = 0.005).
  • Human–AI combinations did show human augmentation versus humans alone: g = 0.64 (95% CI, 0.53 to 0.74; P = 0.000).
  • Task type moderated synergy: decision tasks had losses (g = −0.27; 95% CI, −0.44 to −0.10; P = 0.002), while creation tasks had positive but non-significant gains (g = 0.19; 95% CI, −0.09 to 0.48; P = 0.180), with a significant difference between task types.
  • Relative human/AI performance moderated synergy: when humans alone outperformed AI alone, synergy was positive (g = 0.46; 95% CI, 0.28 to 0.66); when AI alone outperformed humans alone, synergy was negative (g = −0.54; 95% CI, −0.71 to −0.37). Explanation, confidence, participant type, and division of labour were not statistically significant moderators.

Methods

  • Preregistered systematic review and meta-analysis following Kitchenham and PRISMA guidelines; searched ACM Digital Library, Web of Science, and AIS eLibrary for studies published 1 January 2020–30 June 2023.
  • From 5,126 initial papers, 74 met inclusion criteria, yielding 106 experiments and 370 unique effect sizes; included studies had to report performance for human alone, AI alone, and human–AI combination.
  • Used Hedges’ g and a three-level random-effects meta-analytic model with robust variance estimation, Knapp–Hartung adjustment, and separate meta-regressions for moderators; sensitivity analyses clustered at paper level, excluded outliers, and used leave-one-out checks.

Limitations

  • The paper states that results apply only to studies reporting all three conditions, so tasks that neither humans nor AI can perform alone are excluded; samples and outcome measures varied across studies, limiting comparability, and measurement error was not fully modelled.
  • The paper states possible publication bias remains possible, though tests did not indicate it for synergy; tests did indicate potential publication bias for human augmentation. It also notes possible research topic selection bias, variation in study rigour, and high unexplained heterogeneity.
  • Evident from the text: most effect sizes concerned decision tasks (~85%) and most human–AI systems had humans making final decisions after AI input (>95%), so findings may not generalize to other human–AI configurations.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41562-024-02024-1

Cites

95 of 95 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

1 of 11 cited sources checked · 1 supported · 0 flagged

  • Study design and conductreasoning

    A three-level meta-analytic model was adopted in which effect sizes are nested within experiments, explicitly taking dependencies into account.

    “We thus adopted a three-level meta-analytic model in which effect sizes are nested within experiments, so we explicitly took the dependencies in our data into account in the model63,64.”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [63] Van den Noortgate, W., López-López, J. A., Marín-Martínez, F. & Sánchez-Meca, J. Meta-analysis of multiple outcomes: a multilevel approach. Behav. Res. Methods 47, 1274–1294 (2015). source
    • cites [64] Cheung, M. W.-L. Modeling dependent effect sizes with three-level meta-analyses: a structural equation modeling approach. Psychol. Methods 19, 211–229 (2014). source
  • Findingown data

    The year of publication significantly moderated human-AI synergy (F3,102 = 3.65, P = 0.015).

    “We also found that the type of AI involved in the experiment (F2,103 = 3.77, two-tailed P = 0.026) and the year of publication (F3,102 = 3.65, two-tailed P = 0.015) moderated human–AI synergy, and the experimental design moderated human augmentation (F1,104 = 4.90, two-tailed P = 0.029).”
    Quote in the paperNot a substantial claim
  • Findingown data

    The difference between losses for decision tasks and gains for creation tasks was statistically significant.

    “Even though the average performance gains for creation tasks were not significantly different from 0 (presumably because of the relatively small sample size of n = 34), the difference between losses for decision tasks and gains for creation tasks was statistically significant.”
    Quote in the paperSubstantial claim
  • Findingown data

    Only 3 of the 100+ experiments explored predetermined delegation of separate subtasks to humans and AI.

    “Only 3 of the 100+ experiments in our analysis explore such processes with a predetermined delegation of separate subtasks to humans and AI.”
    Quote in the paperSubstantial claim
  • Findingown data

    The type of AI involved in the experiment significantly moderated human-AI synergy (F2,103 = 3.77, P = 0.026).

    “We also found that the type of AI involved in the experiment (F2,103 = 3.77, two-tailed P = 0.026) and the year of publication (F3,102 = 3.65, two-tailed P = 0.015) moderated human–AI synergy, and the experimental design moderated human augmentation (F1,104 = 4.90, two-tailed P = 0.029).”
    Quote in the paperNot a substantial claim
  • May combine two claims. 23
  • Classified as probably not a substantial claim. 16

  • May combine two claims.

    Findingown data

    There was substantial heterogeneity of effect sizes for human-AI synergy (I2 = 97.7%) and human augmentation (I2 = 93.8%).

    “We also found evidence for substantial heterogeneity of effect sizes in our estimations of human–AI synergy (I2 = 97.7%) and human augmentation (I2 = 93.8%) (see Supplementary Tables 5 and 6 for more details).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    With the four effect sizes from these 3 experiments, average human-AI synergy was g = 0.22, but the result was not statistically significant (P = 0.494; 95% CI, −0.42 to 0.87).

    “With the four effect sizes from these 3 experiments, we found that, on average, human–AI synergy (g = 0.22, t104 = 0.69; two-tailed P = 0.494; 95% CI, −0.42 to 0.87) occurred, but the result was not statistically significant (see Supplementary Information section 2.6 for a more detailed discussion of these experiments).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Explanation, confidence, participant type and division of labour did not significantly moderate human-AI synergy or human augmentation.

    “The remaining moderators we investigated were not statistically significant for human–AI synergy or human augmentation (explanation, confidence, participant type and division of labour).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The relative performance of human and AI significantly impacted both human-AI synergy (F1,104 = 81.79, P = 0.000) and human augmentation (F1,104 = 24.35, P = 0.000).

    “Second, we found that the performance of the human and AI relative to each other impacted both human–AI synergy (F1,104 = 81.79, two-tailed P = 0.000) and human augmentation (F1,104 = 24.35, two-tailed P = 0.000).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The type of data involved in the task significantly moderated both human-AI synergy (F4,101 = 15.24, P = 0.000) and human augmentation (F4,101 = 6.52, P = 0.000).

    “Relatedly, we found that the type of data involved in the task significantly moderated both human–AI synergy (F4,101 = 15.24, two-tailed P = 0.000) and human augmentation (F4,101 = 6.52, two-tailed P = 0.000).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    When the human alone outperformed the AI alone, the combined human-AI system outperformed both alone, with an average pooled effect size for human-AI synergy of g = 0.46 (95% CI, 0.28 to 0.66).

    “As shown in Fig. 2, when the human alone outperformed the AI alone, the combined human–AI system outperformed both alone with an average pooled effect size for human–AI synergy of g = 0.46 (t104 = 5.06; two-tailed P = 0.000; 95% CI, 0.28 to 0.66), a medium-sized effect21.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    When the AI outperformed the human alone, greater performance gains occurred in the human-AI systems relative to the human alone, with a pooled effect size for human augmentation of g = 0.74 (95% CI, 0.63 to 0.85).

    “When the AI outperformed the human alone, greater performance gains tended to occur in the human–AI systems relative to the human alone, and the pooled effect size for human augmentation was positive and medium to large in magnitude (g = 0.74; t104 = 13.50; two-tailed P = 0.000; 95% CI, 0.63 to 0.85) (see Supplementary Figs. 4–7 for a more detailed visualization of this result for decision tasks).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    Measurement error can vary across experiments.

    “The measurement error can also vary across experiments.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Among creation tasks, the pooled effect size for human-AI synergy was positive (g = 0.19) but not significantly different from 0 (P = 0.180; 95% CI, −0.09 to 0.48).

    “In contrast, among creation tasks—those in which participants created some sort of open-response content—the pooled effect size for human–AI synergy was positive (g = 0.19; t104 = 1.35; two-tailed P = 0.180; 95% CI, −0.09 to 0.48), pointing to synergy between humans and AI.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Findingnone given

    The experiments in the analysis recruited different sets of participants and entailed different tasks or interventions.

    “We find this assumption plausible because the experiments recruited different sets of participants and entailed different tasks or interventions.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    No significant asymmetry or regions of missing data were observed in the funnel plot for the primary outcome, human-AI synergy.

    “Supplementary Fig. 9 displays the funnel plot of the included effect sizes, and we did not observe significant asymmetry or regions of missing data in the plot for our primary outcome, human–AI synergy.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    Many experiments in the analysis evaluate performance according to a single measure of overall accuracy, which corresponds to different things depending on the situation and omits other important criteria for human-AI systems.

    “Many of the experiments in our analysis evaluate performance according to a single measure of overall accuracy, but this measure corresponds to different things depending on the situation, and it omits other important criteria for human–AI systems.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    Studies used different attention check mechanisms and performance incentive structures, which can affect response quality and introduce noise.

    “For example, studies used different attention check mechanisms and performance incentive structures, which can both affect the quality of responses and thus introduce another source of noise into our data.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    Researchers mostly used their own experimental platforms and stimuli, introducing sources of variation between studies.

    “For example, researchers mostly used their own experimental platforms and stimuli, which naturally introduce sources of variation between their studies.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    11 outliers were identified for human-AI synergy and 9 outliers for human augmentation.

    “Using this approach, we identified 11 outliers for human–AI synergy and 9 outliers for human augmentation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    The initial literature search yielded 5,126 papers, and 74 met the inclusion criteria.

    “Our initial literature search yielded 5,126 papers, and, per the review process described in the Methods (see ‘Literature review’), we identified 74 that met our inclusion criteria (Supplementary Fig. 1).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The search covered the Association for Computing Machinery Digital Library, the Web of Science and the Association for Information Systems eLibrary for studies published between 1 January 2020 and 30 June 2023.

    “We searched an interdisciplinary set of databases (the Association for Computing Machinery Digital Library, the Web of Science and the Association for Information Systems eLibrary) for studies published between 1 January 2020 and 30 June 2023.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Each included study was required to include an original human-participants experiment that evaluated the performance of humans alone, AI alone and human-AI combinations.

    “Each study was required to include an original human-participants experiment that evaluated the performance of humans alone, AI alone and human–AI combinations.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The systematic review and meta-analysis was preregistered.

    “Here we addressed this question by conducting a preregistered systematic review and meta-analysis of 106 experimental studies reporting 370 effect sizes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductcitation

    The meta-analysis was conducted in accord with Kitchenham guidelines for systematic reviews and followed PRISMA standards.

    “We conducted this meta-analysis in accord with the guidelines from Kitchenham55 on systematic reviews, and we followed the standards set forth by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses56.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Kitchenham55

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [56] Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G. & PRISMA Group Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann. Intern. Med. 151, 264–269 (2009). source

  • May combine two claims.

    Premisecitation

    Hedges' g is unitless, allowing comparison across metrics, and corrects for upward bias in Cohen's d.

    “We chose Hedges’ g as our measure of effect size because it is unitless, so it allows us to compare human–AI performance across different metrics, and it corrects for upward bias in the raw standardized mean difference (Cohen’s d)61.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [61] Hedges, L. V. Distribution theory for Glass’s estimator of effect size and related estimators. J. Educ. Stat. 6, 107–128 (1981). source

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The data were synthesized in a three-level meta-analytic model.

    “We synthesized these data in a three-level meta-analytic model (see ‘Data analysis’ in Methods).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationnone given

    Although the paper did not find evidence of publication biases, it remains possible that they exist and would impact the literature base and meta-analytic results.

    “Third, although we did not find evidence of publication biases, it remains possible that they exist, which would impact our literature base and, by extension, our meta-analytic results.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given

    The results only apply to the tasks, processes and participant pools that researchers have chosen to study, and these configurations may not be representative of practical uses of AI outside the laboratory.

    “Fourth, our results only apply to the tasks, processes and participant pools that researchers have chosen to study, and these configurations may not be representative of the ways human–AI systems are configured in practical uses of AI outside the laboratory.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    If publication bias existed, it would favor studies showing significant gains from combining humans and AI, and since the overall results showed the opposite, it is unlikely the results are due to publication bias.

    “However, we expect that if there were a publication bias operating here, it would be a bias to publish studies that showed significant gains from combining humans and AI. And since our overall results showed the opposite, it seems unlikely that they are a result of publication bias.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductcitation

    The three-level model weighted effect sizes by variance across participants and did not account for variation in measurement error.

    “As is typical for meta-analyses43, in our three-level model, we weighted effect sizes as a function of their variance across participants, so we did not account for this other source of variation in measurement.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [43] Borenstein, M., Hedges, L. V., Higgins, J. P. & Rothstein, H. R. Introduction to Meta-analysis (John Wiley & Sons, 2021). source

  • May combine two claims.

    Interpretationown data

    The human-AI systems studied were, on average, better than humans alone but not better than both humans alone and AI alone.

    “In other words, the human–AI systems we analysed were, on average, better than humans alone but not better than both humans alone and AI alone.”
    • Also stated in: § Main · plain
      “On average, we found evidence of human augmentation, meaning that the average human–AI systems performed better than the human alone. But we did not find human–AI synergy on average, meaning that the average human–AI systems performed worse than at least one of the human alone or the AI alone.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Materials required to reproduce the results are publicly accessible through an Open Science Framework repository.

    “We have made the materials required to reproduce our results publicly accessible through an Open Science Framework repository.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The quality of the analysis depends on the quality of the studies synthesized.

    “Fifth, the quality of our analysis depends on the quality of the studies we synthesized.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The moderators investigated account for some but not all of the high heterogeneity.

    “The moderators we investigated account for some of this heterogeneity, but much remains unexplained.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    The discrepancy between symmetry in the funnel plot for human-AI synergy versus asymmetry for human augmentation may reflect how many researchers and journals implicitly assume an interest in human augmentation.

    “The discrepancy between the symmetry in the funnel plot for human–AI synergy versus asymmetry in the funnel plot for human augmentation may reflect how many researchers and journals implicitly assume an interest in human augmentation, comparing the human–AI system to the human alone.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Human-AI synergy requires that humans be better at some parts of a task, AI be better at other parts, and the system as a whole be good at appropriately allocating subtasks to whichever partner is best for that subtask.

    “Additionally, as discussed in ref. 33, human–AI synergy requires that humans be better at some parts of a task, AI be better at other parts of the task and the system as a whole be good at appropriately allocating subtasks to whichever partner is best for that subtask.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [33] Donahue, K., Chouldechova, A. & Kenthapadi, K. Human–algorithm collaboration: achieving complementarity and avoiding unfairness. In Proc. 2022 ACM Conference on Fairness, Accountability, and Transparency 1639–1656 (Association for Computing Machinery, 2022); https://doi.org/10.1145/3531146.3533221 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    A forward and backward search was conducted on all studies that met inclusion criteria to ensure comprehensive coverage.

    “To further ensure comprehensive coverage, we also conducted a forward and backward search on all studies we found that met our inclusion criteria.”
    Quote in the paperNot a substantial claim
33 of 33

  • The paper relies on this without support.

    There is a lack of broad conceptual understanding of when combinations of humans and AI are better than either alone.

    “Despite such a large body of work, we lack a broad conceptual understanding of when combinations of humans and AI are better than either alone.”

  • The authors state this as an assumption.

    The experiments in the analysis are independent from each other even if they come from the same paper.

    “It does, however, consider the experiments in our analysis as independent from each other, even if they come from the same paper. We find this assumption plausible because the experiments recruited different sets of participants and entailed different tasks or interventions.”

  • The paper relies on this without support.

    The participant samples overlapped only on the level of the experiment.

    “because the participant samples overlapped only on the level of the experiment.”
3 of 3

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Over-reliance on Gen-AI leads to logical complacency and selective cognitive atrophy.

    “While Gen-AI as an MKO can improve learning process, whereas over-reliance on these tools leads to logical complacency and selective cognitive atrophy (Vaccaro, , Almaatouq, & Malone, 2024).”

    From The Double-Edged Sword of Automation: Impact of Generative AI Adoption on Student's Critical Thinking Skills in Higher Education Considering Cognitive Offloading as a Mediator (2026) · cites this paper as Vaccaro, , Almaatouq, & Malone, 2024

    The Discussion section, under “Performance losses from human–AI collaboration” (page unavailable), says people sometimes over-rely on AI suggestions without seeking or processing more information. That supports a concern about reduced scrutiny, but the paper does not establish that over-reliance on generative AI causes logical complacency or selective cognitive atrophy. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Human-AI combinations often outperform either alone in creative tasks.

    “Vaccaro et al. (2024) showed that human-AI combinations often outperform either alone in creative tasks.”

    From Metadiscourse Patterns in Human-Written vs. Generative AI-Authored Research Abstracts: A Comparative Corpus-Based Analysis (2026) · cites this paper as Vaccaro et al. (2024)

    The Results (Heterogeneity of human–AI synergy; page unavailable) report a positive pooled effect for creation tasks, where synergy means outperforming both humans and AI alone. But that effect was not statistically distinguishable from zero (g = 0.19; P = 0.180), and the paper does not establish that combinations often outperform either alone in such tasks. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Human-AI augmentation is more inclusive and better established according to medical literature.

    “To support the introduction of AI in clinical practice in intensive care, we propose starting with the concept of human-AI augmentation, which is more inclusive and better established according to medical literature [34].”

    From Implementing Artificial Intelligence in Critical Care Medicine: a consensus of 22. (2025) · cites this paper as [34]

    The Main section (page unavailable) defines augmentation as outperforming the human alone, a less restrictive criterion than synergy, which requires outperforming both the human and AI alone. It also says many studies focus on augmentation, but does not establish that the concept is better established specifically in medical literature. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

  • Recent meta-analyses of human-AI collaboration show that combinations sometimes underperform either party in isolation.

    “Recent meta-analyses of human-AI collaboration show that combinations sometimes underperform either party in isolation (Vaccaro et al., 2024)”

    From Indie at Scale: How Large Language Models Make Community-Centered Game Development Economically Viable (2026) · cites this paper as Vaccaro et al., 2024

  • Human-AI collaboration has been described in two forms: human-AI augmentation and human-AI synergy.

    “This collaboration has been described in two forms: human-AI augmentation (when human–AI interface enhances clinical performance compared to human alone) and human-AI synergy (where the combined performance exceeds that of both the human and the AI individually) [34].”

    From Implementing Artificial Intelligence in Critical Care Medicine: a consensus of 22. (2025) · cites this paper as [34]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Division of labour called significant after being reported nonsignificant

    Major

    The discussion lists division of labour among the significant moderators, but the results explicitly list it among moderators that were not statistically significant.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. Wilson, H. J. & Daugherty, P. R. Collaborative intelligence: humans and AI are joining forces. Harv. Bus. Rev. 96, 114–123 (2018).
  2. Hedges, L. V. & Olkin, I. Statistical Methods for Meta-analysis (Academic Press, 2014).

Cited by

Sorted by citations

Related

Authors
Vaccaro, M., Almaatouq, A., & Malone, T.
Affiliations
Massachusetts Institute of Technology (US)
ORCID
Venue
Nature Human Behaviour
Publisher
Springer Science and Business Media LLC
Published
2024-10-28
Volume
8
Issue
12
Pages
2293-2303
ISSN
2397-3374 (ISSN-L)
DOI
10.1038/s41562-024-02024-1
PMC
PMC11659167
OpenAlex
W4403839497
Semantic Scholar
02aec976011a7d595d61db9639209968f4b93177
Cited by
570
References
95
Citations by year
2026: 404 · 2025: 156 · 2024: 9 · 2020: 1
FWCI
51.29
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Human Behaviour (publishedVersion, cc-by)
  • Research Square (acceptedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • PubMed Central (submittedVersion, cc-by)
  • RePEc: Research Papers in Economics (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
  • Accenture Technology Convergence Fellowship
  • Toyota Research Institute; MIT Quest for Intelligence; National Research Foundation, Prime Minister's Office, Singapore
Topics
  • Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
human-AI collaboration, human-AI comparison, decision-making, content creation, performance outcomes, AI augmentation, task type effects, human-AI systems
MeSH terms
Artificial Intelligence, Decision Making, Humans

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-20.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-20.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review