The DOI matches its Crossref record; 96 of the 96 references it cites were found in a registry or an index; 1 of the 11 sources its claims cite were checked for support.
When combinations of humans and AI are useful: A systematic review and meta-analysis
Open accessPublished version of a preprint
Abstract
Inspired by the increasing use of artificial intelligence (AI) to augment humans, researchers have studied human–AI systems involving different tasks, systems and populations. Despite such a large body of work, we lack a broad conceptual understanding of when combinations of humans and AI are better than either alone. Here we addressed this question by conducting a preregistered systematic review and meta-analysis of 106 experimental studies reporting 370 effect sizes. We searched an interdisciplinary set of databases (the Association for Computing Machinery Digital Library, the Web of Science and the Association for Information Systems eLibrary) for studies published between 1 January 2020 and 30 June 2023. Each study was required to include an original human-participants experiment that evaluated the performance of humans alone, AI alone and human–AI combinations. First, we found that, on average, human–AI combinations performed significantly worse than the best of humans or AI alone (Hedges’ g = −0.23; 95% confidence interval, −0.39 to −0.07). Second, we found performance losses in tasks that involved making decisions and significantly greater gains in tasks that involved creating content. Finally, when humans outperformed AI alone, we found performance gains in the combination, but when AI outperformed humans alone, we found losses. Limitations of the evidence assessed here include possible publication bias and variations in the study designs analysed. Overall, these findings highlight the heterogeneity of the effects of human–AI collaboration and point to promising avenues for improving human–AI systems.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This preregistered systematic review and meta-analysis of 106 experiments (370 effect sizes) finds that, on average, human–AI combinations perform worse than the best of humans or AI alone, though they do outperform humans alone. Effects vary by task type and by whether humans or AI perform better alone.
Key findings
- Human–AI combinations showed negative overall synergy versus the better of human alone or AI alone: Hedges’ g = −0.23 (95% CI, −0.39 to −0.07; P = 0.005).
- Human–AI combinations did show human augmentation versus humans alone: g = 0.64 (95% CI, 0.53 to 0.74; P = 0.000).
- Task type moderated synergy: decision tasks had losses (g = −0.27; 95% CI, −0.44 to −0.10; P = 0.002), while creation tasks had positive but non-significant gains (g = 0.19; 95% CI, −0.09 to 0.48; P = 0.180), with a significant difference between task types.
- Relative human/AI performance moderated synergy: when humans alone outperformed AI alone, synergy was positive (g = 0.46; 95% CI, 0.28 to 0.66); when AI alone outperformed humans alone, synergy was negative (g = −0.54; 95% CI, −0.71 to −0.37). Explanation, confidence, participant type, and division of labour were not statistically significant moderators.
Methods
- Preregistered systematic review and meta-analysis following Kitchenham and PRISMA guidelines; searched ACM Digital Library, Web of Science, and AIS eLibrary for studies published 1 January 2020–30 June 2023.
- From 5,126 initial papers, 74 met inclusion criteria, yielding 106 experiments and 370 unique effect sizes; included studies had to report performance for human alone, AI alone, and human–AI combination.
- Used Hedges’ g and a three-level random-effects meta-analytic model with robust variance estimation, Knapp–Hartung adjustment, and separate meta-regressions for moderators; sensitivity analyses clustered at paper level, excluded outliers, and used leave-one-out checks.
Limitations
- The paper states that results apply only to studies reporting all three conditions, so tasks that neither humans nor AI can perform alone are excluded; samples and outcome measures varied across studies, limiting comparability, and measurement error was not fully modelled.
- The paper states possible publication bias remains possible, though tests did not indicate it for synergy; tests did indicate potential publication bias for human augmentation. It also notes possible research topic selection bias, variation in study rigour, and high unexplained heterogeneity.
- Evident from the text: most effect sizes concerned decision tasks (~85%) and most human–AI systems had humans making final decisions after AI input (>95%), so findings may not generalize to other human–AI configurations.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
36 Own result · 14 Own interpretation · 7 Own method · 6 No full text of the sources · 6 No basis given · 1 Supported so far · 1 Sources not found · 1 Not checked yet
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“We thus adopted a three-level meta-analytic model in which effect sizes are nested within experiments, so we explicitly took the dependencies in our data into account in the model63,64.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Methods > Data analysis
What the sources say
- Supportedcites [63] Van den Noortgate, W., López-López, J. A., Marín-Martínez, F. & Sánchez-Meca, J. Meta-analysis of multiple outcomes: a multilevel approach. Behav. Res. Methods 47, 1274–1294 (2015). source
The Abstract describes a three-level meta-analytic model for dependence among multiple effect sizes from the same study. The Conclusions likewise state that the approach accounts for dependence between effect sizes within studies. The citing authors’ adoption of the model is their own methodological choice, not a finding attributed to this source. (LLM, conf=0.92)
- Not checked — no full text of the sourcecites [64] Cheung, M. W.-L. Modeling dependent effect sizes with three-level meta-analyses: a structural equation modeling approach. Psychol. Methods 19, 211–229 (2014). source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We also found that the type of AI involved in the experiment (F2,103 = 3.77, two-tailed P = 0.026) and the year of publication (F3,102 = 3.65, two-tailed P = 0.015) moderated human–AI synergy, and the experimental design moderated human augmentation (F1,104 = 4.90, two-tailed P = 0.029).”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Results > Heterogeneity of human–AI synergy
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Even though the average performance gains for creation tasks were not significantly different from 0 (presumably because of the relatively small sample size of n = 34), the difference between losses for decision tasks and gains for creation tasks was statistically significant.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Results > Heterogeneity of human–AI synergy
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Only 3 of the 100+ experiments in our analysis explore such processes with a predetermined delegation of separate subtasks to humans and AI.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Discussion > Moderating effect of task type
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We also found that the type of AI involved in the experiment (F2,103 = 3.77, two-tailed P = 0.026) and the year of publication (F3,102 = 3.65, two-tailed P = 0.015) moderated human–AI synergy, and the experimental design moderated human augmentation (F1,104 = 4.90, two-tailed P = 0.029).”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Results > Heterogeneity of human–AI synergy
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“First, we found that the type of task significantly moderated human–AI synergy (F1,104 = 7.84, two-tailed P = 0.006).”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Results > Heterogeneity of human–AI synergy
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, when we compared the performance of the human–AI systems to a different baseline—the humans alone—we found substantial evidence of human augmentation. The human–AI systems performed significantly better than humans alone, and this pooled effect size was positive (g = 0.64; t98 = 11.87; two-tailed P = 0.000; 95% CI, 0.53 to 0.74) and medium to large21.”
- Also stated in: § Main · plain
“On average, we found evidence of human augmentation, meaning that the average human–AI systems performed better than the human alone.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Results > Overall levels of human–AI synergy
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Main · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Among decision tasks—those in which participants decided between a finite set of options—the pooled effect size for human–AI synergy was significantly negative (g = −0.27; t104 = −3.20; two-tailed P = 0.002; 95% CI, −0.44 to −0.10), which indicates performance losses from combining humans and AI.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Results > Heterogeneity of human–AI synergy
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“But when the AI alone outperformed the human alone, performance losses occurred in the combined system relative to the AI alone, with a negative effect size for human–AI synergy of g = −0.54 (t104 = −6.20; two-tailed P = 0.000; 95% CI, −0.71 to −0.37), a medium-sized effect21.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Results > Heterogeneity of human–AI synergy
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Here we addressed this question by conducting a preregistered systematic review and meta-analysis of 106 experimental studies reporting 370 effect sizes.”
- Also stated in: § Results · plain
“These papers reported the results of 106 unique experiments, and many of the experiments had multiple conditions, so we collected a total of 370 unique effect sizes measuring the impact of human–AI collaboration on task performance.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Results · plain
- It may state two separate claims as one. 23
- Probably not a substantial claim. 16
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We also found evidence for substantial heterogeneity of effect sizes in our estimations of human–AI synergy (I2 = 97.7%) and human augmentation (I2 = 93.8%) (see Supplementary Tables 5 and 6 for more details).”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Results > Heterogeneity of human–AI synergy
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With the four effect sizes from these 3 experiments, we found that, on average, human–AI synergy (g = 0.22, t104 = 0.69; two-tailed P = 0.494; 95% CI, −0.42 to 0.87) occurred, but the result was not statistically significant (see Supplementary Information section 2.6 for a more detailed discussion of these experiments).”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Discussion > Moderating effect of task type
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The remaining moderators we investigated were not statistically significant for human–AI synergy or human augmentation (explanation, confidence, participant type and division of labour).”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Results > Heterogeneity of human–AI synergy
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Second, we found that the performance of the human and AI relative to each other impacted both human–AI synergy (F1,104 = 81.79, two-tailed P = 0.000) and human augmentation (F1,104 = 24.35, two-tailed P = 0.000).”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Results > Heterogeneity of human–AI synergy
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Relatedly, we found that the type of data involved in the task significantly moderated both human–AI synergy (F4,101 = 15.24, two-tailed P = 0.000) and human augmentation (F4,101 = 6.52, two-tailed P = 0.000).”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Results > Heterogeneity of human–AI synergy
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Fig. 2, when the human alone outperformed the AI alone, the combined human–AI system outperformed both alone with an average pooled effect size for human–AI synergy of g = 0.46 (t104 = 5.06; two-tailed P = 0.000; 95% CI, 0.28 to 0.66), a medium-sized effect21.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Results > Heterogeneity of human–AI synergy
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When the AI outperformed the human alone, greater performance gains tended to occur in the human–AI systems relative to the human alone, and the pooled effect size for human augmentation was positive and medium to large in magnitude (g = 0.74; t104 = 13.50; two-tailed P = 0.000; 95% CI, 0.63 to 0.85) (see Supplementary Figs. 4–7 for a more detailed visualization of this result for decision tasks).”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Results > Heterogeneity of human–AI synergy
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The measurement error can also vary across experiments.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Discussion > Limitations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast, among creation tasks—those in which participants created some sort of open-response content—the pooled effect size for human–AI synergy was positive (g = 0.19; t104 = 1.35; two-tailed P = 0.180; 95% CI, −0.09 to 0.48), pointing to synergy between humans and AI.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Results > Heterogeneity of human–AI synergy
hedgedn = 34- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We find this assumption plausible because the experiments recruited different sets of participants and entailed different tasks or interventions.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Methods > Sensitivity analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Supplementary Fig. 9 displays the funnel plot of the included effect sizes, and we did not observe significant asymmetry or regions of missing data in the plot for our primary outcome, human–AI synergy.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Methods > Bias tests
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Many of the experiments in our analysis evaluate performance according to a single measure of overall accuracy, but this measure corresponds to different things depending on the situation, and it omits other important criteria for human–AI systems.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Discussion > A roadmap for future work: finding human–AI synergy
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“For example, studies used different attention check mechanisms and performance incentive structures, which can both affect the quality of responses and thus introduce another source of noise into our data.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Discussion > Limitations
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“For example, researchers mostly used their own experimental platforms and stimuli, which naturally introduce sources of variation between their studies.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Discussion > Limitations
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Using this approach, we identified 11 outliers for human–AI synergy and 9 outliers for human augmentation.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Methods > Sensitivity analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our initial literature search yielded 5,126 papers, and, per the review process described in the Methods (see ‘Literature review’), we identified 74 that met our inclusion criteria (Supplementary Fig. 1).”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We searched an interdisciplinary set of databases (the Association for Computing Machinery Digital Library, the Web of Science and the Association for Information Systems eLibrary) for studies published between 1 January 2020 and 30 June 2023.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Each study was required to include an original human-participants experiment that evaluated the performance of humans alone, AI alone and human–AI combinations.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Here we addressed this question by conducting a preregistered systematic review and meta-analysis of 106 experimental studies reporting 370 effect sizes.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“We conducted this meta-analysis in accord with the guidelines from Kitchenham55 on systematic reviews, and we followed the standards set forth by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses56.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Methods
What the sources say
- Not matchedcites Kitchenham55
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not checked — no full text of the sourcecites [56] Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G. & PRISMA Group Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann. Intern. Med. 151, 264–269 (2009). source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Not matched
In the paper
“We chose Hedges’ g as our measure of effect size because it is unitless, so it allows us to compare human–AI performance across different metrics, and it corrects for upward bias in the raw standardized mean difference (Cohen’s d)61.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Methods > Data analysis
What the sources say
- Not checked — no full text of the sourcecites [61] Hedges, L. V. Distribution theory for Glass’s estimator of effect size and related estimators. J. Educ. Stat. 6, 107–128 (1981). source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We synthesized these data in a three-level meta-analytic model (see ‘Data analysis’ in Methods).”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Third, although we did not find evidence of publication biases, it remains possible that they exist, which would impact our literature base and, by extension, our meta-analytic results.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Discussion > Limitations
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Fourth, our results only apply to the tasks, processes and participant pools that researchers have chosen to study, and these configurations may not be representative of the ways human–AI systems are configured in practical uses of AI outside the laboratory.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Discussion > Limitations
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, we expect that if there were a publication bias operating here, it would be a bias to publish studies that showed significant gains from combining humans and AI. And since our overall results showed the opposite, it seems unlikely that they are a result of publication bias.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Discussion > Limitations
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“As is typical for meta-analyses43, in our three-level model, we weighted effect sizes as a function of their variance across participants, so we did not account for this other source of variation in measurement.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Discussion > Limitations
What the sources say
- Not checked — no full text of the sourcecites [43] Borenstein, M., Hedges, L. V., Higgins, J. P. & Rothstein, H. R. Introduction to Meta-analysis (John Wiley & Sons, 2021). source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In other words, the human–AI systems we analysed were, on average, better than humans alone but not better than both humans alone and AI alone.”
- Also stated in: § Main · plain
“On average, we found evidence of human augmentation, meaning that the average human–AI systems performed better than the human alone. But we did not find human–AI synergy on average, meaning that the average human–AI systems performed worse than at least one of the human alone or the AI alone.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Results > Overall levels of human–AI synergy
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § Main · plain
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We have made the materials required to reproduce our results publicly accessible through an Open Science Framework repository.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Fifth, the quality of our analysis depends on the quality of the studies we synthesized.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Discussion > Limitations
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The moderators we investigated account for some of this heterogeneity, but much remains unexplained.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Discussion > Limitations
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The discrepancy between the symmetry in the funnel plot for human–AI synergy versus asymmetry in the funnel plot for human augmentation may reflect how many researchers and journals implicitly assume an interest in human augmentation, comparing the human–AI system to the human alone.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Methods > Bias tests
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
In the paper
“Additionally, as discussed in ref. 33, human–AI synergy requires that humans be better at some parts of a task, AI be better at other parts of the task and the system as a whole be good at appropriately allocating subtasks to whichever partner is best for that subtask.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Discussion > A roadmap for future work: finding human–AI synergy
What the sources say
- Not checked — no full text of the sourcecites [33] Donahue, K., Chouldechova, A. & Kenthapadi, K. Human–algorithm collaboration: achieving complementarity and avoiding unfairness. In Proc. 2022 ACM Conference on Fairness, Accountability, and Transparency 1639–1656 (Association for Computing Machinery, 2022); https://doi.org/10.1145/3531146.3533221 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“To further ensure comprehensive coverage, we also conducted a forward and backward search on all studies we found that met our inclusion criteria.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Methods > Literature review
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper relies on this without support.
There is a lack of broad conceptual understanding of when combinations of humans and AI are better than either alone.
“Despite such a large body of work, we lack a broad conceptual understanding of when combinations of humans and AI are better than either alone.”
The authors state this as an assumption.
The experiments in the analysis are independent from each other even if they come from the same paper.
“It does, however, consider the experiments in our analysis as independent from each other, even if they come from the same paper. We find this assumption plausible because the experiments recruited different sets of participants and entailed different tasks or interventions.”
The paper relies on this without support.
The participant samples overlapped only on the level of the experiment.
“because the participant samples overlapped only on the level of the experiment.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While Gen-AI as an MKO can improve learning process, whereas over-reliance on these tools leads to logical complacency and selective cognitive atrophy (Vaccaro, , Almaatouq, & Malone, 2024).”
From The Double-Edged Sword of Automation: Impact of Generative AI Adoption on Student's Critical Thinking Skills in Higher Education Considering Cognitive Offloading as a Mediator (2026) · cites this paper as Vaccaro, , Almaatouq, & Malone, 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Vaccaro et al. (2024) showed that human-AI combinations often outperform either alone in creative tasks.”
From Metadiscourse Patterns in Human-Written vs. Generative AI-Authored Research Abstracts: A Comparative Corpus-Based Analysis (2026) · cites this paper as Vaccaro et al. (2024)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“To support the introduction of AI in clinical practice in intensive care, we propose starting with the concept of human-AI augmentation, which is more inclusive and better established according to medical literature [34].”
From Implementing Artificial Intelligence in Critical Care Medicine: a consensus of 22. (2025) · cites this paper as [34]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Recent meta-analyses of human-AI collaboration show that combinations sometimes underperform either party in isolation (Vaccaro et al., 2024)”
From Indie at Scale: How Large Language Models Make Community-Centered Game Development Economically Viable (2026) · cites this paper as Vaccaro et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“This collaboration has been described in two forms: human-AI augmentation (when human–AI interface enhances clinical performance compared to human alone) and human-AI synergy (where the combined performance exceeds that of both the human and the AI individually) [34].”
From Implementing Artificial Intelligence in Critical Care Medicine: a consensus of 22. (2025) · cites this paper as [34]
The cited source says what the claim attributes to it.
In the citing paper
“Vaccaro et al .'s meta-analysis of 106 experimental studies found that human-AI combinations performed worse than the best individual decision-maker on average for decision tasks 34 .”
From AI-induced never-skilling in medical education (2026) · cites this paper as [34]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Such tasks may have been part of the training data in GenAI models, which is a known issue in GenAI research [61]”
From Generative AI and Creativity: A Systematic Literature Review and Meta-Analysis (2025) · cites this paper as [61]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Discussion > Surprisingly insignificant moderators
“the significant and less researched moderators we identified: the baseline performance of the human and AI alone, the type of task they perform, and the division of labour between them.”
§ When combinations of humans and AI are useful: A systematic review and meta-analysis > Results > Heterogeneity of human–AI synergy
“The remaining moderators we investigated were not statistically significant for human–AI synergy or human augmentation (explanation, confidence, participant type and division of labour).”
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722 · cited 1×
“We visually inspected the plots and performed Egger’s regression test 70 as well as the rank correlation test 71 to evaluate the results in the funnel plots.”
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
Statistical Power Analysis for the Behavioral Sciences
Cohen, J. (2013) · doi:10.4324/9780203771587 · Cited by 25,373
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955 · cited 1×
“We visually inspected the plots and performed Egger’s regression test 70 as well as the rank correlation test 71 to evaluate the results in the funnel plots.”
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
Research works printed but not linked: 2
- Wilson, H. J. & Daugherty, P. R. Collaborative intelligence: humans and AI are joining forces. Harv. Bus. Rev. 96, 114–123 (2018).
- Hedges, L. V. & Olkin, I. Statistical Methods for Meta-analysis (Academic Press, 2014).
Cited by
Sorted by citations
Items 301–400 of 578
Impact of Distrust Priming on Artificial Intelligence Attitudes: A Context-Based InvestigationPreprint
Dan, K., & Hoshino, T. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5967417
Tomanek, K. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-02999-7_16
Collective intelligence: Conceptualization, mechanism, and measurement
Chu, G., Zhimou, W. A. N. G., Jing, H. U., & Zhan, P. (2026) · Advances in Psychological Science · doi:10.3724/sp.j.1042.2026.0515
When Your Reviewer is an LLM: Biases, Divergence, and Prompt Injection Risks in Peer ReviewPreprint
Zhu, C., Xiong, J., Ma, R., Lu, Z., Lu, Z., Liu, Y., & Li, L. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6067034
Developing a collaborative tool to foster communication in sustainability research
Hunter, N., Perret, N.-L., & Klepal, M. (2026) · Sustainability Science · doi:10.1007/s11625-025-01791-8
Artificial intelligence in biomedical team science: perceptions, practices, and training needs
Slade, E., Karnik, K. N., Phan, C. V., Hall, M. E., Feygin, Y. B., & McQuerry, K. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1720970
Coherence Under Acceleration Judgment Integrity In A High-Velocity Convergence EraPreprint
Calloway, D. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.5960914
When Better Performance Feels the Same: Performance Underestimation in Human-AI CollaborationPreprint
Voigt, J., & Berger, S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6164246
The Use of AI in Design Thinking
Graff, D., Clark, M. A., Li, D., et al. (2026) · Palgrave executive essentials · doi:10.1007/978-3-032-10543-1_3
Durmaz, Ş., & Çivilidağ, A. (2026) · Multidisciplinary Reviews · doi:10.31893/multirev.2026389
When robots reshape teams: neurodynamic insights into taskwork and teamwork in search and rescue
Spenceley, R. J., & Mehta, R. K. (2026) · Frontiers in Neuroergonomics · doi:10.3389/fnrgo.2026.1771753
Human–machine hybrid deep reinforcement learning for autonomous navigation in unknown environments
Li, Y., Zhang, Q., Kang, Y., & Zhao, Y.-B. (2026) · Advanced Engineering Informatics · doi:10.1016/j.aei.2026.104379
Grammenos, D., & Lubart, T. (2026) · Research Square · doi:10.21203/rs.3.rs-8718021/v1
How Forced Intervention Facilitates AI Adoption
Cao, X., Hu, C., Sun, J., & Zhang, D. (2026) · Manufacturing & Service Operations Management · doi:10.1287/msom.2024.1137 · Cited by 1
Beyond Hand Offs: Policy Driven Collaboration Between Human Agents and LLM Co-Pilots
GUDIPATI, S. K., ANKAM, G., MISHRA, N. A., Mohammed, A. R., & AKKISETTI, S. R. (2026) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc67433.2026.11393821
The Role of Perceived Social Identity in Human-AI Collaboration
Barfield, J. K. (2025) · IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA) · doi:10.1109/tps-isa67132.2025.00086
The Role of Artificial Intelligence in Shaping Human Life
Kate, P., Dagadkar, S. N., & Sontakke, A. (2026) · International Conference on Mobile Computing and Sustainable Informatics (ICMCSI) · doi:10.1109/icmcsi67283.2026.11412858
Critical friction: neurosymbolic support in the offloading-boosting competence tradeoff
Aldo Gangemi, A., & Lucifora, C. (2026) · Cognitive Systems Research · doi:10.1016/j.cogsys.2026.101511
ChatGPT in Nursing Education: Who Benefits Most?
Laun, M., & Wolff, F. (2026) · The Journal of Experimental Education · doi:10.1080/00220973.2026.2637671
Financial Decision Making in the Age of AIPreprint
Heimer, R. Z. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6249619
Luo, D., Zhu, X., He, F., & Yan, S. (2026) · Innovations in Education and Teaching International · doi:10.1080/14703297.2026.2635408
Franke, I. F., van der Wal, C. N., Comes, T., & Hoogendoorn, S. P. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6384381
Chung, M., Jeon, S., Yang, J., et al. (2026) · Proceedings, IEEE micro electro mechanical systems · doi:10.1109/mems64181.2026.11419442
Pompei, A., Battistella, C., & Nonino, F. (2026) · International Journal of Managing Projects in Business · doi:10.1108/ijmpb-10-2025-0451
Randomised study of human machine collaboration for cardiotocography interpretation during labour
M’Barek, I. B., M’Barek, B. B., Jauvion, G., Whelehan, V., Papageorghiou, A., Pennec, E. L., & Stirnemann, J. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02556-y
Access Without Displacement: An Access-Displacement Framework for AI Economic TransformationPreprint
Henjoto, V. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6324578
Optimal Human AI Coordination in Decision Workflows: Collaboration Paradox and Automation CliffsPreprint
Gu, W., Li, M. L., & Zhu, S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6417798 · Cited by 2
Sha, L., Liang, Z., Zhao, L., Li, Y., Fernandez-Nieto, G. M., Tsai, Y., Swiecki, Z., Chen, G., & Gašević, D. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2643354 · Cited by 2
Distributed Teaching Agency–AI in the University: A Typology Based on Student Voice
Fontaines-Ruíz, T., Rojo, A. P., Fabre-Merchán, P., Urcos, W. H. C., & Rincón, L. C. (2026) · Multimodal Technologies and Interaction · doi:10.3390/mti10040034
Artificial Influence: How AI Role and Gender Reshape Team DynamicsPreprint
Traeger, M. L., Kormylo, C., Hubbard, T. D., & Villano, M. A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6469423
Agrawal, A., Ravuri, S., Rani, M. J., Singh, A. P., Rajitha, B., Singh, S., Kumar, A., & Gupta, D. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-2878-2.ch015
Fu, M. (2026) · Asia Pacific Business Review · doi:10.1080/13602381.2026.2651845
Ludwig, B. (2026) · doi:10.1515/9783111386690-025
Human-Machine Conflict in Human-Machine TeamingPreprint
Chen, M., Xiong, W., Tsao, L., & Liang Ma, A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6508019
ChatGPT Is a People Pleaser, Especially When Taking Human vs. AI Advice
Rieger, T., & Roesler, E. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798923
Cheatle, C., & Banks, A. P. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798384
Exploring a Collaborative Gamified Approach to Vision-Language Model Evaluation
Gao, F., Wang, J. R. M., Sayuda, T., Pennington, M., & Sugano, Y. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3799410
What ethical AI and robots could mean: a conceptual and ethical reflection
Sedláková, J. (2026) · Frontiers in Robotics and AI · doi:10.3389/frobt.2026.1769361
Kuang, E., Shen, L., Soure, E. J., Fan, M., & Shinohara, K. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2638554
IA en reumatología: Aplicaciones clínicas actuales y consideraciones prácticas
Chacón, M., Hidalgo-Esparza, A. A., Fernández-Manrique, M. D. L. Á., Bustos, I. G., Cedeño, M. M. F., Figueroa, G. R. S., & Vargas-Morales, D. (2026) · Revista Reumatología al Día. · doi:10.62484/rad.v20i1.12402
Danti, A., Banerjee, J., & Nassoura, A. B. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-1348-9.ch001
AI-Augmented Scenario Design: Experiences from a National Cybersecurity Exercise
Reuter, L., Smith, P., & Skopik, F. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-19540-1_16
Wang, L., Kong, J., & Zhang, M. (2026) · Acta Psychologica · doi:10.1016/j.actpsy.2026.106826 · Cited by 1
Modeling the joint impact of human and AI metacognitive sensitivity on human–AI collaboration
Li, Z., & Steyvers, M. (2026) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2026.102988
Artificial intelligence in genomic medicine: dispelling three myths
Solomon, B. D. (2026) · npj Genomic Medicine · doi:10.1038/s41525-026-00575-y
Implementing responsible artificial intelligence-driven precision medicine in critical care
Ennouri, E., Boussarsar, M., Saida, I. B., Meddeb, K., & Toumi, R. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-36554-6.00021-0
Hara, T., Chen, W., & Ota, J. (2026) · CIRP Annals · doi:10.1016/j.cirp.2026.04.094
Derdouri, A., Oyama, T., Murayama, Y., & Masago, Y. (2026) · Research Square · doi:10.21203/rs.3.rs-9582053/v1
Zhang, X. (2026) · Journal of Innovation & Knowledge · doi:10.1016/j.jik.2026.101050 · Cited by 1
Transforming project management: the impact and potential of AI agents in shaping future practice
Hughes, L., Hughes, R., Al-Ahmadi, M. S., Jeon, I., & Dwivedi, Y. K. (2026) · Industrial Management & Data Systems · doi:10.1108/imds-03-2025-0295
Liu, Q. (2026) · International Journal of Digital Law and Governance · doi:10.1515/ijdlg-2025-0021
Dai, A. X. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2667475
Edwards, C. R., Murphy, A., Gunn, T. A., Singh, A., Arruzza, E. S., Makanjee, C. R., Geritz, E., & Chamunyonga, C. (2026) · Radiography · doi:10.1016/j.radi.2026.103438
Erb, D. T. (2026) · Open reports series · doi:10.11647/obp.0530.08
Tkachyk, F. P. (2026) · Economic Analysis · doi:10.35774/econa2026.01.249
Kumar, P., Mastana, A. S., Khanijou, S., & Luanrattana, R. (2026) · ShodhKosh Journal of Visual and Performing Arts · doi:10.29121/shodhkosh.v7.i10s.2026.7927
A mesterséges intelligencia munkaerőpiaci hatásai a globális értékláncok elméletének tükrében
Szalavetz, A. (2026) · Közgazdasági Szemle · doi:10.18414/ksz.2026.5.501
From AI use to positive functioning: The roles of trust and need satisfaction
Watermann, L., Lermer, E., & Kubowitsch, S. (2026) · Gruppe. Interaktion. Organisation: Zeitschrift für Angewandte Organisationspsychologie (GIO) · doi:10.1007/s11612-026-00886-9 · Cited by 1
Gargiulo, A., Leone, D., Claudio, L., & Dominici, G. (2026) · Management Decision · doi:10.1108/md-07-2025-1894
AI-Assisted Moot Courts: Simulating Justice-Specific Questioning in Oral Arguments
Zhang, K., Nadeem, N., Zheng, L., Stammbach, D., & Henderson, P. (2026) · ACM Symposium on Computer Science and Law (CSLAW) · doi:10.1145/3788646.3789536 · Cited by 2
Evaluating Generative AI in Benefits Administration: A Demonstration Project
Martin, O., Magesh, V., Surani, F., Rodolfa, K. T., Ho, D. E., & Perez, A. (2026) · ACM Symposium on Computer Science and Law (CSLAW) · doi:10.1145/3788646.3789527
Ebrahimi, N., Ghozloujeh, Z. G., Loo, L. K., et al. (2026) · Journal of Nephrology · doi:10.1093/joneph/aajag002
Yang, H., Liu, J., Chen, S., Mao, Y., Lai, K.-H., Wu, X., Jiang, R., & Cheng, Y. (2026) · Research Square · doi:10.21203/rs.3.rs-9664215/v1
Fang, M., Yang, W., Tan, Y., Xu, J., & Yin, J. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1852192
BODNAR, D., & Koldovskyy, V. (2026) · Advances in Cyber-Physical Systems · doi:10.23939/acps2026.01.012
AI Error Difficulty Modulates the Effectiveness of Explainability in Decision Support Systems
Rieger, T., Schindler, H., Koch, K., & Onnasch, L. (2026) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3817603 · Cited by 1
Li, Y., Yu, R., & Hu, X. (2026) · Technology in Society · doi:10.1016/j.techsoc.2026.103416
Outperforming but Not Collaborating: Fluency Gaps in Human–AI Teams
Lounis, C., Castells, M., Drougard, N., Gateau, T., Pinto, A. H. M., & Chanel, C. P. C. (2026) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai68641.2026.11536304
Generative AI and Organizational Decision-Making: A Systematic Review of Performance Effects
Nassiri, E. M., & Mzabi, A. E. (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.04.182
Brândaș, C., Fragnière, E., Andrei, A. L. B. U., & Plesa, M. (2026) · Systems · doi:10.3390/systems14060626
Designing cognition-driven Human-AI teaming: A Three-Layer Cognitive Framework (TriC)
Yin, Y., Jin, E., Pan, H., (324865), X. J., Zhang, Y., Liu, Y., Shen, M., & Gao, Z. (2026) · Computers in Human Behavior Reports · doi:10.2139/ssrn.6890561
The Crowd Without People: Multi-agent Large Language Models as Idea Evaluators
Probost, F., Schoch, M., Voucko-Glockner, H., Gimpel, H., & Laubacher, R. (2026) · Group Decision and Negotiation · doi:10.1007/s10726-026-09993-w · Cited by 1
Cabitza, F., & Vicente, L. (2026) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2026.103855 · Cited by 2
Yang, W., Li, S., Du, Y., Chen, M., Yang, F., Zhang, F., Zhao, J., Li, Y., & Xu, X. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02729-9
Lekhi, N., Patten, T., Patil, P., Li, M., & Alsaid, A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6910509
Meyer, F., Miggelbrink, J., & Nguyễn, P. (2026) · International Journal of Qualitative Methods · doi:10.1177/16094069261459023
Using the Blackbox in Embodied AI Art Practice
Yuan, D., Graham, C., Yakura, H., & Lim, A. (2026) · Designing Interactive Systems Conference · doi:10.1145/3800645.3813020
Sustainability via Large Language Model Right-Sizing
Haase, J., Klessascheck, F., Mendling, J., & Pokutta, S. (2026) · IEEE Transactions on Sustainable Computing · doi:10.1109/tsusc.2026.3697261
Human Robot Collaborative Creativity: Challenges and Opportunities
Babaei, Z., Kalateh, S., Nikghadam-Hojjati, S., Leitao, P., & Barata, J. (2026) · IFIP advances in information and communication technology · doi:10.1007/978-3-032-28939-1_22
Rethinking AI aversion in tourism: A meta-analysis of causal effects and contextual boundaries
Peng, C., Sun, J., Feng, C., Chen, Z., & Li, X. (2026) · Tourism Management · doi:10.1016/j.tourman.2026.105475 · Cited by 1
Wen, R., Zhao, Z., & Liu, S. (2026) · Computers & Industrial Engineering · doi:10.1016/j.cie.2026.112200
Pelechano, V., Mestre, A., Albert, M., & Gil, M. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6958934
Strategic Decision-Making Frameworks With Human-AI Systems
Roos, G. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-7036-1.ch004
Dhahbi, W., Briki, W., & Chamari, K. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6956772
AI-Driven Biostatistics in Healthcare
Jose, A., Mohan, R., & Kaithathara, S. T. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-2373-0.ch005
Watanabe, H., Morisaka, H., Aoyagi, K., Tozuka, R., Wumu, T., Ii, T., Sakamoto, K., Yato, T., & Ichikawa, S. (2026) · Japanese Journal of Radiology · doi:10.1007/s11604-026-02029-y · Cited by 1
AI as Arbiter? The Role of Artificial Intelligence in Cyberbullying Report Review Process
Lee, S., Kim, W., & Moon, W.-K. (2026) · Cyberpsychology Behavior and Social Networking · doi:10.1177/21522715261460961
Fraser, A. G. (2026) · European Heart Journal - Digital Health · doi:10.1093/ehjdh/ztag095
AI-Augmented Leadership and Digital Decision-Making Models
Roos, G. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-2541-3.ch002
Zhao, J. Y., & Shi, J. (2026) · Acta Psychologica · doi:10.1016/j.actpsy.2026.107295 · Cited by 1
Puutio, T. A., & Do, M. (2026) · Group Decision and Negotiation · doi:10.1007/s10726-026-10006-z · Cited by 1
Taş, E., Memmert, L., & Bittner, E. A. C. (2026) · Electronic Markets · doi:10.1007/s12525-026-00915-x
Large reasoning models as thinking machines for medicine
Zhou, H.-Y., Rodman, A. M., Liu, P., Rajpurkar, P., Hu, Y., Wong, T. Y., & Topol, E. J. (2026) · Nature Biomedical Engineering · doi:10.1038/s41551-026-01701-y
AIStories: Exploring Human–Artificial Intelligence Collaboration Through Narratives and Games
Sapone, C., Triberti, S., & Riva, G. (2026) · Cyberpsychology Behavior and Social Networking · doi:10.1177/21522715261462213
How Can AI Augment Access to Justice? Public Defenders' Perspectives on Responsible AI Adoption
Cheong, I., Liu, P., Stammbach, D., & Henderson, P. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806526 · Cited by 3
Automation Gains, Human Costs: Deskilling Pressure in Human–AI Task AllocationPreprint
Pelechano, V., Mestre, A., Albert, M., & Gil, M. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6993655
ІННОВАЦІЙНІ МЕТОДИ ТА ІНСТРУМЕНТИ ОПТИМІЗАЦІЇ ВІДДАЛЕНОЇ РОБОТИ КРЕАТИВНИХ КОМАНД
Kovshova, I., & Канєвська, К. (2026) · РОЗВИТОК МІСТА · doi:10.32782/city-development.2026.2-10
Making Teacher-AI Complementarity Visible with Task-, Skill- and Knowledge Analysis
Kramer, A., Falhs, A.-C., Topali, P., Suraworachet, W., Zhou, Q., Cukurova, M., Ley, T., Rummel, N., & Molenaar, I. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-29791-4_35
Chen, W.-H. (2026) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2026.103095
Metadata
- Authors
- Vaccaro, M., Almaatouq, A., & Malone, T.
- Affiliations
- Massachusetts Institute of Technology (US)
- Venue
- Nature Human Behaviour
- Published
- 2024-10-28
- Volume
- 8
- Issue
- 12
- Pages
- 2293-2303
- ISSN
- 2397-3374 (ISSN-L)
- PMC
- PMC11659167
- OpenAlex
- W4403839497
- Semantic Scholar
- 02aec976011a7d595d61db9639209968f4b93177
- Cited by
- 570
- References
- 95
- Citations by year
- 2026: 404 · 2025: 156 · 2024: 9
- FWCI
- 51.29
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Nature Human Behaviour (publishedVersion, cc-by)
- Research Square (acceptedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- PubMed Central (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Accenture Technology Convergence Fellowship
- Toyota Research Institute; MIT Quest for Intelligence; National Research Foundation, Prime Minister's Office, Singapore
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- human-AI collaboration, human-AI comparison, decision-making, content creation, performance outcomes, AI augmentation, task type effects, human-AI systems
- MeSH terms
- Artificial Intelligence, Decision Making, Humans
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-20, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-20 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
