Cited by 63Forecasting Techniques and ApplicationsImpact of AI and Big Data on Business and SocietySupply Chain and Inventory Management

Human–artificial intelligence collaboration in prediction: a field experiment in the retail industry

Revilla, Elena, Saenz, Maria Jesus, Seifert, Matthias, Ma, Ye·2023·Journal of Management Information Systems

Abstract

This study investigates the role of human intervention in artificial intelligence/machine learning (AIML)-driven predictions. By doing so, we distinguish between three different types of human-AIML collaboration: automation, adjustable automation, and augmentation. We theorize that prediction uncertainty and time horizon represent two critical determinants of forecast accuracy. Based on a field experiment involving AIML-driven demand forecasts approximately 1,888 stock-keeping units in the retail industry, we rely on a multivalued treatment effect methodology to measure the effects of human-AIML collaboration on forecast accuracy. Our findings show that human intervention complements AIML-driven forecasts most effectively (augmentation) in predictions with long time horizons and low uncertainty. However human intervention is least likely to contribute to the effectiveness of AIML predictions (automation) in environments with short time horizons and high uncertainty. We discuss implications for extant theory and propose a framework outlining the conditions in which human intervention is most likely to add predictive value to human-AIML collaborations.

Integrity signals

Registry

Crossref · 10.1080/07421222.2023.2267317

Cites

70 of 70 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Human partners possess distinctive advantages in contextual understanding, commonsense reasoning, and emotional support, thereby enhancing the usefulness and practical value of collaborative outcomes.

    “by contrast, human partners possess distinctive advantages in contextual understanding, commonsense reasoning, and emotional support, thereby enhancing the usefulness and practical value of collaborative outcomes (Huang et al., 2024; Markovitch et al., 2024; Revilla et al., 2023).”
  • This result directly challenges the conventional assumption that humans are more useful whereas AI is more novel, offering a new theoretical lens for adjusting role perceptions in human-AI collaboration.

    “This result directly challenges the conventional assumption that “humans are more useful whereas AI is more novel” (Chen and Chan, 2024; Revilla et al., 2023), offering a new theoretical lens for adjusting role perceptions in human–AI collaboration.”

Cites

Flagged first, then by citations

1 of the 71 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Revilla, E., Saenz, M. J., Seifert, M., & Ma, Y.
Affiliations
  • IE Business School, Madrid, Spain
  • MIT Digital Supply Chain Transformation (MIT CTL), Cambridge, MA, USA
  • Massachusetts Institute of Technology (US)
Corresponding author
Revilla, Elena
Venue
Journal of Management Information Systems
Published
2023-10-02
Volume
40
Issue
4
Pages
1071-1098
ISSN
  • 0742-1222 (ISSN-L)
  • 1557-928X
OpenAlex
W4389549428
Cited by
63
References
71
Citations by year
2026: 36 · 2025: 19 · 2024: 7 · 2023: 1
FWCI
11.62
Type
journal article
Language
en
Open access
closed
Locations
Journal of Management Information Systems (publishedVersion)
Topics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Forecasting Techniques and Applications
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Impact of AI and Big Data on Business and Society
  • Social Sciences › Business, Management and Accounting › Management Information Systems › Supply Chain and Inventory Management
Keywords
human-AI collaboration, demand forecasting, retail inventory management, forecast uncertainty, time horizon, forecast accuracy, augmentation, automation, human intervention

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review