The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Algorithm-Augmented Work and Domain Experience: The Countervailing Forces of Ability and Aversion
Open accessAbstract
Past research offers mixed perspectives on whether domain experience helps or hurts algorithm-augmented worker performance. Reconciling these perspectives, we theorize that intermediate levels of domain experience are optimal for algorithm-augmented performance, due to the interplay between two countervailing forces—ability and aversion. Although domain experience can increase performance via increased ability to complement algorithmic advice (e.g., identifying inaccurate predictions), it can also decrease performance via increased aversion to accurate algorithmic advice. Because ability developed through learning by doing increases at a decreasing rate, and algorithmic aversion is more prevalent among experts, we theorize that algorithm-augmented performance will first rise with increasing domain experience, then fall. We test this by exploiting a within-subjects experiment in which corporate information technology support workers were assigned to resolve problems both manually and using an algorithmic tool. We confirm that the difference between performance with the algorithmic tool versus without the tool was characterized by an inverted U-shape over the range of domain experience. Only workers with moderate domain experience did significantly better using the algorithm than resolving tickets manually. These findings highlight that, even if greater domain experience increases workers’ ability to complement algorithms, domain experience can also trigger other mechanisms that overcome the positive ability effect and inhibit performance. Additional analyses and participant interviews suggest that, even though the highest experience workers had the greatest ability to complement the algorithmic tool, they rejected its advice because they felt greater accountability for possible unintended consequences of accepting algorithmic advice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Causality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653
The dynamic resource‐based view: capability lifecycles
Helfat, C. E., & Peteraf, M. A. (2003) · Strategic Management Journal · doi:10.1002/smj.332 · Cited by 4,553
Chase, W. G., & Simon, H. A. (1973) · Cognitive Psychology · doi:10.1016/0010-0285(73)90004-2 · Cited by 4,479 · cited 1×
“Domain experience allows people to build context-dependent perceptual structures that help them frame problems, detect relevant signals, and react intuitively and appropriately when making complex evaluations (Chase and Simon 1973, Simon 1991, Salas et al. 2010).”
Why Are There Still So Many Jobs? The History and Future of Workplace Automation
Autor, D. H. (2015) · Journal of Economic Perspectives · doi:10.1257/jep.29.3.3 · Cited by 3,917
Bounded rationality and organizational learning
Simon, H. A. (1991) · Organization Science · doi:10.1287/orsc.2.1.125 · Cited by 3,028
Algorithms at Work: The New Contested Terrain of Control
Kellogg, K. C., Valentine, M. A., & Christin, A. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0174 · Cited by 2,732
Conditions for intuitive expertise: A failure to disagree.
Kahneman, D., & Klein, G. (2009) · American Psychologist · doi:10.1037/a0016755 · Cited by 2,443 · cited 1×
“Domain experience gleaned from learning by doing may have a limited impact on ability in contexts with high causal ambiguity (Kahneman and Klein 2009)—such as geopolitical forecasting (Tetlock 2009).”
BackgroundBarley, S. R. (1986) · Administrative Science Quarterly · doi:10.2307/2392767 · Cited by 2,315
Algorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
Experimental methods: Between-subject and within-subject design
Charness, G., Gneezy, U., & Kuhn, M. A. (2012) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2011.08.009 · Cited by 1,469 · cited 1×
“In a within-subjects design, each participant is subjected to both the treatment and the control conditions, which yields a causal estimate if the treatment and control exposures can be considered independent (Charness et al. 2012).”
BackgroundExpert political judgment: how good is it? how can we know?
(2006) · Choice Reviews Online · doi:10.5860/choice.43-3634 · Cited by 1,260 · cited 1×
“Domain experience gleaned from learning by doing may have a limited impact on ability in contexts with high causal ambiguity (Kahneman and Klein 2009)—such as geopolitical forecasting (Tetlock 2009).”
BackgroundWhat can machine learning do? Workforce implications
Brynjolfsson, E., & Mitchell, T. (2017) · Science · doi:10.1126/science.aap8062 · Cited by 1,131 · cited 1×
“Human complementarity with algorithms arises from humans’ relative advantages in using tacit knowledge, causal reasoning, better understanding of context, and human values to make judgments (Autor 2015; Agrawal et al. 2018, 2019; Brynjolfsson and Mitchell 2017; Brynjolfsson et al. 2018).”
BackgroundOrganizational Decision-Making Structures in the Age of Artificial Intelligence
Shrestha, Y. R., Ben-Menahem, S. M., & von Krogh, G. (2019) · California Management Review · doi:10.1177/0008125619862257 · Cited by 1,108
Estimating and testing mediation and moderation in within-subject designs.
Judd, C. M., Kenny, D. A., & McClelland, G. H. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.2.115 · Cited by 846
What Can Machines Learn and What Does It Mean for Occupations and the Economy?
Brynjolfsson, E., Mitchell, T., & Rock, D. (2018) · AEA Papers and Proceedings · doi:10.1257/pandp.20181019 · Cited by 695
Pustejovsky, J. E., & Tipton, E. (2018) · Journal of Business & Economic Statistics · doi:10.1080/07350015.2016.1247004 · Cited by 492
Algorithms in practice: Comparing web journalism and criminal justice
Christin, A. (2017) · Big Data & Society · doi:10.1177/2053951717718855 · Cited by 379
Machine learning and human capital complementarities: Experimental evidence on bias mitigation
Choudhury, P., Starr, E., & Agarwal, R. (2020) · Strategic Management Journal · doi:10.1002/smj.3152 · Cited by 293
Self-citation
Cited by
Bankins, S., Ocampo, A. C., Marrone, M., Restubog, S. L. D., & Woo, S. E. (2024) · Journal of Organizational Behavior · doi:10.1002/job.2735 · Cited by 799
Dell'Acqua, F., McFowland III, E., Mollick, E. R., Lifshitz-Assaf, H., Kellogg, K., Rajendran, S., Krayer, L., Candelon, F., & Lakhani, K. R. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4573321 · Cited by 794
The Crowdless Future? Generative AI and Creative Problem-Solving
Boussioux, L., Lane, J. N., Zhang, M., Jacimovic, V., & Lakhani, K. R. (2024) · Organization Science · doi:10.1287/orsc.2023.18430 · Cited by 329
Combining Human and Artificial Intelligence: Hybrid Problem-Solving in Organizations
Raisch, S., & Fomina, K. (2025) · Academy of Management Review · doi:10.5465/amr.2021.0421 · Cited by 280
Digital governance: A conceptual framework and research agenda
Hanisch, M., Goldsby, C. M., Fabian, N. E., & Oehmichen, J. (2023) · Journal of Business Research · doi:10.1016/j.jbusres.2023.113777 · Cited by 267
Metadata
- Authors
- Allen, R., & Choudhury, P. (.
- Venue
- Organization Science
- Published
- 2022-01
- Volume
- 33
- Issue
- 1
- Pages
- 149-169
- ISSN
- 1047-7039 (ISSN-L)
- 1526-5455
- OpenAlex
- W4200429321
- Cited by
- 177
- References
- 18
- FWCI
- 16.53
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.