Friend or Foe? Teaming Between Artificial Intelligence and Workers with Variation in Experience
The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
As artificial intelligence (AI) applications become more pervasive, it is critical to understand how knowledge workers with different levels and types of experience can team with AI for productivity gains. We focus on the influence of two major types of human work experience (narrow experience based on the specific task volume and broad experience based on seniority) on the human-AI team dynamics. We developed an AI solution for medical chart coding in a publicly traded company and conducted a field study among the knowledge workers. Based on a detailed analysis performed at the medical chart level, we find evidence that AI benefits workers with greater task-based experience, but senior workers gain less from AI than their junior colleagues. Further investigation reveals that the relatively lower productivity lift from AI is not a result of seniority per se but lower trust in AI, likely triggered by the senior workers’ broader job responsibilities. This study provides new empirical insights into the differential roles of worker experience in the collaborative dynamics between AI and knowledge workers, which have important societal and business implications. This paper was accepted by Kartik Hosanagar, information systems. Funding: This work was supported by Inovalon [Sponsor of the Health Insights AI Laboratory]. Supplemental Material: The data files and online appendix are available at https://doi.org/10.1287/mnsc.2021.00588 .
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561
Machine learning: Trends, perspectives, and prospects
Jordan, M. I., & Mitchell, T. M. (2015) · Science · doi:10.1126/science.aaa8415 · Cited by 9,955
Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., et al. (2016) · JAMA · doi:10.1001/jama.2016.17216 · Cited by 7,697
Developing and validating trust measures for e-commerce: an integrative typology
McKnight, D. H., Choudhury, V., & Kacmar, C. (2002) · Information Systems Research · doi:10.1287/isre.13.3.334.81 · Cited by 5,550
Why Are There Still So Many Jobs? The History and Future of Workplace Automation
Autor, D. H. (2015) · Journal of Economic Perspectives · doi:10.1257/jep.29.3.3 · Cited by 3,917
The Black Box Society: The Secret Algorithms That Control Money and Information
Pasquale, F. (2015) · doi:10.4159/harvard.9780674736061 · Cited by 3,796
International evaluation of an AI system for breast cancer screening
McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1799-6 · Cited by 3,287
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
Artificial Intelligence and Management: The Automation–Augmentation Paradox
Raisch, S., & Krakowski, S. (2021) · Academy of Management Review · doi:10.5465/amr.2018.0072 · Cited by 2,755
Resistance to Medical Artificial Intelligence
Longoni, C., Bonezzi, A., & Morewedge, C. K. (2019) · Journal of Consumer Research · doi:10.1093/jcr/ucz013 · Cited by 2,015
Savova, G. K., Masanz, J. J., Ogren, P. V., Zheng, J., Sohn, S., Kipper-Schuler, K. C., & Chute, C. G. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.001560 · Cited by 1,967
Levels issues in theory development, data collection and analysis
Klein, K. J., Dansereau, F., & Hall, R. J. (1994) · The Academy of Management Review · doi:10.2307/258703 · Cited by 1,607
Competitive learning: From interactive activation to adaptive resonance
Grossberg, S. (1987) · Cognitive Science · doi:10.1111/j.1551-6708.1987.tb00862.x · Cited by 1,471
Mortality and Morbidity in the 21st Century
Case, A., & Deaton, A. (2017) · Brookings Papers on Economic Activity · doi:10.1353/eca.2017.0005 · Cited by 1,448
Extraneous factors in judicial decisions
Danziger, S., Levav, J., & Avnaim-Pesso, L. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1018033108 · Cited by 1,431
Trust in a specific technology: An investigation of its components and measures
Mcknight, D. H., Carter, M., Thatcher, J. B., & Clay, P. F. (2011) · ACM Transactions on Management Information Systems · doi:10.1145/1985347.1985353 · Cited by 1,185
Research Priorities for Robust and Beneficial Artificial Intelligence
Russell, S., Dewey, D., & Tegmark, M. (2015) · AI Magazine · doi:10.1609/aimag.v36i4.2577 · Cited by 637
Adapting to Artificial Intelligence: Radiologists and Pathologists as Information Specialists
Jha, S., & Topol, E. J. (2016) · JAMA · doi:10.1001/jama.2016.17438 · Cited by 622
Will Humans-in-the-Loop Become Borgs? Merits and Pitfalls of Working with AI
Fügener, A., Grahl, J., Gupta, A., & Ketter, W. (2021) · MIS Quarterly · doi:10.25300/misq/2021/16553 · Cited by 469
Commonsense reasoning and commonsense knowledge in artificial intelligence
Davis, E., & Marcus, G. (2015) · Communications of the ACM · doi:10.1145/2701413 · Cited by 434
Cognitive and System Factors Contributing to Diagnostic Errors in Radiology
Lee, C. S., Nagy, P. G., Weaver, S. J., & Newman-Toker, D. E. (2013) · American Journal of Roentgenology · doi:10.2214/ajr.12.10375 · Cited by 415
Does Machine Translation Affect International Trade? Evidence from a Large Digital Platform
Brynjolfsson, E., Hui, X., & Liu, M. (2019) · Management Science · doi:10.1287/mnsc.2019.3388 · Cited by 408
McCarthy, J. (2007) · Artificial Intelligence · doi:10.1016/j.artint.2007.10.009 · Cited by 396
Artificial Intelligence and Its Implications for Income Distribution and Unemployment
Korinek, A., & Stiglitz, J. (2017) · doi:10.3386/w24174 · Cited by 380
Data Analytics, Innovation, and Firm Productivity
Wu, L., Hitt, L., & Lou, B. (2020) · Management Science · doi:10.1287/mnsc.2018.3281 · Cited by 368
Job experience correlates of job performance.
McDaniel, M. A., Schmidt, F. L., & Hunter, J. E. (1988) · Journal of Applied Psychology · doi:10.1037/0021-9010.73.2.327 · Cited by 368
Challenges to Mismeasurement Explanations for the US Productivity Slowdown
Syverson, C. (2017) · Journal of Economic Perspectives · doi:10.1257/jep.31.2.165 · Cited by 302
Ethical limitations of algorithmic fairness solutions in health care machine learning
McCradden, M. D., Joshi, S., Mazwi, M., & Anderson, J. A. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30065-0 · Cited by 291
Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff
Bansal, G., Nushi, B., Kamar, E., Weld, D. S., Lasecki, W. S., & Horvitz, E. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33012429 · Cited by 284
Zhang (2021) · doi:10.1145/3432945 · Cited by 225
Dunlosky, J., & Hertzog, C. (2000) · Psychology and Aging · doi:10.1037/0882-7974.15.3.462 · Cited by 189
On the Utility of Learning about Humans for Human-AI Coordination
Carroll, M., Shah, R., Ho, M. K., Griffiths, T. L., Seshia, S. A., Abbeel, P., & Dragan, A. (2019) · arXiv · doi:10.48550/arxiv.1910.05789 · Cited by 168
Humans rely more on algorithms than social influence as a task becomes more difficult
Bogert, E., Schecter, A., & Watson, R. T. (2021) · Scientific Reports · doi:10.1038/s41598-021-87480-9 · Cited by 144
At your service on the table: Impact of tabletop technology on restaurant performance
Tan, T. F., & Netessine, S. (2020) · Management Science · doi:10.1287/mnsc.2019.3430 · Cited by 111
Characterizing Driving Styles with Deep Learning
Dong, W., Li, J., Yao, R., Li, C., Yuan, T., & Wang, L. (2016) · arXiv · doi:10.48550/arxiv.1607.03611 · Cited by 100
Lebovitz, S., Levina, N., & Lifshitz-Assaf, H. (2021) · MIS Quarterly · doi:10.25300/misq/2021/16564 · Cited by 49
Machine Learning Explainability for External Stakeholders
Bhatt, U., Andrus, M., Weller, A., & Xiang, A. (2020) · arXiv · doi:10.48550/arxiv.2007.05408 · Cited by 41
Cited by
When does AI pay off? AI-adoption intensity, complementary investments, and R&D strategy
Lee, Y. S., Kim, T., Choi, S., & Kim, W. (2022) · Technovation · doi:10.1016/j.technovation.2022.102590 · Cited by 141
Ramaul, L., Ritala, P., Kostis, A., & Aaltonen, P. (2026) · Journal of Management Studies · doi:10.1111/joms.13246 · Cited by 41
Wang, S., & Zhang, H. (2026) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-026-00585-x · Cited by 38
Sikhondze, B., Ye, H. J., Wang, W., Zhan, X., & Santhanam, R. (2025) · Journal of Management Information Systems · doi:10.1080/07421222.2025.2487314 · Cited by 18
“These studies reveal that users are often averse to algorithmic decision-making, largely due to issues of trust, performance expectations, perceived accuracy of advice, lack of autonomy, and implicit prejudices [10, 36, 72, 75].”
Self-citationClinical experience and perception of risk affect the acceptance and trust of using AI in medicine
Schulz, P. J., Kee, K. M., Lwin, M. O., Goh, W. W., Chia, K. Y., Cheung, M. F. K., Lam, T. Y. T., & Sung, J. J. Y. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1620127 · Cited by 12
Metadata
- Authors
- Wang, W., Gao, G. (., & Agarwal, R.
- Venue
- Management Science
- Published
- 2023-10-11
- ISSN
- 0025-1909 (ISSN-L)
- 1526-5501
- OpenAlex
- W4387539693
- Cited by
- 113
- References
- 41
- FWCI
- 31.80
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.