Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality
Open accessThe DOI matches its Crossref record; 88 of the 88 references it cites were found in a registry or an index.
Abstract
We introduce and study the concept of a “jagged technology frontier” to describe the uneven impact of artificial intelligence (AI) capabilities, where AI assistance improves performance for some tasks but worsens it for others, even within the same knowledge workflow and with a seemingly similar level of difficulty. In collaboration with the global management consulting firm Boston Consulting Group, we have developed realistic management consulting tasks and examined the human performance implications of using AI to perform complex and knowledge-intensive work. The preregistered experiment involved 758 knowledge workers. After establishing a performance baseline on similar tasks, subjects were randomly assigned to one of three conditions: no AI access, GPT-4 AI access, or GPT-4 AI access with a prompt engineering overview. For each one of a set of 18 realistic knowledge tasks within the frontier of AI capabilities ranging from creative to analytical tasks, subjects using AI outperformed those not using AI, completing 12.2% more tasks and completing them 25.1% more quickly on average while also delivering solutions of significantly improved quality. However, for a complex managerial task selected to be outside the frontier, subjects using AI were 19% less likely to produce correct solutions compared with those without AI, pointing to potential limitations of AI supporting knowledge workers. We discuss the positive and negative implications of AI-aided human performance in knowledge-intensive tasks.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We already see widespread adoption of AI in business and the world at large [16].”
From Incorporating generative AI into a writing-intensive undergraduate course without off-loading learning (2025) · cites this paper as [16]
In the citing paper
“Previous studies have found that LLMs increase productivity the most for low-performing workers (Brynjolfsson et al. 2023, Noy and Zhang 2023, Dell'Acqua et al. 2023).”
From AI Meets the Classroom: When Do Large Language Models Harm Learning? (2024) · cites this paper as Dell'Acqua et al. 2023
In the citing paper
“In a laboratory experiment on consultants employed by Boston Consulting Group, Dell'Acqua et al. (2023) found that productivity on 18 tasks designed to mimic the day-to-day work at a consulting company increased by 12% to 25%.”
From The Productivity Effects of Generative AI: Evidence from a Field Experiment with GitHub Copilot (2024) · cites this paper as Dell'Acqua et al. (2023)
In the citing paper
“Gains were most impressive for historically below average performers, who improved performance 43%, compared to above average performers increasing performance 17%.”
From Is it all hype? ChatGPT’s performance and disruptive potential in the accounting and auditing industries (2024) · cites this paper as Dell'Acqua et al. (2023)
In the citing paper
“They showed dramatic improvements for tasks that AI can perform-consultants performed tasks 25.1% more quickly and completed 12.2% more tasks with quality 40% higher than a control group.”
From Is it all hype? ChatGPT’s performance and disruptive potential in the accounting and auditing industries (2024) · cites this paper as Dell'Acqua et al. (2023)
15 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
The Skill Content of Recent Technological Change: An Empirical Exploration
Autor, D. H., Levy, F., & Murnane, R. J. (2003) · The Quarterly Journal of Economics · doi:10.1162/003355303322552801 · Cited by 6,205
Kung, T. H., Cheatham, M., ChatGPT, Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., et al. (2022) · doi:10.1101/2022.12.19.22283643 · Cited by 4,115
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083
Skills, Tasks and Technologies: Implications for Employment and Earnings
Acemoglu, D., & Autor, D. (2011) · Handbook of Labor Economics · doi:10.1016/s0169-7218(11)02410-5 · Cited by 3,799
Agarwal, R., & Prasad, J. (1998) · Information Systems Research · doi:10.1287/isre.9.2.204 · Cited by 3,575
Human trust in artificial intelligence: Review of empirical research
Glikson, E., & Woolley, A. W. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0057 · Cited by 2,921
Artificial Intelligence and Management: The Automation–Augmentation Paradox
Raisch, S., & Krakowski, S. (2021) · Academy of Management Review · doi:10.5465/amr.2018.0072 · Cited by 2,755
Algorithms at Work: The New Contested Terrain of Control
Kellogg, K. C., Valentine, M. A., & Christin, A. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0174 · Cited by 2,732
Self-citationAutomation and New Tasks: How Technology Displaces and Reinstates Labor
Acemoglu, D., & Restrepo, P. (2019) · Journal of Economic Perspectives · doi:10.1257/jep.33.2.3 · Cited by 2,612
Research works printed but not linked: 16
- A Agrawal, 2018, Prediction Machines: The Simple Economics of Artificial Intelligence
- A Blandin, 2026, The rapid adoption of generative AI, Management Sci
- Z Buc �inca, 2021, To trust or to think: Cognitive forcing functions can reduce overreliance on AI in
- Dell'acqua, Navigating the Jagged Technological Frontier Organization Science, Articles in Advance, Proc. ACM Human-Comput. Interaction, 5
- E Sogancioglu, 2021, Deep learning for chest X-ray analysis: A survey, Medical Image Anal, 73
- F Dell'acqua, 2022, Falling asleep at the wheel: Human/AI collaboration in a field experiment on HR recruiters. Working paper, Laboratory for Innovation Science
- P F Drucker, 1959, Landmarks of Tomorrow: A Report on the New Post Modern World
- M Iansiti, 2020, Competing in the Age of AI: Strategy and Leadership When Algorithms and Networks Run the World
- A Karpathy, 2024, Commentary on jagged capabilities of large language models, Twitter
- A Karpathy, 2025, 2025 LLM year in review
- L Meincke, 2024, Using large language models for idea generation in innovation. Working paper, Operations, Information and Decisions
- S Pichai, 2025, Interview with Lex Fridman, Lex Fridman Podcast #471
- S Randazzo, 2025, GenAI as a power persuader: How professionals get persuasion bombed when they attempt to validate LLMs
- S Reed, 2022, A generalist agent
- Dell'acqua, Navigating the Jagged Technological Frontier
- K Vlamis, 2024, MBB explained: How hard it is to get hired and what it's like to work for the prestigious strategy consulting firms, McKinsey, Bain, and BCG. Bus. Insider Africa
Cited by
Brynjolfsson, E., Li, D., & Raymond, L. (2025) · The Quarterly Journal of Economics · doi:10.1093/qje/qjae044 · Cited by 1,083
Jackson, I., Ivanov, D., Dolgui, A., & Namdar, J. (2024) · International Journal of Production Research · doi:10.1080/00207543.2024.2309309 · Cited by 441
AI literacy and its implications for prompt engineering strategies
Knoth, N., Tolzin, A., Janson, A., & Leimeister, J. M. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100225 · Cited by 368
“Dell ’ Acqua et al. (2023) showed the positive effects on consultant work outcomes, thus, this technology will be adopted industry-wide and we need to train our students in prompting LLMs.”
BackgroundThe Crowdless Future? Generative AI and Creative Problem-Solving
Boussioux, L., Lane, J. N., Zhang, M., Jacimovic, V., & Lakhani, K. R. (2024) · Organization Science · doi:10.1287/orsc.2023.18430 · Cited by 329
Self-citationLarge Legal Fictions: Profiling Legal Hallucinations in Large Language Models
Dahl, M., Magesh, V., Suzgun, M., & Ho, D. E. (2024) · Journal of Legal Analysis · doi:10.1093/jla/laae003 · Cited by 559
Metadata
- Authors
- Dell'Acqua, F., McFowland III, E., Mollick, E. R., Lifshitz-Assaf, H., Kellogg, K., Rajendran, S., Krayer, L., Candelon, F., & Lakhani, K. R.
- Affiliations
- University of Pennsylvania (US)
- Massachusetts Institute of Technology (US)
- Boston Consulting Group (United States) (US)
- Harvard University (US)
- Venue
- SSRN Electronic Journal
- Publisher
- Elsevier BV
- Published
- 2023
- ISSN
- 1556-5068 (ISSN-L)
- OpenAlex
- W4386827556
- Semantic Scholar
- 409dc9f1801947f85c53231cba346f0e314cb208
- Cited by
- 794
- References
- 67
- Citations by year
- 2026: 240 · 2025: 305 · 2024: 230 · 2023: 18
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Locations
- SSRN Electronic Journal (acceptedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Social Sciences › Decision Sciences › Management Science and Operations Research › Impact of AI and Big Data on Business and Society
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › AI and HR Technologies
- Keywords
- artificial intelligence, knowledge workers, GPT-4, prompt engineering, human-AI collaboration, task performance, solution quality, knowledge-intensive work
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.