Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting
Open accessAbstract
AI assistance in expertise-intensive work presents a tension between performance and learning: AI tools may raise the quality of the work a professional delivers while stunting the expertise they would otherwise develop in producing it.In a pre-registered three-month randomized controlled trial, we gave 133 practicing patent lawyers at eleven U.S. intellectual property law firms access to a custom AI patent drafting assistant and measured both their performance while using AI and their professional judgment afterward without it.All work was scored by blinded expert patent attorneys.AI access raised the quality of work delivered on benchmark patent drafting tasks at 10 days (0.34 SD, p = 0.03) and 90 days (0.38 SD, p = 0.01), with larger gains among junior lawyers.After three months, all subjects redlined an existing patent application without AI, a core task of patent practice that requires expert judgment.Treated lawyers outperformed controls by 0.32 SD (p = 0.04), but this advantage was concentrated entirely among senior lawyers (0.45 SD, p = 0.02).Junior lawyers showed no average gain.Their scores instead bifurcated, with sharply fewer mediocre scores offset by more poor and more good ones, suggesting that AI assistance served as a springboard for some juniors and a cushion for others.Foundational expertise may be a prerequisite for extracting durable skill from AIassisted practice.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A few studies examine whether AI-assisted work improves subsequent unassisted performance, with mixed results.
“A few studies examine whether AI-assisted work improves subsequent unassisted performance, with mixed results.”
From Experimental Evidence on the Learning Impact of Generative AI (2026) · cites this paper as Autor et al. (2026)
Autor et al. (2026) find that an AI drafting assistant improves subsequent unassisted performance among senior patent lawyers, though not among junior patent lawyers.
“Autor et al. (2026) also find that an AI drafting assistant improves subsequent unassisted performance among senior patent lawyers, though not among junior patent lawyers.”
From Experimental Evidence on the Learning Impact of Generative AI (2026) · cites this paper as Autor et al. (2026)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 31
- Abadie, A., Athey, S., Imbens, G. W., and Wooldridge, J. M. (2023). When should you adjust standard errors for clustering? The Quarterly Journal of Economics, 138(1):1-35.
- Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., and Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122.
- Bick, A., Blandin, A., Deming, D. J., Fuchs-Schündeln, N., and Jessen, J. (2026). Mind the gap: AI adoption in Europe and the US. Brookings papers on economic activity conference draft, Brookings Papers on Economic Activity.
- Brynjolfsson, E., Li, D., and Raymond, L. R. (2024). Generative AI at work. The Quarterly Journal of Economics, 139(4):1721-1784.
- Budzyń, K., Romańczyk, M., Kitala, D., Kołodziej, P., Bugajski, M., Adami, H. O., Blom, J., Buszkiewicz, M., Halvorsen, N., Hassan, C., et al. (2025). Endoscopist deskilling risk after exposure to artificial intelligence in colonoscopy: a multicentre, observational study. The Lancet Gastroenterology & Hepatology, 10(10):896-903.
- Chien, C. V. and Kim, M. (2025). Generative AI and legal aid: Results from a field study and 100 use cases to bridge the access to justice gap. Loyola of Los Angeles Law Review, 57(4):903.
- Chiong, K. and Xie, Y. (2024). Generative AI and job satisfaction: Evidence from glassdoor employee reviews. Working Paper, Available at SSRN.
- Choi, J. H., Monahan, A. B., and Schwarcz, D. (2024). Lawyering in the age of artificial intelligence. Minnesota Law Review, 109(1):147-218.
- Choi, J. H. and Schwarcz, D. (2025). AI assistance in legal analysis: An empirical study. Journal of Legal Education, 73. Available at SSRN.
- Dell'Acqua, F., McFowland III, E., Mollick, E. R., Lifshitz-Assaf, H., Kellogg, K., Rajendran, S., Krayer, L., Candelon, F., and Lakhani, K. R. (2026). Navigating the jagged technological frontier: Field experimental evidence of the effects of Artificial Intelligence on knowledge worker productivity and quality. Organization Science, 37(2):403-423.
- Emanuel, N., Harrington, E., and Pallais, A. (2026). The power of proximity to coworkers: Training for tomorrow or productivity today? The Quarterly Journal of Economics, 141(3):1825-1870.
- Goh, E., Gallo, R., Hom, J., Strong, E., et al. (2024). Large language model influence on diagnostic reasoning: A randomized clinical trial. JAMA Network Open, 7(10):e2440969.
- Haslberger, M. et al. (2025). Rage against the machine? generative AI exposure, subjective risk, and policy preferences. Journal of European Public Policy.
- Holt, C. A. and Sullivan, S. P. (2023). Permutation tests for experimental data. Experimental Economics, pages 1-38. Epub ahead of print. PMID: 37363160; PMCID: PMC10066020.
- Hosseini, S. M. M. and Lichtinger, G. (2025). Generative AI as seniority-biased technological change: Evidence from U.S. résumé and job posting data. Working Paper 5425555, SSRN.
- Humlum, A. and Vestergaard, E. (2025b). The unequal adoption of ChatGPT exacerbates existing inequalities among workers. Proceedings of the National Academy of Sciences.
- Lambert, P. J. and Schindler, Y. (2026). The broken ladder: AI, remote work, and early-career hiring. Working Paper 6787638, SSRN.
- Lee, H.-P., Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R., and Wilson, N. (2025). The impact of generative AI on critical thinking: Self-reported reductions in cognitive effort and confidence effects from a survey of knowledge workers. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. ACM.
- Martin, L., Whitehouse, N., Yiu, S., Catterson, L., and Perera, R. (2024). Better call gpt, comparing large language models against lawyers. arXiv preprint arXiv:2401.16212. · arXiv
- Nielsen, P. A. et al. (2024). Building a better lawyer: Experimental evidence that artificial intelligence can increase legal work efficiency. Journal of Empirical Legal Studies, 21(4):979-1022.
- Noy, S. and Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science, 381(6654):187-192.
- Peng, S., Kalliamvakou, E., Cihon, P., and Demirer, M. (2023). The impact of AI on developer productivity: Evidence from github copilot. arXiv:2302.06590. · arXiv
- Rapoport, N. B. and Tiano, Jr., J. R. (2025). Fighting the hypothetical: Why law firms should rethink the billable hour in the generative AI era. Washington Journal of Law, Technology & Arts, 20(2):41.
- Remus, D. and Levy, F. (2017). Can robots be lawyers? computers, lawyers, and the practice of law. Georgetown Journal of Legal Ethics, 30(3):501-558.
- Robert, A. (2026). Sanctions ramping up in cases involving AI hallucinations. ABA Journal.
- Schwarcz, D., Manning, S., Barry, P. J., Cleveland, D. R., Prescott, J. J., and Rich, B. (2026). AI-powered lawyering: AI reasoning models, retrieval augmented generation, and the future of legal practice. Journal of Law and Empirical Analysis. Forthcoming. Minnesota Legal Studies Research Paper No. 25-16; U of Michigan Public Law Research Paper No. 24-058.
- Usdan, M., Connell Pensky, A. E., and Chang, R. (2024). Generative AI's impact on graduate student writing productivity and quality. Working paper, SSRN.
- Vaccaro, M., Almaatouq, A., and Malone, T. (2024). When combinations of humans and AI are useful: A systematic review and meta-analysis. Nature Human Behaviour, 8:2293-2303.
- Wiles, E., Krayer, L., Abbadi, M., Awasthi, U., Kennedy, R., Mishkin, P., Sack, D., and Can- delon, F. (2024). GenAI as an exoskeleton: Experimental evidence on knowledge work- ers using GenAI on new skills. Working paper, Boston University. Available at SSRN: https://ssrn.com/abstract=4944588.
- Wrzesniowska, L. (2023). Can ai make a case? ai vs. lawyer in the dutch legal context. International Journal of Law, Ethics, and Technology. Reminder 2: When you finish the task, please go to https://www.tickcounter.com/stopwatch and click "Stop" on the timer, then please record your time on task at the top of your submission. Reminder 3: Do not use InFlow or any other form of AI for this task.
- Overly Structured Feedback: Submitting the "top-down feedback" as a highly stylized, excessively nested outline (e.g., multi-level bullet points summarizing the review) that looks machine-generated rather than a human's free-form thoughts.
Cited by
Sorted by citations
Experimental Evidence on the Learning Impact of Generative AI
Contractor, Z., & Reyes, G. (2026) · arXiv · doi:10.48550/arxiv.2607.08849
- Authors
- Autor, D., Rodchenko, T., Martin, J., Iscenko, Z., Strand, S., Pearl, D., & Ferere, M.
- Venue
- National Bureau of Economic Research
- Publisher
- National Bureau of Economic Research
- Published
- 2026-09
- OpenAlex
- W7211893772
- Semantic Scholar
- 37075a006bd662b869f62447629d789d066e8b32
- Cited by
- 0
- References
- 0
- Type
- report
- Language
- en
- Open access
- green
- doi.org
- Locations
- National Bureau of Economic Research (publishedVersion)
- SSRN Electronic Journal (acceptedVersion)
- Funders
- Smith Richardson Foundation
- Sid W. Richardson Foundation
- NOMIS Stiftung
- Topics
- Social Sciences › Political Science and International Relations › Artificial Intelligence in Law
- Physical Sciences › Computer Science › Artificial Intelligence › Law, AI, and Intellectual Property
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › AI and HR Technologies
- Keywords
- AI-assisted drafting, patent drafting, professional expertise, professional judgment, skill erosion, white-collar work
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
