CoChat

Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts

Open access
Zamfirescu-Pereira, J.D., Wong, Richmond Y., Hartmann, Bjoern, Yang, Qian · 2023 · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index; 18 of the 33 sources its claims cite were checked for support.

Abstract

Pre-trained large language models (“LLMs”) like GPT-3 can engage in fluent, multi-turn instruction-taking out-of-the-box, making them attractive materials for designing natural language interactions. Using natural language to steer LLM outputs (“prompting”) has emerged as an important design technique potentially accessible to non-AI-experts. Crafting effective prompts can be challenging, however, and prompt-based interactions are brittle. Here, we explore whether non-AI-experts can successfully engage in “end-user prompt engineering” using a design probe—a prototype LLM-based chatbot design tool supporting development and systematic evaluation of prompting strategies. Ultimately, our probe participants explored prompt designs opportunistically, not systematically, and struggled in ways echoing end-user programming systems and interactive machine learning systems. Expectations stemming from human-to-human instructional experiences, and a tendency to overgeneralize, were barriers to effective prompt design. These findings have implications for non-AI-expert-facing LLM-based tool design and for improving LLM-and-prompt literacy among programmers and the public, and present opportunities for further research.

Overview

CoChat Summary

TL;DR

This design-probe study found that 10 non-AI experts could iterate on LLM prompts opportunistically but not systematically, and their struggles echoed end-user programming and interactive machine learning barriers. Barriers included overgeneralizing from single successes/failures and applying human-to-human social expectations to prompt design.

Key findings

  • All 10 participants engaged in ad hoc, opportunistic prompt iteration with local testing; two out of ten needed assistance to begin, but then could iterate on their own.
  • Zero participants used BotDesigner’s systematic prompt-testing interface for the prompt design task, and none elicited more than one or two conversations before jumping in to fix errors.
  • Participants overgeneralized from single observations, often declaring success after one correct response or assuming incapability after one failed prompt; all held a strong bias for direct instruction over in-line examples, and only P5 independently added example dialog to the preamble.
  • Human-human interaction expectations shaped behavior: some mixed instructions to the bot with instructions to the bot’s user, expected memory across conversations, used polite phrasing such as “please,” and avoided repetition even when encouraged.

Methods

  • The authors built BotDesigner, a no-code React/node.js prompt-design probe using OpenAI’s GPT-3 `text-davinci-002` with temperature set to 0, supporting prompt-template authoring, conversation testing, error labeling, and systematic replay.
  • They conducted a think-aloud user study with 10 participants with little to no prompt-design experience, asking them to improve a baseline instructional chatbot (“CarlaBot”) toward behaviors from a chef transcript, with up to 1 hour for the task.
  • Analysis involved transcribing videos, exploratory data analysis, affinity diagrams, and a service blueprint to compare prompt-design approaches and struggles across participants.

Limitations

  • Stated: the participant pool was small and skewed toward professionals and graduate students in STEM-related fields, likely early adopters, so findings are not claimed to be universal or complete; the time given for prompt design was relatively limited.
  • Stated: interviewer advice was not intentionally varied across participants, and because participants focused on different errors, not all received the same advice or advice in the same order.
  • Evident from the text: the study used one tool, one underlying model, and a chatbot domain, so the authors note open questions about generalization to other domains and broader populations.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3544548.3581388

Cites

51 of 51 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

12 Valid · 6 Not valid · 8 Not checked · 83 Own results and interpretations

45 more claims

Create a free account to see all 109 claims and to verify the 1 reference still open.

See all claims — free account
  • Probably not a substantial claim. 38
  • It may state two separate claims as one. 28
  • The quote only partly states this claim. 2
  • Borderline: one of the tests for a substantial claim is unclear. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

6 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 51 references

Research works printed but not linked: 1

  1. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (Eds.). Vol. 33. Curran Associates, Inc., 1877–1901.

Cited by

See all 1,274

Metadata

Authors
Zamfirescu-Pereira, J. D., Wong, R. Y., Hartmann, B., & Yang, Q.
Affiliations
  • University of California, Berkeley (US)
  • Georgia Institute of Technology (US)
  • EECS, UC Berkeley, United States
  • Cornell University (US)
Venue
Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2023-04-19
Pages
1-21
OpenAlex
W4366548330
Cited by
1,274
References
37
Citations by year
2026: 484 · 2025: 407 · 2024: 319 · 2023: 63
FWCI
144.17
Type
conference paper
Language
en
Open access
Locations
CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by-nc)
Funders
USAF/DARPA (FA8750-20-C-0156, FA8750-20-C-0074, and FA8750-20-C0155 (SDCPS Program))
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
Keywords
prompt engineering, large language models, non-expert users, end-user programming, chatbot design, design probes, overgeneralization, natural language interfaces, human-computer interaction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts. https://app.cochat.ai/papers/10.1145/3544548.3581388
The paper
Zamfirescu-Pereira, J. D., Wong, R. Y., Hartmann, B., & Yang, Q. (2023). Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544548.3581388

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.