PromptChainer: Chaining Large Language Model Prompts through Visual Programming
Open accessAbstract
While LLMs have made it possible to rapidly prototype new ML functionalities, many real-world applications involve complex tasks that cannot be easily handled via a single run of an LLM. Recent work has found that chaining multiple LLM runs together (with the output of one step being the input to the next) can help users accomplish these more complex tasks, and in a way that is perceived to be more transparent and controllable. However, it remains unknown what users need when authoring their own LLM chains – a key step to lowering the barriers for non-AI-experts to prototype AI-infused applications. In this work, we explore the LLM chain authoring process. We find from pilot studies that users need support transforming data between steps of a chain, as well as debugging the chain at multiple granularities. To address these needs, we designed PromptChainer, an interactive interface for visually programming chains. Through case studies with four designers and developers, we show that PromptChainer supports building prototypes for a range of applications, and conclude with open questions on scaling chains to even more complex tasks, as well as supporting low-fi chain prototyping.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AI Chains successfully enables designers to explore prompt and chaining strategies more efficiently and strategically.
“The tool successfully enables designers to explore prompt and chaining strategies more efciently and strategically [48].”
From Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts (2023) · cites this paper as [48]
The Study Results (pp. 5–6) show that PromptChainer supported different chain-construction strategies and helped participants refine prompts. They do not establish that designers explored strategies more efficiently; the Discussion and Limitations (p. 7) treats rapid exploration of alternative chains as a goal for future work. (LLM, conf=0.88)
Experimental — accuracy not yet measuredMany users write their LLM pipelines as chains of multiple LLM calls.
“Second, many users write their LLM pipelines as chains of multiple LLM calls [1, 11, 14, 53].”
From Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences (2024) · cites this paper as [53]
The Introduction (p. 2) describes chains of multiple LLM steps and reports that four designers and developers built chains in the study. This supports the practice described, but the paper does not establish that many users write their pipelines this way. (LLM, conf=0.86)
Experimental — accuracy not yet measuredProcessing fluency-the subjective ease with which a cognitive task is performed-can influence people's confidence in information accuracy.
“One relevant type of cue-'processing fluency',”
From The Metacognitive Demands and Opportunities of Generative AI (2024) · cites this paper as [201]
The Introduction (p. 2) and Study Results (p. 6) discuss LLM prompt chaining, not processing fluency or judgments of information accuracy. Searches for “processing fluency” and for ease of processing affecting confidence found no passage addressing the claim in our copy. (LLM, conf=0.76)
Experimental — accuracy not yet measuredSelf-explanation prompts improved students' accuracy in evaluating their own understanding of information.
“Likewise, self-explanation prompts improved students' accuracy in evaluating their own understanding of information [201].”
From The Metacognitive Demands and Opportunities of Generative AI (2024) · cites this paper as [201]
The Study Design (p. 5) describes four designers and developers building LLM prompt chains, not students responding to self-explanation prompts. The Study Results (pp. 5–6) discuss chain construction and debugging, but do not assess students’ accuracy in judging their own understanding. Searches for self-explanation and metacognitive accuracy found no relevant passage. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Wu, T. S., Terry, M., & Cai, C. (2021) · International Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517582 · Cited by 757 · see citing works
Self-citationRe-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design
Yang, Q., Steinfeld, A., Rosé, C., & Zimmerman, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376301 · Cited by 622 · see citing works
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
Lu, Y., Bartolo, M., Moore, A., Riedel, S., & Stenetorp, P. (2021) · arXiv · doi:10.48550/arxiv.2104.08786 · Cited by 615 · cited 1× · see citing works
“To invoke the desired functionalities, users need to write prompts [1, 11, 12] that are appropriate for the task.”
BackgroundParallel prototyping leads to better design results, more divergence, and increased self-efficacy
Dow, S. P., Glassco, A., Kass, J., Schwarz, M., Schwartz, D. L., & Klemmer, S. R. (2010) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1879831.1879836 · Cited by 367 · see citing works
Kittur, A., Smus, B., Khamkar, S., & Kraut, R. E. (2011) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2047196.2047202 · Cited by 309 · see citing works
Burnett, M., Cook, C. R., & Rothermel, G. (2004) · Communications of the ACM · doi:10.1145/1015864.1015889 · Cited by 214 · see citing works
Story Centaur: Large Language Model Few Shot Learning as a Creative Writing Tool
Swanson, B., Mathewson, K. W., Pietrzak, B., Chen, S., & Dinalescu, M. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-demos.29 · Cited by 56 · see citing works
Decision Tree Design for Classification in Crowdsourcing Systems
Geng, B., Li, Q., & Varshney, P. K. (2018) · 2018 52nd Asilomar Conference on Signals, Systems, and Computers · doi:10.1109/acssc.2018.8645073 · Cited by 12 · see citing works
End-User Software Engineering and Why it Matters
Burnett, M. (2010) · Journal of Organizational and End User Computing · doi:10.4018/joeuc.2010101904 · Cited by 11 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505 · see citing works
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187 · see citing works
“…environment like the one we inhabit [10, 59], and strive to behave in ways that exhibit emergent behaviors grounded in social interactions with users or other agents with the aim of becoming believable proxies of our behavior in hypothetical simulations of individuals and communities [20, 36, 71].”
BackgroundSelf-citationWhy Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts
Zamfirescu-Pereira, J. D., Wong, R. Y., Hartmann, B., & Yang, Q. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581388 · Cited by 893 · see citing works
Graph of Thoughts: Solving Elaborate Problems with Large Language Models
Besta, M., Blach, N., Kubicek, A., Gerstenberger, R., Podstawski, M., Gianinazzi, L., Gajda, J., Lehmann, T., Niewiadomski, H., Nyczyk, P., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29720 · Cited by 556 · see citing works
Katz, D. M., Bommarito, M. J., Gao, S., & Arredondo, P. (2024) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2023.0254 · Cited by 420 · see citing works
The Metacognitive Demands and Opportunities of Generative AI
Tankelevitch, L., Kewenig, V., Simkute, A., Scott, A. E., Sarkar, A., Sellen, A., & Rintel, S. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642902 · Cited by 280 · see citing works
Dynamic prompt-based virtual assistant framework for BIM information search
Zheng, J., & Fischer, M. (2023) · Automation in Construction · doi:10.1016/j.autcon.2023.105067 · Cited by 154 · see citing works
Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences
Shankar, S., Zamfirescu-Pereira, J. D., Hartmann, B., Parameswaran, A., & Arawjo, I. (2024) · Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3654777.3676450 · Cited by 151 · see citing works
ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing
Arawjo, I., Swoopes, C., Vaithilingam, P., Wattenberg, M., & Glassman, E. (2023) · arXiv · doi:10.48550/arxiv.2309.09128 · Cited by 146
Character-LLM: A Trainable Agent for Role-Playing
Shao, Y., Li, L., Dai, J., & Qiu, X. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.814 · Cited by 140 · see citing works
Prompting Is Programming: A Query Language for Large Language Models
Beurer-Kellner, L., Fischer, M., & Vechev, M. (2023) · Proceedings of the ACM on Programming Languages · doi:10.1145/3591300 · Cited by 117 · see citing works
Simple techniques to bypass GenAI text detectors: implications for inclusive education
Perkins, M., Roe, J., Vu, B. H., Postma, D., Hickerson, D., McGaughran, J., & Khuat, H. Q. (2024) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-024-00487-w · Cited by 117 · see citing works
“Examples include the use of delimiters (Sweenor & Ramanathan, 2023) and prompt chaining (T. Wu et al., 2022).”
AngleKindling: Supporting Journalistic Angle Ideation with Large Language Models
Petridis, S., Diakopoulos, N., Crowston, K., Hansen, M., Henderson, K., Jastrzebski, S., Nickerson, J. V., & Chilton, L. B. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580907 · Cited by 98 · see citing works
Wan, Q., Hu, S., Zhang, Y., Wang, P., Wen, B., & Lu, Z. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637361 · Cited by 88 · see citing works
Dang, H., Goller, S., Lehmann, F., & Buschek, D. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580969 · Cited by 83 · see citing works
Cognitive Architectures for Language Agents
Sumers, T. R., Yao, S., Narasimhan, K., & Griffiths, T. L. (2023) · arXiv · doi:10.48550/arxiv.2309.02427 · Cited by 72 · see citing works
Jang, S., Lee, G., Oh, J., Lee, J., & Koo, B. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.102532 · Cited by 77 · see citing works
LaMPost: Design and Evaluation of an AI-assisted Email Writing Prototype for Adults with Dyslexia
Goodman, S. M., Buehler, E., Clary, P., Coenen, A., Donsbach, A., Horne, T. N., Lahav, M., MacDonald, R., Michaels, R. B., Narayanan, A., et al. (2022) · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3517428.3544819 · Cited by 71 · see citing works
Self-citationLiu, Y., Chen, S., Cheng, H., Yu, M., Ran, X., Mo, A., Tang, Y., & Huang, Y. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642698 · Cited by 61 · see citing works
Generating Multiple Choice Questions for Computing Courses Using Large Language Models
Tran, A., Angelikas, K., Rama, E., Okechukwu, C., Smith, D. H., & MacNeil, S. (2023) · 2023 IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie58773.2023.10342898 · Cited by 60 · see citing works
Reza, M., Laundry, N. M., Musabirov, I., Dushniku, P., Yu, Z. Y. “., Mittal, K., Grossman, T., Liut, M., Kuzminykh, A., & Williams, J. J. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641899 · Cited by 52 · see citing works
Beyond Text-to-Image: Multimodal Prompts to Explore Generative AI
Liu, V. (2023) · Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544549.3577043 · Cited by 38 · see citing works
Shen, J. J., King Chen, J., Findlater, L., & Dietz Smith, G. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713405 · Cited by 24 · see citing works
Enhancing memory retrieval in generative agents through LLM-trained cross attention networks
Hong, C., & He, Q. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1591618 · Cited by 7 · see citing works
“LLM-based agents primarily utilize prompt chains (Wu T. et al., 2022) to generate concise natural language descriptions and actions for characters within prototype systems, thereby creating populated prototypes for social computing systems (Park J. S. et al., 2022).”
What Does It Take to Be a Good AI Research Agent? Studying the Role of Ideation Diversity
Audran-Reiss, A., Armengol-Estapé, J., Hambardzumyan, K., Budhiraja, A., Josifoski, M., Toledo, E., Hazra, R., Magka, D., Shvartsman, M., Pathak, P., et al. (2025) · arXiv · doi:10.48550/arxiv.2511.15593
- Authors
- Wu, T., Jiang, E., Donsbach, A., Gray, J., Molina, A., Terry, M., & Cai, C. J.
- Affiliations
- University of Washington (US)
- Google (United States) (US)
- Venue
- CHI Conference on Human Factors in Computing Systems Extended Abstracts
- Publisher
- ACM
- Published
- 2022-04-27
- Pages
- 1-10
- OpenAlex
- W4221159404
- Cited by
- 198
- References
- 9
- Citations by year
- 2026: 30 · 2025: 66 · 2024: 63 · 2023: 35 · 2022: 4
- FWCI
- 31.68
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Software › Spreadsheets and End-User Computing
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Data Visualization and Analytics
- Keywords
- prompt chaining, visual programming, large language models, interactive interface design, non-expert users
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
