The DOI matches its Crossref record; 78 of the 78 references it cites were found in a registry or an index.
AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts
Open accessAbstract
Although large language models (LLMs) have demonstrated impressive potential on simple tasks, their breadth of scope, lack of transparency, and insufficient controllability can make them less effective when assisting humans on more complex tasks. In response, we introduce the concept of Chaining LLM steps together, where the output of one step becomes the input for the next, thus aggregating the gains per step. We first define a set of LLM primitive operations useful for Chain construction, then present an interactive system where users can modify these Chains, along with their intermediate results, in a modular way. In a 20-person user study, we found that Chaining not only improved the quality of task outcomes, but also significantly enhanced system transparency, controllability, and sense of collaboration. Additionally, we saw that users developed new ways of interacting with LLMs through Chains: they leveraged sub-tasks to calibrate model expectations, compared and contrasted alternative strategies by observing parallel downstream effects, and debugged unexpected model outputs by “unit-testing” sub-components of a Chain. In two case studies, we further explore how LLM Chains may be used in future applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361 · cited 1×
“These features can help facilitate various downstream applications, through a “pre-train, prompt, and predict” paradigm [45]: rather than having to train a new model for every custom task, users can just customize the model by feeding it a natural language prompt at run time.”
BackgroundLiu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352 · cited 1×
“These features can help facilitate various downstream applications, through a “pre-train, prompt, and predict” paradigm [45]: rather than having to train a new model for every custom task, users can just customize the model by feeding it a natural language prompt at run time.”
Background5th International Conference on Learning Representations (ICLR 17)
Havrylov, S., & Titov, I. (2017) · Cited by 3,503
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
GPT-3: Its Nature, Scope, Limits, and Consequences
Floridi, L., & Chiriatti, M. (2020) · Minds and Machines · doi:10.1007/s11023-020-09548-1 · Cited by 2,519
Guidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 3×
“Prior work has noted core challenges of the interaction, such as a lack of transparency, controllability, and user agency [5, 14, 33].”
BackgroundInfluentialSequence Level Training with Recurrent Neural Networks
Ranzato, M., Chopra, S., Auli, M., & Zaremba, W. (2015) · arXiv · doi:10.48550/arxiv.1511.06732 · Cited by 1,426
AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., & Singh, S. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.346 · Cited by 1,279
Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data
Bender, E. M., & Koller, A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.463 · Cited by 1,082 · cited 2×
“Because LLMs are designed to grasp the form of language, rather than the meaning [7], they can struggle on tasks like sequential arithmetic problems, multi-hop question answering, recognizing and comparing sentences, or those that require branching logic [9, 11, 26, 66, 67].”
BackgroundPrompt Programming for Large Language Models: Beyond the Few-Shot Paradigm
Reynolds, L., & McDonell, K. (2021) · Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411763.3451760 · Cited by 952 · cited 5×
“To overcome users’ own systematic omissions [74], an upstream unit could be designed to help users create sub-tasks to begin with, similar to metaprompting [58].”
BackgroundInfluentialVega-Lite: A Grammar of Interactive Graphics
Satyanarayan, A., Moritz, D., Wongsuphasawat, K., & Heer, J. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2016.2599030 · Cited by 883
Exploring the Limits of Language Modeling
Józefowicz, R., Vinyals, O., Schuster, M., Shazeer, N., & Wu, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.02410 · Cited by 823
LaMDA: Language Models for Dialog Applications
Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos, T., Baker, L., Du, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2201.08239 · Cited by 778
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757 · cited 1×
“Human-AI interaction has been explored in domains such as classification [7, 62], drawing [26, 54], translation [30], creative writing [25, 29], and design ideation [38].”
BackgroundSelf-citationDoes the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2006.14779 · Cited by 754
Self-citationSoylent: a word processor with a crowd inside
Bernstein, M. S., Little, G., Miller, R. C., Hartmann, B., Ackerman, M. S., Karger, D. R., Crowell, D., & Panovich, K. (2010) · Proceedings of the 23nd annual ACM symposium on User interface software and technology · doi:10.1145/1866029.1866078 · Cited by 686
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
Lu, Y., Bartolo, M., Moore, A., Riedel, S., & Stenetorp, P. (2021) · arXiv · doi:10.48550/arxiv.2104.08786 · Cited by 615 · cited 3×
“They tend to favor certain prompt formats, paraphrases [45, 51], or even certain information in the input.”
BackgroundTowards a Human-like Open-Domain Chatbot
Adiwardana, D., Luong, M.-T., So, D. R., Hall, J., Fiedel, N., Thoppilan, R., Yang, Z., Kulshreshtha, A., Nemade, G., Lu, Y., et al. (2020) · arXiv · doi:10.48550/arxiv.2001.09977 · Cited by 614
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
Lu, Y., Bartolo, M., Moore, A., Riedel, S., & Stenetorp, P. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.556 · Cited by 610 · cited 3×
“They tend to favor certain prompt formats, paraphrases [45, 51], or even certain information in the input.”
BackgroundSequence Level Training with Recurrent Neural Networks
Ranzato, M., Chopra, S., Auli, M., & Zaremba, W. (2016) · International Conference on Learning Representations · Cited by 572
What Makes Good In-Context Examples for GPT-3?
Liu, J., Shen, D., Zhang, Y., Dolan, B., Carin, L., & Chen, W. (2022) · Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures · doi:10.18653/v1/2022.deelio-1.10 · Cited by 563
(2020) · arXiv (Cornell University) · Cited by 461
CrowdForge: crowdsourcing complex work
Kittur, A., Smus, B., & Kraut, R. (2011) · CHI '11 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/1979742.1979902 · Cited by 434
Human-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making
Cai, C. J., Reif, E., Hegde, N., Hipp, J., Kim, B., Smilkov, D., Wattenberg, M., Viegas, F., Corrado, G. S., Stumpe, M. C., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300234 · Cited by 405
Self-citationProgram Synthesis with Large Language Models
Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07732 · Cited by 388
Self-citationFormalizing Visualization Design Knowledge as Constraints: Actionable and Extensible Models in Draco
Moritz, D., Wang, C., Nelson, G. L., Lin, H., Smith, A. M., Howe, B., & Heer, J. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2865240 · Cited by 317
CrowdForge: crowdsourcing complex work
Kittur, A., Smus, B., Khamkar, S., & Kraut, R. E. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047202 · Cited by 309
TurKit: human computation algorithms on mechanical turk
Little, G., Chilton, L. B., Goldman, M., & Miller, R. C. (2010) · Proceedings of the 23nd annual ACM symposium on User interface software and technology · doi:10.1145/1866029.1866040 · Cited by 307
Oh, C., Song, J., Choi, J., Kim, S., Lee, S., & Suh, B. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174223 · Cited by 301
Measuring Massive Multitask Language Understanding
Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., & Steinhardt, J. (2020) · arXiv · doi:10.48550/arxiv.2009.03300 · Cited by 291
Carroll, J. M., & Olson, J. R. (1988) · Handbook of Human-Computer Interaction · doi:10.1016/b978-0-444-70536-5.50007-5 · Cited by 274
CHI '86 Conference on Human Factors in Computing Systems
(1985) · Computer · doi:10.1109/mc.1985.1662775 · Cited by 272
Novice-AI Music Co-Creation via AI-Steering Tools for Deep Generative Models
Louie, R., Coenen, A., Huang, C. Z., Terry, M., & Cai, C. J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376739 · Cited by 271 · cited 1×
“Contrary to recent work in human-AI interaction, which primarily examined how to increase AI controllability through exposing knobs within a model [44, 49], our work opens up the possibility of steering AI using the model itself as units to control.”
BackgroundSelf-citationCross-Task Generalization via Natural Language Crowdsourcing Instructions
Mishra, S., Khashabi, D., Baral, C., & Hajishirzi, H. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.244 · Cited by 269 · cited 1×
“Pretrained on billions of inputs from the Internet [29], generative models like GPT-3 can now perform a wide variety of tasks [10], ranging from translation [12], to question answering [47], and even advanced story writing [60].”
BackgroundCreative Writing with a Machine in the Loop: Case Studies on Slogans and Stories
Clark, E., Ross, A. S., Tan, C., Ji, Y., & Smith, N. A. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172983 · Cited by 257
Cascade: crowdsourcing taxonomy creation
Chilton, L. B., Little, G., Edge, D., Weld, D. S., & Landay, J. A. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466265 · Cited by 234
PromptChainer: Chaining Large Language Model Prompts through Visual Programming
Wu, T., Jiang, E., Donsbach, A., Gray, J., Molina, A., Terry, M., & Cai, C. J. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519729 · Cited by 198 · cited 1×
“Though beyond the scope of this paper, this raises the question of how to support users in not just using Chains, but also authoring their own Chains, to improve user agency and intuitiveness of Chaining [69].”
BackgroundSelf-citationDetecting Hate Speech with GPT-3Preprint
Chiu, K.-L., Collins, A., & Alexander, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.12407 · Cited by 188
May AI? Design Ideation with Cooperative Contextual Bandits
Koch, J., Lucero, A., Hegemann, L., & Oulasvirta, A. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300863 · Cited by 174
Break It Down: A Comparison of Macro- and Microtasks
Cheng, J., Teevan, J. B., Iqbal, S. T., & Bernstein, M. S. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702146 · Cited by 167
Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models
Wu, T., Ribeiro, M. T., Heer, J., & Weld, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.523 · Cited by 161
Self-citationMicroMandarin: mobile language learning in context
Edge, D., Searle, E., Chiu, K., Zhao, J. H., & Landay, J. A. (2011) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979413 · Cited by 152
Metaphoria: An Algorithmic Companion for Metaphor Creation
Gero, K. I., & Chilton, L. B. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300526 · Cited by 146
Davis, N., Hsiao, C.-P., Yashraj Singh, K., Li, L., & Magerko, B. (2016) · Proceedings of the 21st International Conference on Intelligent User Interfaces · doi:10.1145/2856767.2856795 · Cited by 138
A Recipe for Arbitrary Text Style Transfer with Large Language Models
Reif, E., Ippolito, D., Yuan, A., Coenen, A., Callison-Burch, C., & Wei, J. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-short.94 · Cited by 123
Smith-Renner, A., Fan, R., Birchfield, M., Wu, T., Boyd-Graber, J., Weld, D. S., & Findlater, L. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376624 · Cited by 112 · cited 1×
“Human-AI interaction has been explored in domains such as classification [6, 59], drawing [24, 52], translation [28], creative writing [23, 27], and design ideation [36].”
BackgroundSelf-citationStory Centaur: Large Language Model Few Shot Learning as a Creative Writing Tool
Swanson, B., Mathewson, K. W., Pietrzak, B., Chen, S., & Dinalescu, M. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-demos.29 · Cited by 99
No Workflow Can Ever Be Enough: How Crowdsourcing Workflows Constrain Complex Work
Retelny, D., Bernstein, M. S., & Valentine, M. (2017) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3134724 · Cited by 96
AI Song Contest: Human-AI Co-Creation in Songwriting
Huang, C.-Z. A., Koops, H. V., Newton-Rex, E., Dinculescu, M., & Cai, C. J. (2020) · arXiv · doi:10.48550/arxiv.2010.05388 · Cited by 90
Self-citationConsistency of a Recurrent Language Model With Respect to Incomplete Decoding
Welleck, S., Kulikov, I., Kim, J., Pang, R. Y., & Cho, K. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.448 · Cited by 85
Mechanical Novel: Crowdsourcing Complex Work through Reflection and Revision
Kim, J. O., Sterman, S., Cohen, A. A. B., & Bernstein, M. S. (2017) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2998181.2998196 · Cited by 83
Turkomatic: automatic recursive task and workflow design for mechanical turk
Kulkarni, A. P., Can, M., & Hartmann, B. (2011) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1979742.1979865 · Cited by 81
Juxtapeer: Comparative Peer Review Yields Higher Quality Feedback and Promotes Deeper Reflection
Cambre, J., Klemmer, S., & Kulkarni, C. E. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173868 · Cited by 79
Levy, A., Agrawal, M., Satyanarayan, A., & Sontag, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445522 · Cited by 73 · cited 1×
“In a collaboration, AI can play various roles, such as casual creators that encourage exploration [26] or assistants that compensate for humanweaknesses [41, 74].”
BackgroundVizLinter: A Linter and Fixer Framework for Data Visualization
Chen, Q., Sun, F., Xu, X., Chen, Z., Wang, J., & Cao, N. (2021) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2021.3114804 · Cited by 73
Nine Potential Pitfalls when Designing Human-AI Co-Creative SystemsPreprint
Buschek, D., Mecke, L., Lehmann, F., & Dang, H. (2021) · ERef Bayreuth (University of Bayreuth) · doi:10.48550/arxiv.2104.00358 · Cited by 72
Progressive Generation of Long Text with Pretrained Language Models
Tan, B., Yang, Z., Al-Shedivat, M., Xing, E., & Hu, Z. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.341 · Cited by 69
Chain Reactions: The Impact of Order on Microtask Chains
Cai, C. J., Iqbal, S. T., & Teevan, J. B. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858237 · Cited by 66
Self-citationTurkomatic: automatic, recursive task and workflow design for mechanical turk
Kulkarni, A. P., Can, M., & Hartmann, B. (2011) · National Conference on Artificial Intelligence · Cited by 64
Predictive translation memory: a mixed-initiative system for human language translation
Green, S., Chuang, J., Heer, J., & Manning, C. D. (2014) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2642918.2647408 · Cited by 57
Mechanical Novel: Crowdsourcing Complex Work Through Reflection and Revision
Kim, J. O., Sterman, S., Cohen, A. A. B., & Bernstein, M. S. (2017) · Understanding innovation · doi:10.1007/978-3-319-60967-6_5 · Cited by 56
Natural Instructions: Benchmarking Generalization to New Tasks from Natural Language InstructionsPreprint
Mishra, S. R., Khashabi, D., Baral, C. R., & Hajishirzi, H. (2021) · arXiv (Cornell University) · Cited by 43
An evaluation of Dasher with a high-performance language model as a gaze communication method
Rough, D. J., Vertanen, K., & Kristensson, P. O. (2014) · International Conference on Advanced Visual Interfaces (AVI) · doi:10.1145/2598153.2598157 · Cited by 41
Thinking Aloud: Dynamic Context Generation Improves Zero-Shot Reasoning Performance of GPT-2Preprint
Betz, G., Richardson, K., & Voigt, C. C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.13033 · Cited by 34
Law, E., & Zhang, H. (2011) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v25i1.8092 · Cited by 30
AI song contest: Human-AI co-creation in songwritingPreprint
Huang, C.-Z. A., Koops, H. V., Newton-Rex, E., Dinculescu, M., & Cai, C. J. (2020) · arXiv (Cornell University) · doi:10.5281/zenodo.4245529 · Cited by 30
Self-citationGPT3-to-plan: Extracting plans from text using GPT-3
Olmo, A., Sreedharan, S., & Kambhampati, S. (2021) · arXiv · doi:10.48550/arxiv.2106.07131 · Cited by 27
GenLine and GenForm: Two Tools for Interacting with Generative Language Models in a Code Editor
Jiang, E., Toh, E., Molina, A., Donsbach, A., Cai, C. J., & Terry, M. (2021) · Adjunct Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3474349.3480209 · Cited by 27 · cited 1×
“Finally, the closest work to ours might be online interfaces for users to interactively create prompts1, or interfaces enabling users to perform natural language programming of code using a large language model [34].”
BackgroundSelf-citationExploring Generalization Ability of Pretrained Language Models on Arithmetic and Logical Reasoning
Wang, C., Zheng, B., Niu, Y., & Zhang, Y. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-88480-2_61 · Cited by 27
Context Trees: Crowdsourcing Global Understanding from Local Views
Verroios, V., & Bernstein, M. S. (2014) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v2i1.13149 · Cited by 22
Productivity Decomposed: Getting Big Things Done with Little Microtasks
Teevan, J. B., Iqbal, S. T., Cai, C. J., Bigham, J. P., Bernstein, M. S., & Gerber, E. M. (2016) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2851581.2856480 · Cited by 22
Self-citationPolyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving ModelsPreprint
Wu, T., Ribeiro, M. T., Heer, J., & Weld, D. S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.00288 · Cited by 19
Self-citationCai, C. J., Guo, P. J., Glass, J. R., & Miller, R. C. (2014) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2559206.2581183 · Cited by 15
Self-citationUnderstanding Emails and Drafting Responses -- An Approach Using GPT-3
Thiergart, J., Huber, S., & Übellacker, T. (2021) · arXiv · doi:10.48550/arxiv.2102.03062 · Cited by 14
Accelerating Text Communication via Abbreviated Sentence Input
Adhikary, J. K., Berger, J., & Vertanen, K. (2021) · doi:10.18653/v1/2021.acl-long.514 · Cited by 12
What Context Features Can Transformer Language Models Use?Preprint
O’Connor, J., & Andreas, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.08367 · Cited by 11
Research works printed but not linked: 6
- Tom B. Brown , Benjamin Mann , Nick Ryder , Melanie Subbiah , Jared Kaplan , Prafulla Dhariwal , Arvind Neelakantan , Pranav Shyam , Girish Sastry , Amanda Askell , Sandhini Agarwal , Ariel Herbert-Voss , Gretchen Krueger , Tom Henighan , Rewon Child , Aditya Ramesh , Daniel M. Ziegler , Jeffrey Wu , Clemens Winter , Christopher Hesse , Mark Chen , Eric Sigler , Mateusz Litwin , Scott Gray , Benjamin Chess , Jack Clark , Christopher Berner , Sam McCandlish , Alec Radford , Ilya Sutskever , and Dario Amodei . 2020 . Language Models are Few-Shot Learners . In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020 , NeurIPS 2020, December 6-12, 2020, virtual, Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.). https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.). https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html
- Lydia B Chilton , James A Landay , and Daniel S Weld . 2016 . Humortools: A microtask workflow for writing news satire. El Paso, Texas : ACM ( 2016). Lydia B Chilton, James A Landay, and Daniel S Weld. 2016. Humortools: A microtask workflow for writing news satire. El Paso, Texas: ACM (2016).
- Päivi Majaranta and Kari-Jouko Räihä . 2007. Text entry by gaze: Utilizing eye-tracking. Text entry systems: Mobility, accessibility, universality ( 2007 ), 175–187. Päivi Majaranta and Kari-Jouko Räihä. 2007. Text entry by gaze: Utilizing eye-tracking. Text entry systems: Mobility, accessibility, universality (2007), 175–187.
- Hegde Narayan , Jason D Hipp , Yun Liu , Michael Emmert-Buck , Emily Reif , Daniel Smilkov , Michael Terry , Carrie J Cai , Mahul B Amin , Craig H Mermel , 2019. Similar image search for histopathology: SMILY. NPJ Digital Medicine 2, 1 ( 2019 ). Hegde Narayan, Jason D Hipp, Yun Liu, Michael Emmert-Buck, Emily Reif, Daniel Smilkov, Michael Terry, Carrie J Cai, Mahul B Amin, Craig H Mermel, 2019. Similar image search for histopathology: SMILY. NPJ Digital Medicine 2, 1 (2019).
- Besmira Nushi , Ece Kamar , Eric Horvitz , and Donald Kossmann . 2017 . On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems . In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence , February 4-9, 2017, San Francisco, California, USA, Satinder P. Singh and Shaul Markovitch (Eds.). AAAI Press, 1017–1025. http://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/15032 Besmira Nushi, Ece Kamar, Eric Horvitz, and Donald Kossmann. 2017. On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA, Satinder P. Singh and Shaul Markovitch (Eds.). AAAI Press, 1017–1025. http://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/15032
- Bingjun Xie , Jia Zhou , and Huilin Wang . 2017. How influential are mental models on interaction performance? exploring the gap between users ’ and designers’ mental models through a new quantitative method. Advances in Human-Computer Interaction 2017 (2017). Bingjun Xie, Jia Zhou, and Huilin Wang. 2017. How influential are mental models on interaction performance? exploring the gap between users’ and designers’ mental models through a new quantitative method. Advances in Human-Computer Interaction 2017 (2017).
Cited by
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · arXiv · doi:10.48550/arxiv.2304.03442 · Cited by 2,173
Self-citationGenerative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187
“…environment like the one we inhabit [10, 59], and strive to behave in ways that exhibit emergent behaviors grounded in social interactions with users or other agents with the aim of becoming believable proxies of our behavior in hypothetical simulations of individuals and communities [20, 36, 71].”
BackgroundSelf-citationThe rise and potential of large language model based agents: a survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970
“Of course, the collaborative nature of human-agent interaction also allows humans to directly improve the agents’ output by modifying intermediate links [176, 362] or adjusting the conversation content [363].”
Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts
Zamfirescu-Pereira, J. D., Wong, R. Y., Hartmann, B., & Yang, Q. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581388 · Cited by 1,274
“In this work, we use these previously-identifed challenges to better understand why we observe some of the struggles with end-user prompt engineering that we do.”
Method
Metadata
- Authors
- Wu, T., Terry, M., & Cai, C. J.
- Venue
- CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2022-04-29
- Pages
- 1-22
- OpenAlex
- W3203321135
- Cited by
- 483
- References
- 78
- FWCI
- 80.48
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.