Towards Designing Conversational Agents for Pair Programming: Accounting for Creativity Strategies and Conversational Styles
The DOI matches its Crossref record; 91 of the 92 references it cites were found in a registry or an index.
Abstract
Established research on pair programming reveals benefits, including increasing communication, creativity, self-efficacy, and promoting gender inclusivity. However, research has reported limitations such as finding a compatible partner, scheduling sessions between partners, and resistance to pairing. Further, pairings can be affected by predispositions to negative stereotypes. These problems can be addressed by replacing one human member of the pair with a conversational agent. To investigate the design space of such a conversational agent, we conducted a controlled remote pair programming study. Our analysis found various creative problem-solving strategies and differences in conversational styles. We further analyzed the transferable strategies from human-human collaboration to human-agent collaboration by conducting a Wizard of Oz study. The findings from the two studies helped us gain insights regarding design of a programmer conversational agent. We make recommendations for researchers and practitioners for designing pair programming conversational agent tools.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We envision that a conversational agent interface could be a natural fit for AI assistant for code or co-programming tools, as explored in recent work [42,43]”
From Investigating Explainability of Generative AI for Code through Scenario-based Design (2022) · cites this paper as [42]
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
WEST, C., & ZIMMERMAN, D. H. (1987) · Gender & Society · doi:10.1177/0891243287001002002 · Cited by 8,954
(1997) · Choice Reviews Online · doi:10.5860/choice.34-3702 · Cited by 4,887
Heuristic evaluation of user interfaces
Nielsen, J., & Molich, R. (1990) · Proceedings of the SIGCHI conference on Human factors in computing systems Empowering people - CHI '90 · doi:10.1145/97243.97281 · Cited by 3,533 · cited 1×
“In the human-human study, participants said they did not want to interrupt their partner as they considered it to be impolite, had low confidence, or did not want to be distracting.”
BackgroundBERGER, C. R., & CALABRESE, R. J. (1975) · Human Communication Research · doi:10.1111/j.1468-2958.1975.tb00258.x · Cited by 3,006
Guidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 2×
“Our in-depth qualitative analysis revealed human-human pair programmer’s creativity strategies and communication styles resulting in 12 design guidelines for designing a conversational agent.”
BackgroundMethodDialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech
Stolcke, A., Ries, K., Coccaro, N., Shriberg, E., Bates, R., Jurafsky, D., Taylor, P., Martin, R., Ess-Dykema, C. V., & Meteer, M. (2000) · Computational Linguistics · doi:10.1162/089120100561737 · Cited by 1,108
Applied Imagination: Principles and Procedures of Creative Thinking
Osborn, A. F. (1953) · Medical Entomology and Zoology · Cited by 860
Peopleware (2nd ed.): productive projects and teams
DeMarco, T., & Lister, T. (1987) · Dorset House Publishing Co., Inc. eBooks · Cited by 816
Strengthening the case for pair programming
Williams, L., Kessler, R. R., Cunningham, W., & Jeffries, R. (2000) · IEEE Software · doi:10.1109/52.854064 · Cited by 780
Research works printed but not linked: 18
- swartz, 2003, Why people hate the paperclip Labels appearance behavior and social responses to user interface agents Tech rep
- edelman, 2010, Inc Creativity and education Why it matters
- gallis, 2002, A transition from partner programming to pair programming-an industrial case study
- 2009, Copyright page, Building Intelligent Interactive Tutors, iii
- belshee, 2005, Promiscuous pairing and beginner’s mind Embrace inexperience, 125
- paul green, 0, THE WIZARD OF OZ A TOOL FOR RAPID DEVELOPMENT OF USER INTERFACES
- 2018, FACSvatar
- 2019, Google text-to-speech python library
- 2019, Teamview
- 2019, Eclipse IDE
- lewis, 1982, Using the “thinking Aloud” Method in Cognitive Interface Design
- 2019, Morae
- jaccard, 1901, Etude de la distribution florale dans une portion des alpes et du jura, Bulletin de la Societe Vaudoise des Sciences Naturelles, 37, 547
- tannen, 2005, THE POWER of Talk Who Gets Heard and why HBR Onpoint [Harvard Business Review] Harvard Business Review
- 2019, Explain Everything
- 2019, RealTimeBoard
- 2019, LiveBoard
- 2020, Human-Agent study protocol
Cited by
Investigating Explainability of Generative AI for Code through Scenario-based Design
Sun, J., Liao, Q. V., Muller, M., Agarwal, M., Houde, S., Talamadupula, K., & Weisz, J. D. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511119 · Cited by 219
“We envision that a conversational agent interface could be a natural fit for AI assistant for code or co-programming tools, as explored in recent work [42, 43] We also noticed that the explainability needs and user needs in general may differ for users with varied levels of programming skills.”
BackgroundCo-designing conversational agents: A comprehensive review and recommendations for best practices
Sadek, M., Calvo, R. A., & Mougenot, C. (2023) · Design Studies · doi:10.1016/j.destud.2023.101230 · Cited by 59
Designing PairBuddy—A Conversational Agent for Pair Programming
Robe, P., & Kuttal, S. K. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3498326 · Cited by 44
“Lack of Support for Creativity: As identified in previous research [134], programmers follow the Osborn–Parnes Creative Problem Solving Process when working together.”
BackgroundMethodSelf-citationKuttal, S. K., Ong, B., Kwasny, K., & Robe, P. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445659 · Cited by 44
“Giving direct instruction as a backseat driver [97] Apologizing when the answer is unknown or a mistake is made [97]”
BackgroundInfluentialSelf-citationBots in software engineering: a systematic mapping study
Santhanam, S., Hecking, T., Schreiber, A., & Wagner, S. (2022) · PeerJ Computer Science · doi:10.7717/peerj-cs.866 · Cited by 44
“Kuttal et al. (2020) designed a conversational agent (CA) for pair-programming task through WOZ study. Results convey that the CAs are to be developed with social intelligence, to include embodiments (face and voice) and trustable traits. Moreover, when Agents interrupt humans, humans don’t like it unless there is a h…”
BackgroundInfluential
Metadata
- Authors
- Kuttal, S. K., Myers, J., Gurka, S., Magar, D., Piorkowski, D., & Bellamy, R.
- Affiliations
- University of Tulsa (US)
- IBM (United States) (US)
- Venue
- 2020 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)
- Publisher
- IEEE
- Published
- 2020-08
- Pages
- 1-11
- ISSN
- 1943-6092 (ISSN-L)
- OpenAlex
- W3043660278
- Cited by
- 27
- References
- 92
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 4 · 2023: 5 · 2022: 9 · 2021: 3
- FWCI
- 4.93
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- pair programming, chatbots, creative strategy, creative problem solving, conversational style, human-agent collaboration, human-human collaboration, Wizard-of-Oz study, remote programming, agent design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Towards Designing Conversational Agents for Pair Programming: Accounting for Creativity Strategies and Conversational Styles. https://app.cochat.ai/papers/10.1109/vl/hcc50065.2020.9127276
- The paper
- Kuttal, S. K., Myers, J., Gurka, S., Magar, D., Piorkowski, D., & Bellamy, R. (2020). Towards Designing Conversational Agents for Pair Programming: Accounting for Creativity Strategies and Conversational Styles. 2020 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). https://doi.org/10.1109/vl/hcc50065.2020.9127276
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.