CoChat

Automating the Development of Task-oriented LLM-based Chatbots

Open access
Sánchez Cuadrado, Jesús, Pérez-Soler, Sara, Guerra, Esther, De Lara, Juan · 2024 · ACM Conversational User Interfaces 2024

The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.

Abstract

Task-oriented chatbots are increasingly used to access all sorts of services – like booking a flight, or setting a medical appointment – through natural language conversation. There are many technologies for implementing task-oriented chatbots, including Dialogflow, Watson, and Rasa. They rely on an explicit definition of the user intents, conversation flows, and chatbot outputs, which is costly to specify, and sometimes results in suboptimal user experiences and artificial conversations with limited diversity of chatbot responses.

Overview

Integrity signals

Registry

Crossref · 10.1145/3640794.3665538

Cites

14 of 14 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Research works printed but not linked: 24

  1. Anthropic. last access in 2024. Claude. https://www.anthropic.com/news/introducing-claude.
  2. Tom Bocklisch, Thomas Werkmeister, Daksh Varshneya, and Alan Nichol. 2024. Task-oriented dialogue with in-context learning. arXiv preprint arXiv:2402.12234 (2024). · arXiv
  3. Y. Cheng, J. Chen, Q. Huang, Z. Xing, X. Xu, and Q. Lu. 2023. Prompt Sapper: A LLM-empowered production tool for building AI chains. ACM Trans. Softw. Eng. Methodol. In press (2023), 22 pages.
  4. R. Clarisó and J. Cabot. 2023. Model-driven prompt engineering. In 2023 ACM/IEEE 26th International Conference on Model Driven Engineering Languages and Systems (MODELS). IEEE, 47–54.
  5. J. de Wit and A. Braggaar. 2023. Tilbot: A visual design platform to facilitate open science research into conversational user interfaces. In 5th International Conference on Conversational User Interfaces, CUI. ACM, 55:1–55:5.
  6. [10] Dialogflow. last access in 2024. https://dialogflow.com/.
  7. A. J. Fiannaca, C. Kulkarni, C. J. Cai, and M. Terry. 2023. Programming without a programming language: Challenges and opportunities for designing developer tools for prompt programming. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, CHI EA. ACM, 235:1–235:7.
  8. Google. last access in 2024. Gemini. https://gemini.google.com/.
  9. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen. 2022. LoRA: Low-rank adaptation of large language models. In 10th International Conference on Learning Representations, ICLR. OpenReview.net.
  10. [14] LangChain. last access in 2024. https://www.langchain.com/.
  11. [15] LangFlow. last access in 2024. https://www.langflow.org/.
  12. P. S. H. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-T. Yih, T. Rocktäschel, S. Riedel, and D. Kiela. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020. 16 pages.
  13. [17] Lex. last access in 2024. https://aws.amazon.com/en/lex/.
  14. Robert J. Moore and Raphael Arar. 2019. Conversational UX Design: A Practitioner’s Guide to the Natural Conversation Framework. ACM Books, Vol. 27. ACM.
  15. [19] OpenAI. last access in 2024. https://openai.com/.
  16. [20] Pandorabots. last access in 2023. https://home.pandorabots.com/.
  17. S. Pérez-Soler, E. Guerra, and J. de Lara. 2020. Model-driven chatbot development. In ER(LNCS, Vol. 12400). Springer, 207–222.
  18. [24] Rasa. last access in 2024. https://rasa.com/.
  19. E. Ruane, R. Smith, D. Bean, M. Tjalve, and A. Ventresque. 2020. Developing a conversational agent with a globally distributed team: An experience report. In 15th IEEE/ACM International Conference on Global Software Engineering, ICGSE. ACM, 122–126.
  20. M. Skjuve, A. Følstad, and P. B. Brandtzaeg. 2023. The user experience of ChatGPT: Findings from a questionnaire study of early users. In 5th International Conference on Conversational User Interfaces, CUI. ACM, 2:1–2:10.
  21. A. Vaswani N. Shazeer N. Parmar J. Uszkoreit L. Jones A. N. Gomez L. Kaiser and I. Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30. 5998–6008.
  22. [33] Watson. last access in 2024. https://www.ibm.com/cloud/watson-assistant/.
  23. Jing Wei Sungdong Kim Hyunhoon Jung and Young-Ho Kim. 2024. Leveraging large language models to power chatbots for collecting user self-reported data. In CSCW. See also https://arxiv.org/abs/2301.05843. · arXiv
  24. J. D. Zamfirescu-Pereira, R. Y. Wong, B. Hartmann, and Q. Yang. 2023. Why Johnny can’t prompt: How non-AI experts try (and fail) to design LLM prompts. In 2023 CHI Conference on Human Factors in Computing Systems, CHI. ACM, Article 437, 21 pages.

Cited by

See all 49

Metadata

Authors
Sánchez Cuadrado, J., Pérez-Soler, S., Guerra, E., & De Lara, J.
Affiliations
  • Universidad de Murcia, Spain
  • Universidad Autónoma de Madrid (ES)
  • Computer Science, Universidad Autonoma de Madrid, Spain
Venue
ACM Conversational User Interfaces 2024
Publisher
ACM
Published
2024-07-08
Pages
1-10
OpenAlex
W4400397477
Cited by
46
References
12
Citations by year
2026: 12 · 2025: 26 · 2024: 8
FWCI
13.60
Type
conference paper
Language
en
Open access
Locations
  • ACM Conference on Conversational User Interfaces (CUI) (publishedVersion)
  • Biblos-e Archivo (Universidad Autónoma de Madrid) (submittedVersion, public-domain)
Funders
Agencia Estatal de Investigación (PID2021-122270OB-I00, PID2022-140109NB-I00, RED2022-134647-T, TED2021-129381B-C22)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
task-oriented chatbots, large language models, conversation flow, service booking, medical appointment scheduling

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.