Automating the Development of Task-oriented LLM-based Chatbots
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
Task-oriented chatbots are increasingly used to access all sorts of services – like booking a flight, or setting a medical appointment – through natural language conversation. There are many technologies for implementing task-oriented chatbots, including Dialogflow, Watson, and Rasa. They rely on an explicit definition of the user intents, conversation flows, and chatbot outputs, which is costly to specify, and sometimes results in suboptimal user experiences and artificial conversations with limited diversity of chatbot responses.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In the wake of this breakthrough, the AI landscape underwent a rapid transformation, shifting from the use of standalone LLMs toward more autonomous, task-oriented frameworks [30].”
From AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges (2026) · cites this paper as [30]
Citations
Cites
A Survey on Evaluation of Large Language Models
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., et al. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3641289 · Cited by 4,254
Ray, P. P. (2023) · Internet of Things and Cyber-Physical Systems · doi:10.1016/j.iotcps.2023.04.003 · Cited by 2,463
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.18223 · Cited by 2,090
ReAct: Synergizing Reasoning and Acting in Language Models
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2022) · arXiv · doi:10.48550/arxiv.2210.03629 · Cited by 608
A systematic evaluation of large language models of code
Xu, F. F., Alon, U., Neubig, G., & Hellendoorn, V. J. (2022) · Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming · doi:10.1145/3520312.3534862 · Cited by 540
Choosing a Chatbot Development Tool
Pérez-Soler, S., Juárez-Puerta, S., Guerra, E., & Lara, J. D. (2021) · IEEE Software · doi:10.1109/ms.2020.3030198 · Cited by 94
Self-citationHallucination Detection: Robustly Discerning Reliable Answers in Large Language Models
Chen, Y., Fu, Q., Yuan, Y., Wen, Z., Fan, G., Liu, D., Zhang, D., Li, Z., & Xiao, Y. (2023) · Proceedings of the 32nd ACM International Conference on Information and Knowledge Management · doi:10.1145/3583780.3614905 · Cited by 94
Herding AI Cats: Lessons from Designing a Chatbot by Prompting GPT-3
Zamfirescu-Pereira, J. D., Wei, H., Xiao, A., Gu, K., Jung, G., Lee, M. G., Hartmann, B., & Yang, Q. (2023) · Proceedings of the 2023 ACM Designing Interactive Systems Conference · doi:10.1145/3563657.3596138 · Cited by 69
Evaluation Techniques for Chatbot Usability: A Systematic Mapping Study
Ren, R., Castro, J. W., Acuña, S. T., & de Lara, J. (2019) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194019400163 · Cited by 68
Research works printed but not linked: 24
- Anthropic. last access in 2024. Claude. https://www.anthropic.com/news/introducing-claude.
- Tom Bocklisch, Thomas Werkmeister, Daksh Varshneya, and Alan Nichol. 2024. Task-oriented dialogue with in-context learning. arXiv preprint arXiv:2402.12234 (2024). · arXiv
- Y. Cheng, J. Chen, Q. Huang, Z. Xing, X. Xu, and Q. Lu. 2023. Prompt Sapper: A LLM-empowered production tool for building AI chains. ACM Trans. Softw. Eng. Methodol. In press (2023), 22 pages.
- R. Clarisó and J. Cabot. 2023. Model-driven prompt engineering. In 2023 ACM/IEEE 26th International Conference on Model Driven Engineering Languages and Systems (MODELS). IEEE, 47–54.
- J. de Wit and A. Braggaar. 2023. Tilbot: A visual design platform to facilitate open science research into conversational user interfaces. In 5th International Conference on Conversational User Interfaces, CUI. ACM, 55:1–55:5.
- [10] Dialogflow. last access in 2024. https://dialogflow.com/.
- A. J. Fiannaca, C. Kulkarni, C. J. Cai, and M. Terry. 2023. Programming without a programming language: Challenges and opportunities for designing developer tools for prompt programming. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, CHI EA. ACM, 235:1–235:7.
- Google. last access in 2024. Gemini. https://gemini.google.com/.
- E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen. 2022. LoRA: Low-rank adaptation of large language models. In 10th International Conference on Learning Representations, ICLR. OpenReview.net.
- [14] LangChain. last access in 2024. https://www.langchain.com/.
- [15] LangFlow. last access in 2024. https://www.langflow.org/.
- P. S. H. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-T. Yih, T. Rocktäschel, S. Riedel, and D. Kiela. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020. 16 pages.
- [17] Lex. last access in 2024. https://aws.amazon.com/en/lex/.
- Robert J. Moore and Raphael Arar. 2019. Conversational UX Design: A Practitioner’s Guide to the Natural Conversation Framework. ACM Books, Vol. 27. ACM.
- [19] OpenAI. last access in 2024. https://openai.com/.
- [20] Pandorabots. last access in 2023. https://home.pandorabots.com/.
- S. Pérez-Soler, E. Guerra, and J. de Lara. 2020. Model-driven chatbot development. In ER(LNCS, Vol. 12400). Springer, 207–222.
- [24] Rasa. last access in 2024. https://rasa.com/.
- E. Ruane, R. Smith, D. Bean, M. Tjalve, and A. Ventresque. 2020. Developing a conversational agent with a globally distributed team: An experience report. In 15th IEEE/ACM International Conference on Global Software Engineering, ICGSE. ACM, 122–126.
- M. Skjuve, A. Følstad, and P. B. Brandtzaeg. 2023. The user experience of ChatGPT: Findings from a questionnaire study of early users. In 5th International Conference on Conversational User Interfaces, CUI. ACM, 2:1–2:10.
- A. Vaswani N. Shazeer N. Parmar J. Uszkoreit L. Jones A. N. Gomez L. Kaiser and I. Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30. 5998–6008.
- [33] Watson. last access in 2024. https://www.ibm.com/cloud/watson-assistant/.
- Jing Wei Sungdong Kim Hyunhoon Jung and Young-Ho Kim. 2024. Leveraging large language models to power chatbots for collecting user self-reported data. In CSCW. See also https://arxiv.org/abs/2301.05843. · arXiv
- J. D. Zamfirescu-Pereira, R. Y. Wong, B. Hartmann, and Q. Yang. 2023. Why Johnny can’t prompt: How non-AI experts try (and fail) to design LLM prompts. In 2023 CHI Conference on Human Factors in Computing Systems, CHI. ACM, Article 437, 21 pages.
Cited by
AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges
Sapkota, R., Roumeliotis, K. I., & Karkee, M. (2025) · arXiv · doi:10.48550/arxiv.2505.10468 · Cited by 295
“This evolution progressed through two major post-generative phases: AI Agents and Agentic AI.”
AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges
Sapkota, R., Roumeliotis, K. I., & Karkee, M. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103599 · Cited by 671
“This evolution progressed through two major post-generative phases: AI Agents and Agentic AI.”
AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges
Sapkota, R., Roumeliotis, K. I., & Karkee, M. (2025) · SuperIntelligence - Robotics - Safety & Alignment · doi:10.70777/si.v2i3.15161 · Cited by 186
How the Choice of LLM and Prompt Engineering Affects Chatbot Effectiveness
Pawlik, Ł. (2025) · Electronics · doi:10.3390/electronics14050888 · Cited by 25
Large Language Models and Rule-Based Approaches in Domain-Specific Communication
Halvoník, D., & Kapusta, J. (2024) · IEEE Access · doi:10.1109/access.2024.3436902 · Cited by 30
“Despite the existence of numerous tools for creating chatbots [37], one of the most popular tools for hybrid models is RASA.”
Method
Metadata
- Authors
- Sánchez Cuadrado, J., Pérez-Soler, S., Guerra, E., & De Lara, J.
- Affiliations
- Universidad de Murcia, Spain
- Universidad Autónoma de Madrid (ES)
- Computer Science, Universidad Autonoma de Madrid, Spain
- Venue
- ACM Conversational User Interfaces 2024
- Publisher
- ACM
- Published
- 2024-07-08
- Pages
- 1-10
- OpenAlex
- W4400397477
- Cited by
- 46
- References
- 12
- Citations by year
- 2026: 12 · 2025: 26 · 2024: 8
- FWCI
- 13.60
- Type
- conference paper
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- Locations
- ACM Conference on Conversational User Interfaces (CUI) (publishedVersion)
- Biblos-e Archivo (Universidad Autónoma de Madrid) (submittedVersion, public-domain)
- Funders
- Agencia Estatal de Investigación (PID2021-122270OB-I00, PID2022-140109NB-I00, RED2022-134647-T, TED2021-129381B-C22)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- task-oriented chatbots, large language models, conversation flow, service booking, medical appointment scheduling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Automating the Development of Task-oriented LLM-based Chatbots. https://app.cochat.ai/papers/10.1145/3640794.3665538
- The paper
- Sánchez Cuadrado, J., Pérez-Soler, S., Guerra, E., & De Lara, J. (2024). Automating the Development of Task-oriented LLM-based Chatbots. ACM Conversational User Interfaces 2024. https://doi.org/10.1145/3640794.3665538
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.