AN AUTOMATIC DIALOG SIMULATION TECHNIQUE TO DEVELOP AND EVALUATE INTERACTIVE CONVERSATIONAL AGENTS
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
During recent years, conversational agents have become a solution to provide straightforward and more natural ways of retrieving information in the digital domain. In this article, we present an agent-based dialog simulation technique for learning new dialog strategies and evaluating conversational agents. Using this technique, the effort necessary to acquire data required to train the dialog model and then explore new dialog strategies is considerably reduced. A set of measures has also been defined to evaluate the dialog strategy that is automatically learned and to compare different dialog corpora. We have applied this technique to explore the space of possible dialog strategies and evaluate the dialogs acquired for a conversational agent that collects monitored data from patients suffering from diabetes. The results of the comparison of these measures for an initial corpus and a corpus acquired using the dialog simulation technique show that the conversational agent reduces the time needed to complete the dialogs and improve their quality, thereby allowing the conversational agent to tackle new situations and generate new coherent answers for the situations already present in an initial model.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Intelligent agents: theory and practice
Wooldridge, M., & Jennings, N. R. (1995) · The Knowledge Engineering Review · doi:10.1017/s0269888900008122 · Cited by 6,810 · cited 1×
“Agents are considered to be autonomous entities characterized by a set of properties including social ability, reactivity, and proactiveness (Wooldridge and Jennings, 1995).”
BackgroundSpoken dialogue technology: enabling the conversational user interface
McTear, M. F. (2002) · ACM Computing Surveys · doi:10.1145/505282.505285 · Cited by 571
SCHATZMANN, J., WEILHAMMER, K., STUTTLE, M., & YOUNG, S. (2006) · The Knowledge Engineering Review · doi:10.1017/s0269888906000944 · Cited by 336 · cited 4×
“…that has currently attracted an increasing interest is based on the automatic generation of dialogs between the dialog manager and an additional module called the user simulator, which represents user interactions with the conversational agent (Schatzmann et al., 2006; Paek and Horvitz, 2000).”
BackgroundInfluentialConversation as Action Under Uncertainty
Paek, T., & Horvitz, E. J. (2013) · arXiv · doi:10.48550/arxiv.1301.3883 · Cited by 145 · cited 2×
“A technique that has currently attracted an increasing interest is based on the automatic generation of dialogs between the dialog manager and an additional module called the user simulator, which represents user interactions with the conversational agent (Schatzmann et al., 2006; Paek and Horvitz, 2000).”
BackgroundAppraisal of a Conversational Artefact and Its Utility in Remote Patient Monitoring
Black, L. A., McTear, M., Black, N., Harper, R., & Lemon, M. (2005) · 18th IEEE Symposium on Computer-Based Medical Systems (CBMS'05) · doi:10.1109/cbms.2005.33 · Cited by 27 · cited 3×
“DI@L-log is a spoken conversational agent that acts as a voice logbook to collect home-monitored data from patients suffering from Type-2 diabetes (Black et al., 2005).”
BackgroundMethod
Research works printed but not linked: 47
- Ai , H. , A. Raux , D. Bohus , M. Eskenazi , and D. Litman . 2007 . Comparing spoken dialog corpora collected with recruited subjects versus real users. In Proceedings of the 8th SIGdial workshop on discourse and dialogue , 124–131. Antwerp, Belgium.
- 10.1016/j.specom.2010.02.015 · DOI
- Ballinas-Hernandez A. L., 2011, Journal of Artificial Societies and Social Simulation, 14, 2, 10.18564/jasss.1789 · doi:10.18564/jasss.1789
- 10.1007/978-1-4020-5060-2_11 · doi:10.1007/978-1-4020-5060-2_11
- Bandini S., 2006, Neuralriets, 231
- Bandini S., 2009, Journal of Artificial Societies and Social Simulation, 12, 97
- Bohus , D. , S. Grau , D. Huggins-Daines , V. Keri , G. Krishna , R. Kumar , A. Raux , and S. Tomko . 2007 . Conquest - An open-source dialog system for conferences. In Proceedings of 7th meeting of the North American chapter of the association for computational linguistics (HLT/NAACL 07), 9–12. Rochester, NY, USA. · doi:10.3115/1614108.1614111
- 10.1007/1-4020-3075-4_12 · DOI
- Bos , J. , E. Klein , O. Lemon , and T. Oka . 2003 . DIPPER: Description and formalisation of an information-state update dialogue system architecture. In Proceedings of the 4th SIGdial workshop on discourse and dialogue , 115–124. Sapporo, Japan.
- Brahnam S., 2009, PsychNology Journal, 7, 9
- Chung , G. 2004 . Developing a flexible spoken dialog system using simulation. In Proceedings of the 42nd annual meeting of the association for computational linguistics (ACL 04) , 63–70. Barcelona, Spain. · DOI
- Cuayáhuitl , H. , S. Renals , O. Lemon , and H. Shimodaira . 2005 . Human-computer Dialogue simulation using hidden Markov models. In Proceedings of the IEEE workshop on automatic speech recognition and understanding (ASRU'05) , 290–295. San Juan, Puerto Rico. · DOI · Paper page
- Eckert , W. , E. Levin , and R. Pieraccini . 1997 . User modeling for spoken dialogue system evaluation. In Proceedings of IEEE automatic speech recognition and understanding workshop (ASRU 97) , 80–87. Santa Barbara, USA. · DOI · Paper page
- Eckert W., 1998, Automatic evaluation of spoken dialogue systems
- 10.1007/978-3-540-39398-6_24 · doi:10.1007/978-3-540-39398-6_24
- Georgila , K. , J. Henderson , and O. Lemon . 2005 . Learning user simulations for information state update dialogue systems. In Proceedings of the 9th European conference on speech communication and technology (Eurospeech 05) , 893–896. Lisbon, Portugal. · DOI · Paper page
- 10.1016/0167-6393(95)00008-C · DOI
- 10.1016/j.specom.2008.04.001 · DOI
- 10.1007/978-3-642-14883-5_36 · DOI
- Heath B., 2009, Journal of Artificial Societies and Social Simulation, 12, 9
- 10.1016/j.csl.2010.06.002 · DOI
- 10.1007/978-3-540-39869-1_2 · doi:10.1007/978-3-540-39869-1_2
- 10.1109/89.817450 · DOI · Paper page
- 10.1109/89.928918 · DOI
- López-Cózar R., 2005, Spoken, multilingual and multimodal dialogue systems
- 10.1016/S0167-6393(02)00126-7 · DOI
- 10.1057/jos.2010.3 · DOI · Paper page
- 10.1007/978-0-85729-414-2 · DOI · Paper page
- Melin , H. , A. Sandell , and M. Ihse . 2001 . CTT-bank: A speech controlled telephone banking system - an initial evaluation. In TMH Quarterly Progress and Status Report (TMH-QPSR) 1:1–27.
- Menezes , P. , F. Lerasle , J. Dias , and T. Germa . 2007 . Humanoid robots, humanlike machines. Towards an interactive humanoid companion with visual tracking modalities , 367–398. Advanced Robotic Systems Int. and I-Tech Education and Publishing. · DOI
- Möller , S. , R. Englert , K. Engelbrecht , V. Hafner , A. Jameson , A. Oulasvirta , A. Raake , and N. Reithinger . 2006 . Memo: Towards automatic usability evaluation of spoken dialogue services by user error simulations. In Proceedings of the 9th international conference on spoken language processing (Interspeech/ICSLP) , 1786–1789. Pittsburgh, PA, USA. · DOI
- Navarro , L. , F. Flacher , and V. Corruble . 2011 . Dynamic level of detail for large scale agent-based urban simulations. In Proceedings of the 10th international conference on autonomous agents and multiagent systems (AAMAS’11) , 701–708. Taipei, Taiwan.
- 10.1145/1122012.1122013 · DOI
- 10.1504/IJAOSE.2008.017315 · doi:10.1504/ijaose.2008.017315
- 10.1109/TSA.2005.855836 · DOI · Paper page
- Sánchez-Pi , N. , V. Fuentes , J. Carbó , and J. Molina . 2007 . Knowledge-based system to define context in commercial applications. In Proceedings of the 8th ACIS international conference on software engineering, artificial intelligence, networking and parallel/distributed computing (SNPD 07) , 694–699. Tsingtao, China. · doi:10.1109/snpd.2007.394
- Schatzmann , J. , K. Georgila , and S. Young . 2005 . Quantitative evaluation of user simulation techniques for spoken dialogue systems. In Proceedings of the 6th SIGdial workshop on discourse and dialogue , 45–54. Lisbon, Portugal.
- Schatzmann , J. , B. Thomson , and S. Young . 2007 . Statistical user simulation with a hidden agenda. In Proceedings of the 8th SIGdial workshop on discourse and dialogue , 273–282. Antwerp, Belgium.
- 10.1016/j.csl.2007.09.002 · doi:10.1016/j.csl.2007.09.002
- Vaquero , C. , O. Saz , E. Lleida , J. Marcos , and C. Canalís . 2006 . VOCALIZA: An application for computer-aided speech therapy in Spanish language. In Proceedings IV Jornadas en Tecnología del Habla , 321–326. Zaragoza, Spain.
- Weng , F. , S. Varges , B. Raghunathan , F. Ratiu , H. Pon-Barry , B. Lathrop , Q. Zhang , T. Scheideck , H. Bratt , K. Xu , M. Purver , R. Mishra , M. Raya , S. Peters , Y. Meng , L. Cavedon , and L. Shriberg . 2006 . CHAT: A conversational helper for automotive tasks. In Proceedings of the 9th international conference on spoken language processing (Interspeech/ICSLP) , 1061–1064. Pittsburgh, PA, USA. · doi:10.21437/interspeech.2006-159
- Weyns , D. , N. Boucké , and T. Holvoet . 2006. Gradient field-based task assignment in an Agv transportation system. In Proceedings of the 5th international conference on autonomous agents and multiagent systems (AAMAS’06) , 842–849. Hakodate, Japan. · DOI
- Wilensky U., 2012, An introduction to agent-based modeling: Modeling natural, social and engineered complex systems with NetLogo
- 10.1016/j.csl.2006.06.008 · DOI · Paper page
- Windrum P., 2007, Journal of Artificial Societies and Social Simulation, 10, 8
- Young , S. 2002 . The statistical approach to the design of spoken dialogue systems (Technical Report, CUED/F-INFENG/TR.433, Cambridge University Engineering Department, Cambridge, UK).
- 10.1109/89.817460 · DOI · Paper page
Cited by
Conversational agents in healthcare: a systematic review
Laranjo, L., Dunn, A. G., Tong, H. L., Kocaballi, A. B., Chen, J., Bashir, R., Surian, D., Gallego, B., Magrabi, F., Lau, A. Y. S., et al. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy072 · Cited by 1,388
“31, 32 and health research measures ( 9 studies”
BackgroundAraujo, T. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.03.051 · Cited by 1,156
Frontline Service Technology infusion: conceptual archetypes and future research directions
De Keyser, A., Köcher, S., Alkire (née Nasr), L., Verbeeck, C., & Kandampully, J. (2019) · Journal of Service Management · doi:10.1108/josm-03-2018-0082 · Cited by 383
The Personalization of Conversational Agents in Health Care: Systematic Review
Kocaballi, A. B., Berkovsky, S., Quiroz, J. C., Laranjo, L., Tong, H. L., Rezazadegan, D., Briatore, A., & Coiera, E. (2019) · Journal of Medical Internet Research · doi:10.2196/15360 · Cited by 332
“The purposes of providing personalized content and conversations were to: (1) improve user engagement [35,37,38] and dialogue quality [42,43,54]; (2) provide timely feedback [39,40], adaptive user support [41], and adaptive training [36,38]; and, (3) support self-reflection [45,46].”
BackgroundCastillo, D., Canhoto, A. I., & Said, E. (2021) · The Service Industries Journal · doi:10.1080/02642069.2020.1787993 · Cited by 274
Metadata
- Authors
- Griol, D., Carbó, J., & Molina, J. M.
- Venue
- Applied Artificial Intelligence
- Publisher
- Informa UK Limited
- Published
- 2013-10-21
- Volume
- 27
- Issue
- 9
- Pages
- 759-780
- ISSN
- 0883-9514 (ISSN-L)
- 1087-6545
- OpenAlex
- W2039133058
- Cited by
- 109
- References
- 5
- FWCI
- 4.05
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: AN AUTOMATIC DIALOG SIMULATION TECHNIQUE TO DEVELOP AND EVALUATE INTERACTIVE CONVERSATIONAL AGENTS. https://app.cochat.ai/papers/10.1080/08839514.2013.835230
- The paper
- Griol, D., Carbó, J., & Molina, J. M. (2013). AN AUTOMATIC DIALOG SIMULATION TECHNIQUE TO DEVELOP AND EVALUATE INTERACTIVE CONVERSATIONAL AGENTS. Applied Artificial Intelligence. https://doi.org/10.1080/08839514.2013.835230
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.