CoChat

AN AUTOMATIC DIALOG SIMULATION TECHNIQUE TO DEVELOP AND EVALUATE INTERACTIVE CONVERSATIONAL AGENTS

Open access
Griol, David, Carbó, Javier, Molina, José M. · 2013 · Applied Artificial Intelligence
Cited by 109

The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.

Abstract

During recent years, conversational agents have become a solution to provide straightforward and more natural ways of retrieving information in the digital domain. In this article, we present an agent-based dialog simulation technique for learning new dialog strategies and evaluating conversational agents. Using this technique, the effort necessary to acquire data required to train the dialog model and then explore new dialog strategies is considerably reduced. A set of measures has also been defined to evaluate the dialog strategy that is automatically learned and to compare different dialog corpora. We have applied this technique to explore the space of possible dialog strategies and evaluate the dialogs acquired for a conversational agent that collects monitored data from patients suffering from diabetes. The results of the comparison of these measures for an initial corpus and a corpus acquired using the dialog simulation technique show that the conversational agent reduces the time needed to complete the dialogs and improve their quality, thereby allowing the conversational agent to tackle new situations and generate new coherent answers for the situations already present in an initial model.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1080/08839514.2013.835230

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 5 references

Research works printed but not linked: 47

  1. Ai , H. , A. Raux , D. Bohus , M. Eskenazi , and D. Litman . 2007 . Comparing spoken dialog corpora collected with recruited subjects versus real users. In Proceedings of the 8th SIGdial workshop on discourse and dialogue , 124–131. Antwerp, Belgium.
  2. 10.1016/j.specom.2010.02.015 · DOI
  3. Ballinas-Hernandez A. L., 2011, Journal of Artificial Societies and Social Simulation, 14, 2, 10.18564/jasss.1789 · doi:10.18564/jasss.1789
  4. 10.1007/978-1-4020-5060-2_11 · doi:10.1007/978-1-4020-5060-2_11
  5. Bandini S., 2006, Neuralriets, 231
  6. Bandini S., 2009, Journal of Artificial Societies and Social Simulation, 12, 97
  7. Bohus , D. , S. Grau , D. Huggins-Daines , V. Keri , G. Krishna , R. Kumar , A. Raux , and S. Tomko . 2007 . Conquest - An open-source dialog system for conferences. In Proceedings of 7th meeting of the North American chapter of the association for computational linguistics (HLT/NAACL 07), 9–12. Rochester, NY, USA. · doi:10.3115/1614108.1614111
  8. 10.1007/1-4020-3075-4_12 · DOI
  9. Bos , J. , E. Klein , O. Lemon , and T. Oka . 2003 . DIPPER: Description and formalisation of an information-state update dialogue system architecture. In Proceedings of the 4th SIGdial workshop on discourse and dialogue , 115–124. Sapporo, Japan.
  10. Brahnam S., 2009, PsychNology Journal, 7, 9
  11. Chung , G. 2004 . Developing a flexible spoken dialog system using simulation. In Proceedings of the 42nd annual meeting of the association for computational linguistics (ACL 04) , 63–70. Barcelona, Spain. · DOI
  12. Cuayáhuitl , H. , S. Renals , O. Lemon , and H. Shimodaira . 2005 . Human-computer Dialogue simulation using hidden Markov models. In Proceedings of the IEEE workshop on automatic speech recognition and understanding (ASRU'05) , 290–295. San Juan, Puerto Rico. · DOI · Paper page
  13. Eckert , W. , E. Levin , and R. Pieraccini . 1997 . User modeling for spoken dialogue system evaluation. In Proceedings of IEEE automatic speech recognition and understanding workshop (ASRU 97) , 80–87. Santa Barbara, USA. · DOI · Paper page
  14. Eckert W., 1998, Automatic evaluation of spoken dialogue systems
  15. 10.1007/978-3-540-39398-6_24 · doi:10.1007/978-3-540-39398-6_24
  16. Georgila , K. , J. Henderson , and O. Lemon . 2005 . Learning user simulations for information state update dialogue systems. In Proceedings of the 9th European conference on speech communication and technology (Eurospeech 05) , 893–896. Lisbon, Portugal. · DOI · Paper page
  17. 10.1016/0167-6393(95)00008-C · DOI
  18. 10.1016/j.specom.2008.04.001 · DOI
  19. 10.1007/978-3-642-14883-5_36 · DOI
  20. Heath B., 2009, Journal of Artificial Societies and Social Simulation, 12, 9
  21. 10.1016/j.csl.2010.06.002 · DOI
  22. 10.1007/978-3-540-39869-1_2 · doi:10.1007/978-3-540-39869-1_2
  23. 10.1109/89.817450 · DOI · Paper page
  24. 10.1109/89.928918 · DOI
  25. López-Cózar R., 2005, Spoken, multilingual and multimodal dialogue systems
  26. 10.1016/S0167-6393(02)00126-7 · DOI
  27. 10.1057/jos.2010.3 · DOI · Paper page
  28. 10.1007/978-0-85729-414-2 · DOI · Paper page
  29. Melin , H. , A. Sandell , and M. Ihse . 2001 . CTT-bank: A speech controlled telephone banking system - an initial evaluation. In TMH Quarterly Progress and Status Report (TMH-QPSR) 1:1–27.
  30. Menezes , P. , F. Lerasle , J. Dias , and T. Germa . 2007 . Humanoid robots, humanlike machines. Towards an interactive humanoid companion with visual tracking modalities , 367–398. Advanced Robotic Systems Int. and I-Tech Education and Publishing. · DOI
  31. Möller , S. , R. Englert , K. Engelbrecht , V. Hafner , A. Jameson , A. Oulasvirta , A. Raake , and N. Reithinger . 2006 . Memo: Towards automatic usability evaluation of spoken dialogue services by user error simulations. In Proceedings of the 9th international conference on spoken language processing (Interspeech/ICSLP) , 1786–1789. Pittsburgh, PA, USA. · DOI
  32. Navarro , L. , F. Flacher , and V. Corruble . 2011 . Dynamic level of detail for large scale agent-based urban simulations. In Proceedings of the 10th international conference on autonomous agents and multiagent systems (AAMAS’11) , 701–708. Taipei, Taiwan.
  33. 10.1145/1122012.1122013 · DOI
  34. 10.1504/IJAOSE.2008.017315 · doi:10.1504/ijaose.2008.017315
  35. 10.1109/TSA.2005.855836 · DOI · Paper page
  36. Sánchez-Pi , N. , V. Fuentes , J. Carbó , and J. Molina . 2007 . Knowledge-based system to define context in commercial applications. In Proceedings of the 8th ACIS international conference on software engineering, artificial intelligence, networking and parallel/distributed computing (SNPD 07) , 694–699. Tsingtao, China. · doi:10.1109/snpd.2007.394
  37. Schatzmann , J. , K. Georgila , and S. Young . 2005 . Quantitative evaluation of user simulation techniques for spoken dialogue systems. In Proceedings of the 6th SIGdial workshop on discourse and dialogue , 45–54. Lisbon, Portugal.
  38. Schatzmann , J. , B. Thomson , and S. Young . 2007 . Statistical user simulation with a hidden agenda. In Proceedings of the 8th SIGdial workshop on discourse and dialogue , 273–282. Antwerp, Belgium.
  39. 10.1016/j.csl.2007.09.002 · doi:10.1016/j.csl.2007.09.002
  40. Vaquero , C. , O. Saz , E. Lleida , J. Marcos , and C. Canalís . 2006 . VOCALIZA: An application for computer-aided speech therapy in Spanish language. In Proceedings IV Jornadas en Tecnología del Habla , 321–326. Zaragoza, Spain.
  41. Weng , F. , S. Varges , B. Raghunathan , F. Ratiu , H. Pon-Barry , B. Lathrop , Q. Zhang , T. Scheideck , H. Bratt , K. Xu , M. Purver , R. Mishra , M. Raya , S. Peters , Y. Meng , L. Cavedon , and L. Shriberg . 2006 . CHAT: A conversational helper for automotive tasks. In Proceedings of the 9th international conference on spoken language processing (Interspeech/ICSLP) , 1061–1064. Pittsburgh, PA, USA. · doi:10.21437/interspeech.2006-159
  42. Weyns , D. , N. Boucké , and T. Holvoet . 2006. Gradient field-based task assignment in an Agv transportation system. In Proceedings of the 5th international conference on autonomous agents and multiagent systems (AAMAS’06) , 842–849. Hakodate, Japan. · DOI
  43. Wilensky U., 2012, An introduction to agent-based modeling: Modeling natural, social and engineered complex systems with NetLogo
  44. 10.1016/j.csl.2006.06.008 · DOI · Paper page
  45. Windrum P., 2007, Journal of Artificial Societies and Social Simulation, 10, 8
  46. Young , S. 2002 . The statistical approach to the design of spoken dialogue systems (Technical Report, CUED/F-INFENG/TR.433, Cambridge University Engineering Department, Cambridge, UK).
  47. 10.1109/89.817460 · DOI · Paper page

Cited by

See all 109

Metadata

Authors
Griol, D., Carbó, J., & Molina, J. M.
Venue
Applied Artificial Intelligence
Published
2013-10-21
Volume
27
Issue
9
Pages
759-780
ISSN
  • 0883-9514 (ISSN-L)
  • 1087-6545
OpenAlex
W2039133058
Cited by
109
References
5
FWCI
4.05
Type
journal article
Language
en
Open access
bronze

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: AN AUTOMATIC DIALOG SIMULATION TECHNIQUE TO DEVELOP AND EVALUATE INTERACTIVE CONVERSATIONAL AGENTS. https://app.cochat.ai/papers/10.1080/08839514.2013.835230
The paper
Griol, D., Carbó, J., & Molina, J. M. (2013). AN AUTOMATIC DIALOG SIMULATION TECHNIQUE TO DEVELOP AND EVALUATE INTERACTIVE CONVERSATIONAL AGENTS. Applied Artificial Intelligence. https://doi.org/10.1080/08839514.2013.835230

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.