CoChat
Sign Up
Cited by 21Speech and dialogue systemsTechnology Use by Older AdultsSocial Robot Interaction and HRI

The MATCH corpus: a corpus of older and younger users’ interactions with spoken dialogue systems

Open access
Georgila, Kallirroi, Wolters, Maria, Moore, Johanna D., Logie, Robert H.·2010·Language Resources and Evaluation
DOI 10.1007/s10579-010-9118-8
Open this paper in CoChat — free accountOpen PDF

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s10579-010-9118-8

Cites

13 of 13 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 70

  1. Anderson, S., Liberman, N., Bernstein, E., Foster, S., Cate, E., Levin, B., & Hudson, R. (1999). Recognition of elderly speech and voice-driven document retrieval. In: Proceedings of the IEEE international conference on acoustics, speech and signal processing (pp. 145-148). Phoenix, AZ, USA.
  2. Arking, R. (2005). Biology of aging 3rd edn. New York, NY, USA: Oxford University Press.
  3. Aylett, M., Pidcock, C., & Fraser, M. (2006). The Cerevoice Blizzard Entry 2006: A prototype database unit selection engine. In: Proceedings of the 2nd BLIZZARD challenge. Pittsburgh, PA, USA.
  4. Baba, A., Yoshizawa, S., Yamada, M., Lee, A., & Shikano, K. (2002). Elderly acoustic models for large vocabulary continuous speech recognition. Transactions of the Institute of Electronics, Information and Communication Engineers, D-II J85D-II, 390-397.
  5. Baeckman, L., Small, B. J., & Wahlin, A. (2001). Aging and memory: Cognitive and biological perspectives. In J. E. Birren & K. W. Schaie (Eds.), Handbook of the psychology of aging (pp. 349- 377). San Diego, CA, USA: Academic Press.
  6. Bailey, P. E., & Henry, J. D. (2008). Growing less empathic with age: Disinhibition of the self- perspective. Journals of Gerontology Series B-Psychological Sciences and Social Sciences, 63(4), 219-226.
  7. Baltes, P., & Baltes, M. M. (1990). Psychological perspectives on successful aging: The model of selective optimization with compensation. In P. B. Baltes & M. M. Baltes (Eds.), Successful aging: perspectives from the behavioral sciences (pp. 1-34). Cambridge, UK: Cambridge University Press.
  8. Black, L. A., McMeel, C., McTear, M., Black, N., Harper, R., & Lemon, M. (2005a). Implementing autonomy in a diabetes management system. Journal of Telemedicine and Telecare, 11(Suppl 1), 6-8.
  9. Black, L. A., McTear, M., Black, N., Harper, R., & Lemon, M. (2005b). Evaluating the DI@L-log system on a cohort of elderly, diabetic patients: Results from a preliminary study. In: Proceedings of Interspeech (pp. 821-824). Lisbon, Portugal.
  10. Bohus, D., & Rudnicky, A. (2005). Sorry, I didn't catch that!-An investigation of non-understanding errors and recovery strategies. In: Proceedings of the 5th SIGdial workshop (pp. 128-143). Lisbon, Portugal.
  11. Burke, D. M., & Shafto, M. A. (2004). Aging and language production. Current Directions in Psychological Science, 13(1), 21-24.
  12. Carletta, J. (1996). Assessing agreement on classification tasks: The kappa statistic. Computational Linguistics, 22(2), 249-254.
  13. Carletta, J. (2007). Unleashing the killer corpus: Experiences in creating the multi-everything AMI meeting corpus. Language Resources and Evaluation, 41(2), 181-190.
  14. Carletta, J., Evert, S., Heid, U., Kilgour, J., Robertson, J., & Voormann, H. (2003). The NITE XML toolkit: Flexible annotation for multi-modal language data. Behavior Research Methods, Instruments, and Computers, 35(3), 353-363.
  15. Catizone, R., Dingli, A., Pinto, H., & Wilks, Y. (2008). Information extraction tools and methods for understanding dialogue in a companion. In: Proceedings of the 6th international conference on language resources and evaluation (LREC). Marrakech, Morocco.
  16. Chu-Carroll, J., & Nickerson, J. S. (2000). Evaluating automatic dialogue strategy adaptation for a spoken dialogue system. In: Proceedings of the annual meeting of the north American chapter of the association for computational linguistics (NAACL) (pp. 202-209). Seattle, WA, USA.
  17. Commarford, P. (2006). Working memory, search, and signal detection: Implications for interactive voice response system menu design. PhD thesis, Florida, USA: University of Central Florida.
  18. Cucchiarini, C., van Hamme, H., van Herwijnen, O., & Smits, F. (2006). JASMIN-CGN: Extension of the spoken Dutch corpus with speech of elderly people, children and non-natives in the human-machine interaction modality. In: Proceedings of the 5th international conference on language resources and evaluation (LREC) (pp. 135-138). Genoa, Italy.
  19. Cucchiarini, C., Driese, J., van Hamme, H., & Sanders, E. (2008). Recording speech of children, non- natives and elderly people for HLT applications: The JASMIN-CGN corpus. In: Proceedings of the 6th international conference on language resources and evaluation (LREC). Marrakech, Morocco.
  20. Duffy, S., & Pisoni, D. (1992). Comprehension of synthetic speech produced by rule: A review and theoretical interpretation. Language and Speech, 35, 351-389.
  21. Eskenazi, M., & Black, A. (2001). A study on speech over the telephone and aging. In: Proceedings of Eurospeech. Aalborg, Denmark.
  22. Fozard, J.L., & Gordon-Salant, S. (2001). Changes in vision and hearing with aging. In J. E. Birren & K. W. Schaie (Eds.), Handbook of the psychology of aging (pp. 241-266). San Diego, CA, USA: Academic Press.
  23. Georgila, K., Henderson, J., & Lemon, O. (2005a). Learning user simulations for Information State Update dialogue systems. In: Proceedings of Interspeech (pp. 893-896). Lisbon, Portugal.
  24. Georgila, K., Lemon, O., & Henderson, J. (2005b). Automatic annotation of COMMUNICATOR dialogue data for learning dialogue strategies and user simulations. In: Proceedings of SEMDIAL:DIALOR (pp. 61-68).
  25. Georgila, K., Henderson, J., & Lemon, O. (2006). User simulation for spoken dialogue systems: Learning and evaluation. In: Proceedings of Interspeech (pp. 1065-1068). Pittsburgh, PA, USA.
  26. Georgila, K., Wolters, M., Karaiskos, V., Kronenthal, M., Logie, R., Mayo N., Moore, J. D., & Watson, M. (2008a). A fully annotated corpus for studying the effect of cognitive ageing on users' interactions with spoken dialogue systems. In: Proceedings of the 6th international conference on language resources and evaluation (LREC) (pp. 938-944). Marrakech, Morocco.
  27. Georgila, K., Wolters, M., & Moore, J. D. (2008b). Simulating the behaviour of older versus younger users. In: Proceedings of the 46th annual meeting of the association for computational linguistics, human language technologies (ACL/HLT) (pp. 49-52). Columbus, OH, USA.
  28. Georgila, K., Lemon, O., Henderson, J., & Moore, J. D. (2009). Automatic annotation of context and speech acts for dialogue corpora. Natural Language Engineering, 15(3), 315-353.
  29. ¨dde, F., Mo ¨ller, S., Engelbrecht, K. P., Ku ¨hnel, C., Schleicher, R., Naumann, A., & Wolters, M. (2008). Study of a speech-based smart home system with older users. In: International workshop on intelligent user interfaces for ambient assisted living (pp. 17-22).
  30. Gregor, P., Newell, A. F., & Zajicek, M. (2002). Designing for dynamic diversity-Interfaces for older people. In: Proceedings of ASSETS 2002, the fifth international ACM conference on assistive technologies (pp. 151-156). Edinburgh, UK.
  31. Hardee, J. (2007). Reexaming synthetic speech: Intelligibility and the effects of age, task, and speech type on recall. MSc thesis, North Carolina, USA: North Carolina State University.
  32. Horton, W. S., & Spieler, D. H. (2007). Age-related differences in communication and audience design. Psychology and Aging, 22(2), 281-290.
  33. Humes, L., & Floyd, S. (2005). Measures of working memory, sequence learning, and speech recognition in the elderly. Journal of Speech, Language, and Hearing Research, 48, 224-235.
  34. Keates, S., & Clarkson, J. (2004). Inclusive design. London: Springer.
  35. Kemper, S., Herman, R. E., & Liu, C. J. (2004). Sentence production by young and older adults in controlled contexts. Journals of Gerontology Series B-Psychological Sciences and Social Sciences, 59(5), 220-224.
  36. Langner, B., & Black, A. (2005). Using speech in noice to improve understandability for elderly listeners. In: Proceedings of IEEE ASRU. San Juan, Puerto Rico.
  37. Larsson, S., & Traum, D. (2000). Information state and dialogue management in the TRINDI dialogue move engine toolkit. Natural Language Engineering, 6(3-4), 323-340.
  38. Lemon, O., & Pietquin, O. (2007). Machine learning for spoken dialogue systems. In: Proceedings of Interspeech (pp. 2685-2688). Antwerp, Belgium.
  39. Lemon, O., Georgila, K., Henderson, J., & Stuttle, M. (2006). An ISU dialogue system exhibiting reinforcement learning of dialogue policies: Generic slot-filling in the talk in-car system. In: Proceedings of the 11th conference of the European chapter of the association for computational linguistics (EACL) (pp. 119-122). Trento, Italy.
  40. Levin, E., Pieraccini, R., & Eckert, W. (2000). A stochastic model of human-machine interaction for learning dialog strategies. IEEE Transactions on Speech and Audio Processing, 8(1), 11-23.
  41. Lines, L., & Hone, K. S. (2006). Multiple voices, multiple choices: Older adults evaluation of speech output to support independent living. Gerontechnology, 5(2), 78-91.
  42. Linville, S. E. (2000). The aging voice. In: Voice quality measurement, singular (pp. 359-376).
  43. Litman, D., Silliman, S. (2004). ITSPOKE: An intelligent tutoring spoken dialogue system. In: 4th meeting of the north American chapter of the association for computational linguistics (NAACL/HLT) (pp. 5-8). Boston, MA, USA.
  44. Mo ¨ller, S., Engelbrecht, K. P., & Oulasvirta, A. (2007). Analysis of communication failures for spoken dialogue systems. In: Proceedings of Interspeech (pp. 134-137). Antwerp, Belgium.
  45. Mo ¨ller, S., Smeele, P., Boland, H., & Krebber, J. (2007). Evaluating spoken dialogue systems according to de-facto standards: A case study. Computer Speech and Language, 21(1), 26-53.
  46. Mo ¨ller, S., Go ¨dde, F., & Wolters, M. (2008). A corpus analysis of spoken smart-home interactions with older users. In: Proceedings of the 6th international conference on language resources and evaluation (LREC) (pp. 735-740). Marrakech, Morocco.
  47. Montemerlo, M., Pineau, J., Roy, N., Thrun, S., & Verma, V. (2002). Experiences with a mobile robotic guide for the elderly. In: Proceedings of the AAAI national conference on artificial intelligence.
  48. Moore, J. D., Foster, M. E., Lemon, O., & White, M. (2004). Generating tailored, comparative descriptions in spoken dialogue. In: Proceedings of the seventeenth international Florida artificial intelligence research society conference.
  49. ¨ller, C., Wittig, F., & Baus, J. (2003). Exploiting speech for recognizing elderly users to respond to their special needs. In: Proceedings of Eurospeech (pp. 1305-1308). Geneva, Switzerland.
  50. Paek, T., & Chickering, D. (2007). Improving command and control speech recognition on mobile devices: Using predictive user models for language modeling. User Modeling and User-Adapted Interaction, Special Issue on Statistical and Probabilistic Methods for User Modeling, 17(1-2), 93-117.
  51. Pietquin, O., & Dutoit, T. (2006). A probabilistic framework for dialog simulation and optimal strategy learning. IEEE Transactions on Audio, Speech and Language Processing, 14(2), 589-599.
  52. Rabbitt, P., & Anderson, M. M. (2005). The lacunae of loss? Aging and the differentiation of human abilities. In E. Bialystok & F. I. Craik (Eds.), Lifespan cognition: Mechanisms of change. New York, NY, USA: Oxford University Press.
  53. Raux, A., Bohus, D., Langner, B., Black, A., & Eskenazi, M. (2006). Doing research on a deployed spoken dialogue system: One year of Let's Go! experience. In: Proceedings of Interspeech.
  54. Roring, R., Hines, F., & Charness, N. (2007). Age differences in identifying words in synthetic speech. Human Factors: The Journal of the Human Factors and Ergonomics Society, 49(1), 25-31.
  55. Roy, N., Pineau, J., & Thrun, S. (2000). Spoken dialog management for robots. In: Proceedings of the annual meeting of the association for computational linguistics (ACL).
  56. Schatzmann, J., Weilhammer, K., Stuttle, M., & Young, S. (2006). A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies. Knowledge Engineering Review, 21(2), 97-126.
  57. Searle, J. (1969). Speech Acts: An Essay in the Philosophy of Language. Cambridge University Press.
  58. Smither, J. A. A. (1993). Short term memory demands in processing synthetic speech by old and young adults. Behaviour and Information Technology, 12(6), 330-335.
  59. Traum, D. (1994). A computational theory of grounding in natural language conversation. PhD thesis, New York, USA: University of Rochester.
  60. Traum, D. (2000). 20 questions for dialogue act taxonomies. Journal of Semantics, 17(1), 7-30.
  61. Traum, D. R., Swartout, W., Gratch, J., & Marsella, S. (2008). A virtual human dialogue model for non- team interaction. In L. Dybkjaer & W. Minker (Eds.), Recent Trends in Discourse and Dialogue (pp. 45-67). New York, NY, USA: Springer.
  62. Walker, M. A., Passonneau, R. J., & Boland, J. (2001). Quantitative and qualitative evaluation of DARPA Communicator spoken dialogue systems. In: Proceedings of the annual meeting of the association for computational linguistics (ACL) (pp. 515-522). Toulouse, France.
  63. Walker, M. A., Langkilde-Geary, I., Wright-Hastie, H., Wright, J. H., & Gorin, A. L. (2002). Automatically training a problematic dialogue predictor for the HMIHY spoken dialogue system. Journal of Artificial Intelligence Research, 16, 293-331.
  64. Wolters, M., Campbell, P., DePlacido, C., Liddell, A., & Owens, D. (2007). Making synthetic speech accessible to older people. In: Proceedings of the 6th ISCA workshop on speech synthesis. Bonn, Germany.
  65. Wolters, M., Georgila, K., Moore, J. D., Logie, R. H., MacPherson, S. E., & Watson, M. (2009a). Reducing working memory load in spoken dialogues: Do we have to limit the number of options?. Interacting with Computers, 21(4), 276-287.
  66. Wolters, M., Georgila, K., Moore, J. D., & MacPherson, S. E. (2009b). Being old doesn't mean acting old: How older users interact with spoken dialog systems. ACM Transactions on Accessible Computing (TACCESS), 2(1, Article 2), 39.
  67. Young, S. (2000). Probabilistic methods in spoken dialogue systems. Philosophical Transactions of the Royal Society (Series A), 358(1769), 1389-1402.
  68. Zajicek, M. (2006). Aspects of HCI research for older people. Universal Access in the Information Society, 5(3), 279-286.
  69. Zajicek, M., Wales, R., & Lee, A. (2004). Speech interaction for older adults. Universal Access in the Information Society, 3(2), 122-130.
  70. Zinn, C., Moore, J. D., & Core, M. G. (2002). A 3-tier planning architecture for managing tutorial dialogue. In: Proceedings of intelligent tutoring systems, sixth international conference (pp. 574-584). Biarritz, France.

Cited by

Sorted by citations

Authors
Georgila, K., Wolters, M., Moore, J. D., & Logie, R. H.
Affiliations
  • University of Southern California (US)
  • University of Edinburgh (GB)
Corresponding author
Georgila, Kallirroi
ORCID
Venue
Language Resources and Evaluation
Publisher
Springer Science and Business Media LLC
Published
2010-03-15
Volume
44
Issue
3
Pages
221-261
ISSN
  • 1574-020X (ISSN-L)
  • 1574-0218
DOI
10.1007/s10579-010-9118-8
OpenAlex
W2110874054
Cited by
21
References
13
Citations by year
2023: 1 · 2022: 1 · 2021: 2 · 2020: 2 · 2019: 3 · 2018: 1 · 2017: 2 · 2016: 1 · 2015: 3 · 2014: 1 · 2012: 1
FWCI
1.35
Type
journal article
Language
en
Open access
Licence
www.springer.com/tdm (tdm)
Locations
  • Language Resources and Evaluation (publishedVersion)
  • ERA (submittedVersion)
Funders
  • Wellcome Trust
  • Research, Development and Engineering Command
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Social Sciences › Demography › Technology Use by Older Adults
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Keywords
spoken dialogue systems, older adults, dialogue act annotation, Wizard of Oz, usability evaluation, cognitive ability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review