Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech
Open accessThe DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index.
Abstract
We describe a statistical approach for modeling dialogue acts in conversational speech, i.e., speech-act-like units such as STATEMENT, Question, BACKCHANNEL, Agreement, Disagreement, and Apology. Our model detects and predicts dialogue acts based on lexical, collocational, and prosodic cues, as well as on the discourse coherence of the dialogue act sequence. The dialogue model is based on treating the discourse structure of a conversation as a hidden Markov model and the individual dialogue acts as observations emanating from the model states. Constraints on the likely sequence of dialogue acts are modeled via a dialogue act n-gram. The statistical dialogue grammar is combined with word n-grams, decision trees, and neural networks modeling the idiosyncratic lexical and prosodic manifestations of each dialogue act. We develop a probabilistic integration of speech recognition with dialogue modeling, to improve both speech recognition and dialogue act classification accuracy. Models are trained and evaluated using a large hand-labeled database of 1,155 conversations from the Switchboard corpus of spontaneous human-to-human telephone speech. We achieved good dialogue act labeling accuracy (65% based on errorful, automatically recognized words and prosody, and 71% based on word transcripts, compared to a chance baseline accuracy of 35% and human accuracy of 84%) and a small reduction in word recognition error.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Nonparametric Statistics for the Behavioral Sciences
Guitton, H., & Siegel, S. (1958) · Revue économique · doi:10.2307/3498751 · Cited by 35,245
Classification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,522
Austin, J. L. (1975) · doi:10.1093/acprof:oso/9780198245537.001.0001 · Cited by 21,496
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 20,825
(1988) · Cited by 16,582
A simplest systematics for the organization of turn-taking for conversation
Sacks, H., Schegloff, E. A., & Jefferson, G. (1974) · Language · doi:10.2307/412243 · Cited by 11,350
Error bounds for convolutional codes and an asymptotically optimum decoding algorithm
Viterbi, A. (1967) · IEEE Transactions on Information Theory · doi:10.1109/tit.1967.1054010 · Cited by 6,759
An introduction to hidden Markov models
Rabiner, L., & Juang, B. (1986) · IEEE ASSP Magazine · doi:10.1109/massp.1986.1165342 · Cited by 4,775
Baum, L. E., Petrie, T., Soules, G., & Weiss, N. (1970) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177697196 · Cited by 4,297
A maximum entropy approach to natural language processing
Berger, A. L., Pietra, V. J. D., & Pietra, S. A. D. (1996) · doi:10.5555/234285.234289 · Cited by 3,065
Attention, intentions, and the structure of discourse
Grosz, B. J., & Sidner, C. L. (1986) · doi:10.5555/12457.12458 · Cited by 2,580
Sequencing in Conversational Openings1
SCHEGLOFF, E. A. (1968) · American Anthropologist · doi:10.1525/aa.1968.70.6.02a00030 · Cited by 2,448
Schegloff, E. A. (1982) · Medical Entomology and Zoology · Cited by 2,317
SWITCHBOARD: telephone speech corpus for research and development
Godfrey, J. J., Holliman, E. C., & McDaniel, J. (1992) · [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp.1992.225858 · Cited by 2,134
Assessing Agreement on Classification Tasks: The Kappa Statistic
Carletta, J. (1996) · arXiv · doi:10.48550/arxiv.cmp-lg/9602004 · Cited by 2,088
De Roeck, A. (1995) · Handbook of Pragmatics · doi:10.1075/hop.m.comm2 · Cited by 1,715
Estimation of probabilities from sparse data for the language model component of a speech recognizer
Katz, S. (1987) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/tassp.1987.1165125 · Cited by 1,637
Bridle, J. S. (1990) · Neurocomputing · doi:10.1007/978-3-642-76153-9_28 · Cited by 1,237
A Maximum Likelihood Approach to Continuous Speech Recognition
Bahl, L. R., Jelinek, F., & Mercer, R. L. (1983) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1983.4767370 · Cited by 1,212
Connectionist Speech Recognition: A Hybrid Approach
Bourlard, H. A., & Morgan, N. H. (1993) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 1,115
Anderson, A. H., Bader, M., Bard, E. G., Boyle, E., Doherty, G., Garrod, S., Isard, S., Kowtko, J., McAllister, J., Miller, J., et al. (1991) · Language and Speech · doi:10.1177/002383099103400404 · Cited by 1,017
A stochastic parts program and noun phrase parser for unrestricted text
Church, K. W. (1988) · Proceedings of the second conference on Applied natural language processing - · doi:10.3115/974235.974260 · Cited by 957
Witten, I. H., & Bell, T. C. (1991) · IEEE Transactions on Information Theory · doi:10.1109/18.87000 · Cited by 729
Connectionist Speech Recognition
Bourlard, H. A., & Morgan, N. (1994) · Springer eBooks · doi:10.1007/978-1-4615-3210-1 · Cited by 585
Getting a Word in Edgewise: Laura Terracina’s Discorsi on the Orlando Furioso
(1998) · Ladies Errant · doi:10.2307/j.ctv11cw36q.8 · Cited by 584
Notes on a systematic deployment of the acknowledgement tokens “Yeah”; and “Mm Hm”;
Jefferson, G. (1984) · Paper in Linguistics · doi:10.1080/08351818409389201 · Cited by 571
Proceedings of the International Conference on Spoken Language Processing.
Lickley, R. J., & Bard, E. G. (1992) · Cited by 562
Sequencing In Conversational Openings
Schegloff, E. A. (1972) · Selected Studies and Applications · doi:10.1515/9783110880434-006 · Cited by 514
Fowler, C. A., & Housum, J. (1987) · Journal of Memory and Language · doi:10.1016/0749-596x(87)90136-7 · Cited by 496
A cache-based natural language model for speech recognition
Kuhn, R., & De Mori, R. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.56193 · Cited by 495
Attention, intention, and the structure of discourse
Grosz, B. J., & Sidner, C. L. (1986) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 487
Dialogue games: An approach to discourse analysis By Lauri Carlson (review)
Frawley, W. (1984) · Language · doi:10.1353/lan.1984.0010 · Cited by 462
Prosody-based automatic segmentation of speech into sentences and topics
Shriberg, E., Stolcke, A., Hakkani-Tür, D., & Tür, G. (2000) · Speech Communication · doi:10.1016/s0167-6393(00)00028-5 · Cited by 424
Self-citationSproat, R., Samuelsson, C., Chu‐Carroll, J., & Carpenter, B. (2003) · The Handbook of Linguistics · doi:10.1002/9780470756409.ch25 · Cited by 376
Empirical Studies on the Disambiguation of Cue Phrases
Hirschberg, J., & Litman, D. J. (1991) · doi:10.7916/d8pv6tgb · Cited by 366
Analysis and synthesis of intonation using the Tilt model
Taylor, P. (2000) · The Journal of the Acoustical Society of America · doi:10.1121/1.428453 · Cited by 355
Self-citationCan Prosody Aid the Automatic Classification of Dialog Acts in Conversational Speech?
Shriberg, E., Stolcke, A., Jurafsky, D., Coccaro, N., Meteer, M., Bates, R., Taylor, P., Ries, K., Martin, R., & van Ess-Dykema, C. (1998) · Language and Speech · doi:10.1177/002383099804100410 · Cited by 307
Self-citationAn Introduction to Hidden Markov Models
Schuster‐Böckler, B., & Bateman, A. (2007) · Current Protocols in Bioinformatics · doi:10.1002/0471250953.bia03as18 · Cited by 188
Automatic detection of discourse structure for speech recognition and understanding
Jurafsky, D., Bates, R., Coccaro, N., Martin, R., Meteer, M., Ries, K., Shriberg, E., Stolcke, A., Taylor, P., & Ess-Dykema, C. V. (1997) · 1997 IEEE Workshop on Automatic Speech Recognition and Understanding Proceedings · doi:10.1109/asru.1997.658992 · Cited by 154
Self-citationDialogue-Games: Metacommunication structures for natural language interaction
Levin, J. A., & Moore, J. A. (1977) · Cognitive Science · doi:10.1207/s15516709cog0104_2 · Cited by 153
Automatic linguistic segmentation of conversational speech
Stolcke, A., & Shriberg, E. E. (2002) · International Conference on Spoken Language Processing (ICSLP) · doi:10.1109/icslp.1996.607773 · Cited by 147
Self-citationJekat, S. J., Klein, A., Maier, E., Maleck, I., Mast, M., & Quantz, J. J. (1995) · Zürcher Hochschule für Angewandte Wissenschaften digital collection (Zurich University of Applied Sciences) · doi:10.22028/d291-25152 · Cited by 138
Integrating Prosodic and Lexical Cues for Automatic Topic Segmentation
Tür, G., Hakkani-Tür, D. Z., Stolcke, A., & Shriberg, E. E. (2001) · Computational Linguistics · doi:10.1162/089120101300346796 · Cited by 135
Self-citationThe power of dialogue dynamics.
Linell, P. (1990) · Cited by 132
Dialogue act classification using language models
Reithinger, N., & Klesen, M. (1997) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.1997-589 · Cited by 126
Prosody in Speech Understanding Systems
Kompe, R. (1997) · Lecture notes in computer science · doi:10.1007/3-540-63580-7 · Cited by 115
The organisation of purposeful dialogues
Power, R. (1979) · Linguistics · doi:10.1515/ling.1979.17.1-2.107 · Cited by 111
Lexical, Prosodic, and Syntactic Cues for Dialog Acts
Jurafsky, D. S., Shriberg, E. E., Fox, B. J., & Curl, T. S. (1998) · Cited by 110
Self-citationFundamental Frequency and Discourse Structure
Menn, L., & Boyce, S. (1982) · Language and Speech · doi:10.1177/002383098202500403 · Cited by 104
Automatic stochastic tagging of natural language texts
Dermatas, E., & Kokkinakis, G. C. (1995) · doi:10.5555/211190.211194 · Cited by 97
Dialogue act tagging with Transformation-Based Learning
Samuel, K., Carberry, S., & Vijay‐Shanker, K. (1998) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/980691.980757 · Cited by 97
Speech recognition using on-line estimation of speaking rate
Morgan, N. H., Fosler, E., & Mirghafori, N. (1997) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.1997-550 · Cited by 87
Automatic grammar induction and parsing free text
Brill, E. (1993) · DARPA Workshop on Speech and Natural Language (HLT) · doi:10.3115/1075671.1075726 · Cited by 71
Nagata, M., & Morimoto, T. (1994) · Speech Communication · doi:10.1016/0167-6393(94)90071-x · Cited by 70
The DCIEM Map Task Corpus: Spontaneous dialogue under sleep deprivation and drug treatment
Bard, E. G., Sotillo, C., Anderson, A., Thompson, H. S., & Taylor, M. M. (1996) · Speech Communication · doi:10.1016/s0167-6393(96)00045-3 · Cited by 60
HMM and neural network based speech act detection
Ries, K. (1999) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1999.758171 · Cited by 52
Self-citationDialogue-Games: Metacommunication structures for natural language interaction
LEVIN, J., & MOORE, J. (1977) · Cognitive Science · doi:10.1016/s0364-0213(77)80016-5 · Cited by 22
Discriminative estimation of interpolation parameters for language model classifiers
Warnke, V., Harbeck, S., Nöth, E., Niemann, H. J., & Levit, M. (1999) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1999.758178 · Cited by 7
Research works printed but not linked: 4
- Carletta Jean, 1996, Computational Linguistics, 22, 249
- Grosz Barbara J, 1986, Computational Linguistics, 12, 175
- Hirschberg Julia B, 1993, Computational Linguistics, 19, 501
- 10.1177/002383099804100411 · doi:10.1177/002383099804100411
Cited by
Inter-Coder Agreement for Computational Linguistics
Artstein, R., & Poesio, M. (2008) · Computational Linguistics · doi:10.1162/coli.07-034-r2 · Cited by 1,665
“Dialogue act tagging is a type of linguistic annotation with which by now the CL community has had extensive experience: Several dialogue-act-annotated spoken language corpora now exist, such as MapTask (Carletta et al. 1997), Switchboard (Stolcke et al. 2000), Verbmobil (Jekat et al. 1995), and Communicator (e.g., Do…”
InfluentialUsing linguistic cues for the automatic recognition of personality in conversation and text
Mairesse, F., Walker, M. A., Mehl, M. R., & Moore, R. K. (2007) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2349 · Cited by 1,046
“This could possibly arise from the small size of the datasets, or from the relatively low accuracy of our hand-crafted automatic tagger, compared to other work using supervised learning methods (Stolcke et al., 2000; Webb et al., 2005).”
Can Large Language Models Transform Computational Social Science?
Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z., & Yang, D. (2023) · arXiv · doi:10.48550/arxiv.2305.03514 · Cited by 839
Zhao, T., Zhao, R., & Eskenazi, M. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1061 · Cited by 736
Summarizing Scientific Articles: Experiments with Relevance and Rhetorical Status
Teufel, S., & Moens, M. (2002) · Computational Linguistics · doi:10.1162/089120102762671936 · Cited by 638
Metadata
- Authors
- Stolcke, A., Ries, K., Coccaro, N., Shriberg, E., Bates, R., Jurafsky, D., Taylor, P., Martin, R., Ess-Dykema, C. V., & Meteer, M.
- Venue
- Computational Linguistics
- Publisher
- MIT Press - Journals
- Published
- 2000-09
- Volume
- 26
- Issue
- 3
- Pages
- 339-373
- ISSN
- 0891-2017 (ISSN-L)
- 1530-9312
- OpenAlex
- W2128970689
- Cited by
- 1,108
- References
- 58
- FWCI
- 21.24
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.