Effective use of prosody in parsing conversational speech
Open accessAbstract
We identify a set of prosodic cues for parsing conversational speech and show how such features can be effectively incorporated into a statistical parsing model. On the Switchboard corpus of conversational speech, the system achieves improved parse accuracy over a state-of-the-art system which uses only lexical and syntactic features. Since removal of edit regions is known to improve downstream parse accuracy, we explore alternatives for edit detection and show that PCFGs are not competitive with more specialized techniques.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
After conversion from speech to text, NLP techniques that perform well on grammatically correct sentences break down with conversational speech because of the lack of punctuation and sentence boundaries, grammatical differences between spoken and written language, and lack of structure.
“After conversion from speech to text, NLP techniques that perform well on grammatically correct sentences break down with conversational speech because of the lack of punctuation and sentence boundaries, grammatical differences between spoken and written language, and lack of structure.7,24,25”
From Challenges of developing a digital scribe to reduce clinical documentation burden (2019) · cites this paper as [24]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Coarse-to-fine n -best parsing and MaxEnt discriminative reranking
Eugene Charniak, D., & Johnson, M. (2005) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1219840.1219862 · Cited by 878 · see citing works
Self-citationDiscriminative Reranking for Natural Language Parsing
Collins, M., & Koo, T. (2005) · Computational Linguistics · doi:10.1162/0891201053630273 · Cited by 702 · see citing works
The use of prosody in syntactic disambiguation
Price, P. J., Mari Ostendorf, E., Shattuck‐Hufnagel, S., & Fong, C. (1991) · The Journal of the Acoustical Society of America · doi:10.1121/1.401770 · Cited by 648 · see citing works
Evaluation of prosodic transcription labeling reliability in the tobi framework
Pitrelli, J. F., Beckman, M. E., & Hirschberg, J. (1994) · International Conference on Spoken Language Processing (ICSLP) · doi:10.21437/icslp.1994-34 · Cited by 287 · see citing works
Heeman, P. A., & Allen, J. F. (1999) · Cited by 187 · see citing works
Edit detection and parsing for transcribed speech
Eugene Charniak, D., & Johnson, M. (2001) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/1073336.1073352 · Cited by 160 · see citing works
Self-citationDeterministic parsing of syntactic non-fluencies
Hindle, D. M. (1983) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/981311.981336 · Cited by 134 · see citing works
On the parameter space of generative lexicalized statistical parsing models
Bikel, D. M., & Marcus, M. P. (2004) · ScholarlyCommons (University of Pennsylvania) · Cited by 126 · see citing works
VERBMOBIL: the use of prosody in the linguistic components of a speech understanding system
Nöth, E., Batliner, A. M., Kiessling, A., Kompe, R., & Niemann, H. J. (2000) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.861370 · Cited by 91 · see citing works
Prosody Modeling for Automatic Speech Recognition and Understanding
Shriberg, E. E., & Stolcke, A. (2004) · The IMA volumes in mathematics and its applications · doi:10.1007/978-1-4419-9017-4_5 · Cited by 82 · see citing works
Use of prosody in syntactic disambiguation
Wightman, C. W., Veilleuz, N. M., & Ostendorf, M. (1991) · DARPA Workshop on Speech and Natural Language (HLT) · doi:10.3115/112405.112740 · Cited by 49 · see citing works
Self-citationMathematical Foundations of Speech and Language Processing
Johnson, M., Khudanpur, S., Mari Ostendorf, E., & Rosenfeld, R. (2004) · The IMA volumes in mathematics and its applications · doi:10.1007/978-1-4419-9017-4 · Cited by 47 · see citing works
Self-citationA syntactic framework for speech repairs and other disruptions
Core, M. G., & Schubert, L. K. (1999) · doi:10.3115/1034678.1034742 · Cited by 33 · see citing works
Bear, J., & Price, P. J. (1990) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.3115/981823.981826 · Cited by 33 · see citing works
Parsing conversational speech using enhanced segmentation
Kahn, J. G., Ostendorf, M., & Chelba, C. I. (2004) · doi:10.3115/1613984.1614016 · Cited by 28 · see citing works
Self-citationMany Uses, Many Annotations for Large Speech Corpora: Switchboard and TDT as Case Studies
Graff, D., & Bird, S. (2000) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/56jpw9c58a2n · Cited by 25 · see citing works
An improved model for recognizing disfluencies in conversational speech
Johnson, M. S., Charniak, E., & Lease, M. (2004) · Cited by 24 · see citing works
Self-citationSentence-Internal Prosody Does not Help Parsing the Way Punctuation Does
Gregory, M., Johnson, M. S., & Charniak, E. (2004) · Cited by 23 · see citing works
Self-citationThe ICSI-SRI-UW metadata extraction system
Shriberg, E. E., Stolcke, A., Hillard, D., Mari Ostendorf, E., Peskin, B., Harper, M. P., & Liu, Y. (2004) · Interspeech (USB) · doi:10.21437/interspeech.2004-222 · Cited by 23 · see citing works
Many uses, many annotations for large speech corpora: Switchboard and TDT as case studiesPreprint
Graff, D., & Bird, S. (2000) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0007024 · Cited by 22 · see citing works
Parsing and disfluency placement
Engel, D., Eugene Charniak, D., & Johnson, M. (2002) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1118693.1118700 · Cited by 11 · see citing works
Self-citationParsing Conversational Speech Using Enhanced Segmentation
Kahn, J. G., Ostendorf, M., & Chelba, C. I. (2004) · doi:10.21236/ada457886 · Cited by 9 · see citing works
Self-citationProsody, empty categories and parsing - a success story
Batliner, A. M., Holler, A., Geißler, S., Kiss, T., Kompe, R., & Nöth, E. (1996) · doi:10.22028/d291-25312 · Cited by 7 · see citing works
Prosody, empty categories and parsing - a success story
Batliner, A. M., Holler, A., Geissler, S., Kiss, T., Kompe, R., & Nöth, E. (1996) · International Conference on Spoken Language Processing (ICSLP) · doi:10.21437/icslp.1996-267 · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- M. Johnson, E. Charniak, and M. Lease. 2004. An im- proved model for recognizing disfluencies in conver- sational speech. Proc. Rich Transcription Workshop.
- J. G. Kahn, M. Ostendorf, and C. Chelba. 2004. Pars- ing conversational speech using enhanced segmenta- tion. Proc. HLT-NAACL 2004, pp. 125-128.
- M.A. thesis, U. Wash. LDC. 2004. Simple metadata annotation specification. Tech. report, Linguistic Data Consortium. Available at http://www.ldc.upenn.edu/Projects/MDE.
- M. Ostendorf et al. 2001. A prosodically labeled database of spontaneous speech. ISCA Workshop on Prosody in Speech Recognition and Understanding, pp. 119-121, 10.
Cited by
Sorted by citations
Challenges of developing a digital scribe to reduce clinical documentation burden
Quiroz, J. C., Laranjo, L., Kocaballi, A. B., Berkovsky, S., Rezazadegan, D., & Coiera, E. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0190-1 · Cited by 162 · see citing works
Sridhar, V. K. R., Bangalore, S., & Narayanan, S. S. (2008) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2008.917071 · Cited by 90 · see citing works
Punctuation: Making a Point in Unsupervised Dependency Parsing
Spitkovsky, V. I., Alshawi, H., & Jurafsky, D. S. (2011) · Cited by 52 · see citing works
Disfluency Detection with a Semi-Markov Model and Prosodic Features
Ferguson, J., Durrett, G., & Klein, D. (2015) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/v1/n15-1029 · Cited by 51 · see citing works
Joint Parsing and Disfluency Detection in Linear Time
Rasooli, M. S., & Tetreault, J. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1013 · Cited by 46 · see citing works
Parsing Speech: a Neural Approach to Integrating Lexical and Acoustic-Prosodic Information
Tran, T., Toshniwal, S., Bansal, M., Gimpel, K., Livescu, K., & Mari Ostendorf, E. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1007 · Cited by 46 · see citing works
A dataset of simulated patient-physician medical interviews with a focus on respiratory cases
Fareez, F., Parikh, T., Wavell, C., Shahab, S., Chevalier, M., Good, S., De Blasi, I., Rhouma, R., McMahon, C., Lam, J.-P., et al. (2022) · Scientific Data · doi:10.1038/s41597-022-01423-1 · Cited by 39 · see citing works
Improving Disfluency Detection by Self-Training a Self-Attentive Model
Lou, P. J., & Johnson, M. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.346 · Cited by 39 · see citing works
Self-citationDisfluency Detection Using Multi-step Stacked Learning
Qian, X., & Liu, Y. (2013) · Cited by 38 · see citing works
Disfluency Detection using Auto-Correlational Neural Networks
Lou, P. J., Anderson, P., & Johnson, M. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1490 · Cited by 36 · see citing works
Self-citationThe impact of language models and loss functions on repair disfluency detection
Zwarts, S., & Johnson, M. (2011) · Cited by 31 · see citing works
Self-citationMulti-domain disfluency and repair detection
Zayats, V., Mari Ostendorf, E., & Hajishirzi, H. (2014) · Interspeech (USB) · doi:10.21437/interspeech.2014-603 · Cited by 27 · see citing works
Early deletion of fillers in processing conversational speech
Lease, M., & Johnson, M. (2006) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/1614049.1614068 · Cited by 25 · see citing works
Self-citationNeural Constituency Parsing of Speech Transcripts
Jamshid Lou, P., Wang, Y., & Johnson, M. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1282 · Cited by 22 · see citing works
Self-citationUnsupervised Dependency Parsing with Acoustic Cues
Pate, J. K., & Goldwater, S. J. (2013) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00210 · Cited by 20 · see citing works
Efficient Disfluency Detection with Transition-based Parsing
Wu, S., Zhang, D., Zhou, M. Q., & Zhao, T. (2015) · doi:10.3115/v1/p15-1048 · Cited by 19 · see citing works
PCFGs with syntactic and prosodic indicators of speech repairs
Hale, J., Snover, M., Stewart, R., Izhak Shafran, J., Lisa Yung, J., Dorr, B. J., Harper, M. P., Anna Krasnyanskaya, I., Lease, M., Liu, Y., et al. (2006) · International Conference on Computational Linguistics · doi:10.3115/1220175.1220196 · Cited by 19 · see citing works
Self-citationUnsupervised Syntactic Chunking with Acoustic Cues: Computational Models for Prosodic Bootstrapping
Pate, J. K., & Goldwater, S. J. (2011) · Cited by 17 · see citing works
Farzana, S., Deshpande, A., & Parde, N. (2022) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2022.bionlp-1.4 · Cited by 16 · see citing works
Exploiting prosody for PCFGs with latent annotations
Dreyer, M., & Izhak Shafran, J. (2007) · Interspeech (USB) · doi:10.21437/interspeech.2007-216 · Cited by 13 · see citing works
A progressive feature selection algorithm for ultra large feature spaces
Zhang, Q., Weng, F., & Feng, Z. (2006) · International Conference on Computational Linguistics · doi:10.3115/1220175.1220246 · Cited by 12 · see citing works
An Empirical Analysis of the Correlation of Syntax and Prosody
Köhn, A., Baumann, T., & Dörfler, O. (2018) · Interspeech (USB) · doi:10.21437/interspeech.2018-2530 · Cited by 12 · see citing works
Joint reranking of parsing and word recognition with automatic segmentation
Kahn, J. G., & Mari Ostendorf, E. (2011) · Computer Speech & Language · doi:10.1016/j.csl.2011.03.002 · Cited by 10 · see citing works
Self-citationAppropriately handled prosodic breaks help PCFG parsing
Huang, Z., & Harper, M. P. (2010) · Cited by 10 · see citing works
Gesture Salience as a Hidden Variable for Coreference Resolution and Keyframe Extraction
Eisenstein, J., Barzilay, R., & Davis, R. (2008) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2450 · Cited by 10 · see citing works
Caines, A., McCarthy, M., & Buttery, P. J. (2017) · doi:10.18653/v1/w17-4604 · Cited by 9 · see citing works
Lessons Learned in Part-of-Speech Tagging of Conversational Speech
Eidelman, V., Huang, Z., & Harper, M. P. (2011) · Cited by 9 · see citing works
Reconstructing spontaneous speech
Jelinek, F., & Fitzgerald, E. (2009) · Cited by 9 · see citing works
(In)variability in the Samoan syntax/prosody interface and consequences for syntactic parsing
Yu, K. M., & Stabler, E. P. (2017) · Laboratory Phonology Journal of the Association for Laboratory Phonology · doi:10.5334/labphon.113 · Cited by 9 · see citing works
Gesture in automatic discourse processing
Barzilay, R., Davis, R., & Eisenstein, J. (2008) · Cited by 9 · see citing works
Exploring features for identifying edited regions in disfluent sentences
Zhang, Q., & Weng, F. (2005) · International Workshop/Conference on Parsing Technologies · doi:10.3115/1654494.1654513 · Cited by 8 · see citing works
Disfluency Detection using Auto-Correlational Neural NetworksPreprint
Lou, P. J., Anderson, P., & Johnson, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.09092 · Cited by 7 · see citing works
Self-citationAudio and ASR-based Filled Pause Detection
Chatziagapi, A., Sgouropoulos, D., Karouzos, C., Μελίστας, Θ., Γιαννακόπουλος, Θ., Katsamanis, A., & Narayanan, S. S. (2022) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii55700.2022.9953889 · Cited by 6 · see citing works
Dobrovoljc, K., & Martinc, M. (2018) · Workshop on Universal Dependencies (UDW) · doi:10.18653/v1/w18-6005 · Cited by 6 · see citing works
Automatic sentence structure annotation for spoken language processing
Mari Ostendorf, E., & Hillard, D. (2008) · Cited by 6 · see citing works
A Beam-Search Decoder for Disfluency Detection
Wang, X., Ng, H. T., & Sim, K. C. (2014) · Cited by 5 · see citing works
Proceedings of the Second Workshop on Universal Dependencies (UDW 2018)
de Marneffe, M.-C., Lynn, T., Schuster, S., & 2018), P. O. T. S. W. O. U. D. (. (2018) · doi:10.18653/v1/w18-60 · Cited by 5 · see citing works
Computing confidence scores for all sub parse trees
Lin, F., & Weng, F. (2008) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1557690.1557752 · Cited by 4 · see citing works
Enriching ASR Lattices with POS Tags for Dependency Parsing
Stiefel, M., & Vu, N. T. (2017) · doi:10.18653/v1/w17-4605 · Cited by 4 · see citing works
Dammalapati, S., Rajkumar, R. P., & Agarwal, S. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop (NAACL SRW) · doi:10.18653/v1/n19-3015 · Cited by 4 · see citing works
Neural Constituency Parsing of Speech TranscriptsPreprint
Lou, P. J., Wang, Y., & Johnson, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.08535 · Cited by 4 · see citing works
Self-citationIntegrating sentence- and word-level error identification for disfluency correction
Fitzgerald, E., Jelinek, F., & Hall, K. (2009) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1699571.1699612 · Cited by 3 · see citing works
Assessing the Use of Prosody in Constituency Parsing of Imperfect Transcripts
Tran, T., & Mari Ostendorf, E. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-373 · Cited by 2 · see citing works
Unediting: Detecting Disfluencies Without Careful Transcripts
Zayats, V., Mari Ostendorf, E., & Hajishirzi, H. (2015) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/v1/n15-1161 · Cited by 2 · see citing works
Kramov, A. A., & Pogorilyy, S. D. (2023) · NaUKMA Research Papers Computer Science · doi:10.18523/2617-3808.2022.5.54-61 · Cited by 1 · see citing works
Prosodic Disambiguation in Spoken Systems Output
Moubayed, S. A. (2009) · Cited by 1 · see citing works
Cascading and Direct Approaches to Unsupervised Constituency Parsing on Spoken Sentences
Tseng, Y., Lai, C.-I., & Lee, H.-Y. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10094575 · Cited by 1 · see citing works
Prosodic segmentation for parsing spoken dialogue
Nielsen, E., Steedman, M. J., & Goldwater, S. J. (2021) · doi:10.18653/v1/2021.acl-long.79
Parsing dialog turns with prosodic features in English
Nielsen, E., Steedman, M. J., & Goldwater, S. J. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-952
Widiaputri, R. F., Purwarianti, A., & Sakti, S. (2025) · IEEE Access · doi:10.1109/access.2025.3648059
Predictability effects in language acquisition
Pate, J. K. (2013) · ERA
- Authors
- Kahn, J. G., Lease, M., Charniak, E., Johnson, M., & Ostendorf, M.
- Affiliations
- University of Washington (US)
- Brown University (US)
- Venue
- Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing - HLT '05
- Published
- 2005
- Pages
- 233-240
- OpenAlex
- W2067030274
- Cited by
- 53
- References
- 23
- Citations by year
- 2025: 1 · 2023: 3 · 2022: 3 · 2021: 2 · 2020: 1 · 2019: 4 · 2018: 6 · 2017: 4 · 2015: 3 · 2014: 2 · 2013: 4
- FWCI
- 2.62
- Type
- conference paper
- Language
- en
- Open access
- gold
- doi.org
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- prosody, prosodic cues, statistical parsing, Switchboard corpus, probabilistic context-free grammars, spoken language understanding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
