CoChat
Sign Up
Cited by 25Language Development and DisordersSpeech Recognition and SynthesisVoice and Speech Disorders

The DOI matches its Crossref record; the references it cites are not checked yet.

An Evaluation of Expedited Transcription Methods for School-Age Children's Narrative Language: Automatic Speech Recognition and Real-Time Transcription

Open access
Fox, Carly B., Israelsen-Augenstein, Megan, Jones, Sharad, Gillam, Sandra Laing·2021·Journal of Speech, Language, and Hearing Research
DOI 10.1044/2021_jslhr-21-00096
Open this paper in CoChat — free account

Abstract

Purpose: This study examined the accuracy and potential clinical utility of two expedited transcription methods for narrative language samples elicited from school-age children (7;5–11;10 [years;months]) with developmental language disorder. Transcription methods included real-time transcription produced by speech-language pathologists (SLPs) and trained transcribers (TTs) as well as Google Cloud Speech automatic speech recognition. Method: The accuracy of each transcription method was evaluated against a gold-standard reference corpus. Clinical utility was examined by determining the reliability of scores calculated from the transcripts produced by each method on several language sample analysis (LSA) measures. Participants included seven certified SLPs and seven TTs. Each participant was asked to produce a set of six transcripts in real time, out of a total 42 language samples. The same 42 samples were transcribed using Google Cloud Speech. Transcription accuracy was evaluated through word error rate. Reliability of LSA scores was determined using correlation analysis. Results: Results indicated that Google Cloud Speech was significantly more accurate than real-time transcription in transcribing narrative samples and was not impacted by speech rate of the narrator. In contrast, SLP and TT transcription accuracy decreased as a function of increasing speech rate. LSA metrics generated from Google Cloud Speech transcripts were also more reliably calculated. Conclusions: Automatic speech recognition showed greater accuracy and clinical utility as an expedited transcription method than real-time transcription. Though there is room for improvement in the accuracy of speech recognition for the purpose of clinical transcription, it produced highly reliable scores on several commonly used LSA metrics. Supplemental Material: https://doi.org/10.23641/asha.15167355

Overview

Integrity signals

Registry

Crossref · 10.1044/2021_jslhr-21-00096

Cites

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

The citation lists haven’t been fetched yet

References 69 · Cited by 25

Research works printed but not linked: 67

  1. 10.1044/2016_LSHSS-15-0090 · DOI
  2. 10.1016/j.jcomdis.2010.05.002 · DOI · Paper page
  3. 10.1044/2017_JSLHR-L-17-0152 · DOI
  4. 10.1044/2017_AJSLP-17-0046 · DOI
  5. 10.1044/2020_AJSLP-19-00176 · DOI
  6. Cicchetti, D. V. , & Sparrow, S. A. (1981). Developing criteria for establishing interrater reliability of specific items: Applications to assessment of adaptive behavior. American Journal of Mental Deficiency, 86(2), 127–137.
  7. 10.1044/2020_JSLHR-19-00017 · DOI
  8. 10.1044/2016_LSHSS-16-0007 · DOI
  9. 10.1044/2014_LSHSS-14-0034 · DOI
  10. 10.3389/fpsyg.2017.02265 · DOI
  11. 10.1044/2019_LSHSS-19-00032 · DOI
  12. 10.1044/1092-4388(2002/008) · DOI
  13. 10.1044/2018_AJSLP-16-0121 · DOI
  14. Gillam, R. B. , & Pearson, N. A. (2004). TNL: Test of Narrative Language. Pro-Ed.
  15. Gillam, S. L. , & Justice, L. (2010). Progress Monitoring Tools for SLPs in Response to Intervention (RTI): Primary grades. The ASHA Leader, 9/ 21 issue: Feature.
  16. 10.1044/2017_LSHSS-17-0047 · DOI
  17. 10.1371/journal.pone.0242511 · DOI
  18. 10.1044/1092-4388(2007/068) · DOI
  19. 10.1044/persp3.SIG1.170 · DOI
  20. Han W. Zhang Z. Zhang Y. Yu J. Chiu C. C. Qin J. Gulati A. Pang R. & Wu Y. (2020). ContextNet: Improving convolutional neural networks for automatic speech recognition with global context. arXiv preprint arXiv:2005.03191. · DOI
  21. 10.1016/j.jbi.2008.08.010 · DOI · Paper page
  22. 10.1044/lle17.1.4 · DOI
  23. 10.1044/0161-1461(2009/09-0023) · DOI
  24. 10.1044/2020_PERSP-19-00079 · DOI
  25. 10.1016/j.jcomdis.2004.10.002 · DOI
  26. 10.4324/9781315650982 · DOI
  27. Justice, L. , Gillam, S. , & McGinty, A. (2010). Teaching students with language disorders. In R. McWilliam , M. Tankersley , & B. Cook (Eds.), Effective practices in special education. Pearson.
  28. 10.1177/026565909701300204 · DOI
  29. 10.1016/S0095-4470(19)31460-3 · DOI
  30. 10.1177/026565909100700102 · DOI
  31. 10.1044/0161-1461(2003/005) · DOI
  32. Leadholm, B. J. , & Miller, J. F. (1994). Language sample analysis: The Wisconsin guide (Bulletin 92424). ERIC.
  33. Lileikyte R. Irvin D. & Hansen J. H. (2020). Assessing child communication engagement via speech recognition in naturalistic active learning spaces. In Proceedings of Odyssey 2020: The speaker and language recognition workshop (pp. 396–401). https://doi.org/10.21437/Odyssey.2020-56 · DOI
  34. MacWhinney, B. (2000). CLAN [Computer software] . Carnegie Mellon University.
  35. 10.1044/2020_LSHSS-19-00120 · DOI
  36. 10.1177/0265659010369991 · DOI
  37. 10.1044/2015_LSHSS-15-0051 · DOI
  38. 10.1097/00011363-199202000-00008 · DOI
  39. Miller, J. F. , & Iglesias, A. (2010). Systematic Analysis of Language Transcripts (SALT), research version [Computer Software] . SALT Software.
  40. 10.1044/2018_JSLHR-L-17-0421 · DOI
  41. 10.1111/j.2041-210x.2012.00261.x · DOI · Paper page
  42. 10.1044/1092-4388(2013/13-0097) · DOI
  43. 10.1080/136820310000108133 · DOI
  44. Park, Y. , Patwardhan, S. , Visweswariah, K. , & Gates, S. C. (2008). An empirical analysis of word error rate and keyword error rate. In 9th Annual Conference of the International Speech Communication Association. ISCA Archive.
  45. 10.1177/0142723705059114 · DOI
  46. Paul, R. (2007). Language disorders from infancy through adolescence: Assessment & intervention (Vol. 324). Elsevier Health Sciences.
  47. 10.1044/2017_LSHSS-17-0022 · DOI
  48. 10.1044/2016_LSHSS-15-0044 · DOI · Paper page
  49. 10.1044/2019_LSHSS-18-0148 · DOI
  50. 10.1109/TSA.2003.818026 · DOI
  51. 10.1044/1092-4388(2006/056) · DOI
  52. Rodrigues, A. , Santos, R. , Abreu, J. , Beça, P. , Almeida, P. , & Fernandes, S. (2019). Analyzing the performance of ASR systems: The effects of noise, distance to the device, age and gender. In Proceedings of the XX International Conference on Human Computer Interaction (pp. 1–8). ACM Digital Library.
  53. 10.1044/leader.FTR1.14032009.10 · DOI
  54. Schneider, W. , Eschman, A. , & Zuccolotto, A. (2002). E-Prime: User's guide. Reference guide. Getting started guide. Psychology Software Tools.
  55. 10.1044/jslhr.4302.324 · DOI
  56. Semel, E. , Wiig, E. , & Secord, W. (2003). Clinical Evaluation of Language Fundamentals–Fourth Edition. The Psychological Corporation.
  57. Shahnawazuddin, S. , Bandarupalli, T. S. , & Chakravarthy, R. (2020). Improving automatic speech recognition by classifying adult and child speakers into separate groups using speech rate rhythmicity parameter. In 2020 International Conference on Signal Processing and Communications (SPCOM) (pp. 1–5). IEEE. https://doi.org/10.1109/SPCOM50965.2020.9179497 · DOI
  58. 10.1044/2019_JSLHR-19-00145 · DOI
  59. 10.1177/014272379801805201 · DOI
  60. 10.1177/1525740108314866 · DOI
  61. 10.1044/jshr.3906.1284 · DOI
  62. Van Rossum, G. , & Drake, F. L. (2009). Python 3 Reference Manual. CreateSpace.
  63. Wallace, G. , & Hammill, D. (1994). Comprehensive Receptive and Expressive Vocabulary Test–Second Edition. Pro-Ed.
  64. 10.3109/17549507.2013.871336 · DOI · Paper page
  65. 10.1080/14417040400010140 · DOI · Paper page
  66. 10.1080/13682820601056228 · DOI
  67. Yeung, G. , & Alwan, A. (2018). On the difficulties of automatic speech recognition for kindergarten-aged children. In Interspeech (pp. 1661–1665). NSF Public Access Repository. · DOI

Metadata

Authors
Fox, C. B., Israelsen-Augenstein, M., Jones, S., & Gillam, S. L.
Affiliations
Utah State University (US)
ORCID
Venue
Journal of Speech, Language, and Hearing Research
Publisher
American Speech Language Hearing Association
Published
2021-09-14
Volume
64
Issue
9
Pages
3533-3548
ISSN
  • 1092-4388 (ISSN-L)
  • 1558-9102
DOI
10.1044/2021_jslhr-21-00096
OpenAlex
W3194191636
Cited by
25
References
69
Citations by year
2026: 8 · 2025: 3 · 2024: 6 · 2023: 3 · 2022: 5
FWCI
5.13
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Speech Language and Hearing Research (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Psychology › Developmental and Educational Psychology › Language Development and Disorders
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
  • Health Sciences › Health Professions › Speech and Hearing › Voice and Speech Disorders
Keywords
speech recognition, real-time transcription, narrative language, elementary school students, specific language impairment, language sample analysis, speech rate, transcription accuracy, word error rate, speech-language pathologists
MeSH terms
Child, Humans, Schools, Speech, Speech-Language Pathology, Speech Perception, Reproducibility of Results

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review