Clinical Annotations for Automatic Stuttering Severity Assessment
Open accessAbstract
Stuttering is a complex disorder that requires specialized expertise for effective assessment and treatment. This paper presents an effort to enhance the FluencyBank dataset with a new stuttering annotation scheme based on established clinical standards. To achieve high-quality annotations, we hired expert clinicians to label the data, ensuring that the resulting annotations mirror real-world clinical expertise. The annotations are multi-modal, incorporating audiovisual features for the detection and classification of stuttering moments, secondary behaviors, and tension scores. In addition to individual annotations, we additionally provide a test set with highly reliable annotations based on expert consensus for assessing individual annotators and machine learning models. Our experiments and analysis illustrate the complexity of this task that necessitates extensive clinical expertise for valid training and evaluation of stuttering assessment models.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Annotating and transcribing stuttered speech remains difficult, even for trained professionals.
“Given the variability of stuttering and differing views on how it manifests, annotating and transcribing stuttered speech remains difficult, even for trained professionals.”
From Disability-First AI Dataset Annotation: Co-designing Stuttered Speech Annotation Guidelines with People Who Stutter (2026) · cites this paper as [102]
Section 4.3 (p. 3) says fine-grained stuttering annotation varies even among expert speech-language pathologists and describes substantial disagreements they had to resolve. Section 4.4 (p. 3) also reports that annotators found the task subjective and time-consuming. The paper does not establish that transcribing stuttered speech was difficult: Section 3.1 (p. 2) says the annotators used existing transcriptions. (LLM, conf=0.87)
Experimental — accuracy not yet measuredAnnotating stuttered speech is inherently challenging and subjective due to its high variability and the ambiguous boundary between stuttered and fluent speech.
“While annotating stuttered speech is inherently challenging and subjective due to its high variability and the ambiguous boundary between stuttered and fluent speech [99, 102]”
From Disability-First AI Dataset Annotation: Co-designing Stuttered Speech Annotation Guidelines with People Who Stutter (2026) · cites this paper as [102]
The Introduction (p. 1) says stuttering is variable and that its assessment is challenging. The Test Set section (p. 3) reports substantial annotation disagreements even among expert clinicians, supporting the subjectivity of the task. The Annotation Guidelines (p. 2) define where stuttering begins and fluent speech resumes, but do not say that the boundary itself is ambiguous. (LLM, conf=0.84)
Experimental — accuracy not yet measuredAnnotating stuttered speech is highly subjective even for SLP professionals.
“Annotating stuttered speech is found to be highly subjective task even for SLP professionals [102].”
From Disability-First AI Dataset Annotation: Co-designing Stuttered Speech Annotation Guidelines with People Who Stutter (2026) · cites this paper as [102]
People who stutter are rarely involved in addressing the complexities of stuttered speech annotation.
“people who stutter (PWS) are rarely involved to address such complexities [6, 58, 83, 102].”
From Disability-First AI Dataset Annotation: Co-designing Stuttered Speech Annotation Guidelines with People Who Stutter (2026) · cites this paper as [102]
The Annotation Process and Guidelines (p. 2) identifies three speech-language pathologists as annotators, and the Introduction (p. 1) discusses the complexity of stuttering annotation. Neither passage says whether people who stutter participated as annotators or establishes that their involvement is rare; searches for their participation found no such statement. (LLM, conf=0.72)
Experimental — accuracy not yet measuredOur perspectives may be skewed toward people with higher socio-economic backgrounds and greater technical literacy, and under-represent those with multiple disabilities.
“Despite our efforts in engaging multiple perspectives and community voices, our perspectives may be skewed toward people with higher socio-economic backgrounds and greater technical literacy, and under-represent those with multiple disabilities [102].”
From Disability-First AI Dataset Annotation: Co-designing Stuttered Speech Annotation Guidelines with People Who Stutter (2026) · cites this paper as [102]
Section 4.4 (p. 3) discusses non-native English speakers and possible comorbidities in speech samples, but not the socioeconomic background or technical literacy of the people whose perspectives were gathered. Searches for those terms and for under-representation of people with multiple disabilities found no supporting passage in our copy. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 9
- [2][2] O. Bloodstein, N. B. Ratner, and S. B. Brundage, A handbook on stuttering . Plural Publishing, 2021, vol. 1. -
- [5][5] B. Guitar, Stuttering: An integrated approach to its nature and treatment . Lippincott Williams & Wilkins, 2019. -
- [12][12] A. R. dos Santos Valente, 'Development of assessment tools to evaluate adults with fluency disorders,' Ph.D. dissertation, Universidade de Aveiro (Portugal), 2018. -
- [17][17] R. Riad, A.-C. Bachoud-L´ evi, F. Rudzicz, and E. Dupoux, 'Identification of primary and collateral tracks in stuttered speech,' in Proceedings of the Twelfth Language Resources and Evaluation Conference , May 2020. [Online]. Available: https://aclanthology.org/2020.lrec-1.208 -
- [18][18] P. Howell, S. Davis, and J. Bartrip, 'The uclass archive of stuttered speech.' Journal of speech, language, and hearing research : JSLHR , 03 2009. -
- [21][21] R. B. Gillam, K. J. Logan, and N. A. Pearson, TOCS: Test of childhood stuttering . Pro-Ed Austin, 2009. -
- [23][23] A. Packman, M. Onslow, and K. Bryant, 'The lexicon of stuttering,' in Proceedings of the Fifth Oxford Dysfluency Conference . KL Baker Leicester, UK, 2000, pp. 53-60. -
- [28][28] A. Braylan, M. Marabella, O. Alonso, and M. Lease, 'A general model for aggregating annotations across simple, complex, and multi-object annotation tasks,' vol. 78, pp. 901-973. [Online]. Available: https://jair.org/index.php/jair/article/view/14388 -
- [30][30] A. Baevski, Y. Zhou, A. Mohamed, and M. Auli, 'wav2vec 2.0: A framework for self-supervised learning of speech representations,' Advances in neural information processing systems , vol. 33, pp. 12 449-12 460, 2020. -
Cited by
Sorted by citations
Tang, X., Li, J., & Wu, S. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790405 · Cited by 4 · see citing works
- Authors
- Valente, A., Marew, R., Toyin, H., Al-Ali, H., Bohnen, A., Becerra, I., Soares, E., Leal, G., & Aldarmaki, H.
- Venue
- Interspeech 2025
- Publisher
- ISCA
- Published
- 2025-08-17
- Pages
- 4318-4322
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W4415433022
- Cited by
- 0
- References
- 0
- FWCI
- 0.00
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Interspeech (USB) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Stuttering Research and Treatment
- Health Sciences › Medicine › Physiology › Voice and Speech Disorders
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- stuttering, clinical annotation, speech disorders, severity rating
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
