The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Perceptual Evaluation of Voice Quality: Review, Tutorial, and a Framework for Future Research
Abstract
The reliability of listeners’ ratings of voice quality is a central issue in voice research because of the clinical primacy of such ratings and because they are the standard against which other measures are evaluated. However, an extensive literature review indicates that both intrarater and interrater reliability fluctuate greatly from study to study. Further, our own data indicate that ratings of vocal roughness vary widely across individual clinicians, with a single voice often receiving nearly the full range of possible ratings. No model or theoretical framework currently exists to explain these variations, although such a model might guide development of efficient, valid, and standardized clinical protocols for voice evaluation. We propose a theoretical framework that attributes variability in ratings to several sources (including listeners’ backgrounds and biases, the task used to gather ratings, interactions between listeners and tasks, and random error). This framework may guide development of new clinical voice and speech evaluation protocols, ultimately leading to more reliable perceptual ratings and a better understanding of the perceptual qualities of pathological voices.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 26,755
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
Analysis, synthesis, and perception of voice quality variations among female and male talkers
Klatt, D. H., & Klatt, L. C. (1990) · The Journal of the Acoustical Society of America · doi:10.1121/1.398894 · Cited by 1,691
平野, 実. (1981) · Springer eBooks · Cited by 1,664
Clinical measurement of speech and voice
Baken, R. J. (1987) · Cited by 1,465
Differential diagnostic patterns of dysarthria
Darley, F. L., Aronson, A. E., & Brown, J. R. (1969) · Journal of Speech and Hearing Research · doi:10.1044/jshr.1202.246 · Cited by 1,323
Estimation of the reliability of ratings.
Ebel, R. L. (1951) · Psychometrika · doi:10.1007/bf02288803 · Cited by 976
Interrater reliability and agreement of subjective judgments.
Tinsley, H. E., & Weiss, D. J. (1975) · Journal of Counseling Psychology · doi:10.1037/h0076640 · Cited by 852
Speech and Language: Advances in Basic Research and Practice
Lass, N. J. (1979) · Medical Entomology and Zoology · Cited by 621
Cited by
Validation of an instrument to measure voice-related quality of life (V-RQOL)
Hogikyan, N. D., & Sethuraman, G. (1999) · Journal of Voice · doi:10.1016/s0892-1997(99)80010-1 · Cited by 1,143
Consensus Auditory-Perceptual Evaluation of Voice: Development of a Standardized Clinical Protocol
Kempster, G. B., Gerratt, B. R., Verdolini Abbott, K., Barkmeier-Kraemer, J., & Hillman, R. E. (2009) · American Journal of Speech-Language Pathology · doi:10.1044/1058-0360(2008/08-0017 · Cited by 1,008
Self-citationWuyts, F. L., Bodt, M. S. D., Molenberghs, G., Remacle, M., Heylen, L., Millet, B., Lierde, K. V., Raes, J., & Heyning, P. H. V. D. (2000) · Journal of Speech, Language, and Hearing Research · doi:10.1044/jslhr.4303.796 · Cited by 578
Kent, R. D. (1996) · American Journal of Speech-Language Pathology · doi:10.1044/1058-0360.0503.07 · Cited by 432
De Bodt, M. S., Wuyts, F. L., Van de Heyning, P. H., & Croux, C. (1997) · Journal of Voice · doi:10.1016/s0892-1997(97)80026-4 · Cited by 432
Maryn, Y., Corthals, P., Van Cauwenberge, P., Roy, N., & De Bodt, M. (2010) · Journal of Voice · doi:10.1016/j.jvoice.2008.12.014 · Cited by 427
How Do You Say ‘Hello’? Personality Impressions from Brief Novel Voices
McAleer, P., Todorov, A., & Belin, P. (2014) · PLoS ONE · doi:10.1371/journal.pone.0090779 · Cited by 388
Acoustic measurement of overall voice quality: A meta-analysis
Maryn, Y., Roy, N., De Bodt, M., Van Cauwenberge, P., & Corthals, P. (2009) · The Journal of the Acoustical Society of America · doi:10.1121/1.3224706 · Cited by 385
Karnell, M. P., Melton, S. D., Childes, J. M., Coleman, T. C., Dailey, S. A., & Hoffman, H. T. (2007) · Journal of Voice · doi:10.1016/j.jvoice.2006.05.001 · Cited by 353
John, A., Sell, D., Sweeney, T., Harding-Bell, A., & Williams, A. (2006) · The Cleft Palate Craniofacial Journal · doi:10.1597/04-141.1 · Cited by 319
Establishing Validity of the Consensus Auditory-Perceptual Evaluation of Voice (CAPE-V)
Zraick, R. I., Kempster, G. B., Connor, N. P., Thibeault, S., Klaben, B. K., Bursac, Z., Thrush, C. R., & Glaze, L. E. (2011) · American Journal of Speech-Language Pathology · doi:10.1044/1058-0360(2010/09-0105 · Cited by 300
Self-citationAuditory-Perceptual Evaluation of Disordered Voice Quality
Oates, J. (2009) · Folia Phoniatrica et Logopaedica · doi:10.1159/000200768 · Cited by 294
Acoustic studies of dysarthric speech
Kent, R. D., Weismer, G., Kent, J. F., Vorperian, H. K., & Duffy, J. R. (1999) · Journal of Communication Disorders · doi:10.1016/s0021-9924(99)00004-0 · Cited by 284
Comparing Internal and External Standards in Voice Quality Judgments
Gerratt, B. R., Kreiman, J., Antonanzas-Barroso, N., & Berke, G. S. (1993) · Journal of Speech, Language, and Hearing Research · doi:10.1044/jshr.3601.14 · Cited by 280
Self-citationAssessment of voice quality: Current state-of-the-art
Barsties, B., & De Bodt, M. (2015) · Auris Nasus Larynx · doi:10.1016/j.anl.2014.11.001 · Cited by 265
Pitch and Communicative Intent in Mother's Speech: Adjustments for Age and Sex in the First Year
Kitamura, C., & Burnham, D. (2003) · Infancy · doi:10.1207/s15327078in0401_5 · Cited by 261
The Effect of Perceptual Training on Inexperienced Listeners' Judgments of Dysphonic Voice
Eadie, T. L., & Baylor, C. R. (2006) · Journal of Voice · doi:10.1016/j.jvoice.2005.08.007 · Cited by 254
Voice in Human–Agent Interaction: A Survey
Seaborn, K., Miyake, N. P., Pennefather, P., & Otake-Matsuura, M. (2022) · ACM Computing Surveys · doi:10.1145/3386867 · Cited by 249
Roy, N., Bless, D. M., Heisey, D., & Ford, C. N. (1997) · Journal of Voice · doi:10.1016/s0892-1997(97)80011-2 · Cited by 240
Baumgartner, C. A., Sapir, S., & Ramig, L. O. (2001) · Journal of Voice · doi:10.1016/s0892-1997(01)00010-8 · Cited by 147
Cordes, A. K., & Ingham, R. J. (1994) · Journal of Speech, Language, and Hearing Research · doi:10.1044/jshr.3702.279 · Cited by 104
The effects of aphasia severity on the ability to assess language disorders via telerehabilitation
Hill, A. J., Theodoros, D. G., Russell, T. G., Ward, E. C., & Wootton, R. (2009) · Aphasiology · doi:10.1080/02687030801909659 · Cited by 85
Braden, M. N., Johns, M. M., Klein, A. M., Delgaudio, J. M., Gilman, M., & Hapner, E. R. (2010) · Journal of Voice · doi:10.1016/j.jvoice.2008.08.003 · Cited by 45
Narasimhan, S., & Rashmi, R. (2022) · Journal of Voice · doi:10.1016/j.jvoice.2020.06.009 · Cited by 38
Chen, Z., Zhu, P., Qiu, W., Guo, J., & Li, Y. (2023) · International Journal of Language & Communication Disorders · doi:10.1111/1460-6984.12783 · Cited by 31
Evidence for primary laryngeal inhalant allergy: a randomized, double‐blinded crossover study
Roth, D. F., Abbott, K. V., Carroll, T. L., & Ferguson, B. J. (2013) · International Forum of Allergy & Rhinology · doi:10.1002/alr.21051 · Cited by 20
Jayakumar, T., Rajasudhakar, R., & Benoy, J. J. (2024) · Journal of Voice · doi:10.1016/j.jvoice.2022.02.019 · Cited by 8
Validation of Acoustic Breathiness Index (ABI) in the South Indian Population
Jayakumar, T., & Benoy, J. J. (2024) · Journal of Voice · doi:10.1016/j.jvoice.2022.05.006 · Cited by 8
Benoy, J. J., & Jayakumar, T. (2026) · Journal of Voice · doi:10.1016/j.jvoice.2023.12.011 · Cited by 7
Lima-Filho, L. M. A., Lopes, L. W., & Filho, T. D. M. E. S. (2024) · Journal of Voice · doi:10.1016/j.jvoice.2024.11.002 · Cited by 3
Strömbergsson, S., Fröjdh, M., Pettersson, M., Grósz, T., Getman, Y., & Kurimo, M. (2025) · Clinical Linguistics & Phonetics · doi:10.1080/02699206.2024.2355470 · Cited by 1
Metadata
- Authors
- Kreiman, J., Gerratt, B. R., Kempster, G. B., Erman, A., & Berke, G. S.
- Venue
- Journal of Speech, Language, and Hearing Research
- Published
- 1993-02
- Volume
- 36
- Issue
- 1
- Pages
- 21-40
- ISSN
- 1092-4388 (ISSN-L)
- 1558-9102
- OpenAlex
- W2011585205
- Cited by
- 737
- References
- 46
- FWCI
- 2.91
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.