CoChat
Sign Up
Cited by 10Neurobiology of Language and BilingualismReading and Literacy DevelopmentLanguage Development and Disorders

Comparison of category and letter fluency tasks through automated analysis

Open access
Gonzalez-Recober, Carmen, Nevler, Naomi, Shellikeri, Sanjana, Cousins, Katheryn A. Q., Rhodes, Emma, Liberman, Mark, Grossman, Murray, Irwin, David +1 more·2023·Frontiers in Psychology
DOI 10.3389/fpsyg.2023.1212793PMC PMC10600440
Open this paper in CoChat — free accountOpen PDF

Abstract

Introduction: Category and letter fluency tasks are commonly used neuropsychological tasks to evaluate lexical retrieval. Methods: This study used validated automated methods, which allow for more expansive investigation, to analyze speech production of both category (“Animal”) and letter (“F”) fluency tasks produced by healthy participants (n = 36) on an online platform. Recordings were transcribed and analyzed through automated pipelines, which utilized natural language processing and automatic acoustic processing tools. Automated pipelines calculated overall performance scores, mean inter-word response time, and word start time; errors were excluded from analysis. Each word was rated for age of acquisition (AoA), ambiguity, concreteness, frequency, familiarity, word length, word duration, and phonetic and semantic distance from its previous word. Results: Participants produced significantly more words on the category fluency task relative to the letter fluency task (p < 0.001), which is in line with previous studies. Wilcoxon tests also showed tasks differed on several mean speech measures of words, and category fluency was associated with lower mean AoA (p<0.001), lower frequency (p < 0.001), lower semantic ambiguity (p < 0.001), lower semantic distance (p < 0.001), lower mean inter-word RT (p = 0.03), higher concreteness (p < 0.001), and higher familiarity (p = 0.02), compared to letter fluency. ANOVAs significant interactions for fluency task on total score and lexical measures showed that lower category fluency scores were significantly related to lower AoA and higher prevalence, and this was not observed for letter fluency scores. Finally, word-characteristics changed over time and significant interactions were noted between the tasks, including word familiarity (p = 0.019), semantic ambiguity (p = 0.002), semantic distance (p=0.001), and word duration (p<0.001). Discussion: These findings showed that certain lexical measures such as AoA, word familiarity, and semantic ambiguity were important for understanding how these tasks differ. Additionally, it found that acoustic measures such as inter-word RT and word duration are also imperative to analyze when comparing the two tasks. By implementing these automated techniques, which are reproducible and scalable, to analyze fluency tasks we were able to quickly detect these differences. In future clinical settings, we expect these methods to expand our knowledge on speech feature differences that impact not only total scores, but many other speech measures among clinical populations.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/fpsyg.2023.1212793

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Such metrics could reveal more subtle deficits than is captured by total correct word count, especially during repeated evaluations of the same subject.

    “Such metrics could reveal more subtle deficits than is captured by total correct word count, especially during repeated evaluations of the same subject. 55”

    From Developing digital health technologies for frontotemporal degeneration (2025) · cites this paper as [55]

    The Introduction and Discussion (page not provided) say automated speech metrics may reveal features beyond total fluency scores and could help characterize clinical deficits in future work. The paper studied healthy participants and changes within individual tasks; it did not evaluate whether the metrics detect subtle deficits during repeated assessments of the same person. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Applying automated speech-language analyses to recorded fluency tasks can automate and standardize scoring and provide additional metrics such as semantic and phonetic distances between consecutive words and reaction times.

    “Applying automated speech‐language analyses to recorded fluency tasks can not only automate and standardize scoring but also provide additional metrics such as semantic and phonetic distances between consecutive words and reaction times.”

    From Developing digital health technologies for frontotemporal degeneration (2025) · cites this paper as [55]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 33

  1. Abdel Aziz K. Khater M. S. Emara T. Tawfik H. M. Rasheedy D. Mohammedin A. S.. (2017). Effects of age, education, and gender on verbal fluency in healthy adult Arabic-speakers in Egypt. Appl. Neuropsychol. Adult 24, 331– 341. doi: 10.1080/23279095.2016.1185424, PMID: 27282630 · DOI
  2. Amunts J. Camilleri J. A. Eickhoff S. B. Patil K. R. Heim S. von Polier G. G.. (2021). Comprehensive verbal fluency features predict executive function performance. Sci. Rep. 11: 6929. doi: 10.1038/s41598-021-85981-1, PMID:, 33767208 PMC7994566 · DOI
  3. Andreou G. Trott K. (2013). Verbal fluency in adults diagnosed with attention-deficit hyperactivity disorder (ADHD) in childhood. Atten. Defic. Hyperact. Disord. 5, 343– 351. doi: 10.1007/s12402-013-0112-z, PMID:, 23749309 · DOI
  4. Brysbaert M. Mandera P. Keuleers E. (2018). Word prevalence norms for 62,000 English lemmas. Behav. Res. Methods 2018, 467– 479. doi: 10.3758/s13428-018-1077-9 29967979 · DOI · Paper page
  5. Brysbaert M. New B. (2009). Moving beyond Kučera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behav. Res. Methods 41, 977– 990. doi: 10.3758/BRM.41.4.977 19897807 · DOI · Paper page
  6. Brysbaert M. Warriner A. B. Kuperman V. (2014). Concreteness ratings for 40 thousand generally known English word lemmas. Behav. Res. Methods 46, 904– 911. doi: 10.3758/s13428-013-0403-5, PMID: 24142837 · DOI · Paper page
  7. Cho S. Nevler N. Parjane N. Cieri C. Liberman M. Grossman M.. (2021). Automated analysis of digitized letter fluency data. Front. Psychol. 12: 654214. doi: 10.3389/fpsyg.2021.654214, PMID: 34393894 PMC8359864 · DOI
  8. Cho S. Nevler N. Shellikeri S. Parjane N. Irwin D. J. Ryant N.. (2021). Lexical and acoustic characteristics of young and older healthy adults. J. Speech Lang. Hear. Res. 64, 302– 314. doi: 10.1044/2020_JSLHR-19-00384, PMID: 33439761 PMC8632482 · DOI
  9. Cook P. A. McMillan C. T. Avants B. B. Peelle J. E. Gee J. C. Grossman M. (2014). Relating brain anatomy and cognitive ability using a multivariate multimodal framework. NeuroImage 99, 477– 486. doi: 10.1016/j.neuroimage.2014.05.008, PMID: 24830834 PMC4151353 · DOI
  10. Elvevåg B. Fisher J. E. Gurd J. M. Goldberg T. E. (2002). Semantic clustering in verbal fluency: Schizophrenic patients versus control participants. Psychol. Med. 32, 909– 917. doi: 10.1017/S0033291702005597, PMID: 12171385 · DOI
  11. Fellbaum C. (2005). “ WordNet and wordnets,” in Encyclopedia of Language and Linguistics, Second Edition. ed. Keith B. (Oxford: Elsevier), 665– 670.
  12. Forbes-Mckay K. E. Ellis A. W. Shanks M. F. Venneri A. (2005). The age of acquisition of words produced in a semantic fluency task can reliably differentiate normal from pathological age related cognitive decline. Neuropsychologia 43, 1625– 1632. doi: 10.1016/j.neuropsychologia.2005.01.008, PMID: 16009244 · DOI
  13. Haugrud N. Crossley M. Vrbanic M. (2011). Clustering and Switching Strategies During Verbal Fluency Performance Differentiate Alzheimer’s Disease and Healthy Aging. J. Int. Neuropsychol. Soc. 17, 1153– 1157. doi: 10.1017/S1355617711001196, PMID: 22014065 · DOI
  14. Honnibal M. Johnson M. (2015). An improved non-monotonic transition system for dependency parsing. EMNLP 2015: Conference on empirical methods in natural language processing, 1373– 1378.
  15. Jiahong Y. Liberman M. (2008). Speaker identification on the SCOTUS corpus. J. Acoust. Soc. Am. 123: 3878. doi: 10.1121/1.2935783 · DOI
  16. Juhasz B. J. Chambers D. Shesler L. W. Haber A. Kurtz M. M. (2012). Evaluating lexical characteristics of verbal fluency output in schizophrenia. Psychiatry Res. 200, 177– 183. doi: 10.1016/j.psychres.2012.06.035, PMID: 22809852 PMC3513518 · DOI
  17. Kim B. J. Lee C. S. Oh B. H. Hong C. H. Lee K. S. Son S. J.. (2013). A normative study of lexical verbal fluency in an educationally-diverse elderly population. Psychiatry Investig. 10, 346– 351. doi: 10.4306/pi.2013.10.4.346, PMID: 24474982 PMC3902151 · DOI
  18. Luo L. Luk G. Bialystok E. (2010). Effect of language proficiency and executive control on verbal fluency performance in bilinguals. Cognition 114, 29– 41. doi: 10.1016/j.cognition.2009.08.014, PMID: 19793584 · DOI · Paper page
  19. Monsch A. U. Bondi M. W. Butters N. Salmon D. P. Katzman R. Thal L. J. (1992). Comparisons of verbal fluency tasks in the detection of dementia of the Alzheimer type. Arch. Neurol. 49, 1253– 1258. doi: 10.1001/archneur.1992.00530360051017, PMID:, 1449404 · DOI
  20. Nevado A. Del Río D. Martín-Aragoneses M. T. Prados J. M. López-Higes R. (2021). Preserved semantic categorical organization in mild cognitive impairment: A network analysis of verbal fluency. Neuropsychologia 157: 107875. doi: 10.1016/j.neuropsychologia.2021.107875, PMID: 33930387 · DOI
  21. Pakhomov S. V. S. Hemmy L. S. (2014). A computational linguistic measure of clustering behavior on semantic verbal fluency task predicts risk of future dementia in the nun study. Cortex 55, 97– 106. doi: 10.1016/j.cortex.2013.05.009, PMID: 23845236 PMC4402214 · DOI
  22. Raoux N. Amieva H. Le Goff M. Auriacombe S. Carcaillon L. Letenneur L.. (2008). Clustering and switching processes in semantic verbal fluency in the course of alzheimer’s disease subjects: Results from the PAQUID Longitudinal Study. Cortex 44, 1188– 1196. doi: 10.1016/j.cortex.2007.08.019, PMID: 18761132 · DOI
  23. Rofes A. de Aguiar V. Ficek B. Wendt H. Webster K. Tsapkini K. (2019). The Role of Word Properties in Performance on Fluency Tasks in People with Primary Progressive Aphasia. J. Alzheimers Dis. 68, 1521– 1534. doi: 10.3233/JAD-180990, PMID:, 30909222 PMC6548439 · DOI
  24. Rohrer D. Wixted J. T. Salmon D. P. Butters N. (1995). Retrieval From Semantic Memory and Its Implications for Alzheimer’s Disease. J. Exp. Psychol. Learn. Mem. Cogn. 21, 1127– 1139. doi: 10.1037/0278-7393.21.5.1127 8744958 · DOI
  25. Sailor K. M. Zimmerman M. E. Sanders A. E. (2011). Differential impacts of age of acquisition on letter and semantic fluency in Alzheimer’s disease patients and healthy older adults. Q. J. Exp. Psychol. 64, 2383– 2391. doi: 10.1080/17470218.2011.596660, PMID: 21851152 PMC3671596 · DOI
  26. Sauzéon H. Raboutet C. Rodrigues J. Langevin S. Schelstraete M. A. Feyereisen P.. (2011). Verbal Knowledge as a Compensation Determinant of Adult Age Differences in Verbal Fluency Tasks over Time. J. Adult Dev. 18, 144– 154. doi: 10.1007/s10804-010-9107-6 · DOI
  27. Shao Z. Janse E. Visser K. Meyer A. S. (2014). What do verbal fluency tasks measure? Predictors of verbal fluency performance in older adults. Front. Psychol. 5, 1– 10. doi: 10.3389/fpsyg.2014.00772, PMID: 25101034 PMC4106453 · DOI
  28. Troyer A. K. Moscovitch M. Winocur G. (1997). Clustering and switching as two components of verbal fluency: Evidence from younger and older healthy adults. Neuropsychology, 11, 138–556 146, doi: 10.1037/0894-4105.11.1.138, PMID: 9055277 · DOI · Paper page
  29. Troyer A. K. Moscovitch M. Winocur G. Leach L. Freedman M. (1998). Clustering and switching on verbal fluency tests in Alzheimer's and Parkinson's disease. J. Int. Neuropsychol. Soc. 4, 137– 143. doi: 10.1017/s1355617798001374, PMID:, 9529823 · DOI · Paper page
  30. van Beilen M. Pijnenborg M. van Zomeren E. H. van den Bosch R. J. Withaar F. K. Bouma A. (2004). What is measured by verbal fluency tests in schizophrenia? Schizophr. Res. 69, 267– 276. doi: 10.1016/j.schres.2003.09.007 15469198 · DOI
  31. Van Den Berg E. Jiskoot L. C. Grosveld M. J. H. Van Swieten J. C. Papma J. M. (2017). Qualitative Assessment of Verbal Fluency Performance in Frontotemporal Dementia. Dement. Geriatr. Cogn. Disord. 44, 35– 44. doi: 10.1159/000477538, PMID: 28624827 · DOI
  32. Vonk J. M. J. Rizvi B. Lao P. J. Budge M. Manly J. J. Mayeux R.. (2019). Letter and Category Fluency Performance Correlates with Distinct Patterns of Cortical Thickness in Older Adults. Cereb. Cortex 29, 2694– 2700. doi: 10.1093/cercor/bhy138, PMID:, 29893804 PMC6519688 · DOI
  33. Zehr J. Schwarz F. (2018). PennController for Internet Based Experiments (IBEX). doi: 10.17605/OSF.IO/MD832 · DOI

Cited by

Sorted by citations

Authors
Gonzalez-Recober, C., Nevler, N., Shellikeri, S., Cousins, K. A. Q., Rhodes, E., Liberman, M., Grossman, M., Irwin, D., & Cho, S.
Affiliations
University of Pennsylvania (US)
Corresponding author
Cho, Sunghye
ORCID
Venue
Frontiers in Psychology
Publisher
Frontiers Media SA
Published
2023-10-11
Volume
14
Pages
1212793-1212793
ISSN
1664-1078 (ISSN-L)
DOI
10.3389/fpsyg.2023.1212793
PMC
PMC10600440
OpenAlex
W4387527026
Cited by
10
References
2
Citations by year
2026: 3 · 2025: 6 · 2024: 1
FWCI
1.50
Type
journal article
Language
en
Open access
Licence
Locations
  • Frontiers in Psychology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • U.S. Department of Defense (W81XWH2010531, W81XWH)
  • Alzheimer's Association (AARF‐D‐619473, AARF-21-851126)
  • National Institutes of Health (W81XWH)
  • National Institute on Aging (P01AG066597)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
  • Social Sciences › Psychology › Developmental and Educational Psychology › Reading and Literacy Development
  • Social Sciences › Psychology › Developmental and Educational Psychology › Language Development and Disorders
Keywords
semantic fluency, phonemic fluency, word retrieval, automated speech analysis, age of acquisition, word familiarity, semantic ambiguity, semantic distance, word concreteness, word frequency, word duration, speech production, neuropsychological assessment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review