Talk2Me: Automated linguistic data collection for personal assessment
Open accessThe DOI matches its Crossref record; 98 of the 101 references it cites were found in a registry or an index.
Abstract
Language is one the earliest capacities affected by cognitive change. To monitor that change longitudinally, we have developed a web portal for remote linguistic data acquisition, called Talk2Me, consisting of a variety of tasks. In order to facilitate research in different aspects of language, we provide baselines including the relations between different scoring functions within and across tasks. These data can be used to augment studies that require a normative model; for example, we provide baseline classification results in identifying dementia. These data are released publicly along with a comprehensive open-source package for extracting approximately two thousand lexico-syntactic, acoustic, and semantic features. This package can be applied arbitrarily to studies that include linguistic data. To our knowledge, this is the most comprehensive publicly available software for extracting linguistic features. The software includes scoring functions for different tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures














Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975) · Journal of Psychiatric Research · doi:10.1016/0022-3956(75)90026-6 · Cited by 88,282 · cited 2×
“Talk2Me collects data through tasks similar to those used in standard assessments of cognition (including the Mini-Mental State Examination [21], the Montréal Cognitive Assessment [22], and the Western Aphasia Battery [23]).”
Method(2008) · Journal of Machine Learning Research · Cited by 33,051
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment
Nasreddine, Z. S., Phillips, N. A., Bédirian, V., Charbonneau, S., Whitehead, V., Collin, I., Cummings, J. L., & Chertkow, H. (2005) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.2005.53221.x · Cited by 25,553 · cited 2×
“Talk2Me collects data through tasks similar to those used in standard assessments of cognition (including the Mini-Mental State Examination [21], the Montréal Cognitive Assessment [22], and the Western Aphasia Battery [23]).”
MethodBLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 1×
“, ‘bilingual evaluation understudy’) [53] to score the vocabulary task.”
BackgroundHirono, N., Mori, E., Ikejiri, Y., Imamura, T., Shimomura, T., Hashimoto, M., Yamashita, H., & Ikeda, M. (1997) · Brain and nerve · Cited by 21,039
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
Kok, J. S., Berg, I. J., & Scherder, E. J. A. (2013) · Dementia and Geriatric Cognitive Disorders Extra · doi:10.1159/000353441 · Cited by 20,589
Older Adults' Reasons for Using Technology while Aging in Place
Peek, S. T. M., Luijkx, K. G., Rijnaard, M. D., Nieboer, M. E., van der Voort, C. S., Aarts, S., van Hoof, J., Vrijhoef, H. J. M., & Wouters, E. J. M. (2016) · Gerontology · doi:10.1159/000430949 · Cited by 19,690
Sensitivity and False Alarm Rate of a Fall Sensor in Long-Term Fall Detection in the Elderly
Kangas, M., Korpelainen, R., Vikman, I., Nyberg, L., & Jämsä, T. (2015) · Gerontology · doi:10.1159/000362720 · Cited by 18,909
Research works printed but not linked: 2
- Francis WN, Kucera H. Brown corpus manual. Brown University. 1979; 2.
- Wu Z, Palmer M. Verbs semantics and lexical selection. In: Proceedings of the 32nd annual meeting on Association for Computational Linguistics. Association for Computational Linguistics; 1994. p. 133–138. · DOI · Paper page
Cited by
A Conversational Robot for Older Adults with Alzheimer's Disease
Pou-Prom, C., Raimondo, S., & Rudzicz, F. (2020) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3380785 · Cited by 59
“…of the kinds of linguistic analysis that could be performed on conversations with a robot, from conversation transcripts, we extract an extensive list of linguistic features using available open-source software, 10 which have previously been used for analyzing cognition and speech impairment [21].”
MethodSelf-citationDetection of Mild Cognitive Impairment Through Natural Language and Touchscreen Typing Processing
Ntracha, A., Iakovakis, D., Hadjidimitriou, S., Charisis, V. S., Tsolaki, M., & Hadjileontiadis, L. J. (2020) · Frontiers in Digital Health · doi:10.3389/fdgth.2020.567158 · Cited by 62
“Tools, like the computer-based software of (16), collecting both textual and acoustic linguistic features from different tasks, facilitate the research toward such direction.”
Bartlett, L., Doherty, K., Farrow, M., Kim, S., Hill, E., King, A. E., Alty, J., Eccleston, C. E. A., Kitsos, A., Bindoff, A. D., et al. (2022) · JMIR Research Protocols · doi:10.2196/34688 · Cited by 51
Repeatability of Commonly Used Speech and Language Features for Clinical Applications
Stegmann, G. M., Hahn, S., Liss, J., Shefner, J., Rutkove, S. B., Kawabata, K., Bhandari, S., Shelton, K., Duncan, C. J., & Berisha, V. (2020) · Digital Biomarkers · doi:10.1159/000511671 · Cited by 47
Jeong, L., Lee, M., Eyre, B., Balagopalan, A., Rudzicz, F., & Gabilondo, C. (2023) · Psychiatric Research and Clinical Practice · doi:10.1176/appi.prcp.20230003 · Cited by 37
“A total of 393 features were obtained (23).”
BackgroundSelf-citationParsapoor, M., Alam, M. R., & Mihailidis, A. (2023) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-023-02122-6 · Cited by 34
“We have extracted the acoustic features using the COre Variable Feature Extraction Feature Extractor (COV-FEFE) tool [57].”
BackgroundMethodJonell, P., Moëll, B., Håkansson, K., Henter, G. E., Kucherenko, T., Mikheeva, O., Hagman, G., Holleman, J., Kivipelto, M., Kjellström, H., et al. (2021) · Frontiers in Computer Science · doi:10.3389/fcomp.2021.642633 · Cited by 33
“In the Talk2Me project anonymous people can contribute with both speech and text via a web interface (Komeili et al., 2019).”
Digital Measurement of Mental Health: Challenges, Promises, and Future Directions
Abbas, A., Schultebraucks, K., & Galatzer‐Levy, I. R. (2021) · Psychiatric Annals · doi:10.3928/00485713-20201207-01 · Cited by 23
Akinyelure, F. M. (2021) · International Journal of Research in Medical Science · doi:10.33545/26648733.2021.v3.i2a.167 · Cited by 22
Lammert, J. M., Roberts, A. C., McRae, K., Batterink, L. J., & Butler, B. E. (2025) · Journal of Speech, Language, and Hearing Research · doi:10.1044/2024_jslhr-24-00515 · Cited by 17
A speech-based prognostic model for dysarthria progression in ALS
Stegmann, G., Charles, S. D., Liss, J., Shefner, J. M., Rutkove, S. B., & Berisha, V. (2023) · Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration · doi:10.1080/21678421.2023.2222144 · Cited by 16
“Several speech analysis tools and software have been utilized in ALS research, including open-source software such as openSMILE (9), Talk2me (10), and Praat (11).”
Miller-Lewis, L. R., Lewis, T. W., Tieman, J., Rawlings, D., Parker, D., & Sanderson, C. R. (2021) · PLoS ONE · doi:10.1371/journal.pone.0242848 · Cited by 15
Stegmann, G., Hahn, S., Bhandari, S., Kawabata, K., Shefner, J. M., Duncan, C. J., Liss, J., Berisha, V., & Mueller, K. D. (2022) · Alzheimer s & Dementia Diagnosis Assessment & Disease Monitoring · doi:10.1002/dad2.12294 · Cited by 9
Towards AI-powered Language Assessment ToolsPreprint
Parsapoor, M., Alam, M. R., & Mihailidis, A. (2021) · Research Square · doi:10.21203/rs.3.rs-246079/v1 · Cited by 10
Huang, G., Li, R., Roccati, E., Lawler, K., Bindoff, A. D., King, A. E., Vickers, J. C., Bai, Q., & Alty, J. (2025) · The Journal of Prevention of Alzheimer s Disease · doi:10.1016/j.tjpad.2025.100081 · Cited by 6
Carmi, L., Abbas, A., Schultebraucks, K., & Galatzer‐Levy, I. R. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-822201-0.00002-2 · Cited by 6
Automated analysis of propositional idea density in older adults
Hill, E., Alty, J., Bartlett, L., Goldberg, L., Park, M., Yeom, S., & Vickers, J. C. (2021) · Cortex · doi:10.1016/j.cortex.2021.09.018 · Cited by 4
“Methods for the automated detection of language deficits in cognitive decline are gaining traction (Komeili et al., 2019; Laguarta et al., 2020), with online administration removing geographical, social and economic barriers to participation.”
Measuring Cognitive Status from Speech in a Smart Home Environment
Fraser, K., & Komeili, M. (2021) · IEEE Instrumentation & Measurement Magazine · doi:10.1109/mim.2021.9513645 · Cited by 5
Self-citationWhat is the retest reliability of computationally extractable speech and language markers?
Çokal, D., Villalba, M., Rui, H., Palominos, C., Böke, A., Homan, P., von Heusinger, K., Kambeitz, J. P., & Hinzen, W. (2026) · Computer Speech & Language · doi:10.1016/j.csl.2026.101981 · Cited by 4
Feature Selection and Text Embedding for Detecting Dementia from Spontaneous Cantonese
Ke, X., Mak, M., & Meng, H. M. L. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095140 · Cited by 2
“We then computed semantic similarity using the mean and minimum cosine distances between the one-hot embeddings of each pair of utterances [12].”
MethodSinclair, J. (2020) · TSpace (University of Toronto) · Cited by 7
Correction: Talk2Me: Automated linguistic data collection for personal assessment
Staff, T. P. O. (2019) · PLoS ONE · doi:10.1371/journal.pone.0216375 · Cited by 2
Machine Learning to Assess Language Impairments in Older AdultsPreprint
Parsapoor, M., Alam, M. R., & Mihailidis, A. (2020) · Research Square · doi:10.21203/rs.3.rs-109610/v1 · Cited by 1
“Table 1: Different Tests/Tasks To Assess Language and Cognitive Deficit Test/Task Name Cognitive function and Language Deficit Reference MMSE Cognitive Impairment [59] MoCA Cognitive Impairment [21] Boston Naming Test Lexical retrieval deficit [60] Letter Fluency Task Lexical retrieval deficit [60] Picture Description…”
BackgroundMethodInfluentialDeMention: A Modular System for Speech-Based Alzheimer's Dementia Screening
Behrendt, J., Brauße, E., Rammohan, R. A., & Schultz, T. (2026) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3816046.3816319
“The web-based portal Talk2Me [12] enables the collection of speech recordings through structured tasks such as picture descriptions.”
Metadata
- Authors
- Komeili, M., Pou-Prom, C., Liaqat, D., Fraser, K. C., Yancheva, M., & Rudzicz, F.
- Affiliations
- Carleton University (CA)
- St. Michael's Hospital (CA)
- University of Toronto (CA)
- National Research Council Canada (CA)
- Corresponding author
- Rudzicz, Frank
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2019-03-27
- Volume
- 14
- Issue
- 3
- Pages
- e0212342
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC6436678
- OpenAlex
- W2925077947
- Semantic Scholar
- f9313ca0ddb194ad1c116aeb1058445a2dbd247e
- Cited by
- 23
- References
- 101
- Citations by year
- 2026: 2 · 2025: 2 · 2023: 4 · 2022: 1 · 2021: 8 · 2020: 5 · 2019: 1
- FWCI
- 2.98
- Type
- journal article
- Language
- en
- Update notices
- correction (2019-04-30)
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- NPARC (submittedVersion)
- Carleton University's Institutional Repository (MacOdrum Library, Carleton University) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- OSF Preprints (OSF Preprints) (submittedVersion)
- Funders
- NSERC (RGPIN 435874)
- Alzheimer Society Research Program (Young Investigator 2015)
- Canadian Institutes of Health Research (CHRP 440 381332)
- Topics
- Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
- Keywords
- online assessment, longitudinal cognitive change, language loss, dementia detection, normative model, acoustic features, semantic features, linguistic feature extraction, open source software, scoring functions
- MeSH terms
- Patient Portals, Adult, Aged, Aged, 80 and over, Data Collection, Female, Humans, Language, Linguistics, Male, Middle Aged, Semantics, Software, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.