Talk2Me: Automated linguistic data collection for personal assessment
Open accessThe DOI matches its Crossref record; 98 of the 101 references it cites were found in a registry or an index.
Abstract
Language is one the earliest capacities affected by cognitive change. To monitor that change longitudinally, we have developed a web portal for remote linguistic data acquisition, called Talk2Me, consisting of a variety of tasks. In order to facilitate research in different aspects of language, we provide baselines including the relations between different scoring functions within and across tasks. These data can be used to augment studies that require a normative model; for example, we provide baseline classification results in identifying dementia. These data are released publicly along with a comprehensive open-source package for extracting approximately two thousand lexico-syntactic, acoustic, and semantic features. This package can be applied arbitrarily to studies that include linguistic data. To our knowledge, this is the most comprehensive publicly available software for extracting linguistic features. The software includes scoring functions for different tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures














Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975) · Journal of Psychiatric Research · doi:10.1016/0022-3956(75)90026-6 · Cited by 88,282 · cited 2×
“Talk2Me collects data through tasks similar to those used in standard assessments of cognition (including the Mini-Mental State Examination [21], the Montréal Cognitive Assessment [22], and the Western Aphasia Battery [23]).”
Method(2008) · Journal of Machine Learning Research · Cited by 33,051
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment
Nasreddine, Z. S., Phillips, N. A., Bédirian, V., Charbonneau, S., Whitehead, V., Collin, I., Cummings, J. L., & Chertkow, H. (2005) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.2005.53221.x · Cited by 25,553 · cited 2×
“Talk2Me collects data through tasks similar to those used in standard assessments of cognition (including the Mini-Mental State Examination [21], the Montréal Cognitive Assessment [22], and the Western Aphasia Battery [23]).”
MethodBLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 1×
“, ‘bilingual evaluation understudy’) [53] to score the vocabulary task.”
BackgroundHirono, N., Mori, E., Ikejiri, Y., Imamura, T., Shimomura, T., Hashimoto, M., Yamashita, H., & Ikeda, M. (1997) · Brain and nerve · Cited by 21,039
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
Kok, J. S., Berg, I. J., & Scherder, E. J. A. (2013) · Dementia and Geriatric Cognitive Disorders Extra · doi:10.1159/000353441 · Cited by 20,589
Older Adults' Reasons for Using Technology while Aging in Place
Peek, S. T. M., Luijkx, K. G., Rijnaard, M. D., Nieboer, M. E., van der Voort, C. S., Aarts, S., van Hoof, J., Vrijhoef, H. J. M., & Wouters, E. J. M. (2016) · Gerontology · doi:10.1159/000430949 · Cited by 19,690
Sensitivity and False Alarm Rate of a Fall Sensor in Long-Term Fall Detection in the Elderly
Kangas, M., Korpelainen, R., Vikman, I., Nyberg, L., & Jämsä, T. (2015) · Gerontology · doi:10.1159/000362720 · Cited by 18,909
Research works printed but not linked: 2
- Francis WN, Kucera H. Brown corpus manual. Brown University. 1979; 2.
- Wu Z, Palmer M. Verbs semantics and lexical selection. In: Proceedings of the 32nd annual meeting on Association for Computational Linguistics. Association for Computational Linguistics; 1994. p. 133–138. · DOI · Paper page
Cited by
A Conversational Robot for Older Adults with Alzheimer's Disease
Pou-Prom, C., Raimondo, S., & Rudzicz, F. (2020) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3380785 · Cited by 59
“…of the kinds of linguistic analysis that could be performed on conversations with a robot, from conversation transcripts, we extract an extensive list of linguistic features using available open-source software, 10 which have previously been used for analyzing cognition and speech impairment [21].”
MethodSelf-citationDetection of Mild Cognitive Impairment Through Natural Language and Touchscreen Typing Processing
Ntracha, A., Iakovakis, D., Hadjidimitriou, S., Charisis, V. S., Tsolaki, M., & Hadjileontiadis, L. J. (2020) · Frontiers in Digital Health · doi:10.3389/fdgth.2020.567158 · Cited by 62
“Tools, like the computer-based software of (16), collecting both textual and acoustic linguistic features from different tasks, facilitate the research toward such direction.”
Bartlett, L., Doherty, K., Farrow, M., Kim, S., Hill, E., King, A. E., Alty, J., Eccleston, C. E. A., Kitsos, A., Bindoff, A. D., et al. (2022) · JMIR Research Protocols · doi:10.2196/34688 · Cited by 51
Repeatability of Commonly Used Speech and Language Features for Clinical Applications
Stegmann, G. M., Hahn, S., Liss, J., Shefner, J., Rutkove, S. B., Kawabata, K., Bhandari, S., Shelton, K., Duncan, C. J., & Berisha, V. (2020) · Digital Biomarkers · doi:10.1159/000511671 · Cited by 47
Jeong, L., Lee, M., Eyre, B., Balagopalan, A., Rudzicz, F., & Gabilondo, C. (2023) · Psychiatric Research and Clinical Practice · doi:10.1176/appi.prcp.20230003 · Cited by 37
“A total of 393 features were obtained (23).”
BackgroundSelf-citation
Metadata
- Authors
- Komeili, M., Pou-Prom, C., Liaqat, D., Fraser, K. C., Yancheva, M., & Rudzicz, F.
- Affiliations
- Carleton University (CA)
- St. Michael's Hospital (CA)
- University of Toronto (CA)
- National Research Council Canada (CA)
- Corresponding author
- Rudzicz, Frank
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2019-03-27
- Volume
- 14
- Issue
- 3
- Pages
- e0212342
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC6436678
- OpenAlex
- W2925077947
- Semantic Scholar
- f9313ca0ddb194ad1c116aeb1058445a2dbd247e
- Cited by
- 23
- References
- 101
- Citations by year
- 2026: 2 · 2025: 2 · 2023: 4 · 2022: 1 · 2021: 8 · 2020: 5 · 2019: 1
- FWCI
- 2.98
- Type
- journal article
- Language
- en
- Update notices
- correction (2019-04-30)
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- NPARC (submittedVersion)
- Carleton University's Institutional Repository (MacOdrum Library, Carleton University) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- OSF Preprints (OSF Preprints) (submittedVersion)
- Funders
- NSERC (RGPIN 435874)
- Alzheimer Society Research Program (Young Investigator 2015)
- Canadian Institutes of Health Research (CHRP 440 381332)
- Topics
- Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
- Keywords
- online assessment, longitudinal cognitive change, language loss, dementia detection, normative model, acoustic features, semantic features, linguistic feature extraction, open source software, scoring functions
- MeSH terms
- Patient Portals, Adult, Aged, Aged, 80 and over, Data Collection, Female, Humans, Language, Linguistics, Male, Middle Aged, Semantics, Software, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.