Automatic Detection of Incoherent Speech for Diagnosing Schizophrenia
Open accessThe DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
Schizophrenia is a mental disorder which afflicts an estimated 0.7% of adults worldwide (Saha et al., 2005).It affects many areas of mental function, often evident from incoherent speech.Diagnosing schizophrenia relies on subjective judgments resulting in disagreements even among trained clinicians.Recent studies have proposed the use of natural language processing for diagnosis by drawing on automatically-extracted linguistic features, and particularly the use of discourse coherence.Here, we present the first benchmark comparison of previously proposed coherence models for detecting symptoms of schizophrenia and evaluate their performance on a new dataset of recorded interviews between subjects and clinicians.We also present two improved coherence metrics based on modern sentence embedding techniques that outperform the previous methods on our dataset.Finally, we propose a novel computational model for reference incoherence based on ambiguous pronoun usage and show that it is a highly predictive feature on our data.While the number of subjects is limited in this pilot study, our results suggest new directions for diagnosing common symptoms of schizophrenia.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Computational analysis of psychotherapy is encouraging researchers and companies to identify patterns of patient symptomology and therapist intervention (36, 37).”
From Key Considerations for Incorporating Conversational AI in Psychotherapy (2019) · cites this paper as [37]
Experimental — accuracy not yet measuredIn the citing paper
“Previous work has shown that ASR accuracy is reduced for both elderly patients and those with dementia [99], [100], [101].”
From A Review of Automated Speech and Language Features for Assessment of Cognitive and Thought Disorders (2020) · cites this paper as [99]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 1×
“Second, we draw on recent advances in word embeddings (Mikolov et al., 2013; Pennington et al., 2014) and sentence embeddings (Arora et al., 2016; Pagliardini et al., 2018).”
MethodDistributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 2×
“Various other computational models have shown features of text and speech that can be automatically extracted and are associated with schizophrenia, including lexical features drawn from lexicons (Hong et al., 2012, 2015; Mitchell et al., 2015) and acoustic features (Covington et al., 2012).”
BackgroundHalliday, M. A. K., & Hasan, R. (2014) · Routledge eBooks · doi:10.4324/9781315836010 · Cited by 6,082
Rhetorical Structure Theory: Toward a functional theory of text organization
MANN, W. C., & THOMPSON, S. A. (1988) · Text - Interdisciplinary Journal for the Study of Discourse · doi:10.1515/text.1.1988.8.3.243 · Cited by 3,853 · cited 2×
“Relational models like rhetorical structure theory define discourse relations that hierarchically structure texts (Mann and Thompson, 1988; Lascaridest and Asher, 1991).”
BackgroundAttention, intentions, and the structure of discourse
Grosz, B. J., & Sidner, C. L. (1986) · doi:10.5555/12457.12458 · Cited by 2,580
A Systematic Review of the Prevalence of Schizophrenia
Saha, S., Chant, D., Welham, J., & McGrath, J. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020141 · Cited by 2,168
(2017) · International Conference on Learning Representations · Cited by 1,075
Lexical cohesion computed by thesaural relations as an indicator of the structure of text
Morris, J., & Hirst, G. (1991) · OLAC - Open Language Archives Community · doi:10.5555/971738.971740 · Cited by 950
End-to-end Neural Coreference Resolution
Lee, K., He, L., Lewis, M., & Zettlemoyer, L. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1018 · Cited by 869 · cited 1×
“Co-references are extracted from the corpus with a pretrained co-reference resolver (Lee et al., 2017).”
MethodThe measurement of textual coherence with latent semantic analysis
Foltz, P. W., Kintsch, W., & Landauer, T. K. (1998) · Discourse Processes · doi:10.1080/01638539809545029 · Cited by 749 · cited 2×
“From the beginning it was clear that this method relied on simplifying assumptions that might be inappropriate for schizophrenia; for example Foltz et al. (1998) notes that a discourse that simply repeated a sentence would be judged as highly coherent, problematic since repetition or perseveration…”
Method
Research works printed but not linked: 2
- Nancy C Andreasen. 1979. Thought, language, and communication disorders: Ii. diagnostic significance. Archives of general Psychiatry, 36(12):1325-1330.
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Cor- rado, and Jeff Dean. 2013. Distributed representa- tions of words and phrases and their compositional- ity. In Advances in neural information processing systems, pages 3111-3119.
Cited by
Detecting formal thought disorder by deep contextualized word representations
Sarzynska-Wawer, J., Wawer, A., Pawlak, A., Szymanowska, J., Stefaniak, I., Jarkiewicz, M., & Okruszek, L. (2021) · Psychiatry Research · doi:10.1016/j.psychres.2021.114135 · Cited by 335
Key Considerations for Incorporating Conversational AI in Psychotherapy
Miner, A. S., Shah, N., Bullock, K. D., Arnow, B. A., Bailenson, J., & Hancock, J. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00746 · Cited by 166
“Computational analysis of psychotherapy is encouraging researchers and companies to identify patterns of patient symptomology and therapist intervention (36, 37).”
BackgroundTang, S. X., Kriz, R., Cho, S., Park, S. J., Harowitz, J., Gur, R. E., Bhati, M. T., Wolf, D. H., Sedoc, J., & Liberman, M. (2021) · Schizophrenia · doi:10.1038/s41537-021-00154-3 · Cited by 172
Parola, A., Lin, J. M., Simonsen, A., Bliksted, V., Zhou, Y., Wang, H., Inoue, L., Koelkebeck, K., & Fusaroli, R. (2022) · Schizophrenia Research · doi:10.1016/j.schres.2022.07.002 · Cited by 121
Grodniewicz, J. P., & Hohol, M. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1190084 · Cited by 100
Metadata
- Authors
- Iter, D., Yoon, J., & Jurafsky, D.
- Affiliations
- Stanford University (US)
- Venue
- Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic
- Published
- 2018
- Pages
- 136-146
- OpenAlex
- W2806168354
- Cited by
- 115
- References
- 26
- Citations by year
- 2026: 9 · 2025: 10 · 2024: 17 · 2023: 17 · 2022: 24 · 2021: 14 · 2020: 13 · 2019: 11
- FWCI
- 17.03
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Workshop on Computational Linguistics and Clinical Psychology (CLPsych) (publishedVersion, cc-by)
- Funders
- National Science Foundation (1514268)
- Topics
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Health Sciences › Medicine › Psychiatry and Mental health › Schizophrenia research and treatment
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Keywords
- schizophrenia diagnosis, discourse coherence, coherence measures, sentence embeddings, ambiguous pronouns, clinical interview, benchmark comparison, computational psychiatry
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Automatic Detection of Incoherent Speech for Diagnosing Schizophrenia. https://app.cochat.ai/papers/10.18653/v1/w18-0615
- The paper
- Iter, D., Yoon, J., & Jurafsky, D. (2018). Automatic Detection of Incoherent Speech for Diagnosing Schizophrenia. Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic. https://doi.org/10.18653/v1/w18-0615
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.