CoChat

Automatic Detection of Incoherent Speech for Diagnosing Schizophrenia

Open access
Iter, Dan, Yoon, Jong, Jurafsky, Dan · 2018 · Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic

The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.

Abstract

Schizophrenia is a mental disorder which afflicts an estimated 0.7% of adults worldwide (Saha et al., 2005).It affects many areas of mental function, often evident from incoherent speech.Diagnosing schizophrenia relies on subjective judgments resulting in disagreements even among trained clinicians.Recent studies have proposed the use of natural language processing for diagnosis by drawing on automatically-extracted linguistic features, and particularly the use of discourse coherence.Here, we present the first benchmark comparison of previously proposed coherence models for detecting symptoms of schizophrenia and evaluate their performance on a new dataset of recorded interviews between subjects and clinicians.We also present two improved coherence metrics based on modern sentence embedding techniques that outperform the previous methods on our dataset.Finally, we propose a novel computational model for reference incoherence based on ambiguous pronoun usage and show that it is a highly predictive feature on our data.While the number of subjects is limited in this pilot study, our results suggest new directions for diagnosing common symptoms of schizophrenia.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/w18-0615

Cites

26 of 26 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Average length of a sentence in each quartile of coherence scores.
Figure 1 Average length of a sentence in each quartile of coherence scores.CC-BY · From the PDF
Figure 2 Baseline Incoherence Metric : Each bar is the coherence score for one subject computed as the mean of the cosine similarities of all adjacent sentences in a response. Each sentence is embedded as the mean of the word vectors.
Figure 2 Baseline Incoherence Metric : Each bar is the coherence score for one subject computed as the mean of the cosine similarities of all adjacent sentences in a response. Each sentence is embedded as the mean of the word vectors.CC-BY · From the PDF
Figure 3 New Incoherence Metric Each bar is the coherence for one subject using the improvements explained in Section 4.3. Verbal filler and sentences entirely composed of stop words are removed. Words are embedded with Word2Vec and sentences are embedded with SIF sentence embedding.
Figure 3 New Incoherence Metric Each bar is the coherence for one subject using the improvements explained in Section 4.3. Verbal filler and sentences entirely composed of stop words are removed. Words are embedded with Word2Vec and sentences are embedded with SIF sentence embedding.CC-BY · From the PDF
Figure 5 Ambiguous pronoun usage scores for all subjects using automatic co-reference resolution.
Figure 5 Ambiguous pronoun usage scores for all subjects using automatic co-reference resolution.CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 26 references

Research works printed but not linked: 2

  1. Nancy C Andreasen. 1979. Thought, language, and communication disorders: Ii. diagnostic significance. Archives of general Psychiatry, 36(12):1325-1330.
  2. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Cor- rado, and Jeff Dean. 2013. Distributed representa- tions of words and phrases and their compositional- ity. In Advances in neural information processing systems, pages 3111-3119.

Cited by

See all 121

Metadata

Authors
Iter, D., Yoon, J., & Jurafsky, D.
Affiliations
Stanford University (US)
Venue
Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic
Published
2018
Pages
136-146
OpenAlex
W2806168354
Cited by
115
References
26
Citations by year
2026: 9 · 2025: 10 · 2024: 17 · 2023: 17 · 2022: 24 · 2021: 14 · 2020: 13 · 2019: 11
FWCI
17.03
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
Workshop on Computational Linguistics and Clinical Psychology (CLPsych) (publishedVersion, cc-by)
Funders
National Science Foundation (1514268)
Topics
  • Social Sciences › Psychology › Social Psychology › Mental Health via Writing
  • Health Sciences › Medicine › Psychiatry and Mental health › Schizophrenia research and treatment
  • Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
Keywords
schizophrenia diagnosis, discourse coherence, coherence measures, sentence embeddings, ambiguous pronouns, clinical interview, benchmark comparison, computational psychiatry

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Automatic Detection of Incoherent Speech for Diagnosing Schizophrenia. https://app.cochat.ai/papers/10.18653/v1/w18-0615
The paper
Iter, D., Yoon, J., & Jurafsky, D. (2018). Automatic Detection of Incoherent Speech for Diagnosing Schizophrenia. Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic. https://doi.org/10.18653/v1/w18-0615

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.