A Collaborative Approach to Identifying Social Media Markers of Schizophrenia by Employing Machine Learning and Clinical Appraisals

“Results from the Recovery After Initial Schizophrenia Episode-Early Treatment Program (RAISE-ETP) trial [31] suggest that the median duration of untreated psychosis is 74 weeks [32] and support the established hypothesis that lengthy duration of untreated psychosis (DUP) leads to worse outcomes [31,33].”

“Results from the Recovery After Initial Schizophrenia Episode-Early Treatment Program (RAISE-ETP) trial [31] suggest that the median duration of untreated psychosis is 74 weeks [32] and support the established hypothesis that lengthy duration of untreated psychosis (DUP) leads to worse outcomes [31,33].”

“Prior work identifying markers of mental illness online used similar filtering techniques based on self-reported diagnoses [22,23].”

“More than 90% of US youth use social media daily [1], placing it ahead of texting, email, and instant messaging, and they disclose considerably more about themselves online than offline [2].”

“Furthermore, classifiers designed to automatically sort individual cases into diagnostic categories have achieved success in recognizing participants with psychotic disorders from healthy controls based on linguistic differences in writing samples [15] and speech [13,18].”

“The classifier was built and evaluated by applying 10-fold cross-validation, an established technique in supervised machine learning [25].”

“Main Findings These data contribute to a growing body of literature using language to automatically identify individuals online who may be experiencing mental illness, including depression [16,22,27], postpartum mood disorders [28], suicide [29], posttraumatic stress disorder [30], and bipolar disorder [23].”

“Identified benefits include developing a sense of belonging, establishing and maintaining relationships, accessing support, challenging stigma, raising awareness, and sharing experiences [4,5].”

“Main Findings These data contribute to a growing body of literature using language to automatically identify individuals online who may be experiencing mental illness, including depression [16,22,27], postpartum mood disorders [28], suicide [29], posttraumatic stress disorder [30], and bipolar disorder [23].”

“Language-based computational models have achieved more than 80% and 90% accuracy [19,20] in correctly identifying users with self-reported schizophrenia from healthy controls.”

“There is now clear and convincing evidence that online activity can be used to reliably monitor and predict health-related behaviors [7] ranging from the spread of the influenza virus across the United States to rates of seasonal allergies, HIV infection, cancer, smoking, and obesity [8-10].”

“Identified benefits include developing a sense of belonging, establishing and maintaining relationships, accessing support, challenging stigma, raising awareness, and sharing experiences [4,5].”

“Youth with newly diagnosed schizophrenia in particular report frequently utilizing social networking sites throughout the course of illness development and treatment, engaging in social media activity several times daily, and spending several hours per day online [6].”

“Results from the Recovery After Initial Schizophrenia Episode-Early Treatment Program (RAISE-ETP) trial [31] suggest that the median duration of untreated psychosis is 74 weeks [32] and support the established hypothesis that lengthy duration of untreated psychosis (DUP) leads to worse outcomes [31,33].”

“Comparison With Prior Work Consistent with prior trials [11-15,18,36], first-person pronouns were found to be significantly increased in the psychosis group, suggesting greater interpersonal focus.”

“Prior work in speech and text analysis has identified reliable linguistic markers associated with schizophrenia, including significant differences in word frequency, word categories, and use of self-referential pronouns [11-15].”

“Prior work in speech and text analysis has identified reliable linguistic markers associated with schizophrenia, including significant differences in word frequency, word categories, and use of self-referential pronouns [11-15].”

“There is now clear and convincing evidence that online activity can be used to reliably monitor and predict health-related behaviors [7] ranging from the spread of the influenza virus across the United States to rates of seasonal allergies, HIV infection, cancer, smoking, and obesity [8-10].”

“Additionally, these data replicate findings that biological processes, including words such as “body” and “health,” are more frequently used in psychosis [17], suggesting a greater awareness or focus on health status.”

“At the same time, there is compelling evidence to suggest that linguistic and behavioral changes manifest on the pages of social media before they are clinically detected, providing the prospect for earlier intervention [22,28,34].”

“Prior work in speech and text analysis has identified reliable linguistic markers associated with schizophrenia, including significant differences in word frequency, word categories, and use of self-referential pronouns [11-15].”

“Prior work in speech and text analysis has identified reliable linguistic markers associated with schizophrenia, including significant differences in word frequency, word categories, and use of self-referential pronouns [11-15].”

“Recently, Birnbaum et al [35] used Google AdWords to explore aspects of digital advertising most effective at engaging individuals online.”

“Furthermore, prior work has demonstrated that words that might have been automatically identified as self-disclosure such as “psychosis,” schizophrenia,” and “delusion” are often used inappropriately online [21] and may represent a major limitation to these computational models.”

“Prior work in speech and text analysis has identified reliable linguistic markers associated with schizophrenia, including significant differences in word frequency, word categories, and use of self-referential pronouns [11-15].”

“Researchers have examined social media analytics and mental health, and they identified markers in social media activity associated with worsening psychotic symptoms [57], schizophrenia [58], risk of suicidal ideation [59], and depression [60].”

“Since then, researchers have proposed a wide range of feature engineering methods and built machine-learning models for the prediction [13, 67, 69, 88, 103].”

“Since then, researchers have proposed a wide range of feature engineering methods and built machine-learning models for the prediction [13, 67, 69, 88, 103].”

“[34] proposed a human-to-computer approach by combining social media content with clinical appraisals to explore social media as a tool to identify individuals with schizophrenia.”

“These collaborations are fruitful and have greatly benefit prior research [13, 6, 44].”

“…to derive distinct markers of shifts to suicidal ideation from Reddit, Inc user data for modeling in a prediction framework, and Birnbaum et al [25] proposed a method that used machine learning in combination with clinical appraisals as a means of identifying social media markers of…”

““Data...distinguish[es] users with schizophrenia from healthy controls ” [23][p. 1] However, “control” was always juxtaposed against an implied “treatment” position — in experimental setups, control groups are verified to not have the effect under observation.”

“prior literature [11, 63, 65]: 1) behaviors signaling affiliation to mental health resources, 2) self-reported diagnoses, and 3) clinically appraised self-reports of diagnoses.”

“It has been observed that searching for symptoms is one method that individuals experiencing the early stages of mental distress or illness come to better understand what they are experiencing, and look for care [46, 56].”

“In this study, we have shown that several behavioral and linguistic features of the tweets in Spanish can be used as a complementary tool to detect signals of depression of their authors, corroborating and extending the findings obtained by studies carried out on English tweets.”

“Birnbaum et al [54], for example, discussed the limits of contemporary ethical standards for research on social media in the mental health context, noting that “[e]xisting ethical principles do not sufficiently guide researchers” and new technological approaches to “illness identification and symptom tracking will lik…”

“…that the top features of these classifiers align with that of similar mental health classifiers built on Twitter to identify depression (De Choudhury et al. 2013), anxiety (Dutta et al. 2018), stress (Lin et al. 2014), psychosis (Birnbaum et al. 2017), and suicidal ideation (Burnap et al. 2015).”

“…techniques on Twitter feeds for a mental health diagnosis, analyzing linguistic properties of self-disclosed communication on social media regarding substance use problems provides a novel opportunity to identify communication themes and unmet needs among people with substance use problems [39].”

“…a billion users, over 2.5 billion active users of social media today, and with expanding penetration in high, medium, and low-income countries [7], these technologies can be ORIGINAL RESEARCH page 2 of 11 TBM construed to be a key element of any technology-facilitated mental health solution [8].”

“Reviewed studies covered a range of mental health topics, including predicting depression diagnosis [8], assessing suicide risk [16, 18, 24, 74-76, 98, 99], and developing a better understanding of users’ experiences of eating disorders [15], schizophrenia [59, 61], grief processes between gang-involved youth [58], re…”

“Anxiety and depression are having similar symptoms with features of depressive disorder [14].”

“We cannot claim clinical validity to our assessments, and building upon prior work 40 is a direction to evaluate in the future.”

“Social media data accompanied by formal clinical diagnoses are needed to provide a definitive identification of individuals with schizophrenia and their linguistic markers that will lead to early detection and diagnosis of schizophrenia [7].”

“These studies aimed to identify health problems, symptoms, and treatments, 14,22,50,61,63– 65,68,70,78,94,101,107,116,119,123 as well as classify users into treatment vs control groups (eg, users with or without mental illness) 5,48,58,62,66,70,71,79,85,92,93,100,102,114,119,122,125 .”

“Next, we adopt the supervised machine learning methodology developed by Birnbaum et al. (2017).”

“There are also traditional studies on brain shape and volume symmetry [146], signs of negative symptoms [147,148] and behavioural anomalies [149,150] as well as novel means of detecting by tracking keywords used on social media [151–153] or upbringing [154].”

“Most recently, a study found that Twitter users who self-disclosed diagnoses of schizophrenia showed significant linguistic differences in their online communications when compared to a randomly selected group of control Twitter users [25].”

“In terms of the balance between cases and controls in the data sets, 2 main approaches were to intentionally balance cases and controls [58-68], or to use the chosen criteria to find the “naturally occurring” number of cases from their data set [6,53,55,64,69-71].”

“…communication on Twitter among users who self-identify as having schizophrenia, as reflected by linguistic differences, compared to control users [17,18]; changes toward more positive sentiment following self-disclosure of mental illness on social media [19]; and greater use of conversation…”

“Social media has been used to study a wide variety of health-related outcomes including depression 4 , stress 17 , and schizophrenia 23 .”

“Such queries have been used in multiple prior works to identify self-reported postings about diagnosis or experience of a mental illness or its symptoms [12, 21, 56].”

“Previous research has demonstrated that we can computationally extract behavioral and linguistic patterns from social media posts of patients with schizophrenia [17,29].”

“Birnbaum et al. (2017) also analyzed linguistic markers of SZ in Twitter data, and built a classifier to distinguish users with SZ from healthy controls.”

“Although computerized machine-learning methods have been tested to automatically identify and classify topics in medical research in social media [56], we counted on the clinical expertise of the raters in Mental Health, which constitutes a qualitative advantage in relation to these automatized…”

“platforms to automatically identify individuals who may be experiencing mental illness (13,14).”

“Other studies used a machine learning approach to predict cognitive function in schizophrenia [46], identify people with schizophrenia through social networks [47,48], and develop a predictive model to identify violence in patients with schizophrenia [49].”

“Similar methods have been employed successfully in psychiatry using digital data extracted from social media sites [26-33].”

“However, similar to the studies discussed previously [39,41], the inability to be able to verify the diagnoses of the users with schizophrenia was cited as a main limitation.”

“2014), shown in figure 2, and that employed expert-generated clinical appraisals from clinical psychologists and psychiatrists to assess and curate the quality of online data related to schizophrenia and psychosis (Birnbaum et al. 2017).”

“Another method is to mine the shared content on social media to establish ground truth, for example searching the user content to ind speciic expressions, such as łI was diagnosed with depressionž [16], or searching for self-report anti-depression medication usage evidence.”

“…— it is also elevated in people with obsessive-compulsive disorder 30,31 , anxiety 30,31 , eating disorders 11 – 13,30, 33 , and schizophrenia 15, 16, As discussed in a recent review 36 , this is a common approach and has been applied to study language-use on more generic social media outlets…”

“Birnbaum et al. (2017) used machine learning along with clinical appraisals to identify markers of schizophrenia in social media.”

“In a study evaluating the linguistic characteristics of people with schizophrenia on the Twitter social network, it was found that people diagnosed with schizophrenia more often used interpersonal pronouns in their texts and were more likely to put less emphasis on friendship and more on biological needs [36].”

“The most ideal datasets are curated by the third scheme, which introduces the experts’ examination [5] or incorporate electronic healthcare records [21], but its effort- and time-consuming nature limit its scale, diversity, and accessibility [22].”

“For example, social media has been shown to be a potentially excellent resource for conducting large-scale mental health surveillance, 19,53,54 and our methods can be used to derive gender-specific insights from such surveillance tasks.”

“These metrics align relatively well with diagnostic metrics in similar work involving social media [17, 83], thus providing a sufficient degree of confidence about our methodological approach.”

“There are several studies that cover a range of mental health topics including predicting depression diagnosis [8], assessing suicide risk [9,10], schizophrenia [11,12], stress [13] and obsessivecompulsive disorder [14].”

“The area has now expanded to many other disorders, such as post-traumatic stress disorder [23], schizophrenia [6, 63], eating disorders [19], and anxiety [91].”

“In contrast, supervised ML can result in higher precision than keyword lists on social media data (Birnbaum et al. 2017) and find patterns that are more generalizable and robust (Zhang, Culbertson, and Paritosh 2017).”

“Individuals who generate online content, such as tweets and blog posts, are often unaware that their contributions may be used in research endeavors, particularly concerning mental health studies [122].”

“A classifier that identified users with schizophrenia with an 88% accuracy using linguistic features was created by combining clinician assessments with machine learning to separate genuine disclosures from noisy data [43].”

“The first category of this research includes computational and predictive approaches using social media data to infer peoples’ mental health states, targeting different conditions [15, 27, 35, 134], platforms [27, 141], and disciplines [16, 17, 34, 72].”

“th inform diagnosis and examine the impact of disease. Internet users with various mental illnesses can be characterized by their social media use, text generation, and other online behaviours (e.g., [33, 34, 35]). Social media analysis can also serve to inform our understanding of the sometimes-difficult changes in sel…”

“To further address the issue of noise in online self-disclosure data, Birnbaum et al. [99] adopted a collaborative approach that combined RF with expert clinical appraisals on Twitter data, successfully improving classification accuracy by identifying differences in personal pronoun usage and…”

“Applying computational approaches on digital traces from social technology platforms, researchers have inferred peoples’ mental health, targeting different conditions [12, 25, 33, 106], platforms [25, 37, 113], and disciplines [13, 15, 32, 59].”

“Recently, the need for individuals with schizophrenia to engage themselves in Web-based activities was demonstrated [23].”

“GetOldTweets, a Python package widely used in social media research, was used to identify 25,558,846 alcohol-hashtagged tweets posted throughout 10 years (between 2010 and 2020) containing at least one significant alcohol-related hashtag [55,86,87].”

“A model based on social media behavior has been recently studied in research to identify depression [5], [13], [32].”

“There is one study on empathic response (Vaughn et al. 2018), and two studies focus on health-related issue in which one of them focus on collaboration between patients (Fernandez et al. 2017), and the other focus on collaboration between doctors and software agent (Birnbaum et al. 2017).”

“LIWC has been shown to be very effective in analysis of social media posts related to mental health [19, 14, 3].”

“Technological advances, such as smart phones, web media, digital wearables, and neuroimaging, have enabled health practitioners, researchers and doctors to acquire a large ensemble of data at an increasing pace [ 3,12,19,21 ].”

“Most previous works introduced in the last section extract their features using NLP modules such as the natural language toolkit [32,35], the linguistic inquiry and word count package [36], or even older methods like the n-gram [31].”

“Posts made by individuals suspected of suffering from psychosis were used to evaluate the nature and impact of COVID-19 on subjective well-being 24 or to study their sleep behavior 20 .”

“We join prior work in evaluating training data’s impacts on ML classifiers’ results and, subsequently, on our understanding of ED communities.”

“…efforts have been made in predicting mental health for things other than stress using social media data, such as detection and prediction of depression [68], [69], [70], [71], [72], [73], [74], [75], suicide ideation [76], [77], [78], [79], [80], schizophrenia [81], [82], [83], [84], and so on.”

“For example, self-disclosures based on specific disease related terms or statements that have been mentioned by platform users (21, 23, 40) or affiliation with mental health related communities (29, 30).”

“…be provided by trained human annotators (Wang et al., 2016b; Coppersmith et al., 2015), annotators with clearly referenced domain expertise (e.g., Birnbaum et al. (2017)’s work employing a clinical psychiatrist and a graduate student from Northwell Health’s Early Treatment Program), user…”

“Adolescents and young adults with mental health challenges are among the most frequent users of social media (Abdel-Baki et al., 2017; Birnbaum et al., 2017; Gayer-Anderson and Morgan, 2013), highlighting the relevance of mental health – specific platforms tailored to this demographic.”

“Secondly, this is a novel approach, as previous research has focused primarily on identifying psychopathology rather than examining the community response to it (Ahmed et al. 2022; Birnbaum et al. 2017; Fodeh et al. 2019; Guntuku et al. 2017; Joshi and Kanoongo 2022; Lejeune et al. 2022).”

“In relation to problems of substance abuse, social media platforms such as Instagram and Twitter provide a multitude of data formats to characterize, in the form of both text and images posted by users on social media platforms [22].”