Mobile Sensing and Support for People With Depression: A Pilot Trial in the Wild
Open accessAbstract
BACKGROUND: Depression is a burdensome, recurring mental health disorder with high prevalence. Even in developed countries, patients have to wait for several months to receive treatment. In many parts of the world there is only one mental health professional for over 200 people. Smartphones are ubiquitous and have a large complement of sensors that can potentially be useful in monitoring behavioral patterns that might be indicative of depressive symptoms and providing context-sensitive intervention support. OBJECTIVE: The objective of this study is 2-fold, first to explore the detection of daily-life behavior based on sensor information to identify subjects with a clinically meaningful depression level, second to explore the potential of context sensitive intervention delivery to provide in-situ support for people with depressive symptoms. METHODS: A total of 126 adults (age 20-57) were recruited to use the smartphone app Mobile Sensing and Support (MOSS), collecting context-sensitive sensor information and providing just-in-time interventions derived from cognitive behavior therapy. Real-time learning-systems were deployed to adapt to each subject's preferences to optimize recommendations with respect to time, location, and personal preference. Biweekly, participants were asked to complete a self-reported depression survey (PHQ-9) to track symptom progression. Wilcoxon tests were conducted to compare scores before and after intervention. Correlation analysis was used to test the relationship between adherence and change in PHQ-9. One hundred twenty features were constructed based on smartphone usage and sensors including accelerometer, Wifi, and global positioning systems (GPS). Machine-learning models used these features to infer behavior and context for PHQ-9 level prediction and tailored intervention delivery. RESULTS: A total of 36 subjects used MOSS for ≥2 weeks. For subjects with clinical depression (PHQ-9≥11) at baseline and adherence ≥8 weeks (n=12), a significant drop in PHQ-9 was observed (P=.01). This group showed a negative trend between adherence and change in PHQ-9 scores (rho=-.498, P=.099). Binary classification performance for biweekly PHQ-9 samples (n=143), with a cutoff of PHQ-9≥11, based on Random Forest and Support Vector Machine leave-one-out cross validation resulted in 60.1% and 59.1% accuracy, respectively. CONCLUSIONS: Proxies for social and physical behavior derived from smartphone sensor data was successfully deployed to deliver context-sensitive and personalized interventions to people with depressive symptoms. Subjects who used the app for an extended period of time showed significant reduction in self-reported symptom severity. Nonlinear classification models trained on features extracted from smartphone sensor data including Wifi, accelerometer, GPS, and phone use, demonstrated a proof of concept for the detection of depression superior to random classification. While findings of effectiveness must be reproduced in a RCT to proof causation, they pave the way for a new generation of digital health interventions leveraging smartphone sensors to provide context sensitive information for in-situ support and unobtrusive monitoring of critical mental health states.
AI snapshot
Written from the paper’s full text.
TL;DR
This pilot trial deployed the MOSS smartphone app to provide context-sensitive CBT-based interventions and passively infer depression level from sensor data in adults with depressive symptoms. Among clinically depressed participants who used the app for ≥8 weeks, PHQ-9 scores dropped significantly, and machine-learning models distinguished PHQ-9≥11 from PHQ-9≤10 with modest accuracy.
Key findings
- Of 126 recruited adults, 36 used MOSS for ≥2 weeks; among those with baseline PHQ-9≥11 and adherence ≥8 weeks (n=12), median PHQ-9 fell from 13.00 at baseline to 10.00 at t8, with a significant Wilcoxon signed-rank result (P=.01).
- In that same clinically depressed group, adherence was negatively correlated with change in PHQ-9 (rho=−.498, P=.099).
- Binary classification of 143 biweekly PHQ-9 samples using a PHQ-9≥11 cutoff achieved 60.1% accuracy for Random Forest and 59.1% for SVM in the abstract; Table 4 reports 61.5% accuracy for Random Forest (ntrees=450) and 59.4% for SVM, with SVM sensitivity 72.5 and specificity 47.3, and Random Forest sensitivity 62.3 and specificity 60.8.
- Participants with PHQ-9<11 at baseline and ≥4 weeks of use (n=8) did not show a significant difference between t0 and t4 (P=.22).
Methods
- Monocentric, single-arm clinical pilot study conducted in Switzerland from January to September 2015; 126 adults aged 20–57 were recruited from the general public, with rolling enrollment and electronic informed consent.
- Participants used the MOSS app, which collected smartphone sensor and usage data (accelerometer, WiFi, GPS, phone use, calls, texts, calendar events) and delivered CBT-derived interventions; PHQ-9 was collected biweekly.
- Analyses included Wilcoxon signed-rank tests for PHQ-9 change, Spearman correlation between MOSS sessions and PHQ-9 change, and supervised machine learning using 120 features with SVM and Random Forest under leave-one-out cross-validation for PHQ-9≥11 vs ≤10 classification.
Limitations
- The paper states the study was nonrandomized, uncontrolled, and single-arm, so it cannot prove a causal link between symptom improvement and MOSS use; it also did not collect control variables such as other current treatments.
- The paper notes PHQ-9 is strongly correlated with depression but not all elevated PHQ-9 subjects are certain to have depression, and the pilot did not quantify the efficacy of the recommendation algorithm.
- Evident from the text: dropout was high (64/126, 50.8%, uninstalled within 2 weeks; only 36 used MOSS ≥2 weeks), the clinical symptom-change analysis included only 12 subjects, and classification accuracy was modest.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 27 cited sources checked · 0 supported · 0 flagged
- Findingown data
In the group with clinical depression at baseline and adherence ≥8 weeks, there was a negative trend between adherence and change in PHQ-9 scores (rho=−.498, P=.099).
“This group showed a negative trend between adherence and change in PHQ-9 scores (rho=−.498, P=.099).”
Quote in the paperNot a substantial claim - Findingown data
Binary classification of biweekly PHQ-9 samples (n=143) with a cutoff of PHQ-9≥11 using Random Forest and Support Vector Machine leave-one-out cross validation achieved 60.1% and 59.1% accuracy, respectively.
“Binary classification performance for biweekly PHQ-9 samples (n=143), with a cutoff of PHQ-9≥11, based on Random Forest and Support Vector Machine leave-one-out cross validation resulted in 60.1% and 59.1% accuracy, respectively.”
Quote in the paperNot a substantial claim - Findingown data
Among subjects with clinical depression (PHQ-9≥11) at baseline and adherence ≥8 weeks (n=12), PHQ-9 scores dropped significantly from before to after intervention (P=.01).
“For subjects with clinical depression (PHQ-9≥11) at baseline and adherence ≥8 weeks (n=12), a significant drop in PHQ-9 was observed (P=.01).”
- Also stated in: § Conclusions · plain
“Subjects who used the app for an extended period of time showed significant reduction in self-reported symptom severity.”
Quote in the paperSubstantial claim - Also stated in: § Conclusions · plain
- Findingown data
A total of 126 subjects were recruited from the general public.
“A total of 126 subjects were recruited from the general public.”
Quote in the paperNot a substantial claim - Findingown data
A Kolmogorov-Smirnov test rejected the normality assumption of PHQ-9 scores.
“A Kolmogorov-Smirnov test [42] rejected the normality assumption”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [42] source
Not checked in the free run.
- Findingown data
The RFC showed the highest accuracy performance with 61.5 at 450 trees.
“The RFC showed the highest accuracy performance with 61.5 at 450 trees in the model (ntrees = 450).”
Quote in the paperNot a substantial claim - Findingown data
The SVM had a sensitivity of 72.5.
“The SVM favored sensitivity over specificity leading to a higher sensitivity score of 72.5 compared with the RFC at 62.3”
Quote in the paperNot a substantial claim - Findingown data
The SVM performed with an average accuracy of 59.4.
“The SVM performed slightly worse with an average accuracy of 59.4.”
Quote in the paperNot a substantial claim - Findingown data
For subjects with PHQ-9<11 at baseline and extended use of at least 4 weeks (n=8), there was no significant difference between t0 and t4 (P=.22).
“Subject’s with a PHQ-9<11 at baseline and an extended time of use of at least 4 weeks (n=8) did not show significant difference between t0 and t4. (P=.22)”
Quote in the paperNot a substantial claim - Findingown data
26/126 (20.6%) subjects uninstalled the app in the following 2 weeks.
“Another 20.6% subjects (26/126) uninstalled the app in the following 2 weeks.”
Quote in the paperNot a substantial claim - Findingown data
The majority of PHQ-9 samples represent a value close to the classification threshold for clinical depression of 11.
“The distribution shows, that the majority of samples represents a PHQ-9 value close to the classification threshold for clinical depression of 11.”
Quote in the paperNot a substantial claim - Findingown data
The SVM had a specificity of 47.3.
“whereas the RFC has a higher specificity score of 60.8 compared with 47.3.”
Quote in the paperNot a substantial claim - Findingown data
28/126 (22.2%) subjects had an adherence of 4 weeks or longer providing at least 2 PHQ-9 measures in addition to baseline.
“Approximately one-fifth of the subjects (28/126, 22.2%) had an adherence of 4 weeks or longer providing at least 2 PHQ-9 measures in addition to the baseline measure (male = 10, female = 18).”
Quote in the paperNot a substantial claim - Findingown data
64/126 (50.8%) subjects uninstalled the MOSS app within the first 2 weeks.
“A large portion of subjects uninstalled the MOSS app within the first 2 weeks (64/126, 50.8%).”
Quote in the paperSubstantial claim - Findingown data
Twelve subjects met the criteria of being clinically depressed at first use and having adherence of at least 4 weeks with 2 PHQ-9 values in addition to baseline.
“Twelve subjects met these criteria, where all of these had an adherence of 8 weeks or longer.”
Quote in the paperSubstantial claim - Findingown data
The RFC had a specificity of 60.8.
“the RFC has a higher specificity score of 60.8”
Quote in the paperNot a substantial claim - Findingown data
The RFC had a sensitivity of 62.3.
“compared with the RFC at 62.3”
Quote in the paperNot a substantial claim - Interpretationown data
Proxies for social and physical behavior derived from smartphone sensor data were successfully deployed to deliver context-sensitive and personalized interventions to people with depressive symptoms.
“Proxies for social and physical behavior derived from smartphone sensor data was successfully deployed to deliver context-sensitive and personalized interventions to people with depressive symptoms.”
Quote in the paperSubstantial claim - Study design and conductown data
A total of 36 subjects used MOSS for ≥2 weeks.
“A total of 36 subjects used MOSS for ≥2 weeks.”
Quote in the paperNot a substantial claim - Interpretationown data
Nonlinear classification models trained on features extracted from smartphone sensor data including Wifi, accelerometer, GPS, and phone use demonstrated a proof of concept for the detection of depression superior to random classification.
“Nonlinear classification models trained on features extracted from smartphone sensor data including Wifi, accelerometer, GPS, and phone use, demonstrated a proof of concept for the detection of depression superior to random classification.”
Quote in the paperSubstantial claim - Premisecitation
A recent review revealed an array of shortcomings still present in most digital CBT approaches, for example, the lack of personalization and missing in-situ support.
“However, a recent review revealed an array of shortcomings still present in most of the approaches, for example, the lack of personalization and missing in-situ support [8].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [8] WahleFKowatschTTowards the design of evidence-based mental health information systems: a preliminary literature review2014Proceedings of the 35th International Conference on Information SystemsDec 12-14, 2014Auckland, NZ
Not checked in the free run.
- Interpretationown data
The study successfully demonstrated a first proof of concept for the detection of clinically relevant PHQ-9 levels using nonlinear models on features extracted from smartphone sensor data.
“Complementary to the work of Saeb et al [17], we could successfully demonstrate a first proof of concept for the detection of clinically relevant PHQ-9 levels using nonlinear models on features extracted from smartphone sensor data.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [17] source
Not checked in the free run.
- Interpretationown data
Two different supervised nonlinear machine learning models trained on multiple features from collected sensor data were able to distinguish between subjects above and below a clinically relevant PHQ-9 score with comparable accuracy exceeding the performance of a random binary classifier.
“Two different, supervised, nonlinear machine learning models trained on multiple features calculated from collected sensor data, were able to distinguish between subjects above and below a clinically relevant PHQ-9 score with comparable accuracy exceeding the performance of a random binary classifier.”
Quote in the paperSubstantial claim - Interpretationcitation
Retention rate was above average retention rate of android apps.
“retention rate was above average retention rate of android apps [54].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [54] ChenANew Data Shows Losing 80% of Mobile Users is Normal, and Why the Best Apps Do Better20152016-05-09http://andrewchen.co/new-data-shows-why-losing-80-of-your-mobile-users-is-normal-and-that-the-best-apps-do-much-better/
Not checked in the free run.
- Premisecitation
Asselbergs et al were able to resemble a subject's day-to-day mood level solely based on passively collected data provided by smartphones with 55% to 76% accuracy.
“In another recent explorative study, Asselbergs et al [21] were able to resemble a subject’s day-to-day mood level solely based on passively collected data provided by the smartphones with 55% to 76% accuracy.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [21] source
Not checked in the free run.
- Study design and conductreasoning
Subjects were not asked to provide information about relevant control variables such as other current treatments to rule out their impact on treatment outcome.
“Additionally, to lower the inhibition threshold, subjects were not asked to provide information about relevant control variables such as other current treatments to rule out their impact on treatment outcome.”
Quote in the paperSubstantial claim - Study design and conductnone given
A set of 80 interventions including social, relaxation, thoughtfulness, and physical activity exercises were designed and implemented following best practice in CBT.
“For MOSS, a set of 80 interventions including social, relaxation, thoughtfulness, and physical activity exercises were designed and implemented following best practice in CBT.”
Quote in the paperNot a substantial claim - Interpretationnone given
Magnitude of behavior proxies over time periods of 24 hours in comparison to assumptions about healthy behavior were successfully used to dynamically provide meaningful interventions to support people with depressive symptoms in their everyday life.
“Magnitude of behavior proxies over time periods of 24 hours in comparison to assumptions about healthy behavior were successfully used to dynamically provide meaningful interventions to support people with depressive symptoms in their everyday life.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The system compares behavior proxies to assumptions about healthy behavior.
- Premisecitation
Canzian et al showed a tendency of correlation between a range of GPS metrics similar to Saeb et al and self-reported depression scores.
“Canzian et al [18] were able to show a tendency of correlation between a range of GPS metrics similar to the ones presented by Saeb et al [17] and a self-reported depression scores.”
Quote in the paperNot a substantial claim0 of 2 sources checked - Premisecitation
CBT is suitable for implementation in digital health interventions.
“Moreover, and with regard to its structure, it is suitable for implementation in digital health interventions [37-40].”
Quote in the paperSubstantial claim0 of 4 sources checked- cites [37] source
Not checked in the free run.
- cites [38] Kraft, P, 2009, Communications of the Association for Information Systems source
Not checked in the free run.
- cites [39] Martin, L, 2010, Health Behavior Change and Treatment Adherence: Evidence-Based Guidelines for Improving Healthcare
Not checked in the free run.
- cites [40] source
Not checked in the free run.
- Study design and conductreasoning
In this first pilot, the efficacy of the proposed recommendation algorithm was not quantified.
“Moreover, in this first pilot we did not quantify the efficacy of the proposed recommendation algorithm, as this would involve detailed feedback from participants in order to judge appropriateness of context-related intervention recommendations.”
Quote in the paperNot a substantial claim
- The quote only partly states this claim. 2
- Classified as probably not a substantial claim. 20
- May combine two claims. 9
The quote only partly states this claim.
Findingown dataThere is a negative correlation between cumulated change in PHQ-9 and total number of MOSS app uses between t0 and tend, with rho=-.498 and P=.099 for the 12 clinically depressed subjects with adherence ≥4 weeks.
“The scatter plots indicates a negative correlation between cumulated change in PHQ-9 and the total number of MOSS app uses between t0 and tend.”
- Also stated in: § Relationship of MOSS Usage and Symptom Reduction · plain
“We observed a negative correlation with rho=-.498 and P=.099.”
- Also stated in: § Principal Findings · hedged
“Among these participants, the relation between frequency of MOSS app usage and change in PHQ-9 scores showed a negative trend.”
Quote in the paperNot a substantial claim- Also stated in: § Relationship of MOSS Usage and Symptom Reduction · plain
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataFigure 9 shows the sample distribution of 143 PHQ-9 samples of 36 subjects with adherence of at least 2 weeks.
“Figure 9 shows the sample distribution of the 143 PHQ-9 samples of the 36 subjects with an adherence of at least 2 weeks collected during the trial.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Findingown dataSpearman correlation was used because both distributions deviated from normality (P<.001, Kolmogorov-Smirnov test).
“Spearman correlation was used because both distributions deviated from normality (P<.001, Kolmogorov-Smirnov test).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
FindingreasoningThe majority of participants came from Switzerland and Germany.
“Because the study was primarily advertised in Switzerland and on German-speaking Internet forums, the majority of participants came from Switzerland and Germany.”
Quote in the paperNot a substantial claimThe quote only partly states this claim.
Findingown dataFor participants with a clinically relevant PHQ-9 score and extended MOSS app adherence, a significant drop in PHQ-9 was observed at t=6 and t=8 compared to t0.
“At t=6 and t=8 we observe a significant difference in means.”
- Also stated in: § Principal Findings · plain
“For participants with a clinically relevant PHQ-9 score and an extended MOSS app adherence, a significant drop in PHQ-9 was observed.”
Quote in the paperSubstantial claim- Also stated in: § Principal Findings · plain
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationCognitive-behavioral therapy (CBT) has been proven to be effective for depression.
“Traditionally, depression is treated with medication and/or face to face psychotherapy using methods such as cognitive-behavioral therapy (CBT), which has been proven to be effective [4].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [4] source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationMany digital versions of CBT presented reasonable effect sizes, sometimes even on a par with face to face therapy.
“Many of these solutions presented reasonable effects sizes [6], sometimes even on a par with face to face therapy [7].”
Quote in the paperNot a substantial claim0 of 2 sources checkedMay combine two claims.
PremisecitationDuration and time of the day of stays at different physical locations are related to a person's mental state and can be approximated by WiFi and global positioning systems (GPS) information.
“duration, and time of the day of stays at different physical locations were shown to be related to a person’s mental state and can be approximated by WiFi and global positioning systems (GPS) information [17,18].”
Quote in the paperSubstantial claim0 of 2 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationDepression is associated with worse global outcomes for the affected individual, including reduced social functioning, lower quality of life in regards to health, inability to return to work, as well as suicide.
“Beyond its burden on society, depression is associated with worse global outcomes for the affected individual, including reduced social functioning, lower quality of life in regards to health, inability to return to work, as well as suicide [3].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [3] Holden, C, 2000, Science source
Not checked in the free run.
Classified as probably not a substantial claim.
Study design and conductcitationLeave-one-out cross validation was conducted for unbiased performance estimation of both classifiers.
“For unbiased performance estimation of both classifiers, leave-one-out cross validation was conducted [51].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [51] Kearns, M, 1999, Neural Comput source
Not checked in the free run.
May combine two claims.
PremisecitationSocial activity is highly related to a subject's mental state and the risk of developing depression.
“It is highly related to a subject’s mental state and the risk of developing depression [19,20].”
Quote in the paperSubstantial claim0 of 2 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe study was conducted in full accordance with the Declaration of Helsinki.
“It was conducted in full accordance with the Declaration of Helsinki”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoningNot all subjects with an elevated PHQ-9 are certain to have a depression.
“not all subjects with an elevated PHQ-9 are certain to have a depression.”
Quote in the paperNot a substantial claim- Rests on an assumption: The PHQ-9 is strongly correlated with depression.
Classified as probably not a substantial claim.
Study design and conductcitationThe average of leave-one-out test scores is reported as the unbiased performance estimate.
“The average of these scores is reported as the unbiased performance estimate [52].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [52] KohaviRA study of cross-validation and bootstrap for accuracy estimation and model selectionProceedings of the 14th international joint conference on Artificial intelligence1995International joint conference on Artificial intelligenceAugust 20, 1995Montreal, CannadaSan FranciscoKaufmann Publishers Inc
Not checked in the free run.
Classified as probably not a substantial claim.
PremisecitationThe standard deviation of the 3D acceleration norm resembles intensity of physical activity of 2 widely used commercial acceleration-based activity trackers with reasonable consensus.
“A recent study showed that the standard deviation of the 3D acceleration norm resembles intensity of physical activity of 2 widely used commercial acceleration-based activity trackers with reasonable consensus [29].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [29] source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenTwo clinically trained psychologists predefined rules to prevent the MOSS app making unreasonable intervention recommendations.
“In addition, 2 clinically trained psychologists predefined rules to prevent the MOSS app making unreasonable intervention recommendations.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationCBT is often regarded as the mental health intervention of choice due to its large evidence on a variety of psychological disorders.
“CBT is often regarded as the mental health intervention of choice due to its large evidence on a variety of psychological disorders [4].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [4] source
Not checked in the free run.
Classified as probably not a substantial claim.
Study design and conductown dataAll subjects provided electronic informed consent prior to participation.
“with all subjects providing their electronic informed consent prior to participation.”
Quote in the paperSubstantial claimMay combine two claims.
Study design and conductcitationThe walking detection approach is robust to variance in the orientation of the smartphone and different walking speeds.
“We found, that this approach is robust to variance in the orientation of the smartphone was confirmed by Kunze and Lukowicz [30] and different walking speeds.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [30] source
Not checked in the free run.
May combine two claims.
Study design and conductreasoningThe clinical study is based on a nonrandomized, uncontrolled single-arm study design, which rules out the possibility to prove a direct causal link between symptom improvement and MOSS app use.
“Even though causation cannot be tested with the study design”
- Also stated in: § Limitations · plain
“The clinical study carried out is based on a nonrandomized, uncontrolled single-arm study design, which rules out the possibility to prove a direct causal link between symptom improvement and MOSS app use.”
Quote in the paperSubstantial claim- Also stated in: § Limitations · plain
May combine two claims.
PremisecitationSaeb et al used GPS sensors and phone use statistics to distinguish people without (PHQ-9<5) from people with signs of depression (PHQ-9≥5) in a lab experiment over 2 weeks with high accuracy.
“Saeb et al [17] used the information of GPS sensors and phone use statistics to distinguish people without (Patient Health Questionnaire, PHQ-9, <5) from people with signs of depression (PHQ-9≥5) in a lab experiment over 2 weeks with high accuracy.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [17] source
Not checked in the free run.
Classified as probably not a substantial claim.
PremisecitationA PHQ-9 cut off value of 11 separates people with (≥11) from people without (≤10) a clinically relevant depression level.
“we chose a PHQ-9 cut off value of 11, in line with the PHQ-9 [22] to separate people with (≥11) from people without (≤10) a clinically relevant depression level.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [22] source
Not checked in the free run.
Classified as probably not a substantial claim.
Study design and conductreasoningThe 2-minute time window for computing standard deviation of 3D acceleration norm was appropriate for the app's needs.
“For this trial, we used a time window of 2 minutes. As we did not aim at classifying micro movement, this window size was appropriate for our app needs and trades of phone memory usage and frequency of computation and information gain.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
PremisecitationCBT is a highly-structured psychological treatment.
“CBT is a highly-structured psychological treatment [35].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [35] source
Not checked in the free run.
Classified as probably not a substantial claim. May combine two claims.
InterpretationreasoningAfter 2 weeks, the MOSS app adapts to the subject's actual behavior and does not suffer from potentially flawed assumptions about a subject's behavior with respect to the general population.
“This way, the MOSS app does not suffer from potentially flawed assumptions about a subject’s behavior with respect to the general population and adapts to the subject’s actual behavior.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThe study was approved by the local ethics committee of the Canton of Zurich and the Swiss Agency for Therapeutic Products.
“The study was approved by the local ethics committee of the Canton of Zurich in Switzerland and the Swiss Agency for Therapeutic Products.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationnone givenTo our best knowledge, this study presents the first trial of a context sensitive smartphone app to support people with depressive symptoms under real life conditions.
“To our best knowledge, this study presents the first trial of a context sensitive smartphone app to support people with depressive symptoms under real life conditions.”
Quote in the paperNot a substantial claim
The authors state this as an assumption.
People with depression have characteristic daily behaviors (e.g., time spent at home, number of calls, walking time) that can be used to tailor interventions.
“In the first phase it delivered interventions based on assumptions about the behavior of the general depressed population and handcrafted weights for appropriate interventions to be delivered depending on the characteristics of the context”
“In Table 1, example assumptions about the general depressed population and the characteristics of a subset of context features are presented.”
The authors state this as an assumption.
Past ratings of an intervention represent the preference for an intervention and therefore should have the highest impact on the scoring function.
“The values follow the assumption that past ratings of an intervention represent the preference for an intervention and therefore should have the highest impact on the scoring function.”
The authors state this as an assumption.
The decision to cancel an intervention early is not necessarily related to a subject's general liking of the intervention.
“The decision to cancel an intervention early, is not necessarily related to a subject’s general liking of the intervention, therefore the impact is significantly lower than past ratings.”
The paper relies on this without support.
Smartphones offer numerous sources of information acting as proxies for social activities such as the frequency and average duration of calls, or the number of different persons being contacted.
“Smartphones offer numerous sources of information acting as proxies for social activities such as the frequency and average duration of calls, or the number of different persons being contacted.”
The authors state this as an assumption.
CBT is based on the assumption that thoughts determine how one feels, behaves, and physically reacts.
“It is based on the assumption that thoughts determine how one feels, behaves, and physically reacts.”
The authors state this as an assumption.
Changing maladaptive thinking leads to change in affect and behavior.
“This form of intervention contains various treatments using cognitive and behavioral techniques with the assumption that changing maladaptive thinking leads to change in affect and behavior.”
The authors state this as an assumption.
The system compares behavior proxies to assumptions about healthy behavior.
“in comparison to assumptions about healthy behavior”
The paper relies on this without support.
The PHQ-9 is strongly correlated with depression.
“although research has shown that the PHQ-9 is strongly correlated with depression”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Qualitative flaws in the before-after studies were mostly missing power or effect size.
“Qualitative flaws were mostly found in missing power or effect size [77,83] and no blinding of assessors [77,82,83].”
From Using Artificial Intelligence to Enhance Ongoing Psychological Interventions for Emotional Problems in Real- or Close to Real-Time: A Systematic Review (2022) · cites this paper as [83]
Two of the before-after studies suffered from potential bias in their loss to follow-up.
“Additionally, two of the studies suffered from potential bias in their loss to follow-up [82,83].”
From Using Artificial Intelligence to Enhance Ongoing Psychological Interventions for Emotional Problems in Real- or Close to Real-Time: A Systematic Review (2022) · cites this paper as [83]
Higher degrees of depression are linked to reduced contact with one's social network, shorter text messages, reduced duration of calls, and less variability in geospatial activity.
“Higher degrees of depression are linked to reduced contact with one’s social network, shorter text messages, reduced duration of calls, and less variability in geospatial activity [6-9].”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Classification accuracies differ between the abstract and results
MajorThe abstract reports 60.1% accuracy for Random Forest and 59.1% for Support Vector Machine, whereas the results report 61.5 and 59.4, respectively.
“Binary classification performance for biweekly PHQ-9 samples (n=143), with a cutoff of PHQ-9≥11, based on Random Forest and Support Vector Machine leave-one-out cross validation resulted in 60.1% and 59.1% accuracy, respectively.”
“The RFC showed the highest accuracy performance with 61.5 at 450 trees in the model (ntrees = 450). The SVM performed slightly worse with an average accuracy of 59.4.”
Random Forest evaluation method is reported inconsistently
MinorThe methods say leave-one-out cross-validation scores were reported for both classifiers, but the results describe the reported Random Forest performance as out-of-bag performance.
“For unbiased performance estimation of both classifiers, leave-one-out cross validation was conducted [51].”
“Table 4 shows the average SVM cross-validation score and the RFC out of bag performance with respect to a binary classification of samples with a PHQ-9≥11 and PHQ-9≤10.”
The citation lists haven’t been fetched yet
References 44 · Cited by 333
Research works printed but not linked: 6
- WahleFKowatschTTowards the design of evidence-based mental health information systems: a preliminary literature review2014Proceedings of the 35th International Conference on Information SystemsDec 12-14, 2014Auckland, NZ
- Kotsiantis, SB, 2007, Emerging Artificial Intelligence Applications in Computer Engineering: Real Word AI Systems with Applications in eHealth, HCI, Information Retrieval ... in Artificial Intelligence and Applications)
- 10.1016/j.jad.2013.01.035 · DOI · Paper page
- RekimotoJMiyakiTIshizawaTWiFi-based continuous location logging for life pattern analysisLecture Notes in Computer Science2007The 3rd International symposium of Location and Context AwarenessSeptember 20-21, 2007Oberpfaafenhofen, GermanyBerlin HeidelbergSpringer3549
- LovelockHMatthewsRMurphyKAustralian Psychological Association2016-05-09Evidence Based Psychological Interventions in the Treatment of Mental Disorders: A Literature Review https://www.psychology.org.au/Assets/Files/Evidence-Based-Psychological-Interventions.pdf
- KohaviRA study of cross-validation and bootstrap for accuracy estimation and model selectionProceedings of the 14th international joint conference on Artificial intelligence1995International joint conference on Artificial intelligenceAugust 20, 1995Montreal, CannadaSan FranciscoKaufmann Publishers Inc
- Authors
- Wahle, F., Kowatsch, T., Fleisch, E., Rufer, M., & Weidt, S.
- Affiliations
- ETH Zurich (CH)
- University of St.Gallen (CH)
- University of Zurich (CH)
- Corresponding author
- Kowatsch, Tobias
- Venue
- JMIR mHealth and uHealth
- Publisher
- JMIR Publications Inc.
- Published
- 2016-09-21
- Volume
- 4
- Issue
- 3
- Pages
- e111
- ISSN
- 2291-5222 (ISSN-L)
- PMC
- PMC5052463
- OpenAlex
- W2414738784
- Cited by
- 333
- References
- 44
- Citations by year
- 2026: 12 · 2025: 19 · 2024: 36 · 2023: 70 · 2022: 57 · 2021: 46 · 2020: 29 · 2019: 29 · 2018: 19 · 2017: 14 · 2016: 2
- FWCI
- 32.87
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR mhealth and uhealth (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Repository for Publications and Research Data (ETH Zurich) (submittedVersion, cc-by)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion, cc-by)
- ETH Zürich Research Collection (cc-by)
- Zurich Open Repository and Archive (University of Zurich)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Keywords
- depression, smartphone sensors, mobile sensing, context-aware interventions, cognitive behavioral therapy, PHQ-9, depression detection, GPS, accelerometer, Wi-Fi, personalized interventions, medication adherence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review