Predicting Depression From Smartphone Behavioral Markers Using Machine Learning Methods, Hyperparameter Optimization, and Feature Importance Analysis: Exploratory Study
Open accessPublished version of a preprint
Abstract
Background: Depression is a prevalent mental health challenge. Current depression assessment methods using self-reported and clinician-administered questionnaires have limitations. Instrumenting smartphones to passively and continuously collect moment-by-moment data sets to quantify human behaviors has the potential to augment current depression assessment methods for early diagnosis, scalable, and longitudinal monitoring of depression. Objective: The objective of this study was to investigate the feasibility of predicting depression with human behaviors quantified from smartphone data sets, and to identify behaviors that can influence depression. Methods: Smartphone data sets and self-reported 8-item Patient Health Questionnaire (PHQ-8) depression assessments were collected from 629 participants in an exploratory longitudinal study over an average of 22.1 days (SD 17.90; range 8-86). We quantified 22 regularity, entropy, and SD behavioral markers from the smartphone data. We explored the relationship between the behavioral features and depression using correlation and bivariate linear mixed models (LMMs). We leveraged 5 supervised machine learning (ML) algorithms with hyperparameter optimization, nested cross-validation, and imbalanced data handling to predict depression. Finally, with the permutation importance method, we identified influential behavioral markers in predicting depression. Results: Of the 629 participants from at least 56 countries, 69 (10.97%) were females, 546 (86.8%) were males, and 14 (2.2%) were nonbinary. Participants’ age distribution is as follows: 73/629 (11.6%) were aged between 18 and 24, 204/629 (32.4%) were aged between 25 and 34, 156/629 (24.8%) were aged between 35 and 44, 166/629 (26.4%) were aged between 45 and 64, and 30/629 (4.8%) were aged 65 years and over. Of the 1374 PHQ-8 assessments, 1143 (83.19%) responses were nondepressed scores (PHQ-8 score <10), while 231 (16.81%) were depressed scores (PHQ-8 score ≥10), as identified based on PHQ-8 cut-off. A significant positive Pearson correlation was found between screen status–normalized entropy and depression (r=0.14, P<.001). LMM demonstrates an intraclass correlation of 0.7584 and a significant positive association between screen status–normalized entropy and depression (β=.48, P=.03). The best ML algorithms achieved the following metrics: precision, 85.55%-92.51%; recall, 92.19%-95.56%; F1, 88.73%-94.00%; area under the curve receiver operating characteristic, 94.69%-99.06%; Cohen κ, 86.61%-92.90%; and accuracy, 96.44%-98.14%. Including age group and gender as predictors improved the ML performances. Screen and internet connectivity features were the most influential in predicting depression. Conclusions: Our findings demonstrate that behavioral markers indicative of depression can be unobtrusively identified from smartphone sensors’ data. Traditional assessment of depression can be augmented with behavioral markers from smartphones for depression diagnosis and monitoring.
AI snapshot
Written from the paper’s full text.
TL;DR
This exploratory study used smartphone behavioral markers from 629 Android Carat app participants to predict depression, finding screen and internet connectivity features most influential. Machine learning classifiers achieved high performance, and including age group and gender improved results.
Key findings
- Among 1374 PHQ-8 assessments, 1143 (83.19%) were nondepressed (PHQ-8 <10) and 231 (16.81%) were depressed (PHQ-8 ≥10); mean PHQ-8 score was 5.19 (SD 5.22; range 0-24).
- Screen status–normalized entropy had a significant positive correlation with depression (r=0.14, P<.001) and a significant positive association in LMM (β=.48, P=.03); ICC was 0.7584.
- Best ML algorithms achieved precision 85.55%-92.51%, recall 92.19%-95.56%, F1 88.73%-94.00%, AUC 94.69%-99.06%, Cohen κ 86.61%-92.90%, and accuracy 96.44%-98.14%; including age group and gender improved all performance metrics.
- Screen and internet connectivity features were most influential; top-ranked features included screen regularity index, screen status entropy, internet regularity index, screen status–normalized entropy, and screen off count SD.
Methods
- Longitudinal observational data from the Carat Android app: final sample of 629 participants from at least 56 countries, 13,898 smartphone data days, mean 22.1 days per participant (SD 17.90; range 8-86), with PHQ-8 self-reports every 2 weeks.
- Quantified 22 regularity, entropy, and SD behavioral markers from screen status, internet connectivity, and foreground app usage; used Pearson correlation and bivariate linear mixed models with multiple imputation and multiple-testing adjustment.
- Trained 5 supervised ML classifiers (random forest, SVM with RBF kernel, XGBoost, K-nearest neighbor, logistic regression) with nested stratified cross-validation, grid-search hyperparameter optimization, SMOTE for imbalance, and permutation feature importance; compared against random weighted and decision tree baselines.
Limitations
- Stated: crowdsourced sample was small (N=629) and not representative, biased toward highly educated/professional occupations and males; no clinical diagnosis inclusion criterion, no baseline depression assessment, scarce high depression scores, and unequal repeated assessments across participants.
- Stated: correlations/associations do not imply causality, the screen status–normalized entropy relationship was not strong and may not generalize, and data were Android-only with possible long-term Carat app effects on behavior.
- Evident from text: depression labels relied on self-reported PHQ-8 rather than clinician diagnosis; the sample was male-dominated (546/629, 86.8%) and had class imbalance (16.81% depressed); mean monitoring was only 22.1 days and the feature set was limited to screen, internet, and app data without GPS or wearables.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 80 cited sources checked · 0 supported · 0 flagged
- Findingown data
The best ML algorithms achieved AUC between 94.69% and 99.06%.
“The best ML algorithms achieved the following metrics: precision, 85.55%-92.51%; recall, 92.19%-95.56%; F1, 88.73%-94.00%; area under the curve receiver operating characteristic, 94.69%-99.06%; Cohen κ, 86.61%-92.90%; and accuracy, 96.44%-98.14%.”
Quote in the paperSubstantial claim- Rests on an assumption: The missing data in the data set are missing at random.
- Findingown data
The best ML algorithms achieved Cohen κ between 86.61% and 92.90%.
“The best ML algorithms achieved the following metrics: precision, 85.55%-92.51%; recall, 92.19%-95.56%; F1, 88.73%-94.00%; area under the curve receiver operating characteristic, 94.69%-99.06%; Cohen κ, 86.61%-92.90%; and accuracy, 96.44%-98.14%.”
Quote in the paperSubstantial claim- Rests on an assumption: The missing data in the data set are missing at random.
- Findingown data
A significant positive Pearson correlation was found between screen status-normalized entropy and depression (r=0.14, P<.001).
“We found a significant positive correlation between screen status–normalized entropy and depression (r=0.14, P<.001).”
Quote in the paperSubstantial claim- Rests on an assumption: The missing data in the data set are missing at random.
- Findingown data
The LMM intraclass correlation is 0.7584.
“we found an ICC of 0.7584; thus, 75.84% of the variations in the features are explainable by the interindividual differences.”
Quote in the paperNot a substantial claim- Rests on an assumption: The missing data in the data set are missing at random.
- Findingown data
A significant positive association was found between screen status-normalized entropy and depression (β=.48, P=.03) in the LMM.
“We found a significant positive association between screen status–normalized entropy and depression (β=.48, P=.03).”
Quote in the paperSubstantial claim- Rests on an assumption: The missing data in the data set are missing at random.
- Findingown data
Screen and internet connectivity features were the most influential in predicting depression.
“Screen and internet connectivity features were the most influential in predicting depression.”
Quote in the paperSubstantial claim- Rests on an assumption: The missing data in the data set are missing at random.
- Findingown data
The best ML algorithms achieved precision between 85.55% and 92.51%.
“The best ML algorithms achieved the following metrics: precision, 85.55%-92.51%; recall, 92.19%-95.56%; F1, 88.73%-94.00%; area under the curve receiver operating characteristic, 94.69%-99.06%; Cohen κ, 86.61%-92.90%; and accuracy, 96.44%-98.14%.”
Quote in the paperSubstantial claim- Rests on an assumption: The missing data in the data set are missing at random.
- Findingown data
The best ML algorithms achieved accuracy between 96.44% and 98.14%.
“The best ML algorithms achieved the following metrics: precision, 85.55%-92.51%; recall, 92.19%-95.56%; F1, 88.73%-94.00%; area under the curve receiver operating characteristic, 94.69%-99.06%; Cohen κ, 86.61%-92.90%; and accuracy, 96.44%-98.14%.”
Quote in the paperSubstantial claim- Rests on an assumption: The missing data in the data set are missing at random.
- Findingown data
Including age group and gender as predictors improved the ML performances.
“When age group and gender were included with features as predictors, we observed a general improvement in all performance metrics for all classifiers, as shown in Table 5.”
Quote in the paperNot a substantial claim- Rests on an assumption: The missing data in the data set are missing at random.
- Findingown data
The best ML algorithms achieved F1 between 88.73% and 94.00%.
“The best ML algorithms achieved the following metrics: precision, 85.55%-92.51%; recall, 92.19%-95.56%; F1, 88.73%-94.00%; area under the curve receiver operating characteristic, 94.69%-99.06%; Cohen κ, 86.61%-92.90%; and accuracy, 96.44%-98.14%.”
Quote in the paperSubstantial claim- Rests on an assumption: The missing data in the data set are missing at random.
- Findingown data
The best ML algorithms achieved recall between 92.19% and 95.56%.
“The best ML algorithms achieved the following metrics: precision, 85.55%-92.51%; recall, 92.19%-95.56%; F1, 88.73%-94.00%; area under the curve receiver operating characteristic, 94.69%-99.06%; Cohen κ, 86.61%-92.90%; and accuracy, 96.44%-98.14%.”
Quote in the paperSubstantial claim- Rests on an assumption: The missing data in the data set are missing at random.
- Findingown data
For the XGB classifier, the top 5 most important features in predicting PHQ-8 depression score were the internet regularity index, screen on count, screen regularity index, screen status entropy, and screen off count.
“For the XGB classifier in Figure 1, the top 5 most important features were the internet regularity index, screen on count, screen regularity index, screen status entropy, and the screen off count.”
Quote in the paperNot a substantial claim - Findingnone given
The data set is biased toward males in gender distribution.
“The data set is also biased toward males in gender distribution.”
Quote in the paperNot a substantial claim - Findingown data
App count SD is the least important feature for all classifiers (XGB, RF, and KNN).
“App count SD is the least important feature for all classifiers (Figures 1-3)”
Quote in the paperNot a substantial claim - Findingown data
Tree-based and nearest neighbor-based classifiers had superior performance over linear classifiers, including SVM with RBF kernel.
“Interestingly, tree-based, nearest neighbor–based classifiers had superior performance over linear classifiers, including SVM with RBF kernel”
Quote in the paperSubstantial claim - Findingown data
The predictive performances of ML classifiers improved when demographic characteristics were included among predictors.
“Moreover, we found that the predictive performances of ML classifiers improved when demographic characteristics were included among predictors”
Quote in the paperSubstantial claim - Findingown data
No significant correlation was found between other screen, app, and internet connectivity features and PHQ-8 depression score.
“We found no significant correlation between other screen, app, and internet connectivity features and PHQ-8 depression score.”
Quote in the paperNot a substantial claim- Rests on an assumption: The missing data in the data set are missing at random.
- Findingown data
No significant association was found between other screen, app, and internet connectivity features and depression in the LMM.
“We found no significant association between other screen, app, and internet connectivity features and depression.”
Quote in the paperNot a substantial claim- Rests on an assumption: The missing data in the data set are missing at random.
- Findingown data
Digital biomarkers extracted from participants' smartphone data set were able to predict participants' depression state (depressed or nondepressed) with high predictive performance using ML models.
“Despite this heterogeneity, digital biomarkers extracted from participants’ smartphone data set were able to predict participants’ depression state (ie, depressed or nondepressed) with high predictive performance using ML models.”
- Also stated in: § Conclusions · plain
“Our findings also establish that behavioral markers extracted from smartphone data sets can predict whether or not a participant is depressed based on the PHQ-8 depression score”
Quote in the paperSubstantial claim- Rests on an assumption: The PHQ-8 score is a valid measure of depression.
- Rests on an assumption: The threshold used to classify participants as depressed or nondepressed from PHQ-8 scores is valid.
- Also stated in: § Conclusions · plain
- Findingnone given
The correlation and association between screen status-normalized entropy and depression are not strong.
“In addition, the correlation and association between screen status–normalized entropy and depression are not strong”
Quote in the paperNot a substantial claim - Findingown data
For the RF classifier, the top 5 most important features in predicting PHQ-8 depression score were screen status-normalized entropy, screen regularity index, screen off count SD, screen off count, and internet regularity index.
“For the RF classifier in Figure 2, the top 5 most important features were screen status–normalized entropy, screen regularity index, screen off count SD, screen off count, and internet regularity index.”
Quote in the paperNot a substantial claim - Findingnone given
Despite a fair distribution of age groups and spread over several countries, the data set is biased toward highly educated and professional occupations.
“Despite the data set having a fair distribution of age groups, with a spread over several countries, it is biased toward highly educated and professional occupations.”
Quote in the paperNot a substantial claim - Findingown data
The correlation finding in this study failed to replicate previous results reported in Saeb et al [33].
“This finding further supported the correlation finding, which failed to replicate previous results reported in Saeb et al [33].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Saeb et al [33]
Not checked in the free run.
- Findingown data
Participants with depression symptoms were more likely to lock and unlock their phone's screen in a random and uncertain manner, as shown by a significant positive correlation between screen status-normalized entropy and depression (r=0.14, P<.001).
“We could only identify that participants with depression symptoms were more likely to lock and unlock their phone’s screen in a random and uncertain manner (ie, significant positive correlation between screen status–normalized entropy and depression, r=0.14, P<.001).”
Quote in the paperSubstantial claim- Rests on an assumption: The PHQ-8 score is a valid measure of depression.
- Findingown data
For the KNN classifier, the top 5 most important features in predicting PHQ-8 depression score were the internet regularity index, screen status-normalized entropy, screen regularity index, internet status-normalized entropy, and internet status entropy.
“Likewise, for the KNN classifier in Figure 3, the top 5 most important features are the internet regularity index, screen status–normalized entropy, screen regularity index, internet status–normalized entropy, and the internet status entropy.”
Quote in the paperNot a substantial claim - Findingown data
For the KNN classifier, app entropy, app count, app distinct count, app regularity index, and app count SD were less important features.
“As shown in Figure 3 app entropy, count, distinct count, regularity index, and count SD were less important to the KNN classifier.”
Quote in the paperNot a substantial claim - Findingown data
Across the KNN, XGB, and RF classifiers, the top 5 most important features overall were the screen regularity index, screen status entropy, internet regularity index, screen status-normalized entropy, and screen off count SD.
“By ranking all important features for KNN, XGB, and RF classifiers, the top 5 most were the screen regularity index, screen status entropy, internet regularity index, screen status–normalized entropy, and the screen off count SD.”
Quote in the paperSubstantial claim - Findingown data
All other indicators were nonsignificant in the correlation analysis.
“All other indicators were nonsignificant.”
Quote in the paperSubstantial claim - Findingown data
The results established a positive statistically significant linear correlation and association between depression and screen status-normalized entropy behavior quantified from smartphone data sets.
“Our results established a positive statistically significant linear correlation and association between depression and screen status–normalized entropy behavior quantified from smartphone data sets.”
Quote in the paperSubstantial claim- Rests on an assumption: The PHQ-8 score is a valid measure of depression.
- Interpretationreasoning
Traditional assessment of depression can be augmented with behavioral markers from smartphones for depression diagnosis and monitoring.
“Traditional assessment of depression can be augmented with behavioral markers from smartphones for depression diagnosis and monitoring.”
Quote in the paperSubstantial claim- Rests on an assumption: The Carat app's sensor data accurately reflect participants' actual behaviors.
- Rests on an assumption: The sample of 629 participants is representative of the population of smartphone users with depression.
- Interpretationown data
Behavioral markers indicative of depression can be unobtrusively identified from smartphone sensors' data.
“Our findings demonstrate that behavioral markers indicative of depression can be unobtrusively identified from smartphone sensors’ data.”
Quote in the paperSubstantial claim- Rests on an assumption: The Carat app's sensor data accurately reflect participants' actual behaviors.
- Rests on an assumption: The sample of 629 participants is representative of the population of smartphone users with depression.
- Study design and conductcitation
LMM reduces the likelihood of Type I error.
“LMM also reduces the likelihood of Type I error [56].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [56] source
Not checked in the free run.
- Premisecitation
For the past 30 years, clinician-administered and self-reported questionnaires remain the gold standard in the assessment and diagnosis of depression.
“For the past 30 years, clinician-administered and self-reported questionnaires remain the gold standard in the assessment and diagnosis of depression [3,13].”
Quote in the paperSubstantial claim0 of 2 sources checked - Premisecitation
Instrumenting smartphones and wearables to capture in situ, fine-grained, and moment-by-moment data sets with sensing apps has made it possible to passively collect data sets in naturalistic settings.
“Instrumenting smartphones and wearables to capture in situ, fine-grained, and moment-by-moment data sets with sensing apps [20-24] has made it possible to passively collect data sets in naturalistic settings.”
Quote in the paperSubstantial claim0 of 5 sources checked - Premisecitation
In many individuals with depression, it remains undiagnosed and untreated due to barriers such as social stigma and inaccurate assessment methods.
“Yet, in many individuals with depression, it remains undiagnosed and untreated due to barriers such as social stigma and inaccurate assessment methods [1,3,12,13].”
Quote in the paperSubstantial claim0 of 4 sources checked - Study design and conductnone given
Clinical diagnosis of depression was not an inclusion criterion for the sample population.
“Clinical diagnosis of depression was not an inclusion criterion for our sample population.”
Quote in the paperSubstantial claim - Study design and conductnone given
The data set was collected from Android participants only.
“Lastly, the data set was collected from Android participants only”
Quote in the paperNot a substantial claim - Interpretationreasoning
The correlation and association between screen status-normalized entropy and depression may not generalize in other populations.
“and may not generalize in other populations.”
Quote in the paperNot a substantial claim - Premisecitation
Correlation coefficients only measure the extent of the linear relationship between variables.
“Moreover, correlation coefficients only measure the extent of the linear relationship between variables [87,88].”
Quote in the paperSubstantial claim0 of 2 sources checked - Premisecitation
Previous studies on mental health relied on the mutual information (MI) method from information theory.
“Instead of correlations, previous studies on mental health relied on the mutual information (MI) method from Information theory [87-89].”
Quote in the paperNot a substantial claim0 of 3 sources checked - Interpretationown data
The superior performance of tree-based and nearest neighbor classifiers corroborates the existence of nonlinear relationships between digital biomarkers and depression.
“corroborating the existence of nonlinear relationships between digital biomarkers and depression.”
Quote in the paperSubstantial claim - Premisecitation
Passively sensed participants' smartphone screen interaction (on and off states) behavior was demonstrated to be an important predictor of mental health.
“Passively sensed participants’ smartphone screen interaction (ie, on and off states) behavior was demonstrated to be an important predictor of mental health [27].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [27] source
Not checked in the free run.
- Premisecitation
The mutual information (MI) method measures both linear and nonlinear statistical dependencies between variables.
“The advantage of using the MI method is that the MI measures both linear and nonlinear statistical dependencies between variables.”
Quote in the paperSubstantial claim0 of 3 sources checked - Interpretationown data
Predictive modeling of mental health using digital biomarkers is possible in a more general population, not only in small homogenous populations.
“These findings show that predictive modeling of mental health using digital biomarkers is not only possible in small homogenous populations [83,84], but also in a more general population”
Quote in the paperSubstantial claim0 of 2 sources checked- Rests on an assumption: The PHQ-8 score is a valid measure of depression.
- Rests on an assumption: The threshold used to classify participants as depressed or nondepressed from PHQ-8 scores is valid.
- Premisecitation
The number of times a participant interacts with their phone, including screen lock and unlocks, was found to correlate with participants' mental health state.
“Similar findings have been reported previously [3,33], where the number of times a participant interacts with their phone, including screen lock and unlocks, was found to correlate with participants’ mental health state.”
Quote in the paperSubstantial claim0 of 2 sources checked - Interpretationreasoning
Encoded in the demographic data of this study's population is additional information that is useful in predicting the depression state of the participants.
“Thus, encoded in the demographic data of this study’s population is additional information that is useful in predicting the depression state of the participants.”
Quote in the paperSubstantial claim - Study design and conductreasoning
Because of the crowdsourced rolling recruitment nature of participants, the data set contained an unequal number of repeated depression assessments for all participants.
“Because of the crowdsourced rolling recruitment nature of participants, the data set contained an unequal number of repeated depression assessments for all participants.”
Quote in the paperNot a substantial claim - Premisecitation
The heterogeneity and inconsistency in correlation findings (linear relationships) in the field are common issues.
“The heterogeneity and inconsistency in correlation findings (ie, linear relationships) in the field are common issues [26].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [26] source
Not checked in the free run.
- May combine two claims. 18
- Classified as probably not a substantial claim. 31
May combine two claims.
FindingreasoningThe final data contained 629 participants, 1374 PHQ-8 responses with 13,898 days of participants' data set.
“Consequently, the final data contained 629 participants, 1374 PHQ-8 responses with 13,898 days of participants’ data set.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe mean number of days per participant is 22.1 (SD 17.90; range 8-86).
“The mean number of days per participant is 22.1 (SD 17.90; range 8-86).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe mean PHQ-8 score is 5.19 (SD 5.22; range 0-24).
“The mean PHQ-8 score is 5.19 (SD 5.22; range 0-24).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data1143/1374 (83.19%) PHQ-8 responses were nondepressed scores (PHQ-8 score <10), while 231/1374 (16.81%) were depressed scores (PHQ-8 score ≥10).
“For the distribution of the PHQ-8 scores, 1143/1374 (83.19%) responses were nondepressed scores (PHQ-8 score <10), while 231/1374 (16.81%) were depressed scores (PHQ-8 score ≥10).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe data set was collected from a heterogeneous geographic (at least 56 countries), occupational, and educational population, with high interindividual differences (75.84% interclass correlation).
“This data set was collected from a heterogeneous geographic (ie, from at least 56 different countries), occupational, and educational population, with high interindividual differences (ie, 75.84% interclass correlation).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe ML models achieved precision 85.55%-92.51%, recall 92.19%-95.56%, F1 88.73%-94.00%, AUC 94.69%-99.06%, Cohen κ 86.61%-92.90%, and accuracy 96.44%-98.14%.
“The ML models achieved the following: precision, 85.55%-92.51%; recall, 92.19%-95.56%; F1, 88.73%-94.00%; AUC, 94.69%-99.06%; Cohen κ, 86.61%-92.90%; and accuracy, 96.44%-98.14%.”
Quote in the paperSubstantial claim- Rests on an assumption: The PHQ-8 score is a valid measure of depression.
- Rests on an assumption: The threshold used to classify participants as depressed or nondepressed from PHQ-8 scores is valid.
May combine two claims.
Findingown dataIn the LMM association analysis, a unit increase in screen status-normalized entropy positively increases the average depression score (β=.48, P=.03), but all other variables remained nonsignificant.
“However, the results from the association analyses further showed that a unit increase in the screen status–normalized entropy positively increases the average depression score (β=.48, P=.03), but all other variables remained nonsignificant.”
Quote in the paperSubstantial claim- Rests on an assumption: The PHQ-8 score is a valid measure of depression.
Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
Interpretationnone givenInstrumenting smartphones to passively and continuously collect moment-by-moment data sets to quantify human behaviors has the potential to augment current depression assessment methods for early diagnosis, scalable, and longitudinal monitoring of depression.
“Instrumenting smartphones to passively and continuously collect moment-by-moment data sets to quantify human behaviors has the potential to augment current depression assessment methods for early diagnosis, scalable, and longitudinal monitoring of depression.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Premisenone givenCurrent depression assessment methods using self-reported and clinician-administered questionnaires have limitations.
“Current depression assessment methods using self-reported and clinician-administered questionnaires have limitations.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable).
PremisecitationMonitoring these biomarkers' fluctuations is essential in the early detection and treatment of mental health disorders.
“Monitoring these biomarkers’ fluctuations is essential in the early detection and treatment of mental health disorders [3,32].”
Quote in the paperNot a substantial claim0 of 2 sources checkedProbably not a substantial claim: may not pass S2 (checkable).
InterpretationcitationThe ability to detect early warning signs of depression, continuously and as effortlessly as possible, by extending current assessment methods could have a significant impact in mitigating or addressing depression and its related negative consequences.
“The ability to detect early warning signs of depression, continuously and as effortlessly as possible, by extending current assessment methods could have a significant impact in mitigating or addressing depression and its related negative consequences [3,10,11,14].”
Quote in the paperNot a substantial claim0 of 4 sources checked- cites [3] source
Not checked in the free run.
- cites [10] source
Not checked in the free run.
- cites [11] Depression costs employers $210 billion InternetBehavioral Health Executive20152021-06-28https://www.psychcongress.com/news-item/depression-costs-employers-210-billion
Not checked in the free run.
- cites [14] source
Not checked in the free run.
Classified as probably not a substantial claim.
Study design and conductcitationSMOTE generates synthetic data for the minority class, resulting in a balanced training data set.
“We handled class imbalance in the training data set with the synthetic minority over-sampling technique (SMOTE) [65,73,74], which generates synthetic data for the minority class, resulting in a balanced training data set.”
Quote in the paperSubstantial claim0 of 3 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitationMultiple imputation was used to handle missing data, taking into account the nested structure of the data set.
“In the LMM, we used multiple imputation to handle missing data, taking into account the nested structure of the data set [57].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [57] source
Not checked in the free run.
- Rests on an assumption: The missing data in the data set are missing at random.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationHeart rate variability has been found to be consistently lower in patients with psychiatric disorders.
“Other promising biomarkers from wearable devices are heart rate variability, which has been found to be consistently lower in patients with psychiatric disorders, electrodermal activity, and skin conductance [36,37].”
Quote in the paperNot a substantial claim0 of 2 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductcitationHolm-Bonferroni method was used to adjust P values for multiple testing, with a false discovery rate of 0.05.
“We used the Holm–Bonferroni [51] method to adjust the P values for multiple testing, with a false discovery rate of 0.05.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [51] Holm, S, 1979, Scandinavian Journal of Statistics
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Study design and conductnone givenNested cross-validation is a state-of-the-art procedure to prevent overfitting and overestimation of the hyperparameters of the ML classifiers.
“The nested cross-validation is a state-of-the-art procedure to prevent overfitting and overestimation of the hyperparameters of the ML classifiers.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
InterpretationreasoningPrevious research has shown the potential of quantifying human behavior from smartphones and wearables data set as biomarkers.
“Taken together, previous research has shown the potential of quantifying human behavior from smartphones and wearables data set as biomarkers.”
Quote in the paperSubstantial claimMay combine two claims.
Study design and conductcitationPHQ-8 is clinically validated for the assessment of depression severity, has high internal consistency (Cronbach α=.82), and has been used in several previous studies.
“Although the depression assessments were self-reported, the PHQ-8 is clinically validated for the assessment of depression severity, has high internal consistency (Cronbach α=.82), and has been used in several previous studies [3,7,41].”
Quote in the paperSubstantial claim0 of 3 sources checkedClassified as probably not a substantial claim.
PremisecitationA PHQ-8 score of 10 or more indicates major depression or other severe forms of depression.
“The total PHQ-8 score ranges from 0 to 24, with a score of 10 or more indicating major depression or other severe forms of depression [41].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [41] source
Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable).
PremisereasoningSmartphones and wearables offer a unique opportunity to overcome limitations in traditional depression assessment methods.
“Today, smartphones and wearables offer a unique opportunity to overcome limitations in traditional depression assessment methods.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationSelf-reported appraisals are affected by memory and recall biases in reconstructing past events and may be prone to socially desirable reporting from individuals.
“Lastly, self-reported appraisals are affected by memory and recall biases in reconstructing past events and may be prone to socially desirable reporting from individuals [12,15-17].”
Quote in the paperSubstantial claim0 of 4 sources checkedProbably not a substantial claim: may not pass S4 (based).
Premisenone givenICC > 0.05 necessitates LMM.
“ICC > 0.05 necessitates LMM.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductcitationBenjamini-Hochberg procedure was used to control the false discovery rate.
“To address multiple comparison problems, we adjusted the P values in the LMM to control the false discovery rate with the Benjamini–Hochberg procedure [60,61].”
Quote in the paperSubstantial claim0 of 2 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationDepression is one of the most prevalent, complex, and heterogeneous mental health challenges of our time.
“Depression is one of the most prevalent, complex, and heterogeneous mental health challenges of our time.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [1] DepressionWorld Health Organisation (2020)2021-06-19https://www.who.int/news-room/fact-sheets/detail/depression
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationActigraphy-based biomarkers that quantify time sequences of rest and active behaviors with accelerometer sensors are known to be useful in predicting mood disorders such as depression and bipolar disorder.
“Actigraphy-based biomarkers that quantify time sequences of rest and active behaviors with accelerometer sensors are known to be useful in predicting mood disorders such as depression and bipolar disorder [34,35].”
Quote in the paperNot a substantial claim0 of 2 sources checkedMay combine two claims.
Premisenone givenTraditional depression assessment methods are applied sparingly (eg, a couple of times within a year), thus missing out on the moment-by-moment behavioral patterns of individuals between health assessments.
“Such methods are applied sparingly (eg, a couple of times within a year), thus missing out on the moment-by-moment behavioral patterns of individuals between health assessments.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenStratified sampling ensures the splitting of the data set into folds that have an equal proportion of each class.
“The stratified sampling ensures the splitting of the data set into folds that have an equal proportion of each class (ie, labels 1 and 0).”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationInherent in these data sets are behavioral patterns: routines, rhythms, activities, and interactions that are useful in complementing traditional depression assessment methods, in studying the mental health of individuals, and in developing timely mental health interventions.
“Inherent in these data sets are behavioral patterns: routines, rhythms, activities, and interactions that are useful in complementing traditional depression assessment methods, in studying the mental health of individuals, and in developing timely mental health interventions [14,22,25-28].”
Quote in the paperNot a substantial claim0 of 6 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitationFull information maximum likelihood was used in the correlation analysis to avoid biases introduced by missing data.
“Full information maximum likelihood [48-50] was used in the correlation analysis to avoid biases introduced by missing data.”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites [48] Enders, C, 2010, Applied Missing Data Analysis
Not checked in the free run.
- cites [49] source
Not checked in the free run.
- cites [50] RevelleWpsych: Procedures for Psychological, Psychometric, and Personality Research202006182021-06-28https://CRAN.R-project.org/package=psych
Not checked in the free run.
- Rests on an assumption: The missing data in the data set are missing at random.
Classified as probably not a substantial claim.
Study design and conductcitationAll ML modeling was performed using stratified and nested cross-validation.
“All the ML modeling was performed using stratified and nested cross-validation [71,72].”
Quote in the paperSubstantial claim0 of 2 sources checkedMay combine two claims.
InterpretationreasoningThe sample size in the data set is small (N=629) and may not be representative of the general population.
“Given the crowdsourced nature of the deployment of the Carat app, the sample size in the data set is small (N=629) and may not be representative of the general population.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationScreen unlocks were found to be important behavioral markers that correlate and predict resting state brain functional connectivity.
“In a neuroscience study [91], screen unlocks were found to be important behavioral markers that correlate and predict resting state brain functional connectivity”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [91] source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationPrevious studies suggest a relationship between demographic factors, smartphone usage behavior, and depression.
“Previous studies suggest a relationship between demographic factors, smartphone usage behavior, and depression [38,85,86].”
Quote in the paperNot a substantial claim0 of 3 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningThe correlation and association between behavioral patterns extracted from the data set in this study and depression do not necessarily imply causal relationships.
“The correlation and association between behavioral patterns extracted from the data set in this study and depression do not necessarily imply causal relationships.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationResearch has demonstrated an association between internet usage patterns and depression.
“On internet usage behaviors, research has demonstrated an association between internet usage patterns and depression [93,94]”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [93] Alavi, SS, 2011, J Res Med Sci
Not checked in the free run.
- cites [94] source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe findings in this study are supported by prior research that investigated the relationship between screen interactions and mental health.
“The findings in this study are also supported by prior research that investigated the relationship between screen interactions and mental health [3,27,28,33].”
Quote in the paperNot a substantial claim0 of 4 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThe permutation importance method is model agnostic and computes the proportional decrease in a model score when features are randomly shuffled.
“The permutation importance method is model agnostic and computes the proportional decrease in a model score when features are randomly shuffled.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataBoth the correlation findings and the feature importance analysis in the prediction models showed that participants' phone screen (lock and unlock) behaviors and internet connectivity behaviors played the most important role in predicting their depression state.
“Nevertheless, both the correlation findings and the feature importance analysis in the prediction models clearly showed that participants’ phone screen (lock and unlock) behaviors, such as routinely and randomly locking and unlocking phone screen, and internet connectivity behaviors played the most important role in predicting their depression state.”
- Also stated in: § Conclusions · plain
“phone screen and internet connectivity behaviors were the most insightful behaviors that influence depression in participants.”
Quote in the paperSubstantial claim- Rests on an assumption: The PHQ-8 score is a valid measure of depression.
- Also stated in: § Conclusions · plain
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationResting state brain functional connectivity is known to be associated with depression.
“which is known to be associated with depression [92].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [92] source
Not checked in the free run.
May combine two claims.
InterpretationcitationConventional statistical methods (eg, correlation or LMM) may not depict the complex nonlinear relationship between digital biomarkers and depression, and more powerful methods such as ML models (eg, XGB) are needed to make better predictions.
“This suggests that conventional statistical methods (eg, correlation or LMM) may not depict the complex nonlinear relationship between digital biomarkers and depression, and indeed more powerful methods such as ML models (eg, XGB) are needed to make better predictions [90].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [90] source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
InterpretationcitationThe long-term use of the Carat app could influence participants' behavior.
“and the long-term use of the Carat app could influence participants’ behavior [38,39].”
Quote in the paperNot a substantial claim0 of 2 sources checkedMay combine two claims.
Study design and conductnone givenThe data set does not contain a clinical or self-reported baseline assessment of depression and has scarce high depression scores.
“The data set also does not contain a clinical or self-reported baseline assessment of depression and has scarce high depression scores.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationResearch has shown that participant's behavior and sociodemographics may differ between Android platforms and other mobile platforms such as iOS.
“Research has shown that participant’s behavior and sociodemographics may differ between Android platforms and other mobile platforms such as iOS [44,95].”
Quote in the paperNot a substantial claim0 of 2 sources checkedProbably not a substantial claim: may not pass S2 (checkable). May combine two claims.
Interpretationown dataThe findings in this study are supported by previous research findings and contribute to compelling evidence on the utility of digital biomarkers in augmenting traditional assessment of depression, thus enabling continuous and passive monitoring of the complex vectors of depression.
“The findings in this study are supported by previous research findings and contribute to compelling evidence on the utility of digital biomarkers in augmenting traditional assessment of depression, thus enabling continuous and passive monitoring of the complex vectors of depression.”
Quote in the paperNot a substantial claim- Rests on an assumption: The PHQ-8 score is a valid measure of depression.
- Rests on an assumption: The threshold used to classify participants as depressed or nondepressed from PHQ-8 scores is valid.
The paper relies on this without support.
The missing data in the data set are missing at random.
“In the LMM, we used multiple imputation to handle missing data, taking into account the nested structure of the data set [57].”
The paper relies on this without support.
The Carat app's sensor data accurately reflect participants' actual behaviors.
“Besides battery consumption data, the Carat Android app unobtrusively collected participants’ time zone and time-stamped data, including foreground app usage (ie, app the participant has interacted with), internet connectivity (ie, connected and disconnected states), and screen lock and unlock logs.”
The paper relies on this without support.
The sample of 629 participants is representative of the population of smartphone users with depression.
“Consequently, the final data contained 629 participants, 1374 PHQ-8 responses with 13,898 days of participants’ data set.”
The paper relies on this without support.
The PHQ-8 score is a valid measure of depression.
“predicting PHQ-8 depression score”
The paper relies on this without support.
The threshold used to classify participants as depressed or nondepressed from PHQ-8 scores is valid.
“depression state (ie, depressed or nondepressed)”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Smart sensing is an innovative diagnostic approach that can support mental health screening and intervention.
“One such method could be smart sensing, with the goal of achieving scalable, precise, and time-effective detection of symptoms (27–29).”
In the EU, reimbursement processes for digital health technologies are beginning to take shape; Germany, Belgium, and France have standardized reimbursement processes in place or soon to be launched, while other EU countries have expressed interest in implementing standardized reimbursement processes for digital health technologies.
“In the EU, reimbursement processes for digital health technologies are beginning to take shape; Germany, Belgium, and France have standardized reimbursement processes in place or soon to be launched, while other EU countries have expressed interest in implementing standardized reimbursement processes for digital health technologies.”
From Digital health technologies and major depressive disorder (2023) · cites this paper as [162]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Demographics-enhanced LR has lower recall than the DT baseline
MinorThe claim of outperforming the DT baseline on all metrics includes recall. Table 5 reports LR recall of 60.00, whereas Table 6 reports DT recall of 74.33.
“The worst performing classifier (ie, LR) had some gains in performance, but it could still barely outperform the DT baseline classifier in Table 6 in all performance metrics.”
“| Recall | 84.79 (0.86) | 74.33 (4.71) |”
The citation lists haven’t been fetched yet
References 89 · Cited by 145
Research works printed but not linked: 8
- 10.4108/icst.pervasivehealth.2015.259034 · DOI · Paper page
- Cohen, J, 1988, Statistical Power Analysis for the Behavioral Sciences (Second Edition)
- Nezlek, J, 2012, APA Handbook of Research Methods in Psychology, 219
- Snijders, T, 2011, An Introduction to Basic and Advanced Multilevel Modeling
- Géron, A, 2019, Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems
- One-Hot Encoding - An Overview2021-06-28https://www.sciencedirect.com/topics/computer-science/one-hot-encoding
- 10.5555/3122009.3122026 · DOI
- Alavi, SS, 2011, J Res Med Sci, 16, 793
- Authors
- Opoku Asare, K., Terhorst, Y., Vega, J., Peltonen, E., Lagerspetz, E., & Ferreira, D.
- Affiliations
- Universität Ulm (DE)
- University of Pittsburgh (US)
- University of Oulu (FI)
- University of Helsinki (FI)
- Venue
- JMIR mHealth and uHealth
- Publisher
- JMIR Publications Inc.
- Published
- 2021-07-12
- Volume
- 9
- Issue
- 7
- Pages
- e26540
- ISSN
- 2291-5222 (ISSN-L)
- PMC
- PMC8314163
- OpenAlex
- W3163328251
- Semantic Scholar
- 530d6a8d24c1bf96c3cfa0d1842c1c3e10959499
- Cited by
- 145
- References
- 89
- Citations by year
- 2026: 27 · 2025: 35 · 2024: 36 · 2023: 21 · 2022: 22 · 2021: 4
- FWCI
- 18.43
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR mhealth and uhealth (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Työväentutkimus Vuosikirja (submittedVersion)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Keywords
- depression prediction, passive sensing, screen time, step count, sleep patterns, display brightness, screen rotation, screen lock, PHQ-8, early diagnosis, long-term monitoring, behavioral patterns, entropy (information theory), feature importance, permutation importance, hyperparameter optimization
- MeSH terms
- Smartphone, Machine Learning, Adolescent, Adult, Depression, Female, Humans, Longitudinal Studies, Male, Young Adult, Self Report
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
