CoChat
Sign Up
PreprintCited by 227Digital Mental Health InterventionsMental Health Research TopicsEmotion and Mood Recognition

Digital Health Tools for the Passive Monitoring of Depression: A Systematic Review of Methods

Open access
de Angel, Valeria, Lewis, Serena, White, Katie, Oetzmann, Carolin, Leightley, Daniel, Oprea, Emanuela, Lavelle, Grace, Matcham, Faith +4 more·2021·npj Digital Medicine
DOI 10.1101/2021.07.19.21260786PMC PMC8752685
Open this paper in CoChat — free accountOpen PDF

Abstract

Background: The use of digital tools to measure physiological and behavioural variables of potential relevance to mental health is a growing field sitting at the intersection between computer science, engineering and clinical science. We aim to summarise the literature on remote measuring technologies, mapping methodological challenges and threats to reproducibility, and to identify leading digital signals for depression. Methods: Medical and computer science databases were searched between January 2007 to November 2019. Published studies linking depression and objective behavioural data obtained from smartphone and wearable device sensors in adults with unipolar depression and healthy subjects were included (PROSPERO registration: 2019 CRD42019159929). A descriptive approach was taken to synthesise study methodologies. Results We included 52 studies and found threats to reproducibility and transparency arising from failure to provide comprehensive descriptions of recruitment strategies, sample information, feature construction and the determination and handling of missing data. The literature is characterised by small sample sizes, short follow-up duration and great variability in quality of reporting, limiting the interpretability of pooled results. Bivariate analyses show some consistency in statistically significant associations between depression and digital features from sleep, physical activity, location, and phone use data. Regression and classification machine learning models found predictive value of aggregated features. Interpretation: Recommendations are put forward to improve aspects of generalisability and reproducibility, such as wider diversity of samples, thorough reporting methodology and the potential for reporting bias in studies with numerous features. Funding: National Institute for Health Research (NIHR) Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King's College London.

AI snapshot

Written from the paper’s full text.

TL;DR

This systematic review maps methods and reproducibility threats in 51 studies using smartphones and wearables to passively monitor depression, finding consistent bivariate signals in sleep, physical activity, location, and phone use but major heterogeneity and reporting gaps. The authors conclude these results are hypothesis-generating and recommend stronger reporting standards, preregistration, and more representative samples.

Key findings

  • Across 51 included studies, the median sample size was 58, median age 38 years, and median percentage female 58%; 63% of studies failed to report ethnicity, and 12% and 8% failed to report age or gender, respectively.
  • Fifteen depression measures were used, most commonly CES-D (n = 12), HAM-D (n = 12), and PHQ-9 (n = 9); 14 device types were used, mostly actigraphy-based wrist-worn devices plus smartphones.
  • Bivariate associations were most consistent for location features: homestay was significantly associated with depression in all included studies reporting it, and entropy in 4 studies; sleep stability had the highest proportion of significant associations among sleep features.
  • Machine learning models combining features showed predictive value, but many lacked information on missing sensor data handling and calibration; only two studies used multiple imputation.

Methods

  • Systematic review searching PubMed, IEEE Xplore, ACM Digital Library, Web of Science, and Embase/PsychInfo via OVID for studies published January 2007–November 2019; protocol registered on PROSPERO (CRD42019159929).
  • Included 51 published, peer-reviewed studies linking validated depression measures to passively collected smartphone or wrist-worn sensor data in adults with unipolar depression and healthy subjects, with at least 3 consecutive days of data; excluded inpatient settings and primarily bipolar samples.
  • Descriptive synthesis only, with no meta-analysis due to heterogeneity; quality assessment combined AXIS and Newcastle–Ottawa Scale items, and feasibility was assessed via engagement, dropout reasons, technology problems, completion, attrition, and missing data.

Limitations

  • Stated by the paper: reporting of feature–depression associations was too opaque and diverse for credible meta-analysis, so the review relies on simple counts of associations not weighted by sample size, follow-up duration, or study quality; reporting bias may emphasize significant findings; combining low-level features into broader categories lost nuance; and overlapping samples (e.g., four papers using the StudentLife dataset) may duplicate some associations.
  • Evident from the text: included studies were mostly opportunistic, with small samples, short follow-up (median 9 days), and many student samples, limiting generalisability; there was no consensus on optimal phenotyping windows, and device validity/reliability for complex behaviours such as sleep and sociability remains uncertain.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1101/2021.07.19.21260786

Cites

94 of 94 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 65 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    The review included 51 studies.

    “Fifty-one studies were included in the review (see Fig. 1).”
    Quote in the paperNot a substantial claim
  • Findingown data

    The median sample size of included studies was 58.

    “The median sample size was 58, the median age of participants was 38 years, and the median percentage of females was 58%.”
    Quote in the paperNot a substantial claim
  • Findingown data

    12% of studies failed to give data on age.

    “with 12% and 8% of studies failing to give data on age or gender, respectively, and 63% failing to include information on ethnicity.”
    Quote in the paperNot a substantial claim
  • Findingcitation

    Higher depressive symptoms were associated with less time spent engaging in physical activity.

    “Higher depressive symptoms were associated with less time spent engaging in physical activity5, movement speed18 and step count27.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [5] Lu, J. et al. Joint modeling of heterogeneous sensing data for depression assessment via multi-task learning. In Proc. ACM Interactive Mobile, Wearable and Ubiquitous Technology. Vol. 2, 21:1–21:21 (ACM, 2018).

      Not checked in the free run.

  • Findingcitation

    Depression was negatively correlated with physical activity.

    “Activity levels were predominantly measured as a gross motor activity within a day, and showed that depression was negatively correlated with physical activity20,34.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [20] Difrancesco S Sleep, circadian rhythm, and physical activity patterns in depressive and anxiety disorders: a 2-week ambulatory assessment study Depress. Anxiety 2019 36 975 986 31348850 PMC6790673

      Not checked in the free run.

    • cites [34] Boukhechba, M. et al. Contextual analysis to understand compliance with smartphone-based ecological momentary assessment. In Proc. 12th EAI International Conference on Pervasive Computing Technologies for Healthcare 232–238 (ACM, 2018).

      Not checked in the free run.

  • Classified as probably not a substantial claim. 41
  • May combine two claims. 29
  • The quote only partly states this claim. 5

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The majority of included articles (n = 45) were published in medical journals.

    “The majority of articles (n = 45) were published in medical journals, and 33 (65%) were from North America.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The median age of participants was 38 years.

    “The median sample size was 58, the median age of participants was 38 years, and the median percentage of females was 58%.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Two studies provided no information on the length of follow-up.

    “Two studies provided no information on the length of follow-up, instead only mentioning that data was obtained from participants providing at least 72 h of consecutive data5,18.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Out of seven studies extracting activity levels, both smartphone studies found a significant difference in depression severity, compared to one of five wrist actigraphy studies.

    “Out of the seven studies extracting ‘activity levels’ as a feature within physical activity, both studies using smartphones found a significant difference in depression severity, compared to one out of the five that used wrist actigraphy.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Reasons for dropouts were provided in four studies.

    “Reasons for dropouts were provided in four studies and were due to equipment malfunction and technical problems using devices19,25–27 .”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    33 (65%) of included studies were from North America.

    “The majority of articles (n = 45) were published in medical journals, and 33 (65%) were from North America.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Depression has been associated with lower daytime activity and higher night-time activity.

    “Depression has been associated with lower daytime activity and higher night-time activity (hour-based activity levels36,37), low intra-daily stability, more fragmented intra-daily movement, e.g., leaving for work and coming back at less regular times7, later acrophase, or later activity peaks38–40; lower amplitude, less difference between the average levels of activity during the peaks vs. the tro”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [36] Ávila Moraes C A new chronobiological approach to discriminate between acute and chronic depression using peripheral temperature, rest-activity, and light exposure parameters BMC Psychiatry 2013 13 77 23510455 PMC3599978

      Not checked in the free run.

    • cites [37] Slyepchenko A Association of functioning and quality of life with objective and subjective measures of sleep and biological rhythms in major depressive and bipolar disorder Aust. N. Z. J. Psychiatry 2019 53 683 696 30759998

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    Features with higher proportions of significant findings are sleep stability, sleep offset, and time in bed.

    “Features with higher proportions of significant findings are features of sleep stability, sleep offset, time in bed; longer time in bed and later sleep offset were associated with higher depression scores.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Computer science journals were more likely to report ethnicity than medical journals.

    “Computer science journals were less likely to report age and gender but more likely to report ethnicity (33% studies failing to report each demographic).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing). The quote only partly states this claim.

    Findingown data

    The most commonly used depression scale was HAM-D (n = 12 studies).

    “Fifteen different measures of depression were used, the most commonly used scales being the Center for Epidemiological Studies Depression Scale (CES-D15; n = 12 studies), Hamilton Rating Scale for Depression (HAM-D16; n = 12 studies), and Patient Health Questionnaire-9 (PHQ-917; n = 9).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).

    Findingown data

    The most commonly used depression scale was PHQ-9 (n = 9 studies).

    “Fifteen different measures of depression were used, the most commonly used scales being the Center for Epidemiological Studies Depression Scale (CES-D15; n = 12 studies), Hamilton Rating Scale for Depression (HAM-D16; n = 12 studies), and Patient Health Questionnaire-9 (PHQ-917; n = 9).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    There were 14 types of devices used across all studies.

    “There were 14 types of devices used across all studies: 12 of them actigraphy-based wrist-worn devices including one Fitbit and a Microsoft band, as well as one pedometer and smartphones (both android and iPhone).”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Computer science journals were less likely to report age and gender than medical journals.

    “Computer science journals were less likely to report age and gender but more likely to report ethnicity (33% studies failing to report each demographic).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Nine different features of sleep are reported.

    “Nine different features of sleep are reported in Fig. 4A.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    Longer time in bed and later sleep offset were associated with higher depression scores.

    “Features with higher proportions of significant findings are features of sleep stability, sleep offset, time in bed; longer time in bed and later sleep offset were associated with higher depression scores.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The median percentage of females was 58%.

    “The median sample size was 58, the median age of participants was 38 years, and the median percentage of females was 58%.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    The most commonly used depression scale was CES-D (n = 12 studies).

    “Fifteen different measures of depression were used, the most commonly used scales being the Center for Epidemiological Studies Depression Scale (CES-D15; n = 12 studies), Hamilton Rating Scale for Depression (HAM-D16; n = 12 studies), and Patient Health Questionnaire-9 (PHQ-917; n = 9).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Those that report missing data handling report simple imputation methods such as mean imputation, with two studies using multiple imputation methods.

    “Those that do, report simple imputation methods such as mean imputation, with two studies using multiple imputation methods6,18.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [6] Ghandeharioun, A. et al. Objective assessment of depressive symptoms with machine learning and wearable sensors data. In Proc.2017 7th International Conference on Affective Computing and Intelligent Interaction (ACII). 325–332 (IEEE, 2017).

      Not checked in the free run.

    • cites [18] Yue, C. et al. Fusing location data for depression prediction. IEEE Trans. Big Data 1–1 10.1109/TBDATA.2018.2872569 (2018). PMC9053381 35498556

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Eleven studies assessed sociability.

    “Eleven studies assessed sociability.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Humidity had a negative correlation with depressed symptoms in men.

    “Another study found humidity to have a significant positive correlation with depressed symptoms (r = 0.4) in women, but a negative correlation in men, suggesting females, but not males might feel worsening in their condition during rainy weeks44.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [44] Doryab, A., Min, J. K., Wiese, J., Zimmerman, J. & Hong, J. Detection of behavior change in people with depression. In Proc. of the 28th AAAI Conference on Artificial Intelligence, Vol. 5 (Québec City, QC, Canada, 2014).

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    The average number of ingoing and outgoing calls was negatively correlated with depressive symptoms in one small study (n = 6), and only in men.

    “The average number of ingoing and outgoing calls was found to be negatively correlated with depressive symptoms in one small study (n = 6), and only in men44.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [44] Doryab, A., Min, J. K., Wiese, J., Zimmerman, J. & Hong, J. Detection of behavior change in people with depression. In Proc. of the 28th AAAI Conference on Artificial Intelligence, Vol. 5 (Québec City, QC, Canada, 2014).

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Twenty-four models in total were presented by all studies.

    “Twenty-four models in total were presented by all studies, the majority of which (n = 18) included features of physical activity, followed by location (n = 14), phone use (n = 11) and sleep (n = 9).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Four studies calculated circadian rhythmicity as a Cosinor model, finding lower circadian rhythmicity more likely to be associated with being depressed.

    “Four studies calculated circadian rhythmicity as a measure of the extent to which a participant’s pattern follows an expected Cosinor model, finding lower circadian rhythmicity more likely to be associated with being depressed37,41–43.”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [37] Slyepchenko A Association of functioning and quality of life with objective and subjective measures of sleep and biological rhythms in major depressive and bipolar disorder Aust. N. Z. J. Psychiatry 2019 53 683 696 30759998

      Not checked in the free run.

    • cites [41] Robillard R Sleep–wake cycle in young and older persons with a lifetime history of mood disorders PLoS ONE 2014 9 e87763 24586290 PMC3934865

      Not checked in the free run.

    • cites [42] Saeb S Lattie EG Schueller SM Kording KP Mohr DC The relationship between mobile phone location sensor data and depressive symptom severity PeerJ 2016 4 e2537 28344895 PMC5361882

      Not checked in the free run.

    • cites [43] Robillard R Sleep–wake profiles predict longitudinal changes in manic symptoms and memory in young people with mood disorders J. Sleep Res. 2016 25 549 555 27242171

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    Location was assessed in 11 studies, measured via GPS.

    “Location was assessed in 11 studies, measured via GPS.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    Both classification and regression models showed predictive value.

    “Both classification and regression models showed predictive value, however, many of them lacked information regarding the handling of missing sensor data and calibration.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Many of the studies on circadian rhythms are excluded, making existing associations even more tenuous.

    “Many of the studies on circadian rhythms are excluded, making existing associations even more tenuous; all studies showing a significant association between mood and intradaily stability or acrophase are lost, as are those finding no association between hour-based activity levels and depression.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingreasoning

    Missing data is one issue specific to this field.

    “One of those issues is missing data.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    For any broad concept (e.g. sleep or circadian rhythm), different sensor types or operating systems were used, and component features were derived using different approaches.

    “For any broad concept (e.g. sleep or circadian rhythm) different sensor types or operating systems were used, and component features were derived using different approaches.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    Several sleep features appear to be consistently associated with depressed mood, with sleep stability showing the highest proportion of significant associations.

    “Several sleep features appear also to be consistently associated with depressed mood, with sleep stability showing the highest proportion of significant associations.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    Location-based features consistently appear to be associated with depression, with homestay and entropy both associated with mood in 4 and 5 studies, respectively.

    “Features that consistently appear to be associated with depression are location-based features, with homestay and entropy both associated with the mood in 4 and 5 studies, respectively.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Some studies in the current review report their top predictors and bivariate associations with depression.

    “Some studies in the current review do report their top predictors and bivariate associations with depression”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The findings of this review highlight the array of potential predictors that sensor data generates.

    “The findings of this review highlight the array of potential predictors that sensor data generates.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    Both iPhone and Android smartphone operating systems were included, and sometimes showed differences in significance levels for the same variables.

    “For example, both iPhone and Android smartphone operating systems were included, and sometimes showed differences in significance levels for the same variables5,18.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [5] Lu, J. et al. Joint modeling of heterogeneous sensing data for depression assessment via multi-task learning. In Proc. ACM Interactive Mobile, Wearable and Ubiquitous Technology. Vol. 2, 21:1–21:21 (ACM, 2018).

      Not checked in the free run.

    • cites [18] Yue, C. et al. Fusing location data for depression prediction. IEEE Trans. Big Data 1–1 10.1109/TBDATA.2018.2872569 (2018). PMC9053381 35498556

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Findingnone given

    There is a general lack of discussion around the extent to which devices used in these research studies are valid or reliable tools to detect the behaviours of interest.

    “Finally, there is a general lack of discussion around the extent to which the devices used in these research studies are valid or reliable tools to detect the behaviours of interest.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    Researchers set different thresholds as to what counts as missing data, and this varies between studies.

    “Additionally, researchers set different thresholds as to what counts as missing data. This varies between studies”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    The extent to which any inaccuracies in the devices reduce the strength of association in depression is unknown.

    “the extent to which any inaccuracies in the devices reduce the strength of association in depression is unknown.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    The quality assessments reflect poor reporting of missing data at both the sample level (attrition and study non-completion) and individual level (missing sensor data from participants).

    “Our quality assessments reflect poor reporting of missing data at both the sample level (e.g. attrition and study non-completion) and individual level (e.g. missing sensor data from participants).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingnone given

    Medical papers have limited description in feature extraction and analysis.

    “and the limited description in feature extraction and analysis in medical papers”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingnone given

    The literature identified derives from both clinical and computer science disciplines.

    “The literature we identified derives from both clinical and computer science disciplines”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Studies have mostly employed opportunistic study designs, with small sample sizes, short follow-up windows and many being conducted on students, which limits generalisability.

    “Studies have mostly employed opportunistic study designs, with small sample sizes, short follow-up windows and many being conducted on students, which limits generalisability.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation

    Different features may reach peak predictability of mood with different sampling timeframes, so shorter follow-ups may harm the prediction abilities for some behaviours.

    “Different features may reach peak predictability of mood with different sampling timeframes, so shorter follow-ups may harm the prediction abilities for some behaviours22.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [22] Sano A Identifying objective physiological markers and modifiable behaviors for self-reported stress and mental health status using wearable sensors and mobile phones: observational study J. Med. Internet Res. 2018 20 e210 29884610 PMC6015266

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    Due to the large number of variables obtained in sensing studies, it is likely that published papers are selective in their reporting, and typically emphasise positive findings over negative ones.

    “Additionally, due to the large number of variables obtained in sensing studies, it is likely that published papers are selective in their reporting, and typically emphasise “positive” findings over “negative” ones.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    This heterogeneity challenges the ability of investigators to reproduce findings and hampered the ability to summarise results in a meta-analysis.

    “This heterogeneity challenges the ability of investigators to reproduce findings and hampered our ability to summarise results in a meta-analysis.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The lack of significant association for circadian features may be due to short follow-up, since median follow-up times for circadian features = 9 days.

    “As previously mentioned, this may be due to short follow-up since median follow-up times for circadian features = 9 days.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Given the heterogeneity in research quality and reporting standards across studies, making inferences from aggregated associations between digital features and mood may be misleading.

    “Given the heterogeneity in research quality and reporting standards across studies, making inferences from aggregated associations between digital features and mood may be misleading.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Actigraphy tends to overestimate sleep and underestimate wake, but this inaccuracy is consistent, maintaining its usefulness as a potential marker of sleep-wake patterns.

    “A more recent systematic review, however, found that while actigraphy tended to overestimate sleep and underestimate wake, this inaccuracy was consistent, thereby maintaining its usefulness as a potential marker of sleep–wake patterns59.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [59] Conley S Agreement between actigraphic and polysomnographic measures of sleep in adults with and without chronic conditions: a systematic review and meta-analysis Sleep Med. Rev. 2019 46 151 160 31154154 PMC6594867

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    There is a need for a comprehensive review of methodologies, with more specific inclusion criteria, to highlight the sources of heterogeneity and methodological shortcomings in the field.

    “There is therefore a need for a comprehensive review of methodologies, with more specific inclusion criteria, to highlight the sources of heterogeneity and methodological shortcomings in the field.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The question of how well these models can be replicated remains.

    “but the question of how well these models can be replicated remains”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    Whether sensors measure the exact ground truth may be less concerning than whether the extracted features are consistent against each other and serve the purpose of detecting changes in health status.

    “whether or not these sensors measure the exact ground truth may be less concerning than whether the features we do extract are consistent against each other and serve the purpose of detecting changes in health status.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Machine learning methods have been criticised for lacking transparency in how the model is built and how individual variables contribute to the overall prediction.

    “However, machine learning methods have been criticised for lacking transparency in how the model is built and how individual variables contribute to the overall prediction65.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [65] Wall R Cunningham P Walsh P Byrne S Explaining the output of ensembles in medical decision support on a case by case basis Artif. Intell. Med. 2003 28 191 206 12893119

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Analysing multimodal data may help cover missing data from one source to another.

    “analysing multimodal data in this way may help cover missing data from one source to another.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Previous reviews have included non-validated measures of depression and a mix of bipolar and unipolar samples, characteristics which show divergent results and add study diversity.

    “Additionally, these reviews have included non-validated measures of depression, and a mix of bipolar and unipolar samples, characteristics which not only show divergent results11,12,14, but add study diversity.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [11] Rohani DA Faurholt-Jepsen M Kessing LV Bardram JE Correlations between objective behavioral features collected from mobile and wearable devices and depressive mood symptoms in patients with affective disorders: systematic review JMIR MHealth UHealth 2018 6 e165 30104184 PMC6111148

      Not checked in the free run.

    • cites [12] Melcher, J., Hays, R. & Torous, J. Digital phenotyping for mental health of college students: a clinical review. Evid. Based Ment. Health ebmental-2020-300180. 10.1136/ebmental-2020-300180 (2020). PMC10231503 32998937

      Not checked in the free run.

    • cites [14] Dogan E Sander C Wagner X Hegerl U Kohls E Smartphone-based monitoring of objective and subjective data in affective disorders: where are we and where are we going? Systematic review J. Med. Internet Res. 2017 19 e262 28739561 PMC5547249

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    These domains are yet to be brought together efficiently, giving rise to large differences in reporting standards.

    “The advantages of a multidisciplinary approach are evident, but these domains are yet to be brought together efficiently11,12, giving rise to large differences in reporting standards with the risk that reproducibility may be threatened13.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [11] Rohani DA Faurholt-Jepsen M Kessing LV Bardram JE Correlations between objective behavioral features collected from mobile and wearable devices and depressive mood symptoms in patients with affective disorders: systematic review JMIR MHealth UHealth 2018 6 e165 30104184 PMC6111148

      Not checked in the free run.

    • cites [12] Melcher, J., Hays, R. & Torous, J. Digital phenotyping for mental health of college students: a clinical review. Evid. Based Ment. Health ebmental-2020-300180. 10.1136/ebmental-2020-300180 (2020). PMC10231503 32998937

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Some of the heterogeneity reported results from these disciplines having distinct conventions, with medical outputs putting more weight on sample and clinical outcome characteristics but often overlooking feature extraction and analysis description.

    “some of the heterogeneity we report results from these disciplines having distinct conventions, with medical outputs putting more weight on sample and clinical outcome characteristics but often overlooking feature extraction and analysis description.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The protocol was registered on PROSPERO 2019 CRD42019159929.

    “The protocol was registered on PROSPERO 2019 CRD42019159929.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    There have been attempts at standardising reporting standards for actively collected questionnaire data on mood, and guidelines exist for the reporting of observational data (STROBE) and multivariable prediction models (TRIPOD).

    “Whilst there have been attempts at standardising reporting standards for actively collected questionnaire data on mood66, and guidelines exist for the reporting of observational data (STROBE67) and multivariable prediction models (TRIPOD statement68)”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [66] Faurholt-Jepsen M Reporting guidelines on remotely collected electronic mood data in mood disorder (eMOOD)—recommendations Transl. Psychiatry 2019 9 1 10 31175283 PMC6555812

      Not checked in the free run.

    • cites [67] Elm Evon The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies Lancet 2007 370 1453 1457 18064739

      Not checked in the free run.

    • cites [68] Collins GS Reitsma JB Altman DG Moons KGM Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement BMJ 2015 350 g7594 25569120

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Disagreements at any stage of eligibility and data extraction were resolved by discussing with an additional reviewer.

    “Disagreements at any stage of eligibility and data extraction were resolved by discussing with an additional reviewer.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    It is possible that the associations reported are due to reporting bias, where investigators emphasise "significant" findings over "non-significant" ones.

    “It is possible that the associations we have reported are due to reporting bias, as mentioned in the previous section, where investigators emphasise “significant” findings over “non-significant” ones.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    These failures have important implications for the interpretation of findings.

    “have important implications for the interpretation of findings.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Agreement of >90% was reached for all reviewer pairs.

    “Agreement of >90% was reached for all reviewer pairs.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Some wearable devices may be more accurate on lighter skin tones and on men.

    “For example, some wearable devices may be more accurate on lighter skin tones69, and on men70.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [69] Colvonen PJ DeYoung PN Bosompra N-OA Owens RL Limiting racial disparities and bias for wearable devices in health science research Sleep 2020 43 zsaal159 PMC8477341 32893865

      Not checked in the free run.

    • cites [70] Nuss KJ Assessment of accuracy of overall energy expenditure measurements for the Fitbit Charge HR 2 and Apple Watch Am. J. Health Behav. 2019 43 498 505 31046881

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    To present low-level features clearly, the review combined them into broader low-level features, and some nuances between them were lost.

    “To present low-level features in a clear and meaningful way in this review, we combined them into broader low-level features and therefore some of the nuances between them were lost.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Potentially eligible studies' full texts were reviewed by one researcher, with a second researcher evaluating a random sample of 10% of all texts for validating purposes.

    “Potentially eligible studies’ full texts were reviewed by one researcher, with a second researcher evaluating a random sample of 10% of all texts for validating purposes.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductreasoning

    No single quality assessment tool was suitable because of the diversity of study types, so the review combined the AXIS tool and the Newcastle-Ottawa Scale (NOS).

    “No single quality assessment tool was suitable because of the ditabversity of study types. We, therefore, combined the Appraisal Tool for Cross-Sectional Studies (AXIS tool74 and the Newcastle–Ottawa Scale (NOS) for longitudinal studies75.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [74] Downes MJ Brennan ML Williams HC Dean RS Development of a critical appraisal tool to assess the quality of cross-sectional studies (AXIS) BMJ Open 2016 6 e011458 27932337 PMC5168618

      Not checked in the free run.

    • cites [75] Wells, G. et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomized studies in meta-analysis. 21, (2000).

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Data extraction was carried out on all studies, regardless of their quality assessment score.

    “Data extraction was carried out on all studies, regardless of their quality assessment score.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The eligibility process was documented according to PRISMA guidelines.

    “The eligibility process was documented according to PRISMA guidelines73.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [73] Moher D Liberati A Tetzlaff J Altman DG Group TP Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement PLoS Med. 2009 6 e1000097 19621072 PMC2707599

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Studies were checked for eligibility by two researchers independently screening titles and abstracts.

    “Studies were checked for eligibility by two researchers independently screening titles and abstracts.”
    Quote in the paperNot a substantial claim
67 of 67

  • The paper relies on this without support.

    Digital technology for tracking mood and behaviour has enormous potential for clinical management and improving depression research.

    “The use of digital technology to track mood and behaviour brings enormous potential for clinical management and the improvement of research in depression.”

  • The paper relies on this without support.

    Remote sensing combined with questionnaires may generate more objective and frequent measures of mood and core dimensions of mental disorders than retrospective accounts.

    “When combined with the completion of questionnaires, remote sensing may generate more objective and frequent measures of mood and other core dimensions of mental disorders, instead of relying on retrospective accounts of patients or participants.”

  • The paper relies on this without support.

    Missing data can arise from issues with technology (device and system failures) or from user-related issues that may be associated with depressed mood.

    “Missing data can arise from issues with technology, such as device and system failures, or from user-related issues which may be associated with depressed mood.”

  • The paper relies on this without support.

    Remote sensing potentially places a considerable burden on study participants.

    “Remote sensing is a relatively new technology that potentially places a considerable burden on study participants”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Entropy is attributed to both four and five studies

    Minor

    The results say entropy was reported in four studies, but the discussion says it was associated with mood in five studies.

  • Caldwell study year differs between table and reference

    Minor

    Table 1 dates the Caldwell study to 2019, but its cited reference dates it to 2009.

  • Cohort row does not add up

    Minor

    Table 2 lists 14 student, 6 community, and 3 clinical cohort studies, which sum to 23, not the reported total of 25.

    Reported 25 · recomputed 23

3 of 3

Cites

Flagged first, then by citations

3 of the 97 references couldn’t be matched to a known paper.

Research works printed but not linked: 7

  1. [44]Doryab, A., Min, J. K., Wiese, J., Zimmerman, J. & Hong, J. Detection of behavior change in people with depression. In Proc. of the 28th AAAI Conference on Artificial Intelligence, Vol. 5 (Québec City, QC, Canada, 2014).
  2. [86]Naismith SL Sleep disturbance relates to neuropsychological functioning in late-life depression J. Affect. Disord. 2011 132 139 145 21435728
  3. [88]Pillai V Steenburg LA Ciesla JA Roth T Drake CL A seven day actigraphy-based study of rumination and sleep disturbance among young adults with depressive symptoms J. Psychosom. Res. 2014 77 70 75 24913345
  4. [89]Pratap A The accuracy of passive phone sensors in predicting daily mood Depress. Anxiety 2019 36 72 81 30129691 PMC8491547
  5. [90]Robillard R Sleep-wake profiles predict longitudinal changes in manic symptoms and memory in young people with mood disorders J. Sleep Res. 2016 25 549 555 27242171
  6. [95]Vanderlind WM Sleep and sadness: exploring the relation among sleep, cognitive control, and depressive symptoms in young adults Sleep Med. 2014 15 144 149 24332565 PMC4217155
  7. [96]Wang, R. et al. Tracking depression dynamics in college students using mobile phone and wearable sensing. In Proc. ACM Interactive Mobile, Wearable and Ubiquitous Technology, Vol. 2, 1–26 (2018). PMC11501090 39449996

Cited by

Sorted by citations

Related

Authors
de Angel, V., Lewis, S., White, K., Oetzmann, C., Leightley, D., Oprea, E., Lavelle, G., Matcham, F., Pace, A., Mohr, D. C., Dobson, R., & Hotopf, M.
Affiliations
  • King's College London (GB)
  • Chelsea and Westminster Hospital NHS Foundation Trust (GB)
  • Northwestern University (US)
ORCID
Venue
npj Digital Medicine
Publisher
openRxiv
Published
2021-07-22
Volume
5
Issue
1
Pages
3-3
ISSN
2398-6352 (ISSN-L)
DOI
10.1101/2021.07.19.21260786
PMC
PMC8752685
OpenAlex
W4221044624
Cited by
227
References
97
Citations by year
2026: 56 · 2025: 54 · 2024: 63 · 2023: 32 · 2022: 22
FWCI
7.61
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • npj Digital Medicine (publishedVersion, cc-by)
  • medRxiv (acceptedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Sussex Research Online (University of Sussex) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Figshare (submittedVersion, cc-by)
  • Research Portal (King's College London) (submittedVersion, cc-by)
Funders
  • South London and Maudsley NHS Foundation Trust
  • Wellcome Trust
  • King's College Hospital NHS Foundation Trust
  • Scottish Government
  • European Federation of Pharmaceutical Industries and Associations (116074)
  • UK Research and Innovation
  • National Institute for Health and Care Research (NF-SI-0515-10102)
  • British Heart Foundation
  • Department of Health and Social Care
  • King's College London
  • University College London
  • European Commission (116074, HORIZON2020)
  • Public Health Agency
  • Innovative Medicines Initiative (116074)
  • Chief Scientist Office, Scottish Government Health and Social Care Directorate
  • NIHR Maudsley Biomedical Research Centre
  • Medical Research Council (MC_PC_17214)
  • Engineering and Physical Sciences Research Council
  • Economic and Social Research Council
  • Health and Social Care Research and Development Division
  • National Institute for Health Research Applied Research Collaboration South London
  • Health Data Research UK
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
Keywords
depression, passive monitoring, digital health tools, wearable devices, smartphone sensors, sleep, physical activity, location data, telephone use, major depressive disorder, reproducibility, reporting quality, machine learning prediction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review