GLOBEM: Cross-Dataset Generalization of Longitudinal Human Behavior Modeling
Open accessThe DOI matches its Crossref record; 81 of the 81 references it cites were found in a registry or an index.
Abstract
There is a growing body of research revealing that longitudinal passive sensing data from smartphones and wearable devices can capture daily behavior signals for human behavior modeling, such as depression detection. Most prior studies build and evaluate machine learning models using data collected from a single population. However, to ensure that a behavior model can work for a larger group of users, its generalizability needs to be verified on multiple datasets from different populations. We present the first work evaluating cross-dataset generalizability of longitudinal behavior models, using depression detection as an application. We collect multiple longitudinal passive mobile sensing datasets with over 500 users from two institutes over a two-year span, leading to four institute-year datasets. Using the datasets, we closely re-implement and evaluated nine prior depression detection algorithms. Our experiment reveals the lack of model generalizability of these methods. We also implement eight recently popular domain generalization algorithms from the machine learning community. Our results indicate that these methods also do not generalize well on our datasets, with barely any advantage over the naive baseline of guessing the majority. We then present two new algorithms with better generalizability. Our new algorithm, Reorder, significantly and consistently outperforms existing methods on most cross-dataset generalization setups. However, the overall advantage is incremental and still has great room for improvement. Our analysis reveals that the individual differences (both within and between populations) may play the most important role in the cross-dataset generalization challenge. Finally, we provide an open-source benchmark platform GLOBEM- short for Generalization of Longitudinal BEhavior Modeling - to consolidate all 19 algorithms. GLOBEM can support researchers in using, developing, and evaluating different longitudinal behavior modeling methods. We call for researchers' attention to model generalizability evaluation for future longitudinal human behavior modeling studies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Large-scale data collection efforts by the wearables community, e.g., the GLOBEM dataset [206] and the UK Biobank project [46,198], demonstrated the relative ease of collecting wearable movement in the wild, albeit without annotations.”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [206]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Diagnostic and Statistical Manual of Mental Disorders
American Psychiatric Association (2013) · doi:10.1176/appi.books.9780890425596 · Cited by 116,369
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 1×
“The second method, Reorder, creates a new task of solving a temporal reordering puzzle in addition to the main depression detection task, which was inspired by selfsupervised techniques in CV and NLP approaches [24, 77].”
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865 · cited 1×
“We used SMOTE to mitigate the data imbalance as datasets have different sizes [17].”
MethodFocal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581 · cited 1×
“Other loss functions that better deals with class imbalance, such as balanced cross entropy and focal loss [51], can be tested and evaluated on our platform.”
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387 · cited 2×
“If so, it provides the opportunity to leverage a wide range of transfer learning techniques in the domain adaptation area [74, 95].”
Domain-Adversarial Training of Neural Networks
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. (2017) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-58347-1_10 · Cited by 8,318
Beck, A. T., Steer, R. A., & Brown, G. (2011) · PsycTESTS Dataset · doi:10.1037/t00742-000 · Cited by 7,535
Vos, T., Allen, C., Arora, M., Barber, R. M., Bhutta, Z. A., Brown, A., Carter, A., Casey, D. C., Charlson, F. J., Chen, A. Z., et al. (2016) · The Lancet · doi:10.1016/s0140-6736(16)31678-6 · Cited by 7,253
An overview of statistical learning theory
Vapnik, V. N. (1999) · IEEE Transactions on Neural Networks · doi:10.1109/72.788640 · Cited by 6,266 · cited 2×
“(1) ERM (Empirical Risk Minimization) [89]: the basic model training techniques without particular design for domain generalization.”
BackgroundSGDR: Stochastic Gradient Descent with Warm Restarts
Loshchilov, I., & Hutter, F. (2016) · arXiv · doi:10.48550/arxiv.1608.03983 · Cited by 5,309 · cited 1×
“For all models, we used Adam as the optimizer and adopted a cosine annealing schedule to repeatedly decrease the learning rate and then restart with a higher learning rate [53], with an initial learning rate of 0.001, an annealing decay of 0.95, and an annealing step size of 100.”
InfluentialDeep CORAL: Correlation Alignment for Deep Domain Adaptation
Sun, B., & Saenko, K. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-49409-8_35 · Cited by 3,757 · cited 1×
“There are more techniques that seem promising, e.g. , CORAL [84], GroupDRO [81], ANDMask [75].”
Milliken, F. J., & Martins, L. L. (1996) · The Academy of Management Review · doi:10.5465/amr.1996.9605060217 · Cited by 3,413 · cited 1×
“In our case, such a shift comes not only from dataset differences ( i.e. , each subpopulation behavior pattern varies [21, 59]), but also from individual differences ( i.e. , each person behaves uniquely [67, 83]).”
2017 USENIX Annual Technical Conference (USENIX ATC'17)
Quoc, D. L., Beck, M. S., Bhatotia, P., Chen, R., Fetzer, C. W., & Strufe, T. (2017) · Cited by 3,091
One-shot learning of object categories
Fei-Fei, L., Fergus, R., & Perona, P. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.79 · Cited by 3,029 · cited 1×
“…( e.g. , [23, 113]); 2) Representation learning, which aims to learn desirable feature representations that can generalize across domains ( e.g. , [2, 28, 31]); 3) Learning strategy, which focuses on exploiting the training procedure to promote a model’s generalizability ( e.g. , [50, 88, 111]).”
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles
Noroozi, M., & Favaro, P. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_5 · Cited by 2,775 · cited 1×
“Moreover, as 10! total possible permutations is an overly large number, we followed the practice in [16] and predetermined a subset of 𝑃 = 200 permutations by following the Hamming-distance-based method [71].”
Proceedings of the 23rd international conference on Machine learning
Cohen, W. W., & Moore, A. (2006) · Cited by 2,647
Löwe, B., Wahl, I., Rose, M., Spitzer, C., Glaesmer, H., Wingenfeld, K., Schneider, A., & Brähler, E. (2010) · Journal of Affective Disorders · doi:10.1016/j.jad.2009.06.019 · Cited by 2,473 · cited 7×
“The weekly surveys include PANAS [22] (Institute1Year1 only), PHQ-4 [54] (remaining datasets).”
BackgroundInfluentialTorralba, A., & Efros, A. A. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995347 · Cited by 2,440 · cited 3×
“To quantify the difference among datasets, we first conducted a “Name-The-Dataset” task on our four datasets [86].”
A survey of mobile phone sensing
Lane, N., Miluzzo, E., Lu, H., Peebles, D., Choudhury, T., & Campbell, A. (2010) · IEEE Communications Magazine · doi:10.1109/mcom.2010.5560598 · Cited by 2,292
Self-citationDeep visual domain adaptation: A survey
Wang, M., & Deng, W. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.05.083 · Cited by 2,209
Nahum-Shani, I., Smith, S. N., Spring, B. J., Collins, L. M., Witkiewitz, K., Tewari, A., & Murphy, S. A. (2018) · Annals of Behavioral Medicine · doi:10.1007/s12160-016-9830-8 · Cited by 2,171 · cited 1×
“Such an adaptation process is necessary if we are moving towards the next step of providing personalized intervention experience [64, 108].”
BackgroundCultural diversity in organizations
Cox, T. (1993) · Medical Entomology and Zoology · Cited by 2,133
Proceedings of The 32nd International Conference on Machine Learning
Germain, M., Gregor, K., Murray, I., & Larochelle, H. (2015) · Cited by 2,020
The Balanced Accuracy and Its Posterior Distribution
Brodersen, K. H., Ong, C. S., Stephan, K. E., & Buhmann, J. M. (2010) · 2010 20th International Conference on Pattern Recognition · doi:10.1109/icpr.2010.764 · Cited by 1,816 · cited 1×
“Second, it has been shown more robust to class-imbalance compared to accuracy or F1 score [12].”
Simultaneous Deep Transfer Across Domains and Tasks
Tzeng, E., Hoffman, J., Darrell, T., & Saenko, K. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.463 · Cited by 1,149
Generalizing to unseen domains: A survey on domain generalization
Wang, J., Lan, C., Liu, C., Ouyang, Y., Qin, T., Lu, W., Chen, Y., Zeng, W., & Yu, P. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3178128 · Cited by 1,046 · cited 10×
“Such a finding is consistent with the results in DomainBed [36] and DeepDG [94].”
InfluentialOn Early Stopping in Gradient Descent Learning
Yao, Y., Rosasco, L., & Caponnetto, A. (2007) · Constructive Approximation · doi:10.1007/s00365-006-0663-2 · Cited by 970 · cited 1×
“Meanwhile, Clustering achieves the highest upper bound when the model training can early stop at the optimal epoch [112].”
BackgroundArjovsky, M., Bottou, L., Gulrajani, I., & Lopez-Paz, D. (2019) · arXiv · doi:10.48550/arxiv.1907.02893 · Cited by 927
The evolution of personality variation in humans and other animals.
Nettle, D. (2006) · American Psychologist · doi:10.1037/0003-066x.61.6.622 · Cited by 885 · cited 1×
“In our case, such a shift comes not only from dataset differences ( i.e. , each subpopulation behavior pattern varies [21, 59]), but also from individual differences ( i.e. , each person behaves uniquely [67, 83]).”
Individual differences in rational thought.
Stanovich, K. E., & West, R. F. (1998) · Journal of Experimental Psychology General · doi:10.1037//0096-3445.127.2.161 · Cited by 867 · cited 1×
“In our case, such a shift comes not only from dataset differences ( i.e. , each subpopulation behavior pattern varies [21, 59]), but also from individual differences ( i.e. , each person behaves uniquely [67, 83]).”
BackgroundSaeb, S., Zhang, M., Karr, C. J., Schueller, S. M., Corden, M. E., Kording, K. P., & Mohr, D. C. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4273 · Cited by 782 · cited 6×
“Similar results were also observed in [80, 98].”
BackgroundMethodInfluentialHuckins, J. F., daSilva, A. W., Wang, W., Hedlund, E., Rogers, C., Nepal, S. K., Wu, J., Obuchi, M., Murphy, E. I., Meyer, M. L., et al. (2020) · Journal of Medical Internet Research · doi:10.2196/20185 · Cited by 759 · cited 1×
“, a period of more than two weeks of depressive symptoms [72], and the prevalence has increased during the pandemic [44, 66].”
BackgroundSelf-citationCross-Domain Weakly-Supervised Object Detection Through Progressive Domain Adaptation
Inoue, N., Furuta, R., Yamasaki, T., & Aizawa, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00525 · Cited by 616
Deep Clustering with Convolutional Autoencoders
Guo, X., Liu, X., Zhu, E., & Yin, J. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-70096-0_39 · Cited by 523 · cited 1×
“Using all training data, we first followed [38, 99] to train a deep clustering model with convolutional auto-encoders (DCEC) that assigns data points with cluster indices.”
Self-citationCanzian, L., & Musolesi, M. (2015) · Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2750858.2805845 · Cited by 477
Ben-Zeev, D., Scherer, E. A., Wang, R., Xie, H., & Campbell, A. T. (2015) · Psychiatric Rehabilitation Journal · doi:10.1037/prj0000130 · Cited by 429 · cited 2×
“Earlier research focused on understanding the statistical relationship, such as correlation, between depressive symptoms and mobile sensing data [11, 80].”
BackgroundSelf-citationCross-dataset learning and person-specific normalisation for automatic Action Unit detection
Baltrusaitis, T., Mahmoud, M., & Robinson, P. (2015) · 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2015.7284869 · Cited by 412 · cited 1×
“Almost any applied ML area will encounter the cross-dataset domain generalization challenge for real-life deployment, such as object recognition [6, 15, 45, 86], NLP [70, 117], affective computing [20, 42], security [41, 116], and intelligent interaction [102, 105, 106].”
AWARE: Mobile Context Instrumentation Framework
Ferreira, D., Kostakos, V., & Dey, A. K. (2015) · Frontiers in ICT · doi:10.3389/fict.2015.00006 · Cited by 409 · cited 1×
“In Institute 1, the app was developed based on the AWARE Framework [29].”
MethodSelf-citationTracking Depression Dynamics in College Students Using Mobile Phone and Wearable Sensing
Wang, R., Wang, W., daSilva, A., Huckins, J. F., Kelley, W. M., Heatherton, T. F., & Campbell, A. T. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3191775 · Cited by 340
Self-citationMobile Sensing and Support for People With Depression: A Pilot Trial in the Wild
Wahle, F., Kowatsch, T., Fleisch, E., Rufer, M., & Weidt, S. (2016) · JMIR mHealth and uHealth · doi:10.2196/mhealth.5960 · Cited by 333 · cited 4×
“Recently, researchers have leveraged the results of correlation analysis to build ML models for depression diagnosis and detection [19, 69, 93].”
BackgroundMethodInfluentialUnobtrusive Sleep Monitoring using Smartphones
Chen, Z., Lin, M., Chen, F., Lane, N., Cardone, G., Wang, R., Li, T., Chen, Y., Choudhury, T., & Cambell, A. (2013) · Proceedings of the ICTs for improving Patients Rehabilitation Research Techniques · doi:10.4108/icst.pervasivehealth.2013.252148 · Cited by 300
State of the Art: Reproducibility in Artificial Intelligence
Gundersen, O. E., & Kjensmo, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11503 · Cited by 293
The ubiquity of habits and rules
Hodgson, G. M. (1997) · Cambridge Journal of Economics · doi:10.1093/oxfordjournals.cje.a013692 · Cited by 279 · cited 1×
“Although individuals have unique routines, these patterns would lead to continuous or even repetitive behavior trajectories from day to day [40].”
BackgroundSelfReg: Self-supervised Contrastive Regularization for Domain Generalization
Kim, D., Yoo, Y., Park, S., Kim, J., & Lee, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00948 · Cited by 262 · cited 2×
“, JiGen [16], SelfReg [46]) and NLP tasks (e.”
Cimtay, Y., & Ekmekcioglu, E. (2020) · Sensors · doi:10.3390/s20072034 · Cited by 251 · cited 1×
“Almost any applied ML area will encounter the cross-dataset domain generalization challenge for real-life deployment, such as object recognition [6, 15, 45, 86], NLP [70, 117], affective computing [20, 42], security [41, 116], and intelligent interaction [102, 105, 106].”
BackgroundA Human Depression Circuit Derived From Focal Brain Lesions
Padmanabhan, J. L., Cooke, D., Joutsa, J., Siddiqi, S. H., Ferguson, M., Darby, R. R., Soussand, L., Horn, A., Kim, N. Y., Voss, J. L., et al. (2019) · Biological Psychiatry · doi:10.1016/j.biopsych.2019.07.023 · Cited by 246 · cited 1×
“Using four datasets, we built a leave-one-dataset-out evaluation pipeline [73, 78].”
MethodChikersal, P., Doryab, A., Tumminia, M. J., Villalba, D. K., Dutcher, J. M., Liu, X., Cohen, S., Creswell, K. G., Mankoff, J. C., Creswell, J. D., et al. (2021) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3422821 · Cited by 216 · cited 7×
“Recently, researchers have leveraged the results of correlation analysis to build ML models for depression diagnosis and detection [19, 69, 93].”
BackgroundMethodInfluentialSelf-citationCross-dataset action detection
Cao, L., Liu, Z., & Huang, T. S. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539875 · Cited by 204 · cited 1×
“Almost any applied ML area will encounter the cross-dataset domain generalization challenge for real-life deployment, such as object recognition [6, 15, 45, 86], NLP [70, 117], affective computing [20, 42], security [41, 116], and intelligent interaction [102, 105, 106].”
Toss 'n' turn: smartphone as sleep and sleep quality detector
Min, J.-K., Doryab, A., Wiese, J., Amini, S., Zimmerman, J., & Hong, J. I. (2014) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557220 · Cited by 201
Risk Prediction for Late-Stage Ovarian Cancer by Meta-analysis of 1525 Patient Samples
Riester, M., Wei, W., Waldron, L., Culhane, A. C., Trippa, L., Oliva, E., Kim, S.-H., Michor, F., Huttenhower, C., Parmigiani, G., et al. (2014) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/dju048 · Cited by 195 · cited 1×
“Using four datasets, we built a leave-one-dataset-out evaluation pipeline [73, 78].”
MethodSmartGPA: how smartphones can assess and predict academic performance of college students
Wang, R., Harari, G., Hao, P., Zhou, X., & Campbell, A. T. (2015) · Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2750858.2804251 · Cited by 190
Self-citationModeling and Understanding Human Routine Behavior
Banovic, N., Buzali, T., Chevalier, F., Mankoff, J., & Dey, A. K. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858557 · Cited by 166 · cited 1×
“For example, people tend to have daily routines, which define the structure of and influence of almost every aspect of everyday behaviors [7].”
Self-citationXu, X., Chikersal, P., Doryab, A., Villalba, D. K., Dutcher, J. M., Tumminia, M. J., Althoff, T., Cohen, S., Creswell, K. G., Creswell, J. D., et al. (2019) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3351274 · Cited by 160 · cited 13×
“, once per one to two weeks or less frequent for depression labels [14, 100]), and the field experiments were not controlled.”
BackgroundMethodResultInfluentialSelf-citationZhang, J., Li, W., Ogunbona, P., & Xu, D. (2019) · ACM Computing Surveys · doi:10.1145/3291124 · Cited by 155
MetaSense: few-shot adaptation to untrained conditions in deep mobile sensing
Gong, T., Kim, Y., Shin, J., & Lee, S.-J. (2019) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3356250.3360020 · Cited by 123
Behavior vs. introspection: refining prediction of clinical depression via smartphone sensing data
Farhan, A. A., Yue, C., Morillo, R., Ware, S., Lu, J., Bi, J., Kamath, J., Russell, A., Bamis, A., & Wang, B. (2016) · 2016 IEEE Wireless Health (WH) · doi:10.1109/wh.2016.7764553 · Cited by 115 · cited 5×
“For instance, Farhan et al. [27] employed location features to detect biweekly depression and their top model achieved an F1 score of 82.0% on an eight-month dataset with 79 college students.”
InfluentialEnabling Hand Gesture Customization on Wrist-Worn Devices
Xu, X., Gong, J., Brum, C., Liang, L., Suh, B., Gupta, S. K., Agarwal, Y., Lindsey, L., Kang, R., Shahsavari, B., et al. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501904 · Cited by 114
Self-citationInferring mobility measures from GPS traces with missing data
Barnett, I., & Onnela, J.-P. “. (2018) · Biostatistics · doi:10.1093/biostatistics/kxy059 · Cited by 105 · cited 1×
“, [8, 14, 25]), and has been leveraged in a range of passive sensing and behavior modeling studies (e.”
BackgroundDetecting Drinking Episodes in Young Adults Using Smartphone-based Sensors
Bae, S. W., Ferreira, D., Suffoletto, B. P., Puyana, J. C., Kurtz, R., Chung, T. A., & Dey, A. K. (2017) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3090051 · Cited by 105 · cited 1×
“In the past decade, a large body of work has demonstrated the capability of longitudinal passive sensing and behavior modeling in many application areas, such as detecting physical health issues [5, 60, 65, 104], monitoring mental health status [96, 101], measuring job performance [57, 61], tracking education outcomes…”
Self-citationDifferentiating Higher and Lower Job Performers in the Workplace Using Mobile Sensing
Mirjafari, S., Masaba, K., Grover, T., Wang, W., Audia, P. G., Campbell, A. T., Chawla, N. V., Swain, V. D., Choudhury, M. D., Dey, A. K., et al. (2019) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3328908 · Cited by 99
Self-citationEarBuddy: Enabling On-Face Interaction via Wireless Earbuds
Xu, X., Shi, H., Yi, X., Liu, W., Yan, Y., Shi, Y., Mariakakis, A. T., Mankoff, J. C., & Dey, A. K. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376836 · Cited by 99 · cited 1×
“Almost any applied ML area will encounter the cross-dataset domain generalization challenge for real-life deployment, such as object recognition [6, 15, 45, 86], NLP [70, 117], affective computing [20, 42], security [41, 116], and intelligent interaction [102, 105, 106].”
BackgroundSelf-citationEvaluating the Reproducibility of Physiological Stress Detection Models
Mishra, V., Sen, S., Chen, G., Hao, T., Rogers, J. L., Chen, C.-H., & Kotz, D. (2020) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3432220 · Cited by 96 · cited 1×
“Perhaps the most relevant work is by Mishra et al. [62].”
BackgroundMattingly, S. M., Gregg, J. M., Audia, P. G., Bayraktaroglu, A. E., Campbell, A. T., Chawla, N. V., Swain, V. D., Choudhury, M. D., D’Mello, S. K., Dey, A. K., et al. (2019) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3290607.3299041 · Cited by 95
Self-citationCross-Dataset and Cross-Cultural Music Mood Prediction: A Case on Western and Chinese Pop Songs
Hu, X., & Yang, Y. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2523503 · Cited by 90 · cited 1×
“Almost any applied ML area will encounter the cross-dataset domain generalization challenge for real-life deployment, such as object recognition [6, 15, 45, 86], NLP [70, 117], affective computing [20, 42], security [41, 116], and intelligent interaction [102, 105, 106].”
Joint modeling of heterogeneous sensing data for depression assessment via multi-task learning
Lu, J., Shang, C., Yue, C., Morillo, R., Ware, S., Kamath, J., Bamis, A., Russell, A., Wang, B., & Bi, J. (2018) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3191753 · Cited by 85 · cited 4×
“4, including four epochs of a day (morning 6 am - 12 pm, afternoon 12 pm - 6 pm, evening 6 pm - 12 am, and night 12 am - 6 am) [19, 82, 101], the whole day [19, 80, 93, 98], and past two weeks [14, 27, 55, 93].”
BackgroundMethodInfluentialXu, X., Chikersal, P., Dutcher, J. M., Sefidgar, Y. S., Seo, W., Tumminia, M. J., Villalba, D. K., Cohen, S., Creswell, K. G., Creswell, J. D., et al. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3448107 · Cited by 65
Self-citationMetaPhys: few-shot adaptation for non-contact physiological measurement
Liu, X., Jiang, Z., Fromm, J., Xu, X., Patel, S., & McDuff, D. (2021) · ACM Conference on Health, Inference, and Learning (CHIL) · doi:10.1145/3450439.3451870 · Cited by 56
Self-citationHoffman, S. J., Sharma, R., & Ross, A. (2018) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00213 · Cited by 54 · cited 1×
“Almost any applied ML area will encounter the cross-dataset domain generalization challenge for real-life deployment, such as object recognition [6, 15, 45, 86], NLP [70, 117], affective computing [20, 42], security [41, 116], and intelligent interaction [102, 105, 106].”
Nickels, S., Edwards, M. D., Poole, S. F., Winter, D., Gronsbell, J. L., Rozenkrants, B., Miller, D. P., Fleck, M. S., McLean, A., Peterson, B. E., et al. (2021) · JMIR Mental Health · doi:10.2196/27589 · Cited by 53 · cited 1×
“Recently, researchers have leveraged the results of correlation analysis to build ML models for depression diagnosis and detection [19, 69, 93].”
BackgroundSelf-citationNepal, S. K., Wang, W., Vojdanovski, V., Huckins, J. F., DaSilva, A. W., Meyer, M. L., & Campbell, A. T. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502043 · Cited by 53
Self-citationTypeOut: Leveraging Just-in-Time Self-Affirmation for Smartphone Overuse Reduction
Xu, X., Zou, T., Han, X. L., Li, Y., Wang, R., Yuan, T., Wang, Y., Shi, Y., Mankoff, J. C., & Dey, A. K. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517476 · Cited by 53 · cited 1×
“Such an adaptation process is necessary if we are moving towards the next step of providing personalized intervention experience [64, 108].”
Self-citationLeveraging Multi-Modal Sensing for Mobile Health: A Case Review in Chronic Pain
Aung, M. H., Alquaddoomi, F., Hsieh, C.-K., Rabbi, M., Yang, L., Pollak, J., Estrin, D. L., & Choudhury, T. (2016) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2016.2565381 · Cited by 45
Understanding User Behavior For Document Recommendation
Xu, X., Awadallah, A. H., Dumais, S., Omar, F., Popp, B., Rounthwaite, R., & Jahanbakhsh, F. (2020) · International World Wide Web Conference (WWW) · doi:10.1145/3366423.3380071 · Cited by 44 · cited 1×
“The success in the last decade of using mobile sensing for depression research has attracted increasing attention of researchers from various communities [49, 103].”
BackgroundSelf-citationPassively-sensed Behavioral Correlates of Discrimination Events in College Students
Sefidgar, Y. S., Seo, W., Kuehn, K. S., Althoff, T., Browning, A., Riskin, E. A., Nurius, P. S., Dey, A. K., & Mankoff, J. C. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359216 · Cited by 38 · cited 3×
“4, including four epochs of a day (morning 6 am - 12 pm, afternoon 12 pm - 6 pm, evening 6 pm - 12 am, and night 12 am - 6 am) [19, 82, 101], the whole day [19, 80, 93, 98], and past two weeks [14, 27, 55, 93].”
BackgroundMethodSelf-citationXu, X., Nemati, E., Vatanparvar, K., Nathan, V., Ahmed, T., Rahman, M. M., McCaffrey, D., Kuang, J., & Gao, J. A. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3448124 · Cited by 38
Self-citationThink Outside the Dataset: Finding Fraudulent Reviews using Cross-Dataset Analysis
Nilizadeh, S., Aghakhani, H., Gustafson, E., Kruegel, C., & Vigna, G. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313647 · Cited by 33 · cited 1×
“Almost any applied ML area will encounter the cross-dataset domain generalization challenge for real-life deployment, such as object recognition [6, 15, 45, 86], NLP [70, 117], affective computing [20, 42], security [41, 116], and intelligent interaction [102, 105, 106].”
Huckins, J. F., DaSilva, A. W., Hedlund, E. L., Murphy, E. I., Rogers, C., Wang, W., Obuchi, M., Holtzheimer, P. E., Wagner, D. D., & Campbell, A. T. (2020) · JMIR Mental Health · doi:10.2196/16684 · Cited by 33 · cited 1×
“The datasets were collected from two different Carnegie-classified R-1 universities in the United States using a similar method [43, 82].”
MethodSelf-citationYan, R., Ringwald, W. R., Vega, J., Kehl, M., Bae, S. W., Dey, A. K., Low, C. A., Wright, A. G. C., & Doryab, A. (2021) · PsyArXiv (OSF Preprints) · doi:10.1016/j.future.2022.02.010 · Cited by 25
Self-citationCross-domain dependency parsing using a deep linguistic grammar
Zhang, Y., & Wang, R. (2009) · doi:10.3115/1687878.1687932 · Cited by 23 · cited 7×
“It has been applied to domain generalization problems in CV tasks ( e.g. , JiGen [16], SelfReg [46]) and NLP tasks ( e.g. , BERT [24]).”
BackgroundInfluentialRecognizing Unintentional Touch on Interactive Tabletop
Xu, X., Yu, C., Wang, Y., & Shi, Y. (2020) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3381011 · Cited by 18
Self-citationVega, J., Bell, B. T., Taylor, C. S., Xie, J., Ng, H., Honary, M., & McNaney, R. (2022) · JMIR Mental Health · doi:10.2196/32146 · Cited by 8
Research works printed but not linked: 38
- Daniel A Adler, Fei Wang, David C Mohr, and Tanzeem Choudhury. 2022. Machine learning for passive mental health symptom prediction: Generalization across different longitudinal mobile sensing studies. PLOS ONE (2022), 20.
- Dror Ben-Zeev, Rachel Brian, Rui Wang, Weichen Wang, Andrew T Campbell, Min SH Aung, Michael Merrill, Vincent WS Tseng, Tanzeem Choudhury, Marta Hauser, et al. 2017. CrossCheck: Integrating self-report, behavioral sensing, and smartphone use to identify digital indicators of psychotic relapse. Psychiatric rehabilitation journal 40, 3 (2017), 266.
- Terry C Camacho, Robert E Roberts, Nancy B Lazarus, George A Kaplan, and Richard D Cohen. 1991. Physical activity and depression: evidence from the Alameda County Study. American journal of epidemiology 134, 2 (1991), 220--231.
- Fabio Maria Carlucci, Antonio D'Innocente, Silvia Bucci, Barbara Caputo, and Tatiana Tommasi. 2019. Domain Generalization by Solving Jigsaw Puzzles. arXiv:1903.06864 [cs] (April 2019). http://arxiv.org/abs/1903.06864 arXiv: 1903.06864. · arXiv
- 10.5555/1622407.1622416 · doi:10.5555/1622407.1622416
- John R Crawford and Julie D Henry. 2004. The Positive and Negative Affect Schedule (PANAS): Construct validity, measurement properties and normative data in a large non-clinical sample. British journal of clinical psychology 43, 3 (2004), 245--265.
- Hal Daumé III. 2007. Frustratingly Easy Domain Adaptation. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics. Association for Computational Linguistics, Prague, Czech Republic, 256--263. https://www.aclweb.org/anthology/P07-1033
- Afsaneh Doryab, Prerna Chikarsel, Xinwen Liu, and Anind K Dey. 2018. Extraction of behavioral features from smartphone and wearable data. arXiv preprint arXiv:1812.10394 (2018). · arXiv
- Qi Dou, Daniel C. Castro, Konstantinos Kamnitsas, and Ben Glocker. 2019. Domain Generalization via Model-Agnostic Learning of Semantic Features. arXiv:1910.13580 [cs] (Oct. 2019). http://arxiv.org/abs/1910.13580 arXiv: 1910.13580. · arXiv
- Eiko I Fried, Jessica K Flake, and Donald J Robinaugh. 2022. Revisiting the theoretical and methodological foundations of depression measurement. Nature Reviews Psychology (2022), 1--11.
- 10.1007/978-3-319-58347-1_10 · DOI · Paper page
- Taesik Gong, Yewon Kim, Adiba Orzikulova, Yunxin Liu, Sung Ju Hwang, Jinwoo Shin, and Sung-Ju Lee. 2021. DAPPER: Performance Estimation of Domain Adaptation in Mobile Sensing. arXiv preprint arXiv:2111.11053 (2021). · arXiv
- Ian H Gotlib and Jutta Joormann. 2010. Cognition and depression: current status and future directions. Annual review of clinical psychology 6 (2010), 285--312.
- Joy He-Yueya, Benjamin Buck, Andrew Campbell, Tanzeem Choudhury, John M Kane, Dror Ben-Zeev, and Tim Althoff. 2020. Assessing the relationship between routine and schizophrenia symptoms with passively sensed measures of behavioral stability. NPJ schizophrenia 6, 1 (2020), 1--8.
- Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. 2021. Wilds: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning. PMLR, 5637--5664.
- Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M. Hospedales. 2017. Learning to Generalize: Meta-Learning for Domain Generalization. arXiv:1710.03463 [cs] (Oct. 2017). http://arxiv.org/abs/1710.03463 arXiv: 1710.03463. · arXiv
- Donatella Marazziti, Giorgio Consoli, Michela Picchetti, Marina Carlini, and Luca Faravelli. 2010. Cognitive impairment in major depression. European journal of pharmacology 626, 1 (2010), 83--86.
- Matthew BA McDermott, Shirly Wang, Nikki Marinsek, Rajesh Ranganath, Luca Foschini, and Marzyeh Ghassemi. 2021. Reproducibility in machine learning for health research: Still a ways to go. Science Translational Medicine 13, 586 (2021), eabb1655.
- Tom M Mitchell and Tom M Mitchell. 1997. Machine learning. Vol. 1. McGraw-hill New York.
- Ebrahim Nemati, Xuhai Xu, Viswam Nathan, Korosh Vatanparvar, Tousif Ahmed, Md Mahbubur Rahman, Dan McCaffrey, Jilong Kuang, and Alex Gao. 2022. Ubilung: Multi-Modal Passive-Based Lung Health Assessment. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 551--555.
- Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nati Srebro. 2017. Exploring generalization in deep learning. Advances in neural information processing systems 30 (2017).
- Giambattista Parascandolo, Alexander Neitz, Antonio Orvieto, Luigi Gresele, and Bernhard Schölkopf. 2020. Learning explanations that are hard to vary. arXiv preprint arXiv:2009.00329 (2020). · arXiv
- Vihari Piratla, Praneeth Netrapalli, and Sunita Sarawagi. 2020. Efficient Domain Generalization via Common-Specific Low-Rank Decomposition. arXiv:2003.12815 [cs, stat] (April 2020). http://arxiv.org/abs/2003.12815 arXiv: 2003.12815. · arXiv
- Alan Ramponi and Barbara Plank. 2020. Neural Unsupervised Domain Adaptation in NLP---A Survey. arXiv:2006.00632 (2020). · arXiv
- Babak Roshanaei-Moghaddam, Wayne J Katon, and Joan Russo. 2009. The longitudinal effects of depression on physical activity. General hospital psychiatry 31, 4 (2009), 306--315.
- Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. 2019. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731 (2019). · arXiv
- 10.1037/0096-3445.127.2.161 · doi:10.1037/0096-3445.127.2.161 · Paper page
- Michael E. Thase. 1998. Depression, sleep, and antidepressants. The Journal of Clinical Psychiatry (1998).
- Norifumi Tsuno, Alain Besset, and Karen Ritchie. 2005. Sleep and depression. The Journal of Clinical Psychiatry (2005).
- Julio Vega, Meng Li, Kwesi Aguillera, Nikunj Goel, Echhit Joshi, Kirtiraj Khandekar, Krina C Durica, Abhineeth R Kunta, and Carissa A Low. 2021. Reproducible Analysis Pipeline for Data Streams: Open-Source Software to Process Data Collected With Mobile Devices. Frontiers in Digital Health 3 (2021).
- Rui Wang, Fanglin Chen, Zhenyu Chen, Tianxing Li, Gabriella Harari, Stefanie Tignor, Xia Zhou, Dror Ben-Zeev, and Andrew T. Campbell. 2014. StudentLife: Assessing mental health, academic performance and behavioral trends of college students using smartphones. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 3--14.
- Xuhai Xu, Han Zhang, Yasaman S Sefidgar, Yiyi Ren, Xin Liu, Woosuk Seo, Jennifer Brown, Kevin Scott Kuehn, Mike A Merrill, Paula S Nurius, Shwetak Patel, Tim Althoff, Margaret E Morris, Eve A. Riskin, Jennifer Mankoff, and Anind Dey. 2022. GLOBEM Dataset: Multi-Year Datasets for Longitudinal Human Behavior Modeling Generalization. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track. https://openreview.net/forum?id=GKOa7yNH8Uh
- Jenny Yang, Andrew AS Soltan, and David A Clifton. 2022. Machine learning generalizability across healthcare settings: insights from multi-site COVID-19 screening. npj Digital Medicine 5, 1 (2022), 1--8.
- Yi Yao and Gianfranco Doretto. 2010. Boosting for transfer learning with multiple sources. In 2010 IEEE computer society conference on computer vision and pattern recognition. IEEE, 1855--1862.
- Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. 2018. mixup: Beyond Empirical Risk Minimization. arXiv:1710.09412 [cs, stat] (April 2018). http://arxiv.org/abs/1710.09412 arXiv: 1710.09412. · arXiv
- Han Zhang, Margaret E. Morris, Paula S. Nurius, Kelly Mack, Jennifer Brown, Kevin S. Kuehn, Yasaman S. Sefidgar, Xuhai Xu, Eve A. Riskin, Anind K. Dey, and Jennifer Mankoff. 2022. Impact of Online Learning in the Context of COVID-19 on Undergraduates with Disabilities and Mental Health Concerns. ACM Transactions on Accessible Computing (July 2022), 3538514.
- Xu Zhang, Junghyun Kim, Qingwei Lin, Keunhak Lim, Shobhit O Kanaujia, Yong Xu, Kyle Jamieson, Aws Albarghouthi, Si Qin, Michael J Freedman, et al. 2019. Cross-dataset time series anomaly detection for cloud systems. In 2019 USENIX Annual Technical Conference (USENIX ATC 19). 1063--1076.
- Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. 2021. Domain Generalization in Vision: A Survey. arXiv:2103.02503 [cs] (July 2021). http://arxiv.org/abs/2103.02503 arXiv: 2103.02503. · arXiv
Cited by
Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Data
Xu, X., Yao, B., Dong, Y., Gabriel, S., Yu, H., Hendler, J., Ghassemi, M., Dey, A. K., & Wang, D. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3643540 · Cited by 278
Self-citationYang, Z., Xu, X., Yao, B., Rogers, E., Zhang, S., Intille, S., Shara, N., Gao, G. G., & Wang, D. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3659625 · Cited by 120
Self-citationZhang, S., Yu, J., Xu, X., Yin, C., Lu, Y., Yao, B., Tory, M., Padilla, L. M., Caterino, J., Zhang, P., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642343 · Cited by 99
Self-citationDigital Phenotyping for Stress, Anxiety, and Mild Depression: Systematic Literature Review
Choi, A., Ooi, A., & Lottridge, D. M. (2023) · JMIR mhealth and uhealth · doi:10.2196/40689 · Cited by 108
Hong, Z., Li, Z., Zhong, S., Lyu, W., Wang, H., Ding, Y., He, T., & Zhang, D. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3659597 · Cited by 67
Metadata
- Authors
- Xu, X., Liu, X., Zhang, H., Wang, W., Nepal, S., Sefidgar, Y., Seo, W., Kuehn, K. S., Huckins, J. F., Morris, M. E., Nurius, P. S., Riskin, E. A., Patel, S., Althoff, T., Campbell, A., Dey, A. K., & Mankoff, J.
- Affiliations
- University of Washington (US)
- Dartmouth College (US)
- Stevens Institute of Technology (US)
- ORCID
- Venue
- Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Published
- 2022-12-21
- Volume
- 6
- Issue
- 4
- Pages
- 1-34
- ISSN
- 2474-9567 (ISSN-L)
- OpenAlex
- W4319837253
- Cited by
- 96
- References
- 77
- Citations by year
- 2026: 26 · 2025: 24 · 2024: 35 · 2023: 10 · 2022: 1
- FWCI
- 16.07
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies (publishedVersion, cc-by)
- Funders
- University of Washington
- the National Science Foundation (EDA-2009977, CHS-2016365, CHS1941537, IIS1816687, IIS7974751)
- the National Institute on Disability, Independent Living and Rehabilitation Research (90DPGE0003-01)
- Samsung Research America
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Keywords
- cross-dataset generalization, smartphone sensing, wearable devices, depression detection, human behavior modeling, domain generalization, individual differences, benchmark platform
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.