ExtraSensory App: Data Collection In-the-Wild with Rich User Interface to Self-Report Behavior
The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
We introduce a mobile app for collecting in-the-wild data, including sensor measurements and self-reported labels describing people's behavioral context (e.g., driving, eating, in class, shower). Labeled data is necessary for developing context-recognition systems that serve health monitoring, aging care, and more. Acquiring labels without observers is challenging and previous solutions compromised ecological validity, range of behaviors, or amount of data. Our user interface combines past and near-future self-reporting of combinations of relevant context-labels. We deployed the app on the personal smartphones of 60 users and analyzed quantitative data collected in-the-wild and qualitative user-experience reports. The interface's flexibility was important to gain frequent, detailed labels, support diverse behavioral situations, and engage different users: most preferred reporting their past behavior through a daily journal, but some preferred reporting what they're about to do. We integrated insights from this work back into the app, which we make available to researchers for conducting in-the-wild studies.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Although many commercial fitness trackers or smartwatches adopt wrist-mountable form due to the ease of accessibility, no consensus has been made regarding the positioning of sensors and their data acquisition details.”
From Sensor Data Acquisition and Multimodal Sensor Fusion for Human Activity Recognition Using Deep Learning (2019) · cites this paper as [16]
In the citing paper
“However, as the dataset adopts heterogeneous sensors on limited position of the human body, it is not sufficient to evaluate the recognition performance according to the sensor position and sampling rate.”
From Sensor Data Acquisition and Multimodal Sensor Fusion for Human Activity Recognition Using Deep Learning (2019) · cites this paper as [16]
Citations
Cites
A Survey Method for Characterizing Daily Life Experience: The Day Reconstruction Method
Kahneman, D., Krueger, A. B., Schkade, D. A., Schwarz, N., & Stone, A. A. (2004) · Science · doi:10.1126/science.1103572 · Cited by 3,422 · cited 4×
“DRM was used in [26] for eating detection, but the participants struggled remembering when they were eating.”
BackgroundMethodInfluentialFlow and the Foundations of Positive Psychology: The Collected Works of Mihaly Csikszentmihalyi
Csikszentmihalyi, M. (2014) · doi:10.1007/978-94-017-9088-8 · Cited by 1,653
Activity sensing in the wild: a field trial of ubifit garden
Consolvo, S., McDonald, D. W., Toscos, T., Chen, M. Y., Froehlich, J., Harrison, B., Klasnja, P., LaMarca, A., LeGrand, L., Libby, R., et al. (2008) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357335 · Cited by 1,131
Mel Frequency Cepstral Coefficients For Music Modeling.
Logan, B. (2000) · doi:10.5281/zenodo.1416443 · Cited by 913
Experience Sampling: Promises and Pitfalls, Strengths and Weaknesses
Scollon, C. N., Kim-Prieto, C., & Diener, E. (2003) · Journal of Happiness Studies · doi:10.1023/a:1023605205115 · Cited by 825 · cited 6×
“Much like survey-based studies with ESM or DRM [23, 11], we address the multi-aspect nature of behavioral context and allow users to report combinations of activities, as well as location, company, body posture and more.”
BackgroundMethodInfluentialStisen, A., Blunck, H., Bhattacharya, S., Prentow, T. S., Kjærgaard, M. B., Dey, A., Sonne, T., & Jensen, M. M. (2015) · Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems · doi:10.1145/2809695.2809718 · Cited by 714 · cited 1×
“Data collection in-the-wild raises technical difficulties related to interruptions in sensor recording and diversity in phonedevices [24] and device placement [15].”
BackgroundDetecting activities of daily living in first-person camera views
Pirsiavash, H., & Ramanan, D. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248010 · Cited by 649 · cited 2×
“In some studies research assistants annotated the images [20, 7], compromising the privacy of the participants and their surrounding.”
BackgroundThe Mobile Sensing Platform: An Embedded Activity Recognition System
Choudhury, T., Borriello, G., Consolvo, S., Haehnel, D., Harrison, B., Hemingway, B., Hightower, J., Klasnja, P. "., Koscher, K., LaMarca, A., et al. (2008) · IEEE Pervasive Computing · doi:10.1109/mprv.2008.39 · Cited by 598
Validity and Reliability of the Experience-Sampling Method
Csikszentmihalyi, M., & Larson, R. (2014) · Flow and the Foundations of Positive Psychology · doi:10.1007/978-94-017-9088-8_3 · Cited by 578
King, A. C., Hekler, E. B., Grieco, L. A., Winter, S. J., Sheats, J. L., Buman, M. P., Banerjee, B., Robinson, T. N., & Cirimele, J. (2013) · PLoS ONE · doi:10.1371/journal.pone.0062613 · Cited by 332 · cited 1×
“) is desirable for many domains, such as health management [10, 12], aging care [14, 16] and office assistant systems [30].”
Background
Research works printed but not linked: 1
- 10.5555/2354409.2355089 · doi:10.5555/2354409.2355089
Cited by
Asynchronous Online Federated Learning for Edge Devices with Non-IID Data
Chen, Y., Ning, Y., Slawski, M., & Rangwala, H. (2020) · 2020 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata50022.2020.9378161 · Cited by 431
“• ExtraSensory Dataset 5 : Mobile device sensor data (e.g., high-frequency motion-reactive sensors, location services, audio, watch compass) and watch sensor data (accelerator) collected from 60 users; performing any of 51 activities [41].”
Chung, S., Lim, J., Noh, K. J., Kim, G., & Jeong, H. (2019) · Sensors · doi:10.3390/s19071716 · Cited by 215
Removal as a Method: A Fourth Wave HCI Approach to Understanding the Experience of Self-Tracking
Homewood, S., Karlsson, A., & Vallgårda, A. (2020) · Proceedings of the 2020 ACM Designing Interactive Systems Conference · doi:10.1145/3357236.3395425 · Cited by 87
Context-Aware Human Activity Recognition (CAHAR) in-the-Wild Using Smartphone Accelerometer
Asim, Y., Azam, M. A., Ehatisham-ul-Haq, M., Naeem, U., & Khalid, A. (2020) · IEEE Sensors Journal · doi:10.1109/jsen.2020.2964278 · Cited by 75
Advanced IoT-Based Human Activity Recognition and Localization Using Deep Polynomial Neural Network
Khan, D., Alshahrani, A. S., Almjally, A., Mudawi, N. A., Algarni, A. M., Alnowaiser, K., & Jalal, A. (2024) · IEEE Access · doi:10.1109/access.2024.3420752 · Cited by 74
Asynchronous Federated Learning for Sensor Data with Concept Drift
Chen, Y., Chai, Z., Cheng, Y., & Rangwala, H. (2021) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata52589.2021.9671924 · Cited by 69
Physique-Based Human Activity Recognition Using Ensemble Learning and Smartphone Sensors
Choudhury, N. A., Moulik, S., & Roy, D. S. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3077563 · Cited by 60
Energy Efficiency Office. Building Technologies Office USDOE Office of Energy Efficiency and Renewable Energy (EERE), Roth, A., USDOE Office of Energy Efficiency and Renewable Energy (EERE), & Reyna, J. (2019) · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · doi:10.2172/1580329 · Cited by 59
MUBS: A Personalized Recommender System for Behavioral Activation in Mental Health
Rohani, D. A., Lopategui, A. Q., Tuxen, N., Faurholt‐Jepsen, M., Kessing, L. V., & Bardram, J. E. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376879 · Cited by 71
“Patients were compensated a lump sum for their participation (which is a standard procedure in most HCI studies [22,48,49,52]).”
MethodTarafdar, P., & Bose, I. (2020) · Decision Support Systems · doi:10.1016/j.dss.2020.113426 · Cited by 54
People Like Me: Designing for Reflection on Aggregate Cohort Data in Personal Informatics Systems
Feustel, C., Aggarwal, S., Lee, B., & Wilcox, L. G. (2018) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3264917 · Cited by 59
“Individuals now have the ability to collect multiple streams of data relating to their behavior [34, 64], mood [22], and physiology [57].”
BackgroundAdaimi, R., & Thomaz, E. (2019) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3351228 · Cited by 59
“Methods that require direct and regular individual input, including daily self-recall and experience sampling [20, 43, 47] are popular and straightforward to implement, but are subject to biases and memory recollection errors.”
InfluentialI Can't Be Myself: Effects of Wearable Cameras on the Capture of Authentic Behavior in the Wild
Alharbi, R., Stump, T. K., Vafaie, N., Pfammatter, A. F., Spring, B. J., & Alshurafa, N. I. (2018) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3264900 · Cited by 56
“In obtaining self-report data related to activities, Vaizman et al. [74] have shown that incorporating simple online machine learning models that predict activities can provide further detailed subjective description of the participants activity.”
MethodAsynchronous Online Federated Learning for Edge Devices.Preprint
Chen, Y., Ning, Y., & Rangwala, H. (2019) · arXiv (Cornell University) · Cited by 49
Sultana, M., Al-Jefri, M., & Lee, J. (2020) · JMIR mHealth and uHealth · doi:10.2196/17818 · Cited by 46
LifeSnaps, a 4-month multi-modal dataset capturing unobtrusive snapshots of our lives in the wild
Yfantidou, S., Karagianni, C., Efstathiou, S., Vakali, A. I., Palotti, J., Giakatos, D. P., Marchioro, T., Kazlouski, A., Ferrari, E., & Girdzijauskas, Š. (2022) · Scientific Data · doi:10.1038/s41597-022-01764-x · Cited by 61
“Sensing data collected in the wild are likely to provide a more accurate and representative picture of an individual’s behavior than a snapshot of data collected in a lab 38 , thus facilitating the development of ecologically valid systems that work well in the real world 39 .”
BackgroundMyMove: Facilitating Older Adults to Collect In-Situ Activity Labels on a Smartwatch with Speech
Kim, Y.-H., Chou, D., Lee, B., Danilovich, M., Lazar, A., Conroy, D. E., Kacorri, H., & Choe, E. K. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517457 · Cited by 39
Rohani, D. A., Tuxen, N., Lopategui, A. Q., Faurholt-Jepsen, M., Kessing, L. V., & Bardram, J. E. (2019) · Proceedings of the 13th EAI International Conference on Pervasive Computing Technologies for Healthcare · doi:10.1145/3329189.3329214 · Cited by 33
Ji, G., Woo, J., Lee, G., Msigwa, C., Bernard, D., & Yun, J. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3400509 · Cited by 41
Lee, S., Lim, Y., & Lim, K. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102153 · Cited by 39
Ehatisham-ul-Haq, M., Azam, M. A., Amin, Y., & Naeem, U. (2020) · IEEE Access · doi:10.1109/access.2020.2964237 · Cited by 34
“For validating our proposed scheme, we opted to use the publicly available ExtraSensory [15] dataset that conforms to the pipeline of C2FHAR model.”
MethodRobotic-Based Well-Being Monitoring and Coaching System for the Elderly in Their Daily Activities
Calatrava-Nicolás, F. M., Maestro, E. G., Bautista‐Salinas, D., Ortiz, F. J., González, J. R., Vera-Repullo, J. A., Buendía, M. J., Méndez, I., Esteban, C. R., & Mozos, Ó. M. (2021) · Sensors · doi:10.3390/s21206865 · Cited by 37
Lee, H., Kang, S., & Lee, U. (2022) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3534623 · Cited by 31
“Accordingly, such potentials are opening up new areas of research across diverse application domains by leveraging multiple streams of in-the-wild data sources, such as, everyday behavior [89, 90], user preferences [86], physical activities [88], personal productivity [60], emotions [54, 107], mental wellness [15, 104…”
BackgroundFederated Multi-task Learning with Hierarchical Attention for Sensor Data Analytics
Chen, Y., Ning, Y., Chai, Z., & Rangwala, H. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207508 · Cited by 32
“, location services, audio, accelerator) collected from 60 users [14].”
BackgroundAlkinani, M. H., Almazroi, A. A., Adhikari, M., & Menon, V. G. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3145403 · Cited by 26
“In this section, the experimental analysis of the proposed method is presented over the transport mode dataset 1 [29], [30].”
MethodRedefining Activity Tracking Through Older Adults' Reflections on Meaningful Activities
Wang, Y., Li, M., Kim, Y., Lee, B., Danilovich, M., Lazar, A., Conroy, D. E., Kacorri, H., & Choe, E. K. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642170 · Cited by 26
CascadeHD: Efficient Many-Class Learning Framework Using Hyperdimensional Computing
Kim, Y., Kim, J., & Imani, M. (2021) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18074.2021.9586235 · Cited by 26
Using Smartphone Accelerometer for Human Physical Activity and Context Recognition in-the-Wild
Ehatisham-ul-Haq, M., Azam, M. A., Asim, Y., Amin, Y., Naeem, U., & Khalid, A. (2020) · Procedia Computer Science · doi:10.1016/j.procs.2020.10.007 · Cited by 16
ContextGPT: Infusing LLMs Knowledge into Neuro-Symbolic Activity Recognition Models
Arrotta, L., Bettini, C., Civitarese, G., & Fiori, M. (2024) · IEEE International Conference on Smart Computing · doi:10.1109/smartcomp61445.2024.00029 · Cited by 19
Ontology driven human activity recognition in heterogeneous sensor measurements
Hooda, D., & Rani, R. (2020) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-020-01835-0 · Cited by 16
Bookly: An Interactive Everyday Artifact Showing the Time of Physically Accumulated Reading Activity
Ju, S., Lee, K.-R., Kim, S., & Park, Y. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300614 · Cited by 19
“In this, there were several approaches to provide self-reflection by tracking certain activity and visualizing the collected data to guide the continuity of behavior [6, 8, 26].”
MethodSensor Positioning and Data Acquisition for Activity Recognition using Deep Learning
Chung, S., Lim, J., Noh, K. J., Kim, G. G., & Jeong, H. T. (2018) · International Conference on Information and Communication Technology Convergence (ICTC) · doi:10.1109/ictc.2018.8539473 · Cited by 13
“The ExtraSensory dataset [1], [2] aims to analyze behavioral context in-the-wild from mobile sensors.”
BackgroundUncovering Bias in Personal Informatics
Yfantidou, S., Sermpezis, P., Vakali, A. I., & Baeza-Yates, R. A. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3610914 · Cited by 25
“Existing open datasets were small- to medium-sized [97, 104] or domain-constrained, e.”
BackgroundAdapting to Unknown Conditions in Learning-Based Mobile Sensing
Gong, T., Kim, Y., Choi, R., Shin, J., & Lee, S.-J. (2021) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2021.3061130 · Cited by 14
“ExtraSensory [56] dataset was collected with 60 participants in the free-living environment for seven days.”
MethodTlachac, M., Gerych, W., Agrawal, K., Litterer, B. R., Jurovich, N., Thatigotla, S., Thadajarassiri, J., & Rundensteiner, E. A. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020224 · Cited by 15
“Consequently, existing publicly available digital phenotype datasets only have between 48 and 369 participants [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17].”
BackgroundDELFI: Mislabelled Human Context Detection Using Multi-Feature Similarity Linking
Mansoor, H., Gerych, W., Buquicchio, L., Chandrasekaran, K., Agu, E., & Rundensteiner, E. A. (2019) · IEEE Visualization in Data Science (VDS) · doi:10.1109/vds48975.2019.8973382 · Cited by 11
“This type of user-labeled data can arise in studies where users are expected to report their physical activities [23].”
BackgroundARGUS: Interactive visual analysis of disruptions in smartphone-detected Bio-Behavioral Rhythms
Mansoor, H., Gerych, W., Alajaji, A. S., Buquicchio, L., Chandrasekaran, K., Agu, E., & Rundensteiner, E. A. (2021) · Visual Informatics · doi:10.1016/j.visinf.2021.07.001 · Cited by 11
“Smartphone sensing has been effectively used in inferring depression levels (Saeb et al., 2016), social anxiety levels (Boukhechba et al., 2018b; Rashid et al., 2020), students’ Grade Point Averages (Wang et al., 2014), and the smartphone owner’s current context/situation (Vaizman et al., 2018a).”
MethodHoelzemann, A., & Laerhoven, K. V. (2024) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1379788 · Cited by 16
“This finding corresponds to what Vaizmann et al. [23] already points out.”
ResultDietz, M., Aslan, I., Schiller, D., Flutura, S., Steinert, A., Klebbe, R., & André, E. (2019) · International Conference on Pervasive Computing Technologies for Healthcare · doi:10.1145/3329189.3329197 · Cited by 11
“While such a distribution might be expected from students and employees as shown in [29], this explanation can not be used in our present study since the participants were all retired.”
InfluentialTowards Responsible Design with Internet of Things Data
Bourgeois, J., & Kortuem, G. (2019) · Proceedings of the ... International Conference on Engineering Design · doi:10.1017/dsi.2019.349 · Cited by 11
“For example, the Extra Sensory dataset Vaizman et al. (2018) provides sensor data from smart phones and smart watches of 60 users, along user activity labelling and states.”
BackgroundChoe, E. K., Klasnja, P., & Pratt, W. (2021) · Springer eBooks · doi:10.1007/978-3-030-58721-5_19 · Cited by 17
Madokoro, H., Nix, S., Woo, H., & Sato, K. (2021) · Applied Sciences · doi:10.3390/app112411807 · Cited by 11
Improving Experience Sampling with Multi-view User-driven Annotation Prediction
Liono, J., Salim, F. D., van Berkel, N., Kostakos, V., & Qin, A. K. (2019) · IEEE International Conference on Pervasive Computing and Communications · doi:10.1109/percom.2019.8767394 · Cited by 9
MixMax: Leveraging Heterogeneous Batteries to Alleviate Low Battery Experience for Mobile Users
Kwak, J., Lee, S., Jeong, D. R., Kumar, A., Shin, D. J., Kim, I., Shin, D., Lee, K., Lee, J., & Shin, I. (2023) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3581791.3596843 · Cited by 7
“Data of 50 users was collected directly via Android dumpsys [24], and the other 50 users’ data was from ExtraSensory Dataset [25, 26], a smartphone sensor-measurement dataset.”
MethodHTAD: A Home-Tasks Activities Dataset with Wrist-Accelerometer and Audio Features
Garcia-Ceja, E., Thambawita, V. L., Hicks, S. A., Jha, D., Jakobsen, P., Hammer, H. L., Halvorsen, P., & Riegler, M. A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-67835-7_17 · Cited by 12
Zheng, H. M., Mahapasuthanon, P., Chen, Y., Rangwala, H., Evmenova, A. S., & Motti, V. G. (2021) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3441852.3471220 · Cited by 9
“The dataset generated by Vaizman et al. (2018) tracked users’ behavior in activities such as walking, sitting, and laying down, using self-reported context labels [62].”
BackgroundKnowledge Infusion for Context-Aware Sensor-Based Human Activity Recognition
Arrotta, L., Civitarese, G., & Bettini, C. (2022) · IEEE International Conference on Smart Computing · doi:10.1109/smartcomp55677.2022.00016 · Cited by 11
Employing Environmental Data and Machine Learning to Improve Mobile Health Receptivity
Theilig, M.-M., Korbel, J. J., Mayer, G., Hoffmann, C., & Zarnekow, R. (2019) · IEEE Access · doi:10.1109/access.2019.2958474 · Cited by 12
“Devices are equipped with capable sensors as the authors of the Extra-Sensory Data Set describe [8].”
MethodKim, D. D. E. (2020) · Journal of Broadcasting & Electronic Media · doi:10.1080/08838151.2020.1849702 · Cited by 9
“Data self-reported through these mobile apps(1) can be provided in-situ, byrecall, or a combination of the two (Vaizman, 2018; Vaizman et al., 2018).”
BackgroundFutures for Health Research Data Platforms From the Participants’ Perspectives
Vilaza, G. N., Maharjan, R., Coyle, D., & Bardram, J. E. (2020) · Nordic Conference on Human-Computer Interaction · doi:10.1145/3419249.3420110 · Cited by 8
“Being constantly monitored can lead to feelings of surveillance [140], paranoia [127], and loss of freedom [24].”
BackgroundDeepSQA: Understanding Sensor Data via Question Answering
Xing, T., García, L. A. R., Cerutti, F., Kaplan, L., Preece, A. D., & Srivastava, M. B. (2021) · IEEE/ACM International Conference on Internet-of-Things Design and Implementation (IoTDI) · doi:10.1145/3450268.3453529 · Cited by 18
“In Section 6, we use SQA-Gen to generate multiple variants of OppQA with different parameter configurations, and also apply SQA-Gen to a new sensory dataset, ExtraSensory [32], to obtain new SQA data in a different domain.”
MethodInteraction Initiation with a Museum Guide Robot—From the Lab into the Field
Daczo, L.-D., Kalova, L., Bonita, K. L. F., Lopez, M. D., & Rehm, M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-85613-7_30 · Cited by 5
A Framework for Daily Living Activity Recognition using Fusion of Smartphone Inertial Sensors Data
Khan, S., Shah, S. M. A., Noorani, S. H., Arsalan, A., Ehatisham-ul-Haq, M., Raheel, A., & Ahmed, W. (2023) · International Conference on Computing, Mathematics and Engineering Technologies (iCoMET) · doi:10.1109/icomet57998.2023.10099271 · Cited by 5
“Extrasensory, a public domain in-the-Wild dataset (containing data from heterogeneous sensors on smartphones), is used in the proposed scheme [17].”
MethodTellTime: An AI-Augmented Calendar with a Voice Interface for Collecting Time-Use Data
Hoefer, M. J. D., Rychecky, R., Gong, M., & Voida, S. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712116 · Cited by 10
Bi, T., Buono, R. A., Olugbade, T., Singh, A., Holloway, C., Costanza, E., Williams, A. C. D. C., Gold, N., & Bianchi‐Berthouze, N. (2021) · Companion Publication of the International Conference on Multimodal Interaction (ICMI Companion) · doi:10.1145/3461615.3485670 · Cited by 7
“, [11-14], see a review in [10]) require people to physically interact with a device to self-report via their hands, such a solution would not be optimal within our scenarios, given that maintaining mobility and free hands are a priority, so as not to disrupt the flow of ongoing physical activities.”
BackgroundDomain Adaptation Methods for Lab-to-Field Human Context Recognition
Alajaji, A. S., Gerych, W., Buquicchio, L., Chandrasekaran, K., Mansoor, H., Agu, E., & Rundensteiner, E. A. (2023) · Sensors · doi:10.3390/s23063081 · Cited by 5
Yu, X., Hu, L., Reichman, B., Chu, D., Chandrupatla, R., Zhang, X., Heck, L. P., & Rosing, T. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3749496 · Cited by 25
UniMiB AAL: An Android Sensor Data Acquisition and Labeling Suite
Ginelli, D., Micucci, D., Mobilio, M., & Napoletano, P. (2018) · Applied Sciences · doi:10.3390/app8081265 · Cited by 5
“ExtraSensory is an open-source project, and the source code for both the server-side and the Android application is available.”
BackgroundMethodInfluentialChen, H., Gouin-Vallerand, C., Bouchard, K., Gaboury, S., Ngankam, H. K., Lussier, M., Couture, M., Bier, N., & Giroux, S. (2024) · International Conference on Information Technology for Social Good (GoodIT) · doi:10.1145/3677525.3678671 · Cited by 8
“Recently, researchers proposed a mobile app for collecting in-the-wild data, including self-reported labels describing people’s behavioral contexts (e.g., driving, eating) [21].”
COMEX: Identifying Mislabeled Human Behavioral Context Data Using Visual Analytics
Mansoor, H., Gerych, W., Buquicchio, L., Chandrasekaran, K., Agu, E., & Rundensteiner, E. A. (2019) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac.2019.10212 · Cited by 4
“We installed the Android version of the ES app [11] on a phone and evaluated the interface to identify”
MethodAn Application Suite: Effectiveness in Tracking and Monitoring of Skill Training Programs
Menon, B. M., Aswathi, P. V., & Shekar, L. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-981-15-7234-0_38 · Cited by 4
A Study on the Influence of Sensors in Frequency and Time Domains on Context Recognition
de Souza, P., Silva, D., de Andrade, I., Dias, J., Lima, J. P., Teichrieb, V., Quintino, J. P., da Silva, F. Q. B., & Santos, A. L. M. (2023) · Sensors · doi:10.3390/s23125756 · Cited by 5
Karahanoğlu, A., & Ludden, G. (2021) · Human-computer interaction series · doi:10.1007/978-3-030-67322-2_6 · Cited by 5
COMPASS: Unsupervised and online clustering of complex human activities from smartphone sensors
Campana, M. G., & Delmastro, F. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115124 · Cited by 4
“…have been studied, including Decision Trees (Fallahzadeh et al., 2016), Support Vector Machines (Li & Chung, 2015), Artificial Neural Networks (Vaizman et al., 2018b), or ensembles of different classifiers trained on specific context information and then combined with a meta-classifier to…”
MethodFeng, T., & Narayanan, S. (2024) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc53108.2024.10782414 · Cited by 5
Benefits of Using Activity Recommender Technology for Self-management of Depressive Symptoms
Rohani, D. A., Faurholt‐Jepsen, M., Kessing, L. V., & Bardram, J. E. (2021) · ACM Transactions on Computing for Healthcare · doi:10.1145/3462212 · Cited by 3
Smartphone based Human Activity Recognition using CNNs and Autoencoder Features
Mitra, S., & Kanungoe, P. (2023) · International Conference on Trends in Electronics and Informatics (ICOEI) · doi:10.1109/icoei56765.2023.10126051 · Cited by 3
“In [11], Y. Vaizman et al. acquired a huge in-the-wild dataset to promote practical applications that work in real-life settings.”
BackgroundLess is more: Efficient behavioral context recognition using Dissimilarity-Based Query Strategy
Akram, A., Farhan, A. A., & Basharat, A. (2023) · PLoS ONE · doi:10.1371/journal.pone.0286919 · Cited by 4
“[12], 2019 C and N Extrasensory [30], Opportunity [31] 22/60 Acc, WAcc, Gyro, Loc, Aud, PS AL [57],2013 C Opportunity [31] 15/4 Acc AL [37], 2010 C Self 3/32 GPS and Acc AL [64], 2010 C MIT [32] 20/20 Acc AL [76], 2017 C Self 7/10 Motion sensors, Comp and Acc AL [24], 2017 C Self 5/12 Acc AL [9], 2016 N Extrasensory […”
BackgroundMethodInfluentialN., A. U. N. S., Prasad, R. R. V., Lúa, A. R., & González, L. I. L. (2018) · ACM Multimedia Systems Conference (MMSys) · doi:10.1145/3204949.3204951 · Cited by 3
“Recently there have been open datasets [37, 38] that collect smartphone/smartwatch data from multiple occupants in diverse situations.”
BackgroundAttention-based Multi-task Learning for Sensor Analytics
Chen, Y., & Rangwala, H. (2019) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata47090.2019.9006207 · Cited by 3
“• ExtraSensory Dataset 2 : A multi-label classification dataset with multiple users, which was previously used for user behavioral context recognition in-the-wild from mobile sensors [12], [13].”
BackgroundFederated Multi-task Hierarchical Attention Model for Sensor AnalyticsPreprint
Chen, Y., Ning, Y., Chai, Z., & Rangwala, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.05142 · Cited by 6
ContextGPT: Infusing LLMs Knowledge into Neuro-Symbolic Activity Recognition Models
Arrotta, L., Bettini, C., Civitarese, G., & Fiori, M. (2024) · arXiv · doi:10.48550/arxiv.2403.06586 · Cited by 3
“2) ExtraSensory: ExtraSensory [31] was collected in an unscripted in-the-wild setting from 60 subjects.”
BackgroundObtaining Labels for In-the-Wild Studies: Using Visual Cues and Recall
Vargo, A. W., Ishimaru, S., Islam, M. R., Tag, B., & Kise, K. (2021) · IEEE Pervasive Computing · doi:10.1109/mprv.2021.3129500 · Cited by 3
Wang, Y., Khayami, H., Lee, B., Lazar, A., Kacorri, H., & Choe, E. K. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3770649 · Cited by 4
Lim, J., Chung, S., Noh, K. J., Kim, G. G., & Jeong, H. T. (2019) · International Conference on Information and Communication Technology Convergence (ICTC) · doi:10.1109/ictc46691.2019.8939976 · Cited by 2
“There are several studies developed self-reporting applications for mobile phone to improve the reliability and objectivity of the ground truth [2,3].”
BackgroundEntropy and Memory Aware Active Transfer Learning in Smart Sensing Systems
Alam, U., Farhan, A. A., Kanwal, S., & Allheeib, N. (2024) · IEEE Access · doi:10.1109/access.2024.3412653 · Cited by 2
Harnessing IoT and Robotics for Comprehensive Human Activity Recognition in Smart Buildings
Toupas, P., Tsamis, G., Zotou, D., Giakoumis, D., Votis, K., & Tzovaras, D. K. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-69344-1_12 · Cited by 3
A data-driven approach for context definition in situation-aware wearable computing systems
D’Aniello, G., Fortino, G., Gaeta, M., & Rehman, Z. U. (2024) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case59546.2024.10711334 · Cited by 2
“We evaluate the data-driven context representation approach using the Extrasensory dataset [8], comparing it against two alternatives: a manual definition of context spaces made by experts; a context representation approach based on Mamdani-type Fuzzy Inference System.”
MethodA Sequential Pattern Mining Approach for Situation-Aware Human Activity Projection
D’Aniello, G., Falcone, R., Gaeta, M., Rehman, Z. U., & Fortino, G. (2024) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc54092.2024.10831159 · Cited by 3
“The evaluation is performed using the Extrasensory dataset [9], collected in a real environment, containing activities and real time situations.”
Understanding User Requirements for Self-Created IoT Health Assistant Systems
Faltaous, S., Altmeyer, M., Kosmalla, F., Lessel, P., Daiber, F., & Schneegaß, S. (2021) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3490632.3490645 · Cited by 2
“These include the psychological factors (e.g., motivation) that would affect the use and adherence of such applications [18], the user interface [81], the relevant physiological attributes [33], and potential technologies (e.g., activity trackers [55]).”
BackgroundINTOSIS: Interactive Observation of Smartphone Inferred Symptoms for In-The-Wild Data
Mansoor, H., Gerych, W., Buquicchio, L., Alajaji, A. S., Chandrasekaran, K., Agu, E., & Rundensteiner, E. A. (2020) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata50022.2020.9378147 · Cited by 2
“Consequently, recent smartphone sensing work has adopted “In-the-wild” data gathering approaches [21], [23].”
BackgroundParticipant Engagement and Data Quality: Lessons Learned from a Mental Wellness Crowdsensing Study
Zhang, E., Trujillo, R., & Poellabauer, C. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711043 · Cited by 2
“Only 2 of these 5 studies, namely BioBase [71] and ExtraSensory [96], focused on college students as their targeted population.”
Clara: Context-Aware RAG-LLM Framework for Anomaly Detection in Mobile Device Sensors
Koh, C. Y., DeMedeiros, K., & Hendawi, A. (2025) · IEEE International Conference on Mobile Data Management (MDM) · doi:10.1109/mdm65600.2025.00035 · Cited by 5
Hoefer, M. J. D., Rychecky, R., & Voida, S. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791113 · Cited by 1
“Participants preferred the DRM over the ESM as it did not interrupt their day [90].”
Le, H., Choube, A., Mishra, V., & Intille, S. S. (2026) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3831653 · Cited by 3
“For instance, Stojchevska et al. allowed participants to review and edit predicted activity labels at the end of the day [91], and ExtraSensory [95], ACAI [57], and Neupane et al. [77] enabled users to refine model predictions, improving labeling accuracy and reducing effort.”
Bracken, B. K., Manjunath, S., German, S., Monnier, C. S., & Farry, M. (2019) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181319631002 · Cited by 2
What Did Our Model Just Learn? Hard Lessons in Applying Deep Learning to Human Factors Data
Weigel, B., Loar, K., Colón, A., & Wright, R. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-80285-1_7 · Cited by 1
Smart Mental Health Guiding in Colleges: from HCI Data Collection to Fuzzy Information Mining
Zhang, N. (2022) · International Conference on Sustainable Computing and Data Communication Systems (ICSCDS) · doi:10.1109/icscds53736.2022.9760756 · Cited by 1
Exploratory Data Analysis of Population Level Smartphone-Sensed Data
Mansoor, H., Gerych, W., Alajaji, A. S., Buquicchio, L., Chandrasekaran, K., Agu, E., & Rundensteiner, E. A. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-25477-2_10 · Cited by 1
On Predicting Emotion From Commercial Smartwatch - A Big Data Narrative
Sarmah, P. P., Paul, P. S., & Hazra, K. K. (2025) · IEEE International Conference on Interdisciplinary Approaches in Technology and Management for Social Innovation (IATMSI) · doi:10.1109/iatmsi64286.2025.10984709 · Cited by 1
S-THAD: a framework for sensor-based temporal human activity detection from continuous data streams
Ehatisham-ul-Haq, M., Azam, M. A., Murtaza, F., Amin, Y., & Fu, Q. (2021) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-021-02931-5 · Cited by 1
Studi Literatur Human Activity Recognition (HAR) Menggunakan Sensor Inersia
Pradani, H. N., & Mahananto, F. (2021) · Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) · doi:10.29207/resti.v5i6.3665 · Cited by 2
“UniMiB SHAR 17 30 SP A [36] MobiAct 13 67 SP A,G,M [37] ExtraSensory 6 60 SP A,G [38] UMAFall 13 19 SP A,G [39] IM-WSHA 11 10 SS A,G [40]”
ActiveSense: A Novel Active Learning Framework for Human Activity Recognition
Shahabi, F., Gao, Y., & Alshurafa, N. I. (2022) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomworkshops53856.2022.9767388 · Cited by 1
“In [8], researchers propose a recall approach for the labeling process, which is fraught with burden and memory recollection error.”
BackgroundNer, R., Das, D. P., Kumar, R., & Garg, S. (2018) · International conference on Fuzzy Theory and Its Applications · doi:10.1109/ifuzzy.2018.8751689
Fuzzy Based Wellness AnalyzerPreprint
Ner, R., Das, D. P., Kumar, R., & Garg, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.07245
“We use the ExtraSensory App [14] for the purpose of collecting the data using the in-the-wild approach.”
MethodCollecting Labels for Rare Anomalies via Direct Human Feedback—An Industrial Application Study
Reich, C., Mansour, A., & Laerhoven, K. V. (2019) · Informatics · doi:10.3390/informatics6030038
Combining Participatory and ESM: A Hybrid Approach to Collecting Annotated Mobility Data
Chang, H.-C., Chang, Y.-J., Newman, M., & Lin, C.-H. (2020) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3334480.3383066
“[6] used a powerful label-reporting interface to engage users with different behavior styles and phone-interaction preferences and to acquire detailed labels.”
DatEthics: Ethical Data-Centric Design of Intelligent Behaviour
Bourgeois, J., Vroon, J., Ding, A. Y., & Peltonen, E. (2020) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/3406324.3424589
“For instance, they facilitate data collection [15], experience sampling [16], participant recruitment [11] and participant interaction [1].”
BackgroundJu, S. (2020) · Scholarworks@UNIST (Ulsan National Institute of Science and Technology)
Recognition of Hygiene Activities by Means of Multimodal Sensors
Polo-Rodríguez, A., Cruciani, F., Nugent, C., & Medina-Quero, J. (2021) · Springer proceedings in complexity · doi:10.1007/978-3-030-84311-3_9
Metadata
- Authors
- Vaizman, Y., Ellis, K., Lanckriet, G., & Weibel, N.
- Affiliations
- University of California San Diego (US)
- Venue
- Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2018-04-21
- Pages
- 1-12
- OpenAlex
- W2795423566
- Cited by
- 115
- References
- 31
- Citations by year
- 2026: 7 · 2025: 11 · 2024: 12 · 2023: 9 · 2022: 9 · 2021: 25 · 2020: 18 · 2019: 17 · 2018: 7
- FWCI
- 5.05
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- mobile application, self-report, context recognition, smartphone sensors, user interface design, user experience, health monitoring, elderly care, ecological validity, daily journal
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.