ExtraSensory App: Data Collection In-the-Wild with Rich User Interface to Self-Report Behavior
The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
We introduce a mobile app for collecting in-the-wild data, including sensor measurements and self-reported labels describing people's behavioral context (e.g., driving, eating, in class, shower). Labeled data is necessary for developing context-recognition systems that serve health monitoring, aging care, and more. Acquiring labels without observers is challenging and previous solutions compromised ecological validity, range of behaviors, or amount of data. Our user interface combines past and near-future self-reporting of combinations of relevant context-labels. We deployed the app on the personal smartphones of 60 users and analyzed quantitative data collected in-the-wild and qualitative user-experience reports. The interface's flexibility was important to gain frequent, detailed labels, support diverse behavioral situations, and engage different users: most preferred reporting their past behavior through a daily journal, but some preferred reporting what they're about to do. We integrated insights from this work back into the app, which we make available to researchers for conducting in-the-wild studies.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Although many commercial fitness trackers or smartwatches adopt wrist-mountable form due to the ease of accessibility, no consensus has been made regarding the positioning of sensors and their data acquisition details.”
From Sensor Data Acquisition and Multimodal Sensor Fusion for Human Activity Recognition Using Deep Learning (2019) · cites this paper as [16]
In the citing paper
“However, as the dataset adopts heterogeneous sensors on limited position of the human body, it is not sufficient to evaluate the recognition performance according to the sensor position and sampling rate.”
From Sensor Data Acquisition and Multimodal Sensor Fusion for Human Activity Recognition Using Deep Learning (2019) · cites this paper as [16]
Citations
Cites
A Survey Method for Characterizing Daily Life Experience: The Day Reconstruction Method
Kahneman, D., Krueger, A. B., Schkade, D. A., Schwarz, N., & Stone, A. A. (2004) · Science · doi:10.1126/science.1103572 · Cited by 3,422 · cited 4×
“DRM was used in [26] for eating detection, but the participants struggled remembering when they were eating.”
BackgroundMethodInfluentialFlow and the Foundations of Positive Psychology: The Collected Works of Mihaly Csikszentmihalyi
Csikszentmihalyi, M. (2014) · doi:10.1007/978-94-017-9088-8 · Cited by 1,653
Activity sensing in the wild: a field trial of ubifit garden
Consolvo, S., McDonald, D. W., Toscos, T., Chen, M. Y., Froehlich, J., Harrison, B., Klasnja, P., LaMarca, A., LeGrand, L., Libby, R., et al. (2008) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357335 · Cited by 1,131
Mel Frequency Cepstral Coefficients For Music Modeling.
Logan, B. (2000) · doi:10.5281/zenodo.1416443 · Cited by 913
Experience Sampling: Promises and Pitfalls, Strengths and Weaknesses
Scollon, C. N., Kim-Prieto, C., & Diener, E. (2003) · Journal of Happiness Studies · doi:10.1023/a:1023605205115 · Cited by 825 · cited 6×
“Much like survey-based studies with ESM or DRM [23, 11], we address the multi-aspect nature of behavioral context and allow users to report combinations of activities, as well as location, company, body posture and more.”
BackgroundMethodInfluentialStisen, A., Blunck, H., Bhattacharya, S., Prentow, T. S., Kjærgaard, M. B., Dey, A., Sonne, T., & Jensen, M. M. (2015) · Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems · doi:10.1145/2809695.2809718 · Cited by 714 · cited 1×
“Data collection in-the-wild raises technical difficulties related to interruptions in sensor recording and diversity in phonedevices [24] and device placement [15].”
BackgroundDetecting activities of daily living in first-person camera views
Pirsiavash, H., & Ramanan, D. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248010 · Cited by 649 · cited 2×
“In some studies research assistants annotated the images [20, 7], compromising the privacy of the participants and their surrounding.”
BackgroundThe Mobile Sensing Platform: An Embedded Activity Recognition System
Choudhury, T., Borriello, G., Consolvo, S., Haehnel, D., Harrison, B., Hemingway, B., Hightower, J., Klasnja, P. "., Koscher, K., LaMarca, A., et al. (2008) · IEEE Pervasive Computing · doi:10.1109/mprv.2008.39 · Cited by 598
Validity and Reliability of the Experience-Sampling Method
Csikszentmihalyi, M., & Larson, R. (2014) · Flow and the Foundations of Positive Psychology · doi:10.1007/978-94-017-9088-8_3 · Cited by 578
King, A. C., Hekler, E. B., Grieco, L. A., Winter, S. J., Sheats, J. L., Buman, M. P., Banerjee, B., Robinson, T. N., & Cirimele, J. (2013) · PLoS ONE · doi:10.1371/journal.pone.0062613 · Cited by 332 · cited 1×
“) is desirable for many domains, such as health management [10, 12], aging care [14, 16] and office assistant systems [30].”
Background
Research works printed but not linked: 1
- 10.5555/2354409.2355089 · doi:10.5555/2354409.2355089
Cited by
Items 101–117 of 117
Inviting Participants’ Peers in a Mobile Assessment Study: An Empirical Investigation
Chang, Y., Lee, H.-P., Chang, Y.-J., & Shen, C.-Y. (2021) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/3447526.3472021
Multimodal and Multicultural Field Agents: Considerations for “outside-the-lab” Studies
Rehm, M. (2021) · Springer eBooks · doi:10.1007/978-981-16-3476-5_4
An Optimization Approach to Multi-Sensor Operation for Multi-Context Recognition
Kain, R., & Hajj, H. M. (2021) · Sensors · doi:10.3390/s21206862
“[48], who developed a mobile application deployed on smartphones to collect in-the-wild data via sensor measurements, requires past and nearfuture self-reporting of combinations of relevant context-labels.”
BackgroundOpportunistic Sensing of Behavioural Data to Prioritise Smartphone Notifications in the Wild
Kamal, R., Nugent, C., Cleland, I., & McCullagh, P. J. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-21333-5_92
Location-Aware Context Detection Based-On Behavior Sensors
Rahin, S. A., Hui, B., & Li, W. (2024) · International Conference on Computer Communication and the Internet (ICCCI) · doi:10.1109/iccci62159.2024.10674531
“In the previous work of Yonatan [12], they provided a detailed explanation of the data, which we provide in this section.”
Comparative analysis of Mobile Crowdsensing and Ecological Momentary Assessment systems
Stach, M., Ramović, A., Reichert, M., & Pryss, R. C. (2025) · Ecological Informatics · doi:10.1016/j.ecoinf.2025.103315 · Cited by 1
Leveraging Customized Heterogeneous Batteries to Alleviate Low Battery Experience for Mobile Users
Kwak, J., Lee, S., Jeong, D. R., Kumar, A., Shin, D. J., Kim, I., Shin, D., Lee, K., Lee, J., & Shin, I. (2025) · IEEE Transactions on Sustainable Computing · doi:10.1109/tsusc.2025.3611159
305On Mental Health Monitoring Using Commercial Wearable Devices and Machine Intelligence
Sarmah, P. P., Choudhari, A., Paul, P. S., & Hazra, K. (2025) · De Gruyter eBooks · doi:10.1515/9783111541235-016
Potter, V., Le, H., Syeda, U. H., Intille, S. S., & Borkin, M. A. (2025) · Visualization · doi:10.1109/vis60296.2025.00048 · Cited by 1
“Although activity recalls are often interpreted from a textural or tabular format, activity recall timelines (ARTs) allow for better patternidentificationandsensemakingoftheunderlyingdata[20,8,29,13, 27].”
Multimodal Situation-aware Data Generation for Activity Recognition in Intelligent Wearable Systems
D’Aniello, G., Fotia, L., Gaeta, M., & Rehman, Z. U. (2025) · IEEE Conference on Pervasive and Intelligent Computing (PICom) · doi:10.1109/picom68402.2025.00018
Range query processing over untrustworthy clouds
Tran, V.-H. (2020) · theses.fr (ABES) · doi:10.70675/0f5060b3z27b6z46cbz9b88z5aa6f9c06d6f
Liu, Y., Liu, H., Li, Z., Zou, Y., Liu, M., Zong‐Hao, C., & Wang, C. (2026) · IEEE Sensors Journal · doi:10.1109/jsen.2026.3697846
“…computing technologies, wearable sensors and mobile devices, such as smartwatches and smartphones, have become key enablers for collecting rich behavioral data, including activity patterns, physiological signals, and environmental information, owing to their ubiq-uity and portability [5], [6].”
A Systematic Review of Machine Learning for Situation Awareness
D’Aniello, G., Gaeta, M., Rehman, Z. U., & Vento, M. (2026) · Cognitive Computation · doi:10.1007/s12559-026-10611-9 · Cited by 1
Ariza-Colpas, P., Piñeres-Melo, M. A., Carrascal, A. I. O., & Díaz-Jiménez, D. (2026) · Sensors · doi:10.3390/s26123751
Federated Active User Participation (FAUP)
Akram, A., Farhan, A. A., & Basharat, A. (2026) · Evolving Systems · doi:10.1007/s12530-026-09850-w
StrengthSense: A Dataset of IMU Signals Capturing Everyday Activities with Diverse Physical Demands
Yang, Z., Liu, Z., Zhang, Y., Leite, C. F. S., & Xiao, Y. (2026) · ACM International Symposium on Wearable Computers (ISWC) · doi:10.1145/3830727.3834811 · Cited by 2
Akram, A., Ismail, S., Zahra, S. R., Nisa, S. Q., Islam, N., Bakhsh, S. T., & Khan, M. A. (2026) · Discover Computing · doi:10.1007/s10791-026-10781-y
“• ExtraSensory Dataset [53]: Contains “wild” environment data from 60 users in natural settings.”
Metadata
- Authors
- Vaizman, Y., Ellis, K., Lanckriet, G., & Weibel, N.
- Affiliations
- University of California San Diego (US)
- Venue
- Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2018-04-21
- Pages
- 1-12
- OpenAlex
- W2795423566
- Cited by
- 115
- References
- 31
- Citations by year
- 2026: 7 · 2025: 11 · 2024: 12 · 2023: 9 · 2022: 9 · 2021: 25 · 2020: 18 · 2019: 17 · 2018: 7
- FWCI
- 5.05
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- mobile application, self-report, context recognition, smartphone sensors, user interface design, user experience, health monitoring, elderly care, ecological validity, daily journal
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.