CrowdSignals: a call to crowdfund the community's largest mobile dataset
The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
Researchers from diverse backgrounds critically depend on mobile datasets. From training and testing activity recognition models, to verifying hypotheses in social science, mobile data is indispensable. Unfortunately, mobile data collection requires significant time and budget for infrastructure as well as subject recruiting, screening, training, legal agreements, equipment, and compensation. We estimate up to 70% of the resources in a study may be spent on data collection. Moreover, this massive investment can combine with institutional, legal, and political issues to create a disincentive to sharing with the community. In this paper, we propose and justify a crowdfunded and crowdsourced methodology for longitudinal mobile data collection that cuts researcher costs by orders of magnitude, removes barriers to data sharing, and boosts data value for all stakeholders. We also present CrowdSignals, a first instantiation which will generate the largest labeled mobile dataset available to the community.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Removing the dependency on engineered features by exploiting convolutional layers is particularly important if the set of activities to recognise is changing over time, for instance as additional labelled data become available (e.g., through crowd-sourcing [48]).”
From Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition (2016) · cites this paper as [48]
Citations
Cites
Reality mining: sensing complex social systems
Eagle, N., & (Sandy) Pentland, A. (2006) · Personal and Ubiquitous Computing · doi:10.1007/s00779-005-0046-3 · Cited by 2,772 · cited 1×
“MIT Reality Mining [2] 100 Students 2004 9 $110K+”
BackgroundMiluzzo, E., Lane, N. D., Fodor, K., Peterson, R., Lu, H., Musolesi, M., Eisenman, S. B., Zheng, X., & Campbell, A. T. (2008) · Proceedings of the 6th ACM conference on Embedded network sensor systems · doi:10.1145/1460412.1460445 · Cited by 1,032 · cited 1×
“Dartmouth CenceMe [10] 9000+ People 2008 N/A $75K+”
Social fMRI: Investigating and shaping social mechanisms in the real world
Aharony, N., Pan, W., Ip, C., Khayal, I., & Pentland, A. (2011) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2011.09.004 · Cited by 459
Twitch crowdsourcing: crowd contributions in short bursts of time
Vaish, R., Wyngarden, K., Chen, J., Cheung, B., & Bernstein, M. S. (2014) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2556996 · Cited by 95 · cited 1×
“The lockscreen solicits frequent, lightweight feedback with a multiple choice question (e.g., “How do you feel?”) every time subjects unlock their phone - about 19 times per day on average [16].”
BackgroundEliciting user preferences using image-based experience sampling and reflection
Intille, S., Kukla, C., & Ma, X. (2002) · CHI '02 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/506443.506573 · Cited by 60 · cited 1×
“Sponsors may solicit ground truth from subjects with a survey framework that offers lockscreen surveys and experience sampling method (ESM) [7] surveys (see Figure 2).”
BackgroundCrowdsourced mobile data collection
Welbourne, E., Wu, P., Bao, X., & Tapia, E. M. (2014) · IEEE Workshop on Mobile Computing Systems and Applications (WMCSA) · doi:10.1145/2565585.2565608 · Cited by 9 · cited 1×
“In the CS approach [17], remote subjects are rapidly recruited, managed, and paid with a crowdsourcing service, but they install a data collection app on their own phone.”
BackgroundSelf-citation
Research works printed but not linked: 1
- Khan , S. and Marzec , E. , Tech Trends 2014: Wearables . Deloitte University Press , http://www.dupress.com/articles/ 2014 -tech-trends-wearables Khan, S. and Marzec, E., Tech Trends 2014: Wearables. Deloitte University Press, http://www.dupress.com/articles/2014-tech-trends-wearables
Cited by
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761
“, through crowd-sourcing [48]).”
Gjoreski, H., Ciliberto, M., Wang, L., Ordonez Morales, F. J., Mekki, S., Valentin, S., & Roggen, D. (2018) · IEEE Access · doi:10.1109/access.2018.2858933 · Cited by 276
“Previous work has shown that data can be collected from volunteers with or without financial incentive [25].”
BackgroundExtraSensory App: Data Collection In-the-Wild with Rich User Interface to Self-Report Behavior
Vaizman, Y., Ellis, K., Lanckriet, G., & Weibel, N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174128 · Cited by 115
“The CrowdSignals project [29] aims to collect phone-sensor data from large crowds of users, with additional sparse probing for labels by using quick multiple-choice questions whenever the user unlocks their phone, combined with more in-depth (and less frequent) ESM questionnaires.”
BackgroundMedical Data Processing and Analysis for Remote Health and Activities Monitoring
Vitabile, S., Marks, M., Stojanovic, D. H., Pllana, S., Molina, J. M., Krzysztoń, M., Sikora, A., Jarynowski, A. K., Hosseinpour, F., Jakóbik, A., et al. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-16272-6_7 · Cited by 90
Quantification of free-living activity patterns using accelerometry in adults with mental illness
Chapman, J., Roberts, J. A., Nguyen, V. T., & Breakspear, M. J. (2017) · Scientific Reports · doi:10.1038/srep43174 · Cited by 48
Kołodziej, J., & González–Vélez, H. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-16272-6 · Cited by 40
Inferring Transportation Mode and Human Activity from Mobile Sensing in Daily Life
Liono, J., Abdallah, Z. S., Qin, A. K., & Salim, F. D. (2018) · EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services (MobiQuitous) · doi:10.1145/3286978.3287006 · Cited by 19
“In order to validate ourmobile context recognition system, Crowdsignal dataset [38] is used in the experiment.”
MethodEnabling Virtual Sensing as a Service
Yang, L., Pandis, I., & Guo, Y. (2016) · Informatics · doi:10.3390/informatics3020003 · Cited by 8
“[19,20] proposed a crowdfunded and crowdsourced methodology that uses crowdfunding to finance the collection of mobile datasets by paying participants who contribute their data.”
BackgroundCognitive Analysis of Data Mining Tools Application in Health Care Services
Raykar, S. S., & Shet, V. N. (2020) · International Conference on Emerging Trends in Information Technology and Engineering (ICETITE) · doi:10.1109/ic-etite47903.2020.442 · Cited by 8
An Unsupervised and Noninvasive Model for Predicting Network Resource Demands
Corno, F., Russis, L. D., Marcelli, A., & Montanaro, T. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2860681 · Cited by 7
“io” initiative [10], and we present positive and promising results concerning the prediction capa-”
Improving Experience Sampling with Multi-view User-driven Annotation Prediction
Liono, J., Salim, F. D., van Berkel, N., Kostakos, V., & Qin, A. K. (2019) · IEEE International Conference on Pervasive Computing and Communications · doi:10.1109/percom.2019.8767394 · Cited by 8
“in recent years, for example, emotion recognition [3], mobile user intimacy and smartphone usage [4], [5], human activity recognition [6], lifelogging [7]–[10] and mobile sensor data collection [11]–[14].”
BackgroundMethodIntelligent personal guidance of human behavior utilizing anticipatory models
Meurisch, C. (2016) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2968219.2971355 · Cited by 6
“In comparison to other community datasets [28] (e.g., MIT Reality Mining [6], Dartmouth CenceMe, Samsung CS, StudentLife dataset [25], or Nokia Lausanna Data Collection Campaign), our dataset is unique in its diversity.”
ResultCapturing Daily Student Life by Recognizing Complex Activities Using Smartphones
Meurisch, C., Gogel, A., Schmidt, B., Nolle, T., Janssen, F. W., Schweizer, I., & Mühlhäuser, M. (2017) · EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services (MobiQuitous) · doi:10.1145/3144457.3144472 · Cited by 7
“In comparison to other community datasets (e.g., MIT Reality Mining [10], Dart-mouth CenceMe, Device Analyzer [39], Samsung CS, StudentLife Table 3: Manually user-annotated data dataset [40], or Nokia Lausanna Data Collection Campaign) [43], our dataset is unique in its diversity.”
Mobile Device Identification via User Behavior Analysis
Dogan, K., & İncel, Ö. D. (2019) · Communications in computer and information science · doi:10.1007/978-3-030-27355-2_3 · Cited by 2
Nelimarkka, M., & Vuorenmaa, V. (2024) · Research Square · doi:10.21203/rs.3.rs-4162467/v1 · Cited by 2
Understanding real-world phenomena from human-generated sensor data
Tsapeli, T. K. (2018) · University of Birmingham Institutional Research Archive (University of Birmingham)
Continuous Risk-Aware Multi-Modal Authentication Across Mobile Devices
Hintze, D. (2019) · University Library Linz repository (Johannes Kepler Universitat Linz)
Metadata
- Authors
- Welbourne, E., & Tapia, E. M.
- Affiliations
- CrowdSignals Campaign, San Francisco, CA
- Venue
- Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication
- Publisher
- ACM
- Published
- 2014-09-13
- Pages
- 873-877
- OpenAlex
- W2145704294
- Cited by
- 15
- References
- 6
- Citations by year
- 2024: 1 · 2020: 1 · 2019: 4 · 2018: 5 · 2017: 1 · 2016: 3
- FWCI
- 0.85
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Transportation › Human Mobility and Location-Based Analysis
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Keywords
- mobile data collection, crowdsourcing, crowdfunding, longitudinal data collection, data sharing, activity recognition, research costs
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.