MyMove: Facilitating Older Adults to Collect In-Situ Activity Labels on a Smartwatch with Speech
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Abstract
Current activity tracking technologies are largely trained on younger adults’ data, which can lead to solutions that are not well-suited for older adults. To build activity trackers for older adults, it is crucial to collect training data with them. To this end, we examine the feasibility and challenges with older adults in collecting activity labels by leveraging speech. Specifically, we built MyMove, a speech-based smartwatch app to facilitate the in-situ labeling with a low capture burden. We conducted a 7-day deployment study, where 13 older adults collected their activity labels and smartwatch sensor data, while wearing a thigh-worn activity monitor. Participants were highly engaged, capturing 1,224 verbal reports in total. We extracted 1,885 activities with corresponding effort level and timespan, and examined the usefulness of these reports as activity labels. We discuss the implications of our approach and the collected dataset in supporting older adults through personalized activity tracking technologies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Physical Activity Guidelines for Americans
Piercy, K. L., Troiano, R. P., Ballard, R. M., Carlson, S. A., Fulton, J. E., Galuska, D. A., George, S. M., & Olson, R. D. (2018) · JAMA · doi:10.1001/jama.2018.14854 · Cited by 9,755
Diary Methods: Capturing Life as it is Lived
Bolger, N., Davis, A., & Rafaeli, E. (2003) · Annual Review of Psychology · doi:10.1146/annurev.psych.54.101601.145030 · Cited by 3,843 · cited 1×
“To collect more ecologically valid self-report data, researchers devised Diary Study [8] and Experience Sampling Method (ESM, often interchangeable with ecological momentary assessment or EMA) [64].”
MethodA Survey on Human Activity Recognition using Wearable Sensors
Lara, O. D., & Labrador, M. A. (2013) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2012.110112.00192 · Cited by 2,662
Assessment of physical activity by self-report: Status, limitations, and future directions
Sallis, J. F., & Saelens, B. E. (2000) · Research Quarterly for Exercise and Sport · doi:10.1080/02701367.2000.11082780 · Cited by 2,351
Letter to the Editor: Standardized use of the terms “sedentary” and “sedentary behaviours”
Networ, S. B. R. (2012) · Applied Physiology, Nutrition, and Metabolism · doi:10.1139/h2012-024 · Cited by 1,873
Psychophysical scaling with applications in physical work and the perception of exertion
Borg, G. (1990) · Scandinavian Journal of Work, Environment & Health · doi:10.5271/sjweh.1815 · Cited by 1,817 · cited 1×
“, the number of repetitions, speed, weight lifted) [10].”
BackgroundFlow and the Foundations of Positive Psychology: The Collected Works of Mihaly Csikszentmihalyi
Csikszentmihalyi, M. (2014) · doi:10.1007/978-94-017-9088-8 · Cited by 1,653
Successful Aging: Perspectives from the Behavioral Sciences
(1990) · doi:10.1017/cbo9780511665684 · Cited by 1,362
The Experience Sampling Method
Larson, R., & Csikszentmihalyi, M. (2014) · Flow and the Foundations of Positive Psychology · doi:10.1007/978-94-017-9088-8_2 · Cited by 1,034
Stisen, A., Blunck, H., Bhattacharya, S., Prentow, T. S., Kjærgaard, M. B., Dey, A., Sonne, T., & Jensen, M. M. (2015) · Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems · doi:10.1145/2809695.2809718 · Cited by 714
Association of Step Volume and Intensity With All-Cause Mortality in Older Women
Lee, I.-M., Shiroma, E. J., Kamada, M., Bassett, D. R., Matthews, C. E., & Buring, J. E. (2019) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2019.0899 · Cited by 604 · cited 1×
“42 steps/day) was consistent with reduced allcause mortality risk from previous studies with older women [67].”
ResultObjective Light-Intensity Physical Activity Associations With Rated Health in Older Adults
Buman, M. P., Hekler, E. B., Haskell, W. L., Pruitt, L., Conway, T. L., Cain, K. L., Sallis, J. F., Saelens, B. E., Frank, L. D., & King, A. C. (2010) · American Journal of Epidemiology · doi:10.1093/aje/kwq249 · Cited by 528
Harari, G. M., Lane, N. D., Wang, R., Crosier, B. S., Campbell, A. T., & Gosling, S. D. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616650285 · Cited by 515
The Experience Sampling Method on Mobile Devices
van Berkel, N., Ferreira, D., & Kostakos, V. (2018) · ACM Computing Surveys · doi:10.1145/3123988 · Cited by 432 · cited 1×
“and frequent interruptions [120], we believe that this is a promising outcome.”
BackgroundPrevalence of Sedentary Behavior in Older Adults: A Systematic Review
Harvey, J., Chastin, S., & Skelton, D. (2013) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph10126645 · Cited by 418
Lee, I.-M., & Shiroma, E. J. (2014) · British Journal of Sports Medicine · doi:10.1136/bjsports-2013-093154 · Cited by 415
Behavior Change Techniques Present in Wearable Activity Trackers: A Critical Analysis
Mercer, K., Li, M., Giangregorio, L., Burns, C., & Grindrod, K. (2016) · JMIR mHealth and uHealth · doi:10.2196/mhealth.4461 · Cited by 304 · cited 1×
“Activity tracking technologies can provide meaningful feedback that supports people’s motivations, playing an important role in enhancing physical activity [32, 80, 122].”
BackgroundUse of Intelligent Voice Assistants by Older Adults with Low Technology Use
Pradhan, A., Lazar, A., & Findlater, L. (2020) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3373759 · Cited by 285
Self-citationUsing Cadence to Study Free-Living Ambulatory Behaviour
Tudor-Locke, C., & Rowe, D. A. (2012) · Sports Medicine · doi:10.2165/11599170-000000000-00000 · Cited by 231
The State of Speech in HCI: Trends, Themes and Challenges
Clark, L., Doyle, P., Garaialde, D., Gilmartin, E., Schlögl, S., Edlund, J., Aylett, M., Cabral, J., Munteanu, C., Edwards, J., et al. (2019) · Interacting with Computers · doi:10.1093/iwc/iwz016 · Cited by 193 · cited 1×
“speech, which is widely embedded in consumer devices and digital systems [22].”
BackgroundExploring the use of speech input by blind people on mobile devices
Azenkot, S., & Lee, N. B. (2013) · Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/2513383.2513440 · Cited by 151
Current and future mobile and wearable device use by people with visual impairments
Ye, H., Malu, M., Oh, U., & Findlater, L. (2014) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557085 · Cited by 148
A smartwatch-based framework for real-time and online assessment and mobility monitoring
Kheirkhahan, M., Nair, S., Davoudi, A., Rashidi, P., Wanigatunga, A. A., Corbett, D. B., Mendoza, T., Manini, T. M., & Ranka, S. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.11.003 · Cited by 138 · cited 2×
“To ease the data entry, researchers have explored more effective input methods such as the ROAMM [59] and PROMPT [77] frameworks, which support radial scales and bezel rotation to specify pain level and activity type.”
MethodMeta-HAR: Federated Representation Learning for Human Activity Recognition
Li, C., Niu, D., Jiang, B., Zuo, X., & Yang, J. (2021) · Proceedings of the Web Conference 2021 · doi:10.1145/3442381.3450006 · Cited by 133 · cited 1×
“We could employ novel model personalization methods like teachable machines [25, 46, 54, 68], which leverage advances in transfer learning [90] and meta learning [30, 70].”
MethodμEMA: Microinteraction-based ecological momentary assessment (EMA) using a smartwatch
Intille, S., Haynes, C., Maniar, D., Ponnada, A., & Manjourides, J. (2016) · Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2971648.2971717 · Cited by 130
Pang, C., Collin Wang, Z., McGrenere, J., Leung, R., Dai, J., & Moffatt, K. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445702 · Cited by 118 · cited 1×
“To minimize the effort for the initial set up [91], we deployed smartwatches and Samsung”
MethodExtraSensory App: Data Collection In-the-Wild with Rich User Interface to Self-Report Behavior
Vaizman, Y., Ellis, K., Lanckriet, G., & Weibel, N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174128 · Cited by 115
Longitudinal study of ASR performance on ageing voices
Vipperla, R., Renals, S., & Frankel, J. (2008) · Interspeech 2008 · doi:10.21437/interspeech.2008-632 · Cited by 75 · cited 1×
“recognition (ASR) technologies can recognize participants’ speech inputs, especially since there is prior evidence on disproportionate ASR word error rates for older adults’ voices [19, 124].”
BackgroundGreen, J. R., MacDonald, R. L., Jiang, P.-P., Cattiau, J., Heywood, R., Cave, R., Seaver, K., Ladewig, M. A., Tobin, J., Brenner, M. P., et al. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-1384 · Cited by 74 · cited 1×
“Recent speech recognition personalization efforts like Google’s Project Euphonia [40, 74] are promising.”
BackgroundDisordered speech data collection: lessons learned at 1 million utterances from project euphonia
MacDonald, R. L., Jiang, P.-P., Cattiau, J., Heywood, R., Cave, R., Seaver, K., Ladewig, M. A., Tobin, J., Brenner, M. P., Nelson, P. C., et al. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-697 · Cited by 72
The bit doesn’t fit: Evaluation of a commercial activity-tracker at slower walking speeds
Wong, C. K., Mentis, H. M., & Kuber, R. (2018) · Gait & Posture · doi:10.1016/j.gaitpost.2017.10.010 · Cited by 46 · cited 1×
“9 m/s, a representative walking speed of older adults [132].”
BackgroundLuo, Y., Lee, B., & Choe, E. K. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376616 · Cited by 44 · cited 1×
“, exercise logging on a smart speaker [72], food journaling on a smartphone [71]).”
BackgroundSelf-citationGlasser, A. T., Kushalnagar, K. R., & Kushalnagar, R. S. (2017) · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3132525.3134819 · Cited by 28
Reviewing Speech Input with Audio: Differences between Blind and Sighted Users
Hong, J., Vaing, C., Kacorri, H., & Findlater, L. (2020) · ACM Transactions on Accessible Computing · doi:10.1145/3382039 · Cited by 11
Self-citation
Cited by
Jo, E., Epstein, D. A., Jung, H., & Kim, Y.-H. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581503 · Cited by 174
“[31] have proposed a speech-based smartwatch app to assist older adults in labeling physical activities with a low capture burden.”
BackgroundSelf-citationLeveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data
Wei, J., Kim, S., Jung, H., & Kim, Y.-H. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637364 · Cited by 84
“Prior research found that when people are allowed to insert data in free-form natural language, they tend to provide detailed answers with surrounding contexts [40, 63].”
BackgroundSelf-citationRay, S. J., Cherian, J., Liberty, A. M., Hammond, T. A., & Shireman, P. K. (2024) · doi:10.2196/preprints.67373 · Cited by 18
Ray, S. J., Cherian, J., Liberty, A. M., Hammond, T. A., & Shireman, P. K. (2025) · Journal of Medical Internet Research · doi:10.2196/67373 · Cited by 18
Speaking of accent: A content analysis of accent misconceptions in ASR research
Prinos, K., Patwari, N., & Power, C. A. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658969 · Cited by 12
Metadata
- Authors
- Kim, Y.-H., Chou, D., Lee, B., Danilovich, M., Lazar, A., Conroy, D. E., Kacorri, H., & Choe, E. K.
- Venue
- CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2022-04-29
- Pages
- 1-21
- OpenAlex
- W4225013596
- Cited by
- 39
- References
- 34
- FWCI
- 3.28
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.