Leveraging Multi-Modal Sensing for Mobile Health: A Case Review in Chronic Pain
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
Active and passive mobile sensing has garnered much attention in recent years. In this paper, we focus on chronic pain measurement and management as a case application to exemplify the state of the art. We present a consolidated discussion on the leveraging of various sensing modalities along with modular server-side and on-device architectures required for this task. Modalities included are: activity monitoring from accelerometry and location sensing, audio analysis of speech, image processing for facial expressions as well as modern methods for effective patient self-reporting. We review examples that deliver actionable information to clinicians and patients while addressing privacy, usability, and computational constraints. We also discuss open challenges in the higher level inferencing of patient state and effective feedback with potential directions to address them. The methods and challenges presented here are also generalizable and relevant to a broad range of other applications in mobile sensing.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797
A behavior model for persuasive design
Fogg, B. (2009) · Proceedings of the 4th International Conference on Persuasive Technology · doi:10.1145/1541948.1541999 · Cited by 2,328 · cited 1×
“Such perceptions of familiarity and low effort increases the likelihood of actualization according to behavioral theory [124].”
A survey of mobile phone sensing
Lane, N., Miluzzo, E., Lu, H., Peebles, D., Choudhury, T., & Campbell, A. (2010) · IEEE Communications Magazine · doi:10.1109/mcom.2010.5560598 · Cited by 2,292
Self-citationUsing Pedometers to Increase Physical Activity and Improve Health: A Systematic Review
Bravata, D. M., Smith-Spangler, C., Sundaram, V., Gienger, A. L., Lin, N., Lewis, R., Stave, C. D., Olkin, I., & Sirard, J. R. (2007) · JAMA · doi:10.1001/jama.298.19.2296 · Cited by 2,064
The IMAGE Study Group, Greaves, C. J., Sheppard, K. E., Abraham, C., Hardeman, W., Roden, M., Evans, P. H., & Schwarz, P. (2011) · BMC Public Health · doi:10.1186/1471-2458-11-119 · Cited by 1,163
Wang, R., Chen, F., Chen, Z., Li, T., Harari, G., Tignor, S., Zhou, X., Ben-Zeev, D., & Campbell, A. T. (2014) · Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2632048.2632054 · Cited by 1,116
Eisenberger, N. I. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3231 · Cited by 1,016
Ehde, D. M., Dillworth, T. M., & Turner, J. A. (2014) · American Psychologist · doi:10.1037/a0035747 · Cited by 879 · cited 2×
“For example, studies have shown that factors such as social isolation [74] are linked to physical pain and can be detrimental to CP management [75] and factors into CBT [45].”
Capturing momentary, self-report data: A proposal for reporting guidelines
Stone, A. A., & Shiffman, S. (2002) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm2403_09 · Cited by 807
Saeb, S., Zhang, M., Karr, C. J., Schueller, S. M., Corden, M. E., Kording, K. P., & Mohr, D. C. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4273 · Cited by 782
Accuracy of Smartphone Applications and Wearable Devices for Tracking Physical Activity Data
Case, M. A., Burwick, H. A., Volpp, K. G., & Patel, M. S. (2015) · JAMA · doi:10.1001/jama.2014.17841 · Cited by 694
Turner, J. A., Holtzman, S., & Mancl, L. (2007) · Pain · doi:10.1016/j.pain.2006.09.005 · Cited by 530
Prkachin, K. M., & Solomon, P. E. (2008) · Pain · doi:10.1016/j.pain.2008.04.010 · Cited by 472
Moskowitz, D. S., & Young, S. N. (2006) · Journal of Psychiatry and Neuroscience · doi:10.1139/jpn.0602 · Cited by 457
A Meta-Analysis of Pedometer-Based Walking Interventions and Weight Loss
Richardson, C. R., Newton, T. L., Abraham, J. J., Sen, A., Jimbo, M., & Swartz, A. M. (2008) · The Annals of Family Medicine · doi:10.1370/afm.761 · Cited by 421 · cited 1×
“Such monitoring was traditionally achieved through specialized wearable devices [22], [23], or special-purpose instruments deployed in clinical environments [24].”
BeWell: A Smartphone Application to Monitor, Model and Promote Wellbeing
Lane, N., Mohammod, M., Lin, M., Yang, X., Lu, H., Ali, S., Doryab, A., Berke, E., Choudhury, T., & Campbell, A. (2011) · Proceedings of the 5th International ICST Conference on Pervasive Computing Technologies for Healthcare · doi:10.4108/icst.pervasivehealth.2011.246161 · Cited by 409
Self-citationJensen, M. P., Turner, J. A., & Romano, J. M. (2007) · Pain · doi:10.1016/j.pain.2006.12.007 · Cited by 393
Smartphone applications for pain management
Rosser, B. A., & Eccleston, C. (2011) · Journal of Telemedicine and Telecare · doi:10.1258/jtt.2011.101102 · Cited by 364
The painful face – Pain expression recognition using active appearance models
Ashraf, A. B., Lucey, S., Cohn, J. F., Chen, T., Ambadar, Z., Prkachin, K. M., & Solomon, P. E. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2009.05.007 · Cited by 355
Unobtrusive Sleep Monitoring using Smartphones
Chen, Z., Lin, M., Chen, F., Lane, N., Cardone, G., Wang, R., Li, T., Chen, Y., Choudhury, T., & Cambell, A. (2013) · Proceedings of the ICTs for improving Patients Rehabilitation Research Techniques · doi:10.4108/icst.pervasivehealth.2013.252148 · Cited by 300
Self-citationLalloo, C., Jibb, L. A., Rivera, J., Agarwal, A., & Stinson, J. N. (2015) · The Clinical Journal of Pain · doi:10.1097/ajp.0000000000000171 · Cited by 257
Continuous Pain Intensity Estimation from Facial Expressions
Kaltwang, S., Rudovic, O., & Pantic, M. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33191-6_36 · Cited by 245 · cited 1×
“In this case, it is not known whether this is because of pain, another ailment, or simply commuting with a friend or colleague who walks more slowly.”
Passive and In-Situ assessment of mental and physical well-being using mobile sensors
Rabbi, M., Ali, S., Choudhury, T., & Berke, E. (2011) · Proceedings of the 13th international conference on Ubiquitous computing · doi:10.1145/2030112.2030164 · Cited by 243
Self-citationRole of psychology in pain management
Eccleston, C. (2001) · British Journal of Anaesthesia · doi:10.1093/bja/87.1.144 · Cited by 235 · cited 1×
“such as emotion, cognition, socialization, function as well as behaviour [3].”
Advancing the science of mHealth
Nilsen, W., Kumar, S., Shar, A., Varoquiers, C., Wiley, T., Riley, W. T., Pavel, M., & Atienza, A. A. (2012) · Journal of Health Communication · doi:10.1080/10810730.2012.677394 · Cited by 226
Automatic coding of facial expressions displayed during posed and genuine pain
Littlewort, G. C., Bartlett, M. S., & Lee, K. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.12.010 · Cited by 216
Automatic detection of pain intensity
Hammal, Z., & Cohn, J. F. (2012) · Proceedings of the 14th ACM international conference on Multimodal interaction · doi:10.1145/2388676.2388688 · Cited by 185
Assessing the quality and usability of smartphone apps for pain self-management
Reynoldson, C., Stones, C., Allsop, M., Gardner, P., Bennett, M. I., Closs, S. J., Jones, R., & Knapp, P. (2014) · Pain Medicine · doi:10.1111/pme.12327 · Cited by 178
The more the merrier: Analysing the affect of a group of people in images
Dhall, A., Joshi, J., Sikka, K., Goecke, R., & Sebe, N. (2015) · 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2015.7163151 · Cited by 93 · cited 1×
“Emotional expressions from the user’s face [12] can be detected and augmented to further contextual factors such as scene type detection [76], the number of people in the image and even the overall mood of the group of people in the image [77].”
Automatic measurement of ad preferences from facial responses gathered over the Internet
McDuff, D., El Kaliouby, R., Senechal, T., Demirdjian, D., & Picard, R. (2014) · Image and Vision Computing · doi:10.1016/j.imavis.2014.01.004 · Cited by 52
Cited by
Tracking Depression Dynamics in College Students Using Mobile Phone and Wearable Sensing
Wang, R., Wang, W., daSilva, A., Huckins, J. F., Kelley, W. M., Heatherton, T. F., & Campbell, A. T. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3191775 · Cited by 340
“Existing work [6, 14, 45, 61, 62, 75] have found relationships between depression and generic behavioral features from passive sensing.”
BackgroundInternet of Health Things: Toward intelligent vital signs monitoring in hospital wards
da Costa, C. A., Pasluosta, C. F., Eskofier, B., da Silva, D. B., & da Rosa Righi, R. (2018) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2018.05.005 · Cited by 273
Wang, R., Aung, M. S. H., Abdullah, S., Brian, R., Campbell, A. T., Choudhury, T., Hauser, M., Kane, J., Merrill, M., Scherer, E. A., et al. (2016) · Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2971648.2971740 · Cited by 257
“Such longitudinal tracking is essential for addressing mental health states that have low frequency changes taking days, weeks or even months [4, 32, 50].”
BackgroundSelf-citationWang, W., Harari, G. M., Wang, R., Müller, S. R., Mirjafari, S., Masaba, K., & Campbell, A. T. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3264951 · Cited by 100
GLOBEM: Cross-Dataset Generalization of Longitudinal Human Behavior Modeling
Xu, X., Liu, X., Zhang, H., Wang, W., Nepal, S., Sefidgar, Y., Seo, W., Kuehn, K. S., Huckins, J. F., Morris, M. E., et al. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3569485 · Cited by 96
Metadata
- Authors
- Aung, M. S. H., Alquaddoomi, F., Hsieh, C.-K., Rabbi, M., Yang, L., Pollak, J. P., Estrin, D., & Choudhury, T.
- Venue
- IEEE Journal of Selected Topics in Signal Processing
- Published
- 2016-08
- Volume
- 10
- Issue
- 5
- Pages
- 962-974
- ISSN
- 1932-4553 (ISSN-L)
- 1941-0484
- OpenAlex
- W2346921725
- Cited by
- 37
- References
- 30
- FWCI
- 3.88
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.