Assessing Mobile Phone Digital Literacy and Engagement in User-Centered Design in a Diverse, Safety-Net Population: Mixed Methods Study
Open accessPublished version of a preprint
The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
Background: Health care systems are rapidly deploying digital tools for disease management; however, few studies have evaluated their usability by vulnerable populations. To understand the barriers to app usage among vulnerable populations, we employed user-centered design (UCD) methods in the development of a new text messaging app. Objective: The study aimed to describe variations in patients’ engagement in the app design process, focusing on limited health literacy (LHL), limited English proficiency (LEP), and limited digital literacy (LDL). Methods: We conducted 20 in-depth semistructured interviews with primary care patients at a public health care system, used open-ended discussions and card sorting tasks to seek input about mobile phones and text messaging, and used open coding to categorize the patterns of mobile phone usage and to evaluate engagement in the card sorting process. We examined qualitative differences in engagement by examining the extensiveness of participant feedback on existing and novel text messaging content and calculated the proportion of patients providing extensive feedback on existing and novel content, overall and by health literacy, English proficiency, and digital literacy. Results: The average age of the 20 participants was 59 (SD 8) years; 13 (65%) were female, 18 (90%) were nonwhite, 16 (80%) had LHL, and 13 (65%) had LEP. All had depression, and 14 (70%) had diabetes. Most participants had smartphones (18/20, 90%) and regularly used text messaging (15/20, 75%), but 14 (70%) of them reported having difficulty texting because of inability to type, physical disability, and low literacy. We identified 10 participants as specifically having LDL; 7 of these participants had LEP, and all 10 had LHL. Half of the participants required a modification of the card sorting activity owing to not understanding it or not being able to read the cards in the allotted time. The proportion of participants who gave extensive feedback on existing content was lower in participants with limited versus adequate English proficiency (4/13, 30% vs 5/7, 71%), limited versus adequate health literacy (7/16, 44% vs 3/4, 75%), and limited versus adequate digital literacy (4/10, 40% vs 6/10, 60%); none of these differences were statistically significant. When examining the proportion of patients who gave extensive feedback for novel messaging content, those with LHL were less engaged than those with adequate health literacy (8/16, 50% vs 4/4, 100%); there were no statistical differences by any subgroup. Conclusions: Despite widespread mobile phone use, digital literacy barriers are common among vulnerable populations. Engagement in the card sorting activity varied among participants and appeared to be lower among those with LHL, LEP, and LDL. Researchers employing traditional UCD methods should routinely measure these communication domains among their end-user samples. Future work is needed to replicate our findings in larger samples, but augmentation of card sorting with direct observation and audiovisual cues may be more productive in eliciting feedback for those with communication barriers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
eHealth Literacy: Essential Skills for Consumer Health in a Networked World.
Norman, C. D., & Skinner, H. A. (2006) · Journal of Medical Internet Research · doi:10.2196/jmir.8.2.e9 · Cited by 2,643 · cited 1×
“In fact, health literacy has been included as a component of digital literacy in some conceptual models [30].”
Whitehead, L., & Seaton, P. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.4883 · Cited by 743 · cited 1×
“Despite the potential promise of health-related mobile apps in improving the health of individuals with chronic diseases [1-4], few studies evaluate their usability and feasibility, and even fewer do so among vulnerable populations [5-7].”
O’Connor, S., Hanlon, P., O’Donnell, C. A., Garcia, S., Glanville, J., & Mair, F. S. (2016) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-016-0359-3 · Cited by 636
Health Literacy and Health Information Technology Adoption: The Potential for a New Digital Divide
Mackert, M., Mabry-Flynn, A., Champlin, S., Donovan, E. E., & Pounders, K. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.6349 · Cited by 503 · cited 5×
“Existing data suggest that populations with limited health literacy (LHL), limited digital literacy (LDL), and limited English proficiency (LEP) are less likely to use health information technology (HIT), including mobile apps [6,8-11].”
InfluentialRathbone, A. L., & Prescott, J. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7740 · Cited by 497 · cited 1×
“Despite the potential promise of health-related mobile apps in improving the health of individuals with chronic diseases [1-4], few studies evaluate their usability and feasibility, and even fewer do so among vulnerable populations [5-7].”
Wu, Y., Yao, X., Vespasiani, G., Nicolucci, A., Dong, Y., Kwong, J., Li, L., Sun, X., Tian, H., & Li, S. (2017) · JMIR mHealth and uHealth · doi:10.2196/mhealth.6522 · Cited by 283
Usability of Commercially Available Mobile Applications for Diverse Patients
Sarkar, U., Gourley, G. I., Lyles, C. R., Tieu, L., Clarity, C., Newmark, L., Singh, K., & Bates, D. W. (2016) · Journal of General Internal Medicine · doi:10.1007/s11606-016-3771-6 · Cited by 257
Self-citationImproving low health literacy and patient engagement: A social ecological approach
McCormack, L., Thomas, V., Lewis, M. A., & Rudd, R. (2017) · Patient Education and Counseling · doi:10.1016/j.pec.2016.07.007 · Cited by 231 · cited 1×
“LHL, specifically, has been associated with lower patient activation and engagement in clinical settings, and this may be relevant in research settings as well [36-38].”
Does mHealth increase adherence to medication? Results of a systematic review
Anglada-Martinez, H., Riu-Viladoms, G., Martin-Conde, M., Rovira-Illamola, M., Sotoca-Momblona, J. M., & Codina-Jane, C. (2015) · International Journal of Clinical Practice · doi:10.1111/ijcp.12582 · Cited by 217
Designing and Evaluating mHealth Interventions for Vulnerable Populations: A Systematic Review
Stowell, E., Lyson, M. C., Saksono, H., Wurth, R. C., Jimison, H., Pavel, M., & Parker, A. G. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173589 · Cited by 169 · cited 2×
“Despite the potential promise of health-related mobile apps in improving the health of individuals with chronic diseases [1-4], few studies evaluate their usability and feasibility, and even fewer do so among vulnerable populations [5-7].”
Aguilera, A., Bruehlman-Senecal, E., Demasi, O., & Avila, P. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6914 · Cited by 122 · cited 1×
“A total of 10 transcripts are from patients who had completed a previous trial evaluating automated text messages as an adjunct to cognitive behavioral therapy for depression (MoodText trial, NCT01083628) [19], and 10 are from a separate group of patients participating in early design sessions to develop a text messag…”
Self-citation
Research works printed but not linked: 25
- 10.2337/dc13-1984 · DOI · Paper page
- 10.1080/10810730.2010.499988 · DOI · Paper page
- 10.2196/diabetes.6916 · DOI · Paper page
- 10.2196/mhealth.4928 · DOI · Paper page
- 10.1145/1358628.1358968 · DOI
- 10.1016/j.amepre.2016.06.015 · DOI · Paper page
- 10.1093/tbm/iby119 · DOI · Paper page
- 10.1016/j.sapharm.2015.09.009 · DOI
- Chew, LD, 2004, Fam Med, 36, 588
- 10.1001/jama.2010.896 · DOI · Paper page
- 10.1007/s11606-010-1552-1 · DOI · Paper page
- 10.1007/s11606-008-0693-y · DOI · Paper page
- 10.2196/humanfactors.3696 · DOI · Paper page
- 10.1145/3027063.3058589 · DOI
- 10.1145/2998581.2998614 · DOI
- 10.3233/978-1-61499-101-4-353 · DOI
- Paul, CL, 2008, J Usability Stud, 4, 7
- Wood, JR, 2008, J Usability Stud, 4, 1
- 10.1145/2207676.2208589 · DOI · Paper page
- 10.1155/2016/7586385 · DOI
- 10.1177/2327857917061010 · DOI
- 10.1007/s11606-013-2466-5 · DOI · Paper page
- 10.1080/10810730.2011.604705 · DOI
- Bledsoe, KL, 2013, The Handbook of Social Research Ethics, 391
- 10.1186/s12913-016-1843-2 · DOI
Cited by
Digital literacy as a new determinant of health: A scoping review
Arias López, M. D. P., Ong, B. A., Borrat Frigola, X., Fernández, A. L., Hicklent, R. S., Obeles, A. J. T., Rocimo, A. M., & Celi, L. A. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000279 · Cited by 422
Kendzerska, T., Zhu, D. T., Gershon, A. S., Edwards, J. D., Peixoto, C., Robillard, R., & Kendall, C. E. (2021) · Risk Management and Healthcare Policy · doi:10.2147/rmhp.s293471 · Cited by 279
Effectiveness of Disease-Specific mHealth Apps in Patients With Diabetes Mellitus: Scoping Review
Eberle, C., Löhnert, M., & Stichling, S. (2021) · JMIR mHealth and uHealth · doi:10.2196/23477 · Cited by 160
“But despite the widespread use of smartphones, digital literacy barriers are common in vulnerable populations, which could reduce the effectiveness of diabetes technologies [80].”
Moulaei, K., Sheikhtaheri, A., Fatehi, F., Shanbehzadeh, M., & Bahaadinbeigy, K. (2023) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-023-02348-4 · Cited by 140
Old-fashioned technology in the era of “bling”: is there a future for text messaging in health care?
Willcox, J. C., Dobson, R., & Whittaker, R. (2019) · Journal of Medical Internet Research · doi:10.2196/16630 · Cited by 95
Metadata
- Authors
- Nouri, S. S., Avila-Garcia, P., Cemballi, A. G., Sarkar, U., Aguilera, A., & Lyles, C. R.
- Venue
- JMIR mHealth and uHealth
- Publisher
- JMIR Publications Inc.
- Published
- 2019-08-29
- Volume
- 7
- Issue
- 8
- Pages
- e14250
- ISSN
- 2291-5222 (ISSN-L)
- OpenAlex
- W2950061073
- Cited by
- 112
- References
- 11
- FWCI
- 15.69
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.