A Prospective Study of Usability and Workload of Electronic Medication Adherence Products by Older Adults, Caregivers, and Health Care Providers
Open accessPublished version of a preprint
The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
Abstract
Background: A decreased capacity to self-manage medications results in nonadherence, medication errors, and drug-related problems in older adults. Previous research identified 80 electronic medication adherence products available to assist patients with self-management of medications. Unfortunately, the usability and workload of these products are unknown. Objective: This study aimed to examine the usability and workload of a sample of electronic medication adherence products. Methods: In a prospective, mixed methods study, a sample of older adults, health care professionals, and caregivers tested the usability and workload of 21 electronic medication adherence products. Each participant tested 5 products, one at a time, after which they completed the system usability scale (SUS) and NASA-task load index (NASA-TLX), instruments that measure the usability and workload involved in using a product. Higher SUS scores indicate more user-friendliness, whereas lower NASA-TLX raw scores indicate less workload when using a product. Results: Electronic medication adherence products required a mean of 12.7 steps (range 5-20) for the appropriate use and took, on average, 15.19 min to complete the setup tasks (range 1-56). Participants were able to complete all steps without assistance 55.3% of the time (103 out of the 186 tests were completed by 39 participants; range 0%-100%). The mean SUS and NASA-TLX raw scores were 52.8 (SD 28.7; range 0-100) and 50.0 (SD 25.7; range 4.2-99.2), respectively, revealing significant variability among the electronic medication adherence products. The most user-friendly products were found to be TimerCap travel size (mean 78.67, SD 15.57; P=.03) and eNNOVEA Weekly Planner with Advanced Auto Reminder (mean 78.13, SD 14.13; P=.049) as compared with MedReady 1700 automated medication dispenser (mean 28.63, SD 21.24). Similarly, MedReady (72.92, SD 18.69) was found to be significantly more work intensive when compared with TimerCap (29.35, SD 20.35; P=.03), e-pill MedGlider home medication management system (28.43, SD 20.80; P=.02), and eNNOVEA (28.65, SD 14.97; P=.03). The e-pill MedTime Station automatic pill dispenser with tipper (71.77, SD 21.98) had significantly more workload than TimerCap (P=.04), MedGlider (P=.03), and eNNOVEA (P=.04). Conclusions: This study demonstrated that variability exists in the usability and workload of different electronic medication adherence products among older adults, caregivers, and clinicians. With few studies having investigated the usability and workload of electronic medication adherence products, no benchmarks exist to compare the usability and workload of these products. However, our study highlights the need to assess the usability and workload of different products marketed to assist with medication taking and provides guidance to clinicians regarding electronic medication adherence product recommendations for their patients. Future development of electronic medication adherence products should ensure that the target populations of patients are able to use these products adequately to improve medication management.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“They reported that the usability of these products is signifcantly lower than the national average usability scale score [30].”
From A Collaborative Approach to Support Medication Management in Older Adults with Mild Cognitive Impairment Using Conversational Assistants (CAs) (2022) · cites this paper as [30]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003
Adherence to Long-Term Therapies: Evidence for Action
De Geest, S., & Sabaté, E. (2003) · European Journal of Cardiovascular Nursing · doi:10.1016/s1474-5151(03)00091-4 · Cited by 6,272
Usability Evaluation In Industry
(1996) · CRC Press eBooks · doi:10.1201/9781498710411 · Cited by 4,354
Clinical consequences of polypharmacy in elderly
Maher, R. L., Hanlon, J., & Hajjar, E. R. (2014) · Expert Opinion on Drug Safety · doi:10.1517/14740338.2013.827660 · Cited by 1,938
Multimorbidity in Older Adults
Salive, M. E. (2013) · Epidemiologic Reviews · doi:10.1093/epirev/mxs009 · Cited by 1,341
Cognitive and usability engineering methods for the evaluation of clinical information systems.
Kushniruk, A. W., & Patel, V. L. (2004) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2004.01.003 · Cited by 760
Gellad, W. F., Grenard, J. L., & Marcum, Z. A. (2011) · The American Journal of Geriatric Pharmacotherapy · doi:10.1016/j.amjopharm.2011.02.004 · Cited by 692
With TOR, Less Is More: A Key Role for the Conserved Nutrient-Sensing TOR Pathway in Aging
Kapahi, P., Chen, D., Rogers, A. N., Katewa, S. D., Li, P. W.-L., Thomas, E. L., & Kockel, L. (2010) · Cell Metabolism · doi:10.1016/j.cmet.2010.05.001 · Cited by 660
Jaspers, M. W. M. (2009) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2008.10.002 · Cited by 649 · cited 1×
“In an effort to measure usability in a multidomain approach, we used both quantitative measures, including validated tools, cognitive walkthroughs, observations, and qualitative one-on-one interviews [35,43].”
MethodEmpirical Studies on Usability of mHealth Apps: A Systematic Literature Review
Zapata, B. C., Fernández-Alemán, J. L., Idri, A., & Toval, A. (2015) · Journal of Medical Systems · doi:10.1007/s10916-014-0182-2 · Cited by 585 · cited 2×
“As 80% of the usability problems can be identified with 5 test-users [34], we aimed to test each product with at least five participants.”
BackgroundPolypharmacy Among Adults Aged 65 Years and Older in the United States: 1988–2010
Charlesworth, C. J., Smit, E., Lee, D. S. H., Alramadhan, F., & Odden, M. C. (2015) · The Journals of Gerontology Series A: Biological Sciences and Medical Sciences · doi:10.1093/gerona/glv013 · Cited by 570 · cited 1×
“Chronic diseases such as heart disease, hypertension, diabetes, and cancer, among others, are typically managed with medications, in many cases, with multiple medications [5-7].”
BackgroundHow High is High? A Meta-Analysis of NASA-TLX Global Workload Scores
Grier, R. A. (2015) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931215591373 · Cited by 460
Determining what individual SUS scores mean
(2009) · Journal of Usability Studies archive · doi:10.5555/2835587.2835589 · Cited by 412
Association between polypharmacy and death: A systematic review and meta-analysis
Leelakanok, N., Holcombe, A. L., Lund, B. C., Gu, X., & Schweizer, M. L. (2017) · Journal of the American Pharmacists Association · doi:10.1016/j.japh.2017.06.002 · Cited by 355
Sauro, J., & Lewis, J. R. (2016) · Quantifying the User Experience · doi:10.1016/b978-0-12-802308-2.00002-3 · Cited by 312
Systematic review of the barriers affecting medication adherence in older adults
Yap, A. F., Thirumoorthy, T., & Kwan, Y. H. (2016) · Geriatrics & Gerontology International · doi:10.1111/ggi.12616 · Cited by 308 · cited 3×
“Among older adults, nonadherence to medications can arise from several different factors, including patient-related, medication-related, health care provider–related, health care system–related, and socioeconomic-related factors [19].”
BackgroundInterventional tools to improve medication adherence: review of literature
Costa, E., Pecorelli, S., Giardini, A., Savin, M., Menditto, E., Lehane, E., Laosa, O., Monaco, A., & Marengoni, A. (2015) · Patient Preference and Adherence · doi:10.2147/ppa.s87551 · Cited by 295
Intentional and Unintentional Nonadherence: A Study of Decision Making
Wroe, A. L. (2002) · Journal of Behavioral Medicine · doi:10.1023/a:1015866415552 · Cited by 283
Cross, A. J., Elliott, R. A., Petrie, K., Kuruvilla, L., & George, J. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012419.pub2 · Cited by 249
The epidemiology of polypharmacy
Payne, R. A. (2016) · Clinical Medicine · doi:10.7861/clinmedicine.16-5-465 · Cited by 209
Grindrod, K. A., Li, M., & Gates, A. (2014) · JMIR mhealth and uhealth · doi:10.2196/mhealth.3048 · Cited by 204
Adverse Drug Events as a Cause of Hospitalization in Older Adults
Salvi, F., Marchetti, A., D’Angelo, F., Boemi, M., Lattanzio, F., & Cherubini, A. (2012) · Drug Safety · doi:10.1007/bf03319101 · Cited by 177
Chiatti, C., Bustacchini, S., Furneri, G., Mantovani, L. G., Cristiani, M., Misuraca, C., & Lattanzio, F. (2012) · Drug Safety · doi:10.1007/bf03319105 · Cited by 172
Maddigan, S. L., Farris, K. B., Keating, N. C., Wiens, C. A., & Johnson, J. A. (2003) · Journal of Aging and Health · doi:10.1177/0898264303251893 · Cited by 138
Practical Problems with Medication Use that Older People Experience: A Qualitative Study
Notenboom, K., Beers, E., van Riet‐Nales, D. A., Egberts, T. C. G., Leufkens, H. G. M., Jansen, P. A. F., & Bouvy, M. L. (2014) · Journal of the American Geriatrics Society · doi:10.1111/jgs.13126 · Cited by 136
Intentional Non-Adherence to Medications by Older Adults
Mukhtar, O., Weinman, J. A., & Jackson, S. H. (2014) · Drugs & Aging · doi:10.1007/s40266-014-0153-9 · Cited by 128
Functional Ability of Patients to manage Medication Packaging: A Survey of Geriatric Inpatients
Atkin, P. A., FINNEGAN, T. P., OGLE, S. J., & Shenfield, G. M. (1994) · Age and Ageing · doi:10.1093/ageing/23.2.113 · Cited by 104
Elliott, R. A., & Marriott, J. L. (2009) · BMC Geriatrics · doi:10.1186/1471-2318-9-27 · Cited by 102
Cárdenas‐Valladolid, J., Martín-Madrazo, C., Salinero-Fort, M. Á., Pau, E. C. D. S., Abánades-Herranz, J. C., & de Burgos‐Lunar, C. (2010) · Drugs & Aging · doi:10.2165/11537320-000000000-00000 · Cited by 93
Vasilopoulos, T., Kotwal, A., Huisingh-Scheetz, M. J., Waite, L. J., McClintock, M. K., & Dale, W. (2014) · The Journals of Gerontology Series B: Psychological Sciences and Social Sciences · doi:10.1093/geronb/gbu025 · Cited by 88
Older adults' satisfaction with a medication dispensing device in home care
Reeder, B., Demiris, G., & Marek, K. D. (2013) · Informatics for Health and Social Care · doi:10.3109/17538157.2012.741084 · Cited by 75
Sino, C. G. M., Sietzema, M., Egberts, T. C. G., & Schuurmans, M. J. (2013) · The journal of nutrition health & aging · doi:10.1007/s12603-013-0359-2 · Cited by 69
A framework for evaluating usability of clinical monitoring technology
Daniels, J. P., Fels, S., Kushniruk, A. W., Lim, J. M.-Y., & Ansermino, J. M. (2007) · Journal of Clinical Monitoring and Computing · doi:10.1007/s10877-007-9091-y · Cited by 67
Effects of nonadherence with prescription drugs among older adults.
Butler, R., Davis, T. K., Johnson, W. G., & Gardner, H. H. (2011) · PubMed · Cited by 62
Drug Use Among Seniors in Canada, 2016
Proulx, J. (2018) · Value in Health · doi:10.1016/j.jval.2018.04.1003 · Cited by 62
How Much Does Medication Nonadherence Cost the Medicare Fee-for-Service Program?
LLOYD, J. T., Maresh, S., Powers, C. A., Shrank, W. H., & Alley, D. E. (2019) · Medical Care · doi:10.1097/mlr.0000000000001067 · Cited by 54
A framework for evaluating mHealth tools for Older Patients on Usability.
Wildenbos, G. A., Peute, L., & Jaspers, M. (2015) · PubMed · Cited by 48
Instruments Assessing Capacity to Manage Medications
Farris, K. B., & Phillips, B. B. (2008) · Annals of Pharmacotherapy · doi:10.1345/aph.1g502 · Cited by 47
Problems experienced by older people when opening medicine packaging
Philbert, D., Notenboom, K., Bouvy, M. L., & van Geffen, E. C. G. (2013) · International Journal of Pharmacy Practice · doi:10.1111/ijpp.12070 · Cited by 44
A cloud medication safety support system using QR code and Web services for elderly outpatients
Tseng, M., & Wu, H.-C. (2014) · Technology and Health Care · doi:10.3233/thc-140778 · Cited by 43
Variation in medication understanding among the elderly
Spiers, M. V., Kutzık, D. M., & Lamar, M. (2004) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/61.4.373 · Cited by 39
Ligons, F. M., Mello-Thoms, C. R., Handler, S. M., Romagnoli, K. M., & Hochheiser, H. (2014) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2014.07.004 · Cited by 32
Medication Management Apps: Usable by Older Adults?
Stuck, R. E., Chong, A. W., Tracy, L. M., & Rogers, W. A. (2017) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213601769 · Cited by 28
Cross, A. J., Elliott, R. A., & George, J. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012419 · Cited by 25
A framework for evaluating mHealth tools for Older Patients on Usability
Wildenbos, G. A., Peute, L., & Jaspers, M. W. M. (2015) · Studies in health technology and informatics · doi:10.3233/978-1-61499-512-8-783 · Cited by 21
Ip, Q., Malone, D. C., Chong, J., Harris, R. B., & Labiner, D. M. (2018) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2018.01.009 · Cited by 21
Cost and Consequences of Nonadherence With Oral Bisphosphonate Therapy
Moser, S. S., Yu, J., Goldshtein, I., Ish‐Shalom, S., Rouach, V., Shalev, V., Modi, A., & Chodick, G. (2016) · Annals of Pharmacotherapy · doi:10.1177/1060028015626935 · Cited by 20
Visual acuity, literacy, and unintentional misuse of nonprescription medications
Mullen, R., Curtis, L. M., O’Conor, R. M., Serper, M., McCarthy, D. M., Bailey, S. C., Parker, R. M., & Wolf, M. S. (2018) · American Journal of Health-System Pharmacy · doi:10.2146/ajhp170303 · Cited by 16
Ligons, F. M., Romagnoli, K. M., Browell, S., Hochheiser, H., & Handler, S. M. (2011) · PubMed Central · Cited by 14
Pharmacy Practice Research Abstracts: Canadian Pharmacists Conference 2017
(2017) · Canadian Pharmacists Journal / Revue des Pharmaciens du Canada · doi:10.1177/1715163517719855 · Cited by 7
Research works printed but not linked: 5
- 2017, World Population Ageing 2017 Highlights
- Butler, R, 2011, Am J Manag Care, 17, 153
- 10.1002/14651858.CD012419/full · doi:10.1002/14651858.cd012419/full
- Ligons, FM, 2011, AMIA Annu Symp Proc, 2011, 795
- Sauro, J, 2016, Quantifying the User Experience: Practical Statistics for User Research. Second Edition
Cited by
Applications of Mixed Reality Technology in Orthopedics Surgery: A Pilot Study
Lu, L., Wang, H., Liu, P., Liu, R., Zhang, J., Xie, Y., Liu, S., Huo, T., Xie, M., Wu, X., et al. (2022) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2022.740507 · Cited by 103
Use of technology and social media in dementia care: Current and future directions
Shu, S., & Woo, B. K. P. (2021) · World Journal of Psychiatry · doi:10.5498/wjp.v11.i4.109 · Cited by 99
Emerging approaches to polypharmacy among older adults
Mehta, R. S., Kochar, B. D., Kennelty, K. A., Ernst, M. E., & Chan, A. T. (2021) · Nature Aging · doi:10.1038/s43587-021-00045-3 · Cited by 84
A Systematic Review on Pill and Medication Dispensers from a Human-Centered Perspective
Gargioni, L., Fogli, D., & Baroni, P. (2024) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-024-00161-w · Cited by 69
Mathur, N., Dhodapkar, K., Zubatiy, T., Li, J., Jones, B., & Mynatt, E. (2022) · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3517428.3544830 · Cited by 45
“than the national average usability scale score [30].”
Background
Metadata
- Authors
- Patel, T., Ivo, J., Faisal, S., McDougall, A., Carducci, J., Pritchard, S., & Chang, F.
- Affiliations
- University of Waterloo (CA)
- Centre for Family Medicine (CA)
- Corresponding author
- Patel, Tejal
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2020-06-02
- Volume
- 22
- Issue
- 6
- Pages
- e18073
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC7298635
- OpenAlex
- W3013803816
- Cited by
- 49
- References
- 50
- Citations by year
- 2026: 10 · 2025: 9 · 2024: 7 · 2023: 7 · 2022: 5 · 2021: 9 · 2020: 2
- FWCI
- 5.98
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Medicine › Family Practice › Medication Adherence and Compliance
- Health Sciences › Medicine › Geriatrics and Gerontology › Technology Use by Older Adults
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- medication adherence, usability, workload, older adults, caregivers, healthcare workers, medication self-management, System Usability Scale, NASA-TLX, medication reminders, medication errors, drug-related problems
- MeSH terms
- Aged, Electronics, Female, Health Personnel, Humans, Male, Prospective Studies, Workload, Drug Monitoring, Caregivers
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.