The Electronic Patient Reported Outcome Tool: Testing Usability and Feasibility of a Mobile App and Portal to Support Care for Patients With Complex Chronic Disease and Disability in Primary Care Settings
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
BACKGROUND: People experiencing complex chronic disease and disability (CCDD) face some of the greatest challenges of any patient population. Primary care providers find it difficult to manage multiple discordant conditions and symptoms and often complex social challenges experienced by these patients. The electronic Patient Reported Outcome (ePRO) tool is designed to overcome some of these challenges by supporting goal-oriented primary care delivery. Using the tool, patients and providers collaboratively develop health care goals on a portal linked to a mobile device to help patients and providers track progress between visits. OBJECTIVES: This study tested the usability and feasibility of adopting the ePRO tool into a single interdisciplinary primary health care practice in Toronto, Canada. The Fit between Individuals, Fask, and Technology (FITT) framework was used to guide our assessment and explore whether the ePRO tool is: (1) feasible for adoption in interdisciplinary primary health care practices and (2) usable from both the patient and provider perspectives. This usability pilot is part of a broader user-centered design development strategy. METHODS: A 4-week pilot study was conducted in which patients and providers used the ePRO tool to develop health-related goals, which patients then monitored using a mobile device. Patients and providers collaboratively set goals using the system during an initial visit and had at least 1 follow-up visit at the end of the pilot to discuss progress. Focus groups and interviews were conducted with patients and providers to capture usability and feasibility measures. Data from the ePRO system were extracted to provide information regarding tool usage. RESULTS: Six providers and 11 patients participated in the study; 3 patients dropped out mainly owing to health issues. The remaining 8 patients completed 210 monitoring protocols, equal to over 1300 questions, with patients often answering questions daily. Providers and patients accessed the portal on an average of 10 and 1.5 times, respectively. Users found the system easy to use, some patients reporting that the tool helped in their ability to self-manage, catalyzed a sense of responsibility over their care, and improved patient-centered care delivery. Some providers found that the tool helped focus conversations on goal setting. However, the tool did not fit well with provider workflows, monitoring questions were not adequately tailored to individual patient needs, and daily reporting became tedious and time-consuming for patients. CONCLUSIONS: Although our study suggests relatively low usability and feasibility of the ePRO tool, we are encouraged by the early impact on patient outcomes and generally positive responses from both user groups regarding the potential of the tool to improve care for patients with CCDD. As is consistent with our user-centered design development approach, we have modified the tool based on user feedback, and are now testing the redeveloped tool through an exploratory trial.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., Yount, S., Amtmann, D., Bode, R., Buysse, D., Choi, S., et al. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.011 · Cited by 5,128
Aging with multimorbidity: A systematic review of the literature
Marengoni, A., Angleman, S., Melis, R., Mangialasche, F., Karp, A., Garmen, A., Meinow, B., & Fratiglioni, L. (2011) · Ageing Research Reviews · doi:10.1016/j.arr.2011.03.003 · Cited by 3,092
Boyd, C. M., Darer, J., Boult, C., Fried, L. P., Boult, L., & Wu, A. W. (2005) · JAMA · doi:10.1001/jama.294.6.716 · Cited by 2,581 · cited 1×
“Beyond the challenges that patients face in managing their own illnesses, primary health care providers struggle to manage the multiple conditions with discordant competing symptoms faced by these patients [5] and lack appropriate clinical practice guidelines to guide decision-making [6].”
BackgroundWriting SMART rehabilitation goals and achieving goal attainment scaling: a practical guide
Bovend'Eerdt, T. J., Botell, R. E., & Wade, D. T. (2009) · Clinical Rehabilitation · doi:10.1177/0269215508101741 · Cited by 754
mHealth 2.0: Experiences, Possibilities, and Perspectives
Becker, S., Miron-Shatz, T., Schumacher, N., Krocza, J., Diamantidis, C., & Albrecht, U.-V. (2014) · JMIR mHealth and uHealth · doi:10.2196/mhealth.3328 · Cited by 354 · cited 1×
“…tools available that help support goal setting and monitoring, most of these are disease-specific (eg, supporting patients with diabetes [20]), and there are few available tools that can address the needs of patients with complex care needs who tend to be high users of the health care system [21].”
BackgroundO’Kane, M. J., Bunting, B., Copeland, M., & Coates, V. E. (2008) · BMJ · doi:10.1136/bmj.39534.571644.be · Cited by 326 · cited 1×
“Other studies have found that health monitoring may lead to adverse effects on mental health and well-being [42].”
BackgroundBrown, W., Yen, P.-Y., Rojas, M., & Schnall, R. (2013) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2013.08.001 · Cited by 299 · cited 2×
“Tools are typically assessed in terms of efficiency (what resources are required by the user to complete tasks [27]), effectiveness (the ability to complete tasks completely and accurately), learnability (how easily users can learn the system), and user satisfaction with the product [27,29,30].”
BackgroundBroderick, J. E., Schneider, S., Junghaenel, D. U., Schwartz, J. E., & Stone, A. A. (2013) · Arthritis Care & Research · doi:10.1002/acr.22025 · Cited by 158 · cited 1×
“Furthermore, these tools have demonstrated validity and reliability in chronic disease populations [37-39].”
Backgroundvan der Weegen, S., Verwey, R., Spreeuwenberg, M., Tange, H., van der Weijden, T., & de Witte, L. (2013) · JMIR mhealth and uhealth · doi:10.2196/mhealth.2526 · Cited by 128
A Persuasive and Social mHealth Application for Physical Activity: A Usability and Feasibility Study
Al Ayubi, S. U., Parmanto, B., Branch, R., & Ding, D. (2014) · JMIR mHealth and uHealth · doi:10.2196/mhealth.2902 · Cited by 108
Yang, C., Linas, B., Kirk, G., Bollinger, R., Chang, L., Chander, G., Siconolfi, D., Braxton, S., Rudolph, A., & Latkin, C. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.4344 · Cited by 73 · cited 1×
“Feasibility refers to the ability of users to adopt a technology or intervention in daily routines (often assessed through use of the tool [28]), which is strongly related to the FITT model intention to assess the ability of a technology to be adopted.”
BackgroundConnelly, K., Siek, K. A., Chaudry, B., Jones, J., Astroth, K., & Welch, J. L. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000732 · Cited by 68
Vermeulen, J., Neyens, J., Spreeuwenberg, van Rossum, E., Sipers, Habets, Hewson, D., & de Witte (2013) · Patient Preference and Adherence · doi:10.2147/ppa.s45897 · Cited by 64 · cited 2×
“As a part of this development strategy, conducting usability and feasibility testing is vital to ensure that tools are understandable and can be adopted by target users in typical settings before running larger more costly evaluations [24,25].”
MethodA novel multimodal tool for telemonitoring patients with COPD
Sánchez-Morillo, D., Crespo, M., León, A., & Crespo Foix, L. F. (2015) · Informatics for Health and Social Care · doi:10.3109/17538157.2013.872114 · Cited by 25 · cited 1×
“As a part of this development strategy, conducting usability and feasibility testing is vital to ensure that tools are understandable and can be adopted by target users in typical settings before running larger more costly evaluations [24,25].”
Cited by
Granja, C., Janssen, W., & Johansen, M. A. (2018) · Journal of Medical Internet Research · doi:10.2196/10235 · Cited by 578
“Multimedia Appendix 1 [19-239] presents the outcomes.”
BackgroundMethodResultInfluentialQudah, B., & Luetsch, K. (2019) · Patient Education and Counseling · doi:10.1016/j.pec.2019.01.021 · Cited by 200
Wallwiener, M., Matthies, L., Simoes, E., Keilmann, L., Hartkopf, A. D., Sokolov, A. N., Walter, C. B., Sickenberger, N., Wallwiener, S., Feisst, M., et al. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.8210 · Cited by 55
Öberg, U., Isaksson, U., Jutterström, L., Orre, C. J., & Hörnsten, Å. (2018) · JMIR Diabetes · doi:10.2196/diabetes.9059 · Cited by 41
Metadata
- Authors
- Steele Gray, C., Gill, A., Khan, A. I., Hans, P. K., Kuluski, K., & Cott, C.
- Venue
- JMIR mHealth and uHealth
- Publisher
- JMIR Publications Inc.
- Published
- 2016-06-02
- Volume
- 4
- Issue
- 2
- Pages
- e58
- ISSN
- 2291-5222 (ISSN-L)
- OpenAlex
- W2410352668
- Cited by
- 98
- References
- 14
- FWCI
- 18.03
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.