App Chronic Disease Checklist: Protocol to Evaluate Mobile Apps for Chronic Disease Self-Management
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
BACKGROUND: The availability of mobile health apps for self-care continues to increase. While little evidence of their clinical impact has been published, there is general agreement among health authorities and authors that consumers' use of health apps assist in self-management and potentially clinical decision making. A consumer's sustained engagement with a health app is dependent on the usability and functionality of the app. While numerous studies have attempted to evaluate health apps, there is a paucity of published methods that adequately recognize client experiences in the academic evaluation of apps for chronic conditions. OBJECTIVE: This paper reports (1) a protocol to shortlist health apps for academic evaluation, (2) synthesis of a checklist to screen health apps for quality and reliability, and (3) a proposed method to theoretically evaluate usability of health apps, with a view towards identifying one or more apps suitable for clinical assessment. METHODS: A Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram was developed to guide the selection of the apps to be assessed. The screening checklist was thematically synthesized with reference to recurring constructs in published checklists and related materials for the assessment of health apps. The checklist was evaluated by the authors for face and construct validity. The proposed method for evaluation of health apps required the design of procedures for raters of apps, dummy data entry to test the apps, and analysis of raters' scores. RESULTS: The PRISMA flow diagram comprises 5 steps: filtering of duplicate apps; eliminating non-English apps; removing apps requiring purchase, filtering apps not updated within the past year; and separation of apps into their core functionality. The screening checklist to evaluate the selected apps was named the App Chronic Disease Checklist, and comprises 4 sections with 6 questions in each section. The validity check verified classification of, and ambiguity in, wording of questions within constructs. The proposed method to evaluate shortlisted and downloaded apps comprises instructions to attempt set-up of a dummy user profile, and dummy data entry to represent in-range and out-of-range clinical measures simulating a range of user behaviors. A minimum score of 80% by consensus (using the Intraclass Correlation Coefficient) between raters is proposed to identify apps suitable for clinical trials. CONCLUSIONS: The flow diagram allows researchers to shortlist health apps that are potentially suitable for formal evaluation. The evaluation checklist enables quantitative comparison of shortlisted apps based on constructs reported in the literature. The use of multiple raters, and comparison of their scores, is proposed to manage inherent subjectivity in assessing user experiences. Initial trial of the combined protocol is planned for apps pertaining to the self-monitoring of asthma; these results will be reported elsewhere.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483
Mobile app rating scale: a new tool for assessing the quality of health mobile apps
Stoyanov, S. R., Hides, L., Kavanagh, D. J., Zelenko, O., Tjondronegoro, D., & Mani, M. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.3422 · Cited by 2,233 · cited 19×
“2 3 with 22, 23, and 6 subcriteria, respectively Academic Quality ranking Rating scale: MARS [16];”
InfluentialEnhancing the explanatory power of usability heuristics
Nielsen, J. (1994) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/191666.191729 · Cited by 1,439 · cited 12×
“Nielsen’s Usability Heuristics 1994 Nielsen [29]”
InfluentialInterrater agreement and interrater reliability: Key concepts, approaches, and applications
Gisev, N., Bell, J. S., & Chen, T. F. (2013) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2012.04.004 · Cited by 924 · cited 1×
“As established in the Methods, 2-way mixed ICC is recommended to measure interrater reliability [49].”
Mental Health Smartphone Apps: Review and Evidence-Based Recommendations for Future Developments
Bakker, D., Kazantzis, N., Rickwood, D., & Rickard, N. (2016) · JMIR Mental Health · doi:10.2196/mental.4984 · Cited by 886 · cited 1×
“The need for evidence-based content when deciding which health app to use is also supported by a 2016 Australian review of mental health apps [51].”
Mobile Health Apps to Facilitate Self-Care: A Qualitative Study of User Experiences
Anderson, K., Burford, O., & Emmerton, L. (2016) · PLOS ONE · doi:10.1371/journal.pone.0156164 · Cited by 502 · cited 7×
“…Construct [10,15,42] Gamification Engagement [10,16,33,43] Customization [10,16] Interactivity [10] Positive Behavior Change [16] Effectiveness [10,16,30] Self-Awareness [10] Health Warning Functionality [10,16,27,29,31,34,39,44] Feedback [10,16,33,34] Intuitive Design [10,16,24] Connection to…”
InfluentialSelf-citationJust a Fad? Gamification in Health and Fitness Apps
Lister, C., West, J. H., Cannon, B., Sax, T., & Brodegard, D. (2014) · JMIR Serious Games · doi:10.2196/games.3413 · Cited by 489 · cited 1×
“The app evaluation checklist was synthesized using peer-reviewed checklists and studies on the usability of health apps [5,15,16,25,27,29-35], supplemented with a qualitative study exploring consumer experiences with health apps [10].”
Kim, J., & Park, H.-A. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.2143 · Cited by 475 · cited 3×
“The TAM confirmed alignment of questions relating to Reminders and Automation within the Ease of Use construct.”
Understanding randomised controlled trials
Akobeng, A. K. (2005) · Archives of Disease in Childhood · doi:10.1136/adc.2004.058222 · Cited by 458 · cited 1×
“The purpose of randomizing apps is to eliminate selection bias by balancing unknown factors [50].”
Kirwan, M., Vandelanotte, C., Fenning, A., & Duncan, M. J. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2588 · Cited by 343
Cited by
Criteria for assessing the quality of mHealth apps: a systematic review
Nouri, R., R Niakan Kalhori, S., Ghazisaeedi, M., Marchand, G., & Yasini, M. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy050 · Cited by 231
A review of mobile chatbot apps for anxiety and depression and their self-care features
Ahmed, A., Ali, N., Aziz, S., Abd-alrazaq, A. A., Hassan, A., Khalifa, M., Elhusein, B., Ahmed, M., Ahmed, M. A. S., & Househ, M. (2021) · Computer Methods and Programs in Biomedicine Update · doi:10.1016/j.cmpbup.2021.100012 · Cited by 112
Integration of Mobile Health Technology in the Treatment of Chronic Pain: A Critical Review
Sundararaman, L. V., Edwards, R. R., Ross, E. L., & Jamison, R. N. (2017) · Regional Anesthesia and Pain Medicine · doi:10.1097/aap.0000000000000621 · Cited by 72
Metadata
- Authors
- Anderson, K., Burford, O., & Emmerton, L.
- Venue
- JMIR Research Protocols
- Publisher
- JMIR Publications Inc.
- Published
- 2016-11-04
- Volume
- 5
- Issue
- 4
- Pages
- e204
- ISSN
- 1929-0748 (ISSN-L)
- OpenAlex
- W2547757901
- Cited by
- 64
- References
- 15
- FWCI
- 9.88
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.