Why Reviewing Apps Is Not Enough: Transparency for Trust (T4T) Principles of Responsible Health App Marketplaces
Open accessThe DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
The overselling of health apps that may provide little benefit and even harm needs the health community's immediate attention. With little formal regulation, a light-touch approach to consumer protection is now warranted to give customers a modicum of information to help them choose from the vast array of so-called health apps. We suggest 4 guiding principles that should be adopted to provide the consumer with information that can guide their choice at the point of download. We call these the Transparency for Trust (T4T) principles, which are derived from experimental studies, systematic reviews, and reports of patient concerns. The T4T principles are (1) privacy and data security, (2) development characteristics, (3) feasibility data, and (4) benefits. All our questions are in a simple form so that all consumers can understand them. We suggest that app stores should take responsibility for providing this information and store it with any app marketed as a health app. Even the absence of information would provide consumers with some understanding and fuel their choice. This would also provide some commercial impetus for app developers to consider this requested information from the outset.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Fitzpatrick, K. K., Darcy, A., & Vierhile, M. (2017) · JMIR Mental Health · doi:10.2196/mental.7785 · Cited by 2,730 · cited 1×
“Many studies rely on small numbers of participants or convenience samples, for example, soliciting feedback from stressed college students rather than individuals with depression [51].”
Mobile app rating scale: a new tool for assessing the quality of health mobile apps
Stoyanov, S. R., Hides, L., Kavanagh, D. J., Zelenko, O., Tjondronegoro, D., & Mani, M. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.3422 · Cited by 2,233 · cited 2×
“Most proposed evaluations (eg, Mobile App Rating Scale, MARS [30], Enlight assessment tools [31]) assess usability, aesthetics, content, user engagement, and available research evidence, and others have been adding to this list [32].”
Health App Use Among US Mobile Phone Owners: A National Survey
Krebs, P., & Duncan, D. T. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.4924 · Cited by 1,274 · cited 2×
“reviews, and experimental studies (eg [4,7-14]).”
Do “brain-training” programs work?
Simons, D. J., Boot, W. R., Charness, N., Gathercole, S. E., Chabris, C. F., Hambrick, D. Z., & Stine-Morrow, E. A. L. (2016) · Psychological Science in the Public Interest · doi:10.1177/1529100616661983 · Cited by 1,183
Mobile Health Technology Evaluation
Kumar, S., Nilsen, W. J., Abernethy, A., Atienza, A., Patrick, K., Pavel, M., Riley, W. T., Shar, A., Spring, B., Spruijt-Metz, D., et al. (2013) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2013.03.017 · Cited by 900
The Lancet Psychiatry Commission on psychological treatments research in tomorrow's science
Holmes, E. A., Ghaderi, A., Harmer, C. J., Ramchandani, P. G., Cuijpers, P., Morrison, A. P., Roiser, J. P., Bockting, C. L. H., O'Connor, R. C., Shafran, R., et al. (2018) · The Lancet Psychiatry · doi:10.1016/s2215-0366(17)30513-8 · Cited by 624
Interaction in 4-second bursts: the fragmented nature of attentional resources in mobile HCI
Oulasvirta, A., Tamminen, S., Roto, V., & Kuorelahti, J. (2005) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1054972.1055101 · Cited by 576 · cited 1×
“People use apps differently, including more frequently and in shorter bursts [60], and these differences could affect their impact.”
Mobile Apps for Bipolar Disorder: A Systematic Review of Features and Content Quality
Nicholas, J., Larsen, M. E., Proudfoot, J., & Christensen, H. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4581 · Cited by 438 · cited 1×
“However, evidence suggests that some apps are not only ineffective, unsafe, and hard to use, but do not meet users’ privacy and security expectations [9,20,21].”
The WPA-Lancet psychiatry commission on the future of psychiatry
Bhugra, D., Tasman, A., Pathare, S., Priebe, S., Smith, S., Torous, J., Arbuckle, M. R., Langford, A., Alarcón, R. D., Chiu, H. F. K., et al. (2017) · The Lancet Psychiatry · doi:10.1016/s2215-0366(17)30333-4 · Cited by 392 · cited 2×
“The use of digital technologies to alleviate, prevent, or maintain health has been recognized for many years, but although it offers enormous potential to rethink how services are provided, there are large roadblocks in its way (eg [19]).”
Self-citationHuckvale, K., Prieto, J. T., Tilney, M., Benghozi, P.-J., & Car, J. (2015) · BMC Medicine · doi:10.1186/s12916-015-0444-y · Cited by 376
Technological innovations in mental healthcare: harnessing the digital revolution
Hollis, C., Morriss, R., Martin, J., Amani, S., Cotton, R., Denis, M., & Lewis, S. (2015) · British Journal of Psychiatry · doi:10.1192/bjp.bp.113.142612 · Cited by 375
Torous, J., & Roberts, L. W. (2017) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2017.0262 · Cited by 362 · cited 1×
“The goal of these principles is to operationalize efforts that can be made by app marketplaces to answer calls for better oversight and backing up products with data and research [3,5,6].”
A systematic assessment of smartphone tools for suicide prevention
Larsen, M. E., Nicholas, J., & Christensen, H. (2016) · PLoS ONE · doi:10.1371/journal.pone.0152285 · Cited by 289 · cited 1×
“However, evidence suggests that some apps are not only ineffective, unsafe, and hard to use, but do not meet users’ privacy and security expectations [9,20,21].”
Simblett, S., Greer, B., Matcham, F., Curtis, H., Polhemus, A., Ferrão, J., Gamble, P., & Wykes, T. (2018) · Journal of Medical Internet Research · doi:10.2196/10480 · Cited by 270
Self-citationInternet-delivered treatment for substance abuse: a multisite randomized controlled trial
Campbell, A. N. C., Nunes, E. V., Matthews, A. G., Stitzer, M., Miele, G. M., Polsky, D., Turrigiano, E., Walters, S., McClure, E. A., Kyle, T. L., et al. (2014) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2014.13081055 · Cited by 263
Usability of Commercially Available Mobile Applications for Diverse Patients
Sarkar, U., Gourley, G. I., Lyles, C. R., Tieu, L., Clarity, C., Newmark, L., Singh, K., & Bates, D. W. (2016) · Journal of General Internal Medicine · doi:10.1007/s11606-016-3771-6 · Cited by 257 · cited 1×
“Again, this may seem obvious, but independent usability evaluations have demonstrated that many popular commercial apps are frustrating and challenging for members of the intended audience, raising questions about their prior involvement and the potential for the app to benefit this community [52].”
Security and Privacy Analysis of Mobile Health Applications: The Alarming State of Practice
Papageorgiou, A., Strigkos, M., Politou, E., Alepis, E., Solanas, A., & Patsakis, C. (2018) · IEEE Access · doi:10.1109/access.2018.2799522 · Cited by 236 · cited 1×
“Although full formal audits are needed to ensure apps follow their stated procedures [48], even requiring developers to list their privacy and data security procedures in simple terms would be a significant step forward on raising standards [49].”
mHealth in the Wild: Using Novel Data to Examine the Reach, Use, and Impact of PTSD Coach
Owen, J. E., Jaworski, B. K., Kuhn, E., Makin-Byrd, K. N., Ramsey, K. M., & Hoffman, J. E. (2015) · JMIR Mental Health · doi:10.2196/mental.3935 · Cited by 216 · cited 1×
“because very few users persist in using a health app after the first week [55].”
Baumel, A., Faber, K., Mathur, N., Kane, J. M., & Muench, F. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7270 · Cited by 206 · cited 1×
“Most proposed evaluations (eg, Mobile App Rating Scale, MARS [30], Enlight assessment tools [31]) assess usability, aesthetics, content, user engagement, and available research evidence, and others have been adding to this list [32].”
Hollis, C., Sampson, S., Simons, L., Davies, E. B., Churchill, R., Betton, V., Butler, D., Chapman, K., Easton, K., Gronlund, T. A., et al. (2018) · The Lancet Psychiatry · doi:10.1016/s2215-0366(18)30296-7 · Cited by 186 · cited 1×
“reviews, and experimental studies (eg [4,7-14]).”
Schueller, S. M., Neary, M., O'Loughlin, K., & Adkins, E. C. (2018) · Journal of Medical Internet Research · doi:10.2196/10141 · Cited by 167
Self-citationPham, Q., Graham, G., Carrion, C., Morita, P. P., Seto, E., Stinson, J. N., & Cafazzo, J. A. (2019) · JMIR mHealth and uHealth · doi:10.2196/11941 · Cited by 131 · cited 1×
“Despite the availability of engagement analytics, few are reported even in the clinical assessments of feasibility [56].”
Data Security and Privacy in Apps for Dementia: An Analysis of Existing Privacy Policies
Rosenfeld, L., Torous, J., & Vahia, I. V. (2017) · The American Journal of Geriatric Psychiatry · doi:10.1016/j.jagp.2017.04.009 · Cited by 100 · cited 1×
“Although full formal audits are needed to ensure apps follow their stated procedures [48], even requiring developers to list their privacy and data security procedures in simple terms would be a significant step forward on raising standards [49].”
Is digital medicine different?
Lancet, T. (2018) · The Lancet · doi:10.1016/s0140-6736(18)31562-9 · Cited by 73 · cited 1×
“The goal of these principles is to operationalize efforts that can be made by app marketplaces to answer calls for better oversight and backing up products with data and research [3,5,6].”
An Assessment Framework for e-Mental Health Apps in Canada: Results of a Modified Delphi Process
Zelmer, J., van Hoof, K., Notarianni, M., van Mierlo, T., Schellenberg, M., & Tannenbaum, C. (2018) · JMIR mHealth and uHealth · doi:10.2196/10016 · Cited by 62 · cited 2×
“Their views do not necessarily coincide with the expert groups’ views, as shown in the Delphi exercise by Zelmer et al [33].”
Proportionate methods for evaluating a simple digital mental health tool
Davies, E. B., Craven, M. P., Martin, J. L., & Simons, L. (2017) · Evidence Based Mental Health · doi:10.1136/eb-2017-102755 · Cited by 15 · cited 1×
“Although triangulation of different sorts of data has been suggested (eg, MindTech [61]), we believe that health apps should undergo a trial to determine their superiority to other treatment options, especially as many unsubstantiated claims have been made [36].”
Cited by
Responsible AI—two frameworks for ethical design practice
Peters, D., Vold, K., Robinson, D., & Calvo, R. A. (2020) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2020.2974991 · Cited by 293
“For example, the transparency for trust principles [68] includes questions around privacy and data security, development characteristics, feasibility, and health benefits, and their creators advocate that all apps should be required to provide information relating to these four principles at minimum.”
BackgroundSecurity and Privacy of mHealth Applications: A Scoping Review
Nurgalieva, L., O'Callaghan, D., & Doherty, G. (2020) · IEEE Access · doi:10.1109/access.2020.2999934 · Cited by 132
“which proposes design principles that responsible Health AppMarketplaces should follow [25].”
BackgroundBetter, Virtually: the Past, Present, and Future of Virtual Reality Cognitive Behavior Therapy
Lindner, P. (2021) · International Journal of Cognitive Therapy · doi:10.1007/s41811-020-00090-7 · Cited by 120
Sorkin, D. H., Janio, E. A., Eikey, E. V., Schneider, M., Davis, K., Schueller, S. M., Stadnick, N. A., Zheng, K., Neary, M., Safani, D., et al. (2021) · Journal of Medical Internet Research · doi:10.2196/26994 · Cited by 108
“Indeed, better regulation and better access to information at point-of-access could be a significant improvement in helping guide consumers [44], who are demonstrating a clear interest in such resources, and might serve an important need in light of the COVID-19 pandemic.”
Self-citationEvaluating evaluation frameworks: a scoping review of frameworks for assessing health apps
Lagan, S., Sandler, L., & Torous, J. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-047001 · Cited by 108
Metadata
- Authors
- Wykes, T., & Schueller, S.
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2019-05-02
- Volume
- 21
- Issue
- 5
- Pages
- e12390
- ISSN
- 1438-8871 (ISSN-L)
- OpenAlex
- W2902996901
- Cited by
- 95
- References
- 26
- FWCI
- 19.86
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.