Here’s something I prepared earlier: a review of the time to publication of cross-sectional reviews of smartphone health apps
Open accessAbstract
Objectives: Across a range of health conditions, apps are increasingly valued as tools for supporting the delivery and coordination of healthcare. Research-led cross-sectional reviews of apps are a potential resource to inform app selection in face of uncertainties around content quality, safety and privacy. However, these peer-reviewed publications only capture a snapshot of highly dynamic app stores and marketplaces. To determine the extent to which marketplace dynamics might impact the interpretation of app reviews, the current study sought to quantify the lag between the reported time of app assessment and publication of the results of these studies. Design: Searches were conducted on MEDLINE, Embase and PsycINFO to identify published cross-sectional reviews of health, fitness or wellness apps. Publication timeline metadata were extracted, allowing the primary outcome measure, the delay between app store search and manuscript publication, to be calculated. A secondary measure, the time between search and manuscript submission, was also calculated where possible. Results: After screening, 136 relevant cross-sectional app review studies were analysed. The median time to publication was 431 days (approximately 14 months, range: 42–1054 days). The median time to submission was 269 days (approximately 9 months, range: 5–874 days). Studies which downloaded apps typically took longer to publish (p=0.010), however the number of apps reviewed did not impact the time to publication (p=0.964). Studies which recommended specific apps were not published more rapidly (p=0.998). Conclusions: Most health app reviews present data that are at least a year out-of-date at the time of publication. Given the high rate of turnover of health apps in public marketplaces, it may not be appropriate, therefore, for these reviews to be presented as a resource concerning specific products for commissioners, clinicians and the public. Alternative sources of information may be better calibrated to the dynamics of the app marketplace.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A strength of this focused approach means our review was less likely to be impacted by the high turnover of marketplace offerings identified as problematic in previous publications; at the time of submission, all reviewed apps remained available in the Google Play and Apple app marketplaces.
“However, a strength of this focused approach means our review was less likely to be impacted by the high turnover of marketplace offerings identified as problematic in previous publications (Larsen et al. 2020); indeed, we note that at the time of submission, all reviewed apps remained available in the Google Play and Apple app marketplaces.”
From Evaluating the quality, safety, and functionality of commonly used smartphone apps for bipolar disorder mood and sleep self-management (2022) · cites this paper as Larsen et al. 2020
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Mobile app rating scale: a new tool for assessing the quality of health mobile apps
Stoyanov, S. R., Hides, L., Kavanagh, D. J., Zelenko, O., Tjondronegoro, D., & Mani, M. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.3422 · Cited by 2,233 · see citing works
Alderwick, H., & Dixon, J. (2019) · British medical journal · doi:10.1136/bmj.l84 · Cited by 1,985 · see citing works
Goodwin, G., Haddad, P. M., Ferrier, I. N., Aronson, J., Barnes, T., Cipriani, A., Coghill, D., Fazel, S., Geddes, J. R., Grunze, H., et al. (2016) · Journal of Psychopharmacology · doi:10.1177/0269881116636545 · Cited by 1,314 · see citing works
Smartphones for smarter delivery of mental health programs: A systematic review
Donker, T., Petrie, K., Proudfoot, J., Clarke, J., Birch, M.-R., & Christensen, H. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2791 · Cited by 1,136 · see citing works
Whitehead, L., & Seaton, P. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.4883 · Cited by 743 · see citing works
Can Mobile Phone Apps Influence People’s Health Behavior Change? An Evidence Review
Zhao, J., Freeman, B., & Li, M. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5692 · Cited by 618 · see citing works
Mobile Apps for Bipolar Disorder: A Systematic Review of Features and Content Quality
Nicholas, J., Larsen, M. E., Proudfoot, J., & Christensen, H. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4581 · Cited by 438 · see citing works
Self-citationEffectiveness of mHealth interventions for patients with diabetes: An overview of systematic reviews
Kitsiou, S., Paré, G., Jaana, M., & Gerber, B. S. (2017) · PLoS ONE · doi:10.1371/journal.pone.0173160 · Cited by 390 · see citing works
Beyond validation: getting health apps into clinical practice
Gordon, W. J., Landman, A., Zhang, H., & Bates, D. W. (2020) · npj Digital Medicine · doi:10.1038/s41746-019-0212-z · Cited by 322 · see citing works
Huckvale, K., Torous, J., & Larsen, M. E. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.2542 · Cited by 312 · see citing works
Self-citationA systematic assessment of smartphone tools for suicide prevention
Larsen, M. E., Nicholas, J., & Christensen, H. (2016) · PLoS ONE · doi:10.1371/journal.pone.0152285 · Cited by 289 · see citing works
Coorey, G., Neubeck, L., Mulley, J., & Redfern, J. (2018) · European Journal of Preventive Cardiology · doi:10.1177/2047487317750913 · Cited by 283 · see citing works
Apps for asthma self-management: a systematic assessment of content and tools
Huckvale, K., Car, M., Morrison, C. P., & Car, J. (2012) · BMC Medicine · doi:10.1186/1741-7015-10-144 · Cited by 252 · see citing works
Self-citationHan, M., & Lee, E. (2018) · Healthcare Informatics Research · doi:10.4258/hir.2018.24.3.207 · Cited by 247 · see citing works
Smartphone apps for calculating insulin dose: a systematic assessment
Huckvale, K., Adomaviciute, S., Prieto, J. T., Leow, M. K., & Car, J. (2015) · BMC Medicine · doi:10.1186/s12916-015-0314-7 · Cited by 200 · see citing works
Self-citationTorous, J. B., Chan, S. R., Gipson, S. Y.-M. T., Kim, J. W., Nguyen, T.-Q., Luo, J., & Wang, P. (2018) · Psychiatric Services · doi:10.1176/appi.ps.201700423 · Cited by 184 · see citing works
Deriving a practical framework for the evaluation of health apps
Henson, P., David, G., Albright, K., & Torous, J. B. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30013-5 · Cited by 182 · see citing works
Quantifying App Store Dynamics: Longitudinal Tracking of Mental Health Apps
Larsen, M. E., Nicholas, J., & Christensen, H. (2016) · JMIR mHealth and uHealth · doi:10.2196/mhealth.6020 · Cited by 177 · see citing works
Self-citationByambasuren, O., Beller, E., & Glasziou, P. (2019) · JMIR mhealth and uhealth · doi:10.2196/13199 · Cited by 177 · see citing works
A Systematic Review of Quality Assessment Methods for Smartphone Health Apps
BinDhim, N. F., Hawkey, A., & Trevena, L. (2015) · Telemedicine and e-Health · doi:10.1089/tmj.2014.0088 · Cited by 142 · see citing works
Free smoking cessation mobile apps available in Australia: a quality review and content analysis
Thornton, L. K., Quinn, C. A., Birrell, L., Guillaumier, A., Shaw, B., Forbes, E., Deady, M., & KAY‐LAMBKIN, F. J. (2017) · Australian and New Zealand Journal of Public Health · doi:10.1111/1753-6405.12688 · Cited by 93 · see citing works
Boyle, L., Grainger, R., Hall, R. M., & Krebs, J. D. (2017) · JMIR mhealth and uhealth · doi:10.2196/mhealth.7263 · Cited by 86 · see citing works
Tariq, S., Akhtar, N., Afzal, H., Khalid, S., Mufti, M. R., Hussain, S., Habib, A., & Ahmad, G. (2019) · IEEE Access · doi:10.1109/access.2019.2953087 · Cited by 80 · see citing works
Moshi, M. R., Tooher, R. L., & Merlin, T. (2018) · International Journal of Technology Assessment in Health Care · doi:10.1017/s026646231800051x · Cited by 74 · see citing works
Rubanovich, C. K., Mohr, D. C., & Schueller, S. M. (2017) · JMIR Mental Health · doi:10.2196/mental.7603 · Cited by 63 · see citing works
What makes a good clinical app? Introducing the RCP Health Informatics Unit checklist
Wyatt, J. C., Thimbleby, H., Rastall, P., Hoogewerf, J., Wooldridge, D., & Williams, J. (2015) · Clinical Medicine · doi:10.7861/clinmedicine.15-6-519 · Cited by 52 · see citing works
IDF Europe’s position on mobile applications in diabetes
Rose, K. J., Petrut, C., L’Heveder, R., & de Sabata, S. (2017) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2017.08.020 · Cited by 32 · see citing works
Best Practices User Guide: Health Communications in Tobacco Prevention and Control
Control, C. F. D. P. A., & Center for Public Health Systems Science (2020) · Open Scholarship Institutional Repository (Washington University in St. Louis) · doi:10.7936/vyy2-d932 · Cited by 10 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Lagan, S., D'Mello, R., Vaidyam, A., Bilden, R., & Torous, J. (2021) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.13306 · Cited by 65 · see citing works
Morton, E., Nicholas, J., Yang, L., Lapadat, L., Barnes, S. J., Provencher, M. D., Depp, C., Chan, M., Kulur, R., & Michalak, E. E. (2022) · International Journal of Bipolar Disorders · doi:10.1186/s40345-022-00256-6 · Cited by 20 · see citing works
Self-citation
- Authors
- Larsen, M., Nicholas, J., Han, J., Lemon, C., Okun, K., Torok, M., Wong, D., Wong, I., Wong, Q., & Huckvale, K.
- Affiliations
- Black Dog Institute (AU)
- The University of Melbourne (AU)
- UNSW Sydney (AU)
- Stanford University (US)
- University of Manchester (GB)
- Western Sydney University (AU)
- Corresponding author
- Larsen, Mark
- ORCID
- Venue
- BMJ Open
- Publisher
- BMJ
- Published
- 2020-12
- Volume
- 10
- Issue
- 12
- Pages
- e039817
- ISSN
- 2044-6055 (ISSN-L)
- PMC
- PMC7713229
- OpenAlex
- W3108660110
- Cited by
- 5
- References
- 28
- Citations by year
- 2025: 1 · 2022: 1 · 2021: 3
- FWCI
- 0.19
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMJ Open (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- HighWire Press Open Archive (acceptedVersion)
- Research Explorer (The University of Manchester) (acceptedVersion)
- UNSWorks (University of New South Wales, Sydney, Australia) (submittedVersion, cc-by-nc-nd)
- Minerva Access (University of Melbourne) (submittedVersion, cc-by-nc)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Keywords
- mobile health apps, fitness apps, wellness apps, time to publication, app selection
- MeSH terms
- Smartphone, Cross-Sectional Studies, Humans, Exercise, Mobile Applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
