The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Digital health: a path to validation
Open accessAbstract
Digital health solutions continue to grow in both number and capabilities. Despite these advances, the confidence of the various stakeholders — from patients and clinicians to payers, industry and regulators — in medicine remains quite low. As a result, there is a need for objective, transparent, and standards-based evaluation of digital health products that can bring greater clarity to the digital health marketplace. We believe an approach that is guided by end-user requirements and formal assessment across technical, clinical, usability, and cost domains is one possible solution. For digital health solutions to have greater impact, quality and value must be easier to distinguish. To that end, we review the existing landscape and gaps, highlight the evolving responses and approaches, and detail one pragmatic framework that addresses the current limitations in the marketplace with a path toward implementation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“evidence-supported apps are slow to be made publicly accessible due to research-to-practice translational lags and complex regulatory systems (Mathews et al. 2019).”
From Evaluating the quality, safety, and functionality of commonly used smartphone apps for bipolar disorder mood and sleep self-management (2022) · cites this paper as Mathews et al. 2019
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
International Organization for Standardization
(2011) · Springer eBooks · doi:10.1007/springerreference_16847 · Cited by 12,504
2. Classification and Diagnosis of Diabetes: Standards of Medical Care in Diabetes—2019
(2018) · Diabetes Care · doi:10.2337/dc19-s002 · Cited by 2,855
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
Kamel Boulos, M. N., Brewer, A. C., Karimkhani, C., Buller, D. B., & Dellavalle, R. P. (2014) · Online Journal of Public Health Informatics · doi:10.5210/ojphi.v5i3.4814 · Cited by 659
Iribarren, S. J., Cato, K., Falzon, L., & Stone, P. W. (2017) · PLOS ONE · doi:10.1371/journal.pone.0170581 · Cited by 519
What It Will Take To Achieve The As-Yet-Unfulfilled Promises Of Health Information Technology
Kellermann, A. L., & Jones, S. S. (2013) · Health Affairs · doi:10.1377/hlthaff.2012.0693 · Cited by 482
Origin and Uses of Primum Non Nocere—Above All, Do No Harm!
Smith, C. M. (2005) · The Journal of Clinical Pharmacology · doi:10.1177/0091270004273680 · Cited by 411
Quinn, C. C., Clough, S. S., Minor, J. M., Lender, D., Okafor, M. C., & Gruber-Baldini, A. (2008) · Diabetes Technology & Therapeutics · doi:10.1089/dia.2008.0283 · Cited by 406
mHealth and Mobile Medical Apps: A Framework to Assess Risk and Promote Safer Use
Lewis, T., & Wyatt, J. C. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3133 · Cited by 281
Boudreaux, E. D., Waring, M. E., Hayes, R. B., Sadasivam, R. S., Mullen, S. P., & Pagoto, S. (2014) · Translational Behavioral Medicine · doi:10.1007/s13142-014-0293-9 · Cited by 281
The GRADE system for rating clinical guidelines
Kavanagh, B. P. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000094 · Cited by 246
Sharman, J. E., Avolio, A. P., Baulmann, J., Benetos, A., Blacher, J., Blizzard, C. L., Boutouyrie, P., Chen, C.-H., Chowienczyk, P., Cockcroft, J. R., et al. (2017) · European Heart Journal · doi:10.1093/eurheartj/ehw632 · Cited by 231
International Consortium for Health Outcomes Measurement (ICHOM)
Kelley, T. A. (2015) · Trials · doi:10.1186/1745-6215-16-s3-o4 · Cited by 200
Validation of the Instant Blood Pressure Smartphone App
Plante, T. B., Urrea, B., MacFarlane, Z. T., Blumenthal, R. S., Miller, E. R., Appel, L. J., & Martin, S. S. (2016) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2016.0157 · Cited by 159
FDA Regulation of Mobile Medical Apps
Shuren, J. E., Patel, B., & Gottlieb, S. L. (2018) · JAMA · doi:10.1001/jama.2018.8832 · Cited by 155
Mitchell, S., Godoy, L., Shabazz, K., & Horn, I. B. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.2673 · Cited by 110
Svendsen, M. T., Andersen, F., Andersen, K. H., Pottegård, A., Johannessen, H., Möller, S., August, B., Feldman, S. R., & Andersen, K. E. (2018) · British Journal of Dermatology · doi:10.1111/bjd.16667 · Cited by 99
Digital Health Care—The Convergence of Health Care and the Internet
Frank, S. R. (2000) · Journal of Ambulatory Care Management · doi:10.1097/00004479-200004000-00003 · Cited by 92
Top-Funded Digital Health Companies And Their Impact On High-Burden, High-Cost Conditions
Safavi, K. C., Mathews, S. C., Bates, D. W., Dorsey, E. R., & Cohen, A. B. (2019) · Health Affairs · doi:10.1377/hlthaff.2018.05081 · Cited by 74
Self-citationUsability of electronic medical records
Smelcer, J. B., Miller-Jacobs, H., & Kantrovich, L. (2009) · doi:10.5555/2835582.2835584 · Cited by 68
Kok, R. N., van Straten, A., Beekman, A. T. F., & Cuijpers, P. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3429 · Cited by 49
The Challenges of Mobile Health Regulation
Schoenfeld, A. J., Sehgal, N. J., & Auerbach, A. D. (2016) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2016.0326 · Cited by 23
Plante, T. B., O’Kelly, A. C., Urrea, B., MacFarlane, Z. T., Blumenthal, R. S., Charleston, J. B., Miller, E. R., Appel, L. J., & Martin, S. S. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0039-z · Cited by 18
Mobile Applications for Self-Management of Diabetes
Veazie, S., Winchell, K. A., Gilbert, J., Paynter, R. A., Ivlev, I., Eden, K., Nussbaum, K., Weiskopf, N. G., Guise, J., & Helfand, M. (2018) · Europe PMC (PubMed Central) · Cited by 17
Hatef, E., Sharfstein, J. M., & Labrique, A. B. (2017) · Journal of Public Health Management and Practice · doi:10.1097/phh.0000000000000665 · Cited by 16
Self-citation
Cited by
Sorted by citations
Digital technology, tele-medicine and artificial intelligence in ophthalmology: A global perspective
Li, J.-P. O., Liu, H., Ting, D. S. J., Jeon, S., Chan, R. V. P., Kim, J. E., Sim, D. A., Thomas, P. B. M., Lin, H., Chen, Y., et al. (2021) · Progress in Retinal and Eye Research · doi:10.1016/j.preteyeres.2020.100900 · Cited by 626
The Recent Progress and Applications of Digital Technologies in Healthcare: A Review
Senbekov, M., Saliev, T. M., Bukeyeva, Z., Almabayeva, A., Zhanaliyeva, M., Джамбуловна, А. Н., Toishibekov, Y., & Fakhradiyev, I. R. (2020) · International Journal of Telemedicine and Applications · doi:10.1155/2020/8830200 · Cited by 513
Beyond validation: getting health apps into clinical practice
Gordon, W. J., Landman, A., Zhang, H., & Bates, D. W. (2020) · npj Digital Medicine · doi:10.1038/s41746-019-0212-z · Cited by 322
The future of sleep health: a data-driven revolution in sleep science and medicine
Perez-Pozuelo, I., Zhai, B., Palotti, J., Mall, R., Aupetit, M., Garcia-Gomez, J. M., Taheri, S., Guan, Y., & Fernandez-Luque, L. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0244-4 · Cited by 311
Guo, C., Ashrafian, H., Ghafur, S., Fontana, G., Gardner, C., & Prime, M. S. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00314-2 · Cited by 345
Metadata
- Authors
- Mathews, S. C., McShea, M. J., Hanley, C. L., Ravitz, A., Labrique, A. B., & Cohen, A. B.
- Affiliations
- Johns Hopkins University (US)
- Corresponding author
- Mathews, Simon C.
- Venue
- npj Digital Medicine
- Published
- 2019-05-13
- Volume
- 2
- Issue
- 1
- Pages
- 38-38
- ISSN
- 2398-6352 (ISSN-L)
- PMC
- PMC6550273
- OpenAlex
- W2946661638
- Cited by
- 508
- References
- 24
- Citations by year
- 2026: 52 · 2025: 81 · 2024: 107 · 2023: 85 · 2022: 73 · 2021: 60 · 2020: 44 · 2019: 6
- FWCI
- 8.41
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- npj Digital Medicine (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Health Sciences › Health Professions › Health Information Management › Health, Technology, Consumer Behavior
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- digital health, validation framework, end-user needs, clinical evaluation, usability, cost evaluation, standards-based evaluation, stakeholder confidence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review