Verification, analytical validation, and clinical validation (V3): The foundation of determining fit-for-purpose for Biometric Monitoring Technologies (BioMeTs)
Open accessPublished version of a preprint
Abstract
Digital medicine is an interdisciplinary field, drawing together stakeholders with expertize in engineering, manufacturing, clinical science, data science, biostatistics, regulatory science, ethics, patient advocacy, and healthcare policy, to name a few. Although this diversity is undoubtedly valuable, it can lead to confusion regarding terminology and best practices. There are many instances, as we detail in this paper, where a single term is used by different groups to mean different things, as well as cases where multiple terms are used to describe essentially the same concept. Our intent is to clarify core terminology and best practices for the evaluation of Biometric Monitoring Technologies (BioMeTs), without unnecessarily introducing new terms. We focus on the evaluation of BioMeTs as fit-for-purpose for use in clinical trials. However, our intent is for this framework to be instructional to all users of digital measurement tools, regardless of setting or intended use. We propose and describe a three-component framework intended to provide a foundational evaluation framework for BioMeTs. This framework includes (1) verification, (2) analytical validation, and (3) clinical validation. We aim for this common vocabulary to enable more effective communication and collaboration, generate a common and meaningful evidence base for BioMeTs, and improve the accessibility of the digital medicine field.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Strong development or vendor-reported performance does not necessarily translate into reliable clinical utility in practice.
“These concerns are reinforced by empirical evidence showing that strong development or vendor-reported performance does not necessarily translate into reliable clinical utility in practice [6,7].”
From Beyond Model Development in Healthcare AI: Post-Development Robustness, Post-Deployment Monitoring, and Lifecycle Governance—A Scoping Review of Reviews (2026) · cites this paper as [7]
In “Moving from current siloed practices to one universal best practice” (page unavailable), the paper says a BioMeT may perform well under V3 yet be useless if the target population cannot use it appropriately in the intended setting. This supports the distinction between performance and clinical utility, but the cited passage does not provide empirical evidence about strong development or vendor-reported performance failing in practice. (LLM, conf=0.84)
Experimental — accuracy not yet measuredThe development and validation of robust and reliable DHTs require the collaboration of diverse experts and a stepwise process analogous to the development of fluid, imaging, and other conventional laboratory-derived biomarkers.
“The development and validation of robust and reliable DHTs require the collaboration of diverse experts and a stepwise process analogous to the development of fluid, imaging, and other conventional laboratory‐derived biomarkers. 23 , 24 , 25”
From Developing digital health technologies for frontotemporal degeneration (2025) · cites this paper as [24]
The Digital Medicine Society V3 framework outlines and explains the typical verification, analytical validation, and clinical validation processes of biometric monitoring technologies.
“Another resource is the Digital Medicine Society V3 framework, 24 which outlines and explains the typical verification, analytical validation, and clinical validation processes of biometric monitoring technologies.”
From Developing digital health technologies for frontotemporal degeneration (2025) · cites this paper as [24]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
1 retracted
RETRACTED: Relationship between Clinic and Ambulatory Blood-Pressure Measurements and Mortality
Banegas, J. R., Ruilope, L. M., de la Sierra, A., Vinyoles, E., Gorostidi, M., de la Cruz, J. J., Ruiz-Hurtado, G., Segura, J., Rodríguez-Artalejo, F., & Williams, B. (2018) · New England Journal of Medicine · doi:10.1056/nejmoa1712231 · Cited by 490 · see citing works
RetractedHeart rate variability: Standards of measurement, physiological interpretation, and clinical use
Malik, M., Bigger, J. T., Camm, A. J., Kleiger, R. E., Malliani, A., Moss, A. J., & Schwartz, P. J. (1996) · European Heart Journal · doi:10.1093/oxfordjournals.eurheartj.a014868 · Cited by 15,289 · see citing works
Digital Phenotyping: Technology for a New Science of Behavior
Insel, T. R. (2017) · JAMA · doi:10.1001/jama.2017.11295 · Cited by 833 · see citing works
Wearables and the Medical Revolution
Dunn, J., Runge, R., & Snyder, M. (2018) · Personalized Medicine · doi:10.2217/pme-2018-0044 · Cited by 654 · see citing works
Self-citationDigital biomarkers for Alzheimer’s disease: the mobile/wearable devices opportunity
Kourtis, L. C., Regele, O. B., Wright, J. M., & Jones, G. B. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0084-2 · Cited by 476 · see citing works
Flagged first, then by citations
Research works printed but not linked: 30
- [2]Perakslis E Coravos A Is health-care data the new blood? Lancet Digital Health 2019 1 e8 e9 10.1016/S2589-7500(19)30001-9 33323242 · DOI
- [3]Coravos A Digital medicine: a primer on measurement Digit Biomark. 2019 3 31 71 10.1159/000500413 32095767 PMC7015383 · DOI
- [5]FDA-NIH Biomarker Working Group. BEST (Biomarkers, EndpointS, and other Tools) Resource. (Food and Drug Administration (US), 2016). 27010052
- [6]Versi E ‘Gold standard’ is an appropriate term BMJ 1992 305 187 10.1136/bmj.305.6846.187-b PMC1883235 1515860 · DOI
- [9]IEEE Computer Society. IEEE Standard for System, Software, and Hardware Verification and Validation. IEEE Std 1012-2016 (Revision of IEEE Std 1012-2012/ Incorporates IEEE Std 1012-2016/Cor1-2017) 1–260 (2017). 10.1109/IEEESTD.2017.8055462. · DOI
- [20]Bignardi GE Validation and verification of automated urine particle analysers J. Clin. Pathol. 2017 70 94 101 10.1136/jclinpath-2016-203958 27802413 · DOI
- [24]Witt DR Kellogg RA Snyder MP Dunn J Windows into human health through wearables data analytics Curr. Opin. Biomed. Eng. 2019 9 28 46 10.1016/j.cobme.2019.01.001 31832566 PMC6907085 · DOI
- [25]McCamley J Donati M Grimpampi E Mazzà C An enhanced estimate of initial contact and final contact instants of time using lower trunk inertial sensor data Gait Posture 2012 36 316 318 10.1016/j.gaitpost.2012.02.019 22465705 · DOI
- [26]Trojaniello D Cereatti A Della Croce U Accuracy, sensitivity and robustness of five different methods for the estimation of gait temporal parameters using a single inertial sensor mounted on the lower trunk Gait Posture 2014 40 487 492 10.1016/j.gaitpost.2014.07.007 25085660 · DOI
- [27]Hutchinson, L. et al. Operator bias errors are reduced using standing marker alignment device for repeated visit studies. J. Biomech. Eng.140, 041001 (2018). 29114765
- [28]Godfrey A Del Din S Barry G Mathers JC Rochester L Instrumenting gait with an accelerometer: a system and algorithm examination Med. Eng. Phys. 2015 37 400 407 10.1016/j.medengphy.2015.02.003 25749552 PMC4381862 · DOI
- [35]Huynh, Q. T., Nguyen, U. D., Irazabal, L. B., Ghassemian, N. & Tran, B. Q. Optimization of an accelerometer and gyroscope-based fall detection algorithm. J. Sens. (2015). 10.1155/2015/452078. · DOI
- [36]Pham MH Validation of a lower back “wearable”-based sit-to-stand and stand-to-sit algorithm for patients with parkinson’s disease and older adults in a home-like environment Front. Neurol. 2018 9 652 10.3389/fneur.2018.00652 30158894 PMC6104484 · DOI
- [37]Kovalchick C Can composite digital monitoring biomarkers come of age? A framework for utilization J. Clin. Transl. Sci. 2017 1 373 380 10.1017/cts.2018.4 29707260 PMC5916505 · DOI
- [39]Wang R Accuracy of wrist-worn heart rate monitors JAMA Cardiol. 2017 2 104 10.1001/jamacardio.2016.3340 27732703 · DOI · Paper page
- [40]Braybrook M An ambulatory tremor Score for parkinson’s disease J. Parkinsons Dis. 2016 6 723 731 10.3233/JPD-160898 27589540 · DOI
- [41]Panescu, D. Medical device development. in 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society 5591–5594 (2009). 10.1109/IEMBS.2009.5333490. 19964134 · DOI
- [43]IEEE Standard for System, Software, and Hardware Verification and Validation. IEEE Std 1012-2016 (Revision of IEEE Std 1012-2012/ Incorporates IEEE Std 1012-2016/Cor1-2017) 1–260 (2017). 10.1109/IEEESTD.2017.8055462. · DOI
- [44]National Academies of Sciences, Engineering, and Medicine. An Evidence Framework for Genetic Testing. (The National Academies Press, 2017). 10.17226/24632. 28418631 · DOI
- [45]Giles D Draper N Neil W Validity of the Polar V800 heart rate monitor to measure RR intervals at rest Eur. J. Appl. Physiol. 2016 116 563 571 10.1007/s00421-015-3303-9 26708360 PMC4751190 · DOI
- [47]Hernando D Garatachea N Almeida R Casajús JA Bailón R Validation of heart rate monitor polar rs800 for heart rate variability analysis during exercise J. Strength Cond. Res. 2018 32 716 10.1519/JSC.0000000000001662 27749728 · DOI
- [48]Frasch, M. G. et al. Can a heart rate variability biomarker identify the presence of autism spectrum disorder in eight year old children? arXiv:1808.08306 [q-bio] (2018). · arXiv
- [49]Karpman C LeBrasseur NK DePew ZS Novotny PJ Benzo RP Measuring gait speed in the out-patient clinic: methodology and feasibility Respir. Care 2014 59 531 537 10.4187/respcare.02688 23983271 · DOI
- [50]Fortune E Lugade V Morrow M Kaufman K Validity of using tri-axial accelerometers to measure human movement – Part II: step counts at a wide range of gait velocities Med. Eng. Phys. 2014 36 659 669 10.1016/j.medengphy.2014.02.006 24656871 PMC4030415 · DOI
- [51]König A Objective measurement of gait parameters in healthy and cognitively impaired elderly using the dual-task paradigm Aging Clin. Exp. Res 2017 29 1181 1189 10.1007/s40520-016-0703-6 28130713 PMC5674109 · DOI
- [54]Parvinian B Scully C Wiyor H Kumar A Weininger S Regulatory considerations for physiological closed-loop controlled medical devices used for automated critical care: food and drug administration workshop discussion topics Anesth. Analg. 2018 126 1916 1925 10.1213/ANE.0000000000002329 28763355 PMC6233305 · DOI
- [55]Allen, N. & Gupta, A. Current Diabetes Technology: Striving for the Artificial Pancreas. Diagnostics (Basel)9, 31 (2019). PMC6468523 30875898
- [57]Watanabe N Development and Validation of a Novel Cuff-Less Blood Pressure Monitoring Device. J Am Coll Cardiol Basic Trans JACC: Basic to Translational Science 2017 2 631 642 30062178 PMC6058997
- [58]IEEE Standard for Wearable Cuffless Blood Pressure Measuring Devices. IEEE Std 1708-2014 1–38 (2014). 10.1109/IEEESTD.2014.6882122. · DOI
- [60]IEEE standard for software verification and validation. (Institute of Electrical and Electronics Engineers, 1998).
Cited by
Sorted by citations
Investigating sources of inaccuracy in wearable optical heart rate sensors
Bent, B., Goldstein, B. A., Kibbe, W. A., & Dunn, J. P. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0226-6 · Cited by 673 · see citing works
Self-citationEthical Considerations of Using ChatGPT in Health Care
Wang, C., Liu, S., Yang, H., Guo, J., Wu, Y., & Liu, J. (2023) · Journal of Medical Internet Research · doi:10.2196/48009 · Cited by 541 · see citing works
The Use of Wearables in Clinical Trials During Cancer Treatment: Systematic Review
Beauchamp, U. L., Pappot, H., & Holländer-Mieritz, C. (2020) · JMIR mHealth and uHealth · doi:10.2196/22006 · Cited by 110 · see citing works
Data quality evaluation in wearable monitoring
Böttcher, S., Vieluf, S., Bruno, E., Joseph, B., Epitashvili, N., Biondi, A., Zabler, N., Glasstetter, M., Dümpelmann, M., Van Laerhoven, K., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-25949-x · Cited by 91 · see citing works
Modernizing and designing evaluation frameworks for connected sensor technologies in medicine
Coravos, A., Doerr, M., Goldsack, J., Manta, C., Shervey, M., Woods, B., & Wood, W. A. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0237-3 · Cited by 82 · see citing works
Self-citationCereatti, A., Gurchiek, R., Mündermann, A., Fantozzi, S., Horak, F., Delp, S., & Aminian, K. (2024) · Journal of Biomechanics · doi:10.1016/j.jbiomech.2024.112225 · Cited by 79 · see citing works
Polk, S. E., Öhman, F., Hassenstab, J., König, A., Papp, K. V., Schöll, M., & Berron, D. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01583-5 · Cited by 73 · see citing works
Berisha, V., & Liss, J. M. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01199-1 · Cited by 41 · see citing works
Wienert, J., & Zeeb, H. (2021) · Frontiers in Public Health · doi:10.3389/fpubh.2021.610237 · Cited by 38 · see citing works
Singh, S., Bennett, M. R., Chen, C., Shin, S., Ghanbari, H., & Nelson, B. W. (2024) · Journal of Medical Internet Research · doi:10.2196/62769 · Cited by 36 · see citing works
“It is vitally important for medical pulse oximetry devices as well as nonregulated research and consumer devices to incorporate the V3 framework [74] for sensor verification and analytical validation of derived SpO 2 and PR values for regulatory submission before these metrics can be responsibly…”
Investigating the accuracy of blood oxygen saturation measurements in common consumer smartwatches
Jiang, Y., Spies, C., Magin, J., Bhosai, S. J., Snyder, L., & Dunn, J. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000296 · Cited by 36 · see citing works
Self-citationMarvasti, T. B., Gao, Y., Murray, K. R., Hershman, S., McIntosh, C., & Moayedi, Y. (2024) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2024.07.009 · Cited by 35 · see citing works
Hamideh, D., & Nebeker, C. (2020) · Current Addiction Reports · doi:10.1007/s40429-020-00325-9 · Cited by 24 · see citing works
Bakker, J. P., Izmailova, E. S., Clement, A., Hoffmann, S., Leptak, C., Menetski, J. P., & Wagner, J. A. (2025) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.3398 · Cited by 26 · see citing works
Self-citationAutomated detection of progressive speech changes in early Alzheimer's disease
Robin, J., Xu, M., Balagopalan, A., Novikova, J., Kahn, L., Oday, A., Hejrati, M., Hashemifar, S., Negahdar, M., Simpson, W., et al. (2023) · Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring · doi:10.1002/dad2.12445 · Cited by 21 · see citing works
Accelerating Parkinson’s Disease drug development with federated learning approaches
Khanna, A., Adams, J., Antoniades, C., Bloem, B. R., Carroll, C., Cedarbaum, J., Cosman, J., Dexter, D. T., Dockendorf, M. F., Edgerton, J., et al. (2024) · npj Parkinson's Disease · doi:10.1038/s41531-024-00837-5 · Cited by 21 · see citing works
Self-citationReply: Matters Arising ‘Investigating sources of inaccuracy in wearable optical heart rate sensors’
Bent, B., Enache, O. M., Goldstein, B., Kibbe, W., & Dunn, J. P. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00409-4 · Cited by 17 · see citing works
Self-citationChallenges and Potential of Using Digital Biomarkers in Healthcare and Clinical Trials
Lieberwirth, J. K., Mittermaier, M., & Stern, A. D. (2026) · Communications Medicine · doi:10.1038/s43856-026-01450-8 · Cited by 13 · see citing works
Thies, T., Mallick, E., Tröger, J., Baykara, E., Mücke, D., & Barbe, M. T. (2025) · npj Parkinson's Disease · doi:10.1038/s41531-025-00959-4 · Cited by 8 · see citing works
Developing digital health technologies for frontotemporal degeneration
Nevler, N., Niehoff, D., Gleixner, A. M., Dodge, S., Vogel, A. P., Hill, D. L., Ducharme, S., Staffaroni, A. M., Yunusova, Y., Au, R., et al. (2025) · Alzheimer's & Dementia · doi:10.1002/alz.70082 · Cited by 7 · see citing works
Benway, N. R., & Preston, J. L. (2025) · Language, Speech, and Hearing Services in Schools · doi:10.1044/2025_lshss-24-00085 · Cited by 12 · see citing works
El Arab, R. A., Mustafa, M. H., Almagharbeh, W. T., Saleem, N. H., Al Abdulmohsen, S., Boathab, R., & Bu Washl, M. (2026) · Healthcare · doi:10.3390/healthcare14111459 · Cited by 4 · see citing works
“These concerns are reinforced by empirical evidence showing that strong development or vendor-reported performance does not necessarily translate into reliable clinical utility in practice [6,7].”
- Authors
- Goldsack, J. C., Coravos, A., Bakker, J. P., Bent, B., Dowling, A. V., Fitzer-Attas, C., Godfrey, A., Godino, J. G., Gujar, N., Izmailova, E., Manta, C., Peterson, B., Vandendriessche, B., Wood, W. A., Wang, K. W., & Dunn, J.
- Affiliations
- Harvard University (US)
- Duke University (US)
- Northumbria University (GB)
- University of California San Diego (US)
- Curis (United States) (US)
- Case Western Reserve University (US)
- University of North Carolina at Chapel Hill (US)
- Corresponding author
- Dunn, Jessilyn
- ORCID
- Venue
- npj Digital Medicine
- Published
- 2020-04-14
- Volume
- 3
- Issue
- 1
- Pages
- 55-55
- ISSN
- 2398-6352 (ISSN-L)
- PMC
- PMC7156507
- OpenAlex
- W3016443966
- Cited by
- 557
- References
- 5
- Citations by year
- 2026: 138 · 2025: 95 · 2024: 108 · 2023: 82 · 2022: 52 · 2021: 54 · 2020: 27 · 2013: 1
- FWCI
- 12.18
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- npj Digital Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Northumbria Research Link (Northumbria University) (acceptedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Duke FORGE Fellowship
- Topics
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Physical Sciences › Engineering › Biomedical Engineering › Non-Invasive Vital Sign Monitoring
- Keywords
- verification, analytical method validation, clinical validation, fit for purpose, digital medicine, terminology standardization, clinical trials, evaluation framework
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
