The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
An open source benchmarked toolbox for cardiovascular waveform and interval analysis
Open accessAbstract
OBJECTIVE: This work aims to validate a set of data processing methods for variability metrics, which hold promise as potential indicators for autonomic function, prediction of adverse cardiovascular outcomes, psychophysiological status, and general wellness. Although the investigation of heart rate variability (HRV) has been prevalent for several decades, the methods used for preprocessing, windowing, and choosing appropriate parameters lacks consensus among academic and clinical investigators. Moreover, many of the important steps are omitted from publications, preventing reproducibility. APPROACH: To address this, we have compiled a comprehensive and open-source modular toolbox for calculating HRV metrics and other related variability indices, on both raw cardiovascular time series and RR intervals. The software, known as the PhysioNet Cardiovascular Signal Toolbox, is implemented in the MATLAB programming language, with standard (open) input and output formats, and requires no external libraries. The functioning of our software is compared with other widely used and referenced HRV toolboxes to identify important differences. MAIN RESULTS: Our findings demonstrate how modest differences in the approach to HRV analysis can lead to divergent results, a factor that might have contributed to the lack of repeatability of studies and clinical applicability of HRV metrics. SIGNIFICANCE: Existing HRV toolboxes do not include standardized preprocessing, signal quality indices (for noisy segment removal), and abnormal rhythm detection and are therefore likely to lead to significant errors in the presence of moderate to high noise or arrhythmias. We therefore describe the inclusion of validated tools to address these issues. We also make recommendations for default values and testing/reporting.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Heart Rate Variability: Standards of Measurement, Physiological Interpretation, and Clinical Use
Electrophysiology, T. F. O. T. E. S. O. C. T. N. A. (1996) · Circulation · doi:10.1161/01.cir.93.5.1043 · Cited by 17,159
Heart rate variability: Standards of measurement, physiological interpretation, and clinical use
Malik, M., Bigger, J. T., Camm, A. J., Kleiger, R. E., Malliani, A., Moss, A. J., & Schwartz, P. J. (1996) · European Heart Journal · doi:10.1093/oxfordjournals.eurheartj.a014868 · Cited by 15,289
Goldberger, A. L., Amaral, L. A. N., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G., Mietus, J. E., Moody, G. B., Peng, C.-K., & Stanley, H. E. (2000) · Circulation · doi:10.1161/01.cir.101.23.e215 · Cited by 14,809
Physiological time-series analysis using approximate entropy and sample entropy
Richman, J. S., & Moorman, J. R. (2000) · American Journal of Physiology-Heart and Circulatory Physiology · doi:10.1152/ajpheart.2000.278.6.h2039 · Cited by 7,880
Scargle, J. D. (1982) · The Astrophysical Journal · doi:10.1086/160554 · Cited by 7,376
Least-squares frequency analysis of unequally spaced data
Lomb, N. R. (1976) · Astrophysics and Space Science · doi:10.1007/bf00648343 · Cited by 5,717
The impact of the MIT-BIH Arrhythmia Database
Moody, G. B., & Mark, R. G. (2001) · IEEE Engineering in Medicine and Biology Magazine · doi:10.1109/51.932724 · Cited by 4,812
Quantification of scaling exponents and crossover phenomena in nonstationary heartbeat time series
Peng, C.-K., Havlin, S., Stanley, H. E., & Goldberger, A. L. (1995) · Chaos: An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.166141 · Cited by 3,855
Multiscale Entropy Analysis of Complex Physiologic Time Series
Costa, M., Goldberger, A. L., & Peng, C.-K. (2002) · Physical Review Letters · doi:10.1103/physrevlett.89.068102 · Cited by 3,207
Kubios HRV–heart rate variability analysis software.
Tarvainen, M. P., Niskanen, J.-P., Lipponen, J. A., Ranta-aho, P. O., & Karjalainen, P. A. (2014) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2013.07.024 · Cited by 2,720 · cited 1×
“The proprietary Kubios HRV software (Tarvainen et al., 2014) is another frequently used and cited HRV analysis tool.”
MethodMultiscale entropy analysis of biological signals
Costa, M., Goldberger, A. L., & Peng, C.-K. (2005) · Physical Review E · doi:10.1103/physreve.71.021906 · Cited by 2,482
A dynamical model for generating synthetic electrocardiogram signals
McSharry, P. E., Clifford, G. D., Tarassenko, L., & Smith, L. A. (2003) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2003.808805 · Cited by 1,224
Self-citationSympathovagal Balance: A Critical Appraisal
Eckberg, D. L. (1997) · Circulation · doi:10.1161/01.cir.96.9.3224 · Cited by 1,087 · cited 1×
“It is generally accepted in the clinical community that the HF band is mostly a measure of the parasympathetic activity (Cerutti et al., 1995) with some sympathetic activity, while the LF band contains mostly sympathetic activation (Eckberg, 1997).”
BackgroundAdvanced Methods And Tools for ECG Data Analysis
Clifford, G. D., Azuaje, F. J., & McSharry, P. (2006) · Cited by 808
Self-citationSchmidt, G., Malik, M., Barthel, P., Schneider, R., Ulm, K., Rolnitzky, L., Camm, A. J., Bigger, J. T., & Schömig, A. (1999) · The Lancet · doi:10.1016/s0140-6736(98)08428-1 · Cited by 716 · cited 9×
“On the analyzed dataset, for some recordings, a larger number of NN intervals before and after PVCs have been ‘filtered’ by the PhysioNet Cardiovascular Signal Toolbox than the Schneider code.”
MethodInfluentialSchmidt, F., Malík, Barthel, & Schneider, R. (1999) · The Lancet · Cited by 594
Bernardi, L., Wdowczyk-Szulc, J., Valenti, C., Castoldi, S., Passino, C., Spadacini, G., & Sleight, P. (2000) · Journal of the American College of Cardiology · doi:10.1016/s0735-1097(00)00595-7 · Cited by 516
Heart Rate Turbulence: Standards of Measurement, Physiological Interpretation, and Clinical Use
Bauer, A., Malik, M., Schmidt, G., Barthel, P., Bonnemeier, H., Cygankiewicz, I., Guzik, P., Lombardi, F., Müller, A., Oto, A., et al. (2008) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2008.07.041 · Cited by 462 · cited 5×
“TO can be also calculated on the averaged tachogram which leads to very similar values (Bauer et al., 2008).”
BackgroundMethodInfluentialSleep in Critically Ill Patients Requiring Mechanical Ventilation
Cooper, A. B., Thornley, K. S., Young, G. B., Slutsky, A. S., Stewart, T. E., & Hanly, P. J. (2000) · Chest · doi:10.1378/chest.117.3.809 · Cited by 457
Li, Q., Mark, R. G., & Clifford, G. D. (2008) · Physiological Measurement · doi:10.1088/0967-3334/29/1/002 · Cited by 426
Self-citationFel, L., & Malik, M. (1994) · Cardiac Pacing and Electrophysiology · doi:10.1007/978-94-011-0872-0_6 · Cited by 370
Malik, M. (1998) · Current Opinion in Cardiology · doi:10.1097/00001573-199801000-00006 · Cited by 368
Quantifying Errors in Spectral Estimates of HRV Due to Beat Replacement and Resampling
Clifford, G. D., & Tarassenko, L. (2005) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2005.844028 · Cited by 285
Self-citationVentricular Fibrillation and Tachycardia Classification Using a Machine Learning Approach
Li, Q., Rajagopalan, C., & Clifford, G. D. (2014) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2013.2275000 · Cited by 274
Self-citationOtzenberger, H., Gronfier, C., Simon, C., Charloux, A., Ehrhart, J., Piquard, F., & Brandenberger, G. (1998) · American Journal of Physiology-Heart and Circulatory Physiology · doi:10.1152/ajpheart.1998.275.3.h946 · Cited by 256
Phase-rectified signal averaging detects quasi-periodicities in non-stationary data
Bauer, A., Kantelhardt, J. W., Bunde, A., Barthel, P., Schneider, R., Malik, M., & Schmidt, G. (2006) · Physica A: Statistical Mechanics and its Applications · doi:10.1016/j.physa.2005.08.080 · Cited by 235
A robust open-source algorithm to detect onset and duration of QRS complexes
Zong, W., Moody, G. B., & Jiang, D. (2003) · Computers in Cardiology, 2003 · doi:10.1109/cic.2003.1291261 · Cited by 208
Signal processing methods for heart rate variability
Clifford, G. D. (2002) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.5287/ora-noa0mqzyn · Cited by 181
Self-citationA Comparison of Single Channel Fetal ECG Extraction Methods
Behar, J., Johnson, A., Clifford, G. D., & Oster, J. (2014) · Annals of Biomedical Engineering · doi:10.1007/s10439-014-0993-9 · Cited by 180
Self-citationKantelhardt, J. W., Bauer, A., Schumann, A. Y., Barthel, P., Schneider, R., Malík, M., & Schmidt, G. P. (2007) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.2430636 · Cited by 133
Signal quality and data fusion for false alarm reduction in the intensive care unit
Li, Q., & Clifford, G. D. (2012) · Journal of Electrocardiology · doi:10.1016/j.jelectrocard.2012.07.015 · Cited by 126
Self-citationHEART RATE MULTISCALE ENTROPY AT THREE HOURS PREDICTS HOSPITAL MORTALITY IN 3,154 TRAUMA PATIENTS
Norris, P. R., Anderson, S. M., Jenkins, J. M., Williams, A. E., & Morris, J. A. (2008) · Shock · doi:10.1097/shk.0b013e318164e4d0 · Cited by 113
Multimodal heart beat detection using signal quality indices
Johnson, A. E. W., Behar, J. A., Andreotti, F., Clifford, G. D., & Oster, J. (2015) · Physiological Measurement · doi:10.1088/0967-3334/36/8/1665 · Cited by 96
Self-citationBauer, A., & Schmidt, G. P. (2003) · Journal of Electrocardiology · doi:10.1016/j.jelectrocard.2003.09.020 · Cited by 84
Relationship between detrended fluctuation analysis and spectral analysis of heart-rate variability
Willson, K., Francis, D. P., Wensel, R., Coats, A. J. S., & Parker, K. H. (2002) · Physiological Measurement · doi:10.1088/0967-3334/23/2/314 · Cited by 78
Norris, P. R., Stein, P. K., & Morris, J. A. (2007) · Journal of Critical Care · doi:10.1016/j.jcrc.2007.08.001 · Cited by 75
(1996) · European Heart Journal · doi:10.1093/eurheartj/17.suppl_2.28 · Cited by 66
Berkowitsch, A., Zaręba, W., Neumann, T., Erdogan, A. R., Nitt, S. M., Moss, A. J., & PITSCHNER, H. F. (2004) · Annals of Noninvasive Electrocardiology · doi:10.1111/j.1542-474x.2004.93600.x · Cited by 65
Impact of the presence of noise on RR interval-based atrial fibrillation detection
Oster, J., & Clifford, G. D. (2015) · Journal of Electrocardiology · doi:10.1016/j.jelectrocard.2015.08.013 · Cited by 60
Self-citationA comparison of entropy approaches for AF discrimination
Liu, C., Oster, J., Reinertsen, E., Li, Q., Zhao, L., Nemati, S., & Clifford, G. D. (2018) · Physiological Measurement · doi:10.1088/1361-6579/aacc48 · Cited by 52
Self-citationR-Peak Estimation using Multimodal Lead Switching
Johnson, A. E. W., Behar, J. A., Andreotti, F., Clifford, G. D., & Oster, J. (2014) · HAL · Cited by 45
Self-citationUncertainty Principle of Signal-Averaged Electrocardiography
Goldberger, J. J., Challapalli, S., Waligora, M., Kadish, A. H., Johnson, D. A., Ahmed, M. W., & Inbar, S. (2000) · Circulation · doi:10.1161/01.cir.101.25.2909 · Cited by 43
Phase-rectified signal averaging as a sensitive index of autonomic changes with aging
Campana, L. M., Owens, R. L., Clifford, G. D., Pittman, S. D., & Malhotra, A. (2010) · Journal of Applied Physiology · doi:10.1152/japplphysiol.00013.2010 · Cited by 42
Self-citationPan, Q., Zhou, G., Wang, R., Cai, G., Yan, J., Fang, L., & Ning, G. (2016) · Medical & Biological Engineering & Computing · doi:10.1007/s11517-016-1486-9 · Cited by 42
Crowd-Sourced Annotation of ECG Signals Using Contextual Information
Zhu, T., Johnson, A. E. W., Behar, J. A., & Clifford, G. D. (2013) · Annals of Biomedical Engineering · doi:10.1007/s10439-013-0964-6 · Cited by 41
Self-citationVentricular ectopic beat detection using a wavelet transform and a convolutional neural network
Li, Q., Liu, C., Li, Q., Shashikumar, S. P., Nemati, S., Shen, Z., & Clifford, G. D. (2019) · Physiological Measurement · doi:10.1088/1361-6579/ab17f0 · Cited by 41
Self-citationMethod for generating an artificial RR tachogram of a typical healthy human over 24-hours
McSharry, P., Clifford, G. D., Tarassenko, L., & Smith, L. A. (2003) · Computers in cardiology · doi:10.1109/cic.2002.1166748 · Cited by 40
Self-citationLin, Y., Huang, H., Chang, Y.-C., Lin, C., Lo, M., Liu, L.-Y. D., Tsai, P.-R., Chen, Y., Ko, W., Ho, Y., et al. (2014) · Critical Care · doi:10.1186/s13054-014-0548-3 · Cited by 38
Evaluation of the Reynolds Pathfinder II system for 24 h heart rate variability analysis
Mølgaard, H. (1991) · European Heart Journal · doi:10.1093/eurheartj/12.11.1153 · Cited by 33
Clifford, G. D., & Tarassenko, L. (2004) · Physiological Measurement · doi:10.1088/0967-3334/25/6/n03 · Cited by 27
Self-citationCardiac output estimation using arterial blood pressure waveforms
Sun, J. X. (2006) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 21
Kisohara, M., Stein, P. K., Yoshida, Y., Suzuki, M. H., Iizuka, N., Carney, R. M., Watkins, L. L., Freedland, K. E., Blumenthal, J. A., & Hayano, J. (2012) · EP Europace · doi:10.1093/europace/eus409 · Cited by 21
Bispectral analysis of heart rate variability signal
Saliu, S., Birand, A., & Kudaiberdieva, G. (2002) · Cited by 18
Lacefield, J. C., Weaver, J., Spence, J. R., Dunmore‐Buyze, J., & Boughner, D. R. (2003) · Physiological Measurement · doi:10.1088/0967-3334/25/1/003 · Cited by 10
Benchmarking heart rate variability toolboxes
Vest, A. N., Li, Q., Liu, C., Nemati, S., Shah, A. J., & Clifford, G. D. (2017) · Journal of Electrocardiology · doi:10.1016/j.jelectrocard.2017.08.006 · Cited by 10
Self-citationThe influence of recording length on time and frequency domain analysis of heart rate variability
Haaksma, J., Dijk, W. A., Brouwer, J., et al. (2002) · Computers in cardiology · doi:10.1109/cic.1998.731818 · Cited by 6
Cited by
Classification of 12-lead ECGs: the PhysioNet/Computing in Cardiology Challenge 2020
Perez Alday, E. A., Gu, A., J Shah, A., Robichaux, C., Ian Wong, A.-K., Liu, C., Liu, F., Bahrami Rad, A., Elola, A., Seyedi, S., et al. (2020) · Physiological Measurement · doi:10.1088/1361-6579/abc960 · Cited by 436
Self-citationWearable Photoplethysmography for Cardiovascular Monitoring
Charlton, P. H., Kyriacou, P. A., Mant, J., Marozas, V., Chowienczyk, P., & Alastruey, J. (2022) · Proceedings of the IEEE · doi:10.1109/jproc.2022.3149785 · Cited by 250
An Ultra Low Power Personalizable Wrist Worn ECG Monitor Integrated With IoT Infrastructure
Beach, C., Krachunov, S., Pope, J., Fafoutis, X., Piechocki, R. J., Craddock, I., & Casson, A. J. (2018) · IEEE Access · doi:10.1109/access.2018.2864675 · Cited by 91
Metadata
- Authors
- Vest, A. N., Da Poian, G., Li, Q., Liu, C., Nemati, S., Shah, A. J., & Clifford, G. D.
- Venue
- Physiological Measurement
- Publisher
- IOP Publishing
- Published
- 2018-10-11
- Volume
- 39
- Issue
- 10
- Pages
- 105004
- ISSN
- 0967-3334 (ISSN-L)
- 1361-6579
- OpenAlex
- W2890527005
- Cited by
- 275
- References
- 57
- FWCI
- 16.34
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.