Statistical considerations for reporting and planning heart rate variability case‐control studies
Open accessPublished version of a preprint
Abstract
The calculation of heart rate variability (HRV) is a popular tool used to investigate differences in cardiac autonomic control between population samples. When interpreting effect sizes to quantify the magnitude of group differences, researchers typically use Cohen's guidelines of small (0.2), medium (0.5), and large (0.8) effects. However, these guidelines were originally proposed as a fallback for when the effect size distribution (ESD) was unknown. Despite the availability of effect sizes from hundreds of HRV studies, researchers still largely rely on Cohen's guidelines to interpret effect sizes and to perform power analyses to calculate required sample sizes for future research. This article describes an ESD analysis of 297 HRV effect sizes from between‐group/case‐control studies, revealing that the 25th, 50th, and 75th effect size percentiles correspond with effect sizes of 0.26, 0.51, and 0.88, respectively. The analyses suggest that Cohen's guidelines may underestimate the magnitude of small and large effect sizes and that HRV studies are generally underpowered. Therefore, to better reflect the observed ESD, effect sizes of 0.25, 0.5, and 0.9 should be interpreted as small, medium, and large effects (after rounding to the closest 0.05). Based on power calculations using the ESD, suggested sample sizes are also provided for planning suitably powered studies that are more likely to replicate. Researchers are encouraged to use the ESD data set or their own collected data sets in tandem with the provided analysis script to perform custom ESD and power analyses relevant to their specific research area.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · see citing works
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567 · see citing works
Effect size guidelines for individual differences researchers
Gignac, G. E., & Szodorai, E. T. (2016) · Personality and Individual Differences · doi:10.1016/j.paid.2016.06.069 · Cited by 2,860 · see citing works
Statistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791 · see citing works
Peters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.11.010 · Cited by 2,053 · see citing works
The Abuse of Power: The Pervasive Fallacy of Power Calculations for Data Analysis
Hoenig, J. M., & Heisey, D. M. (2001) · The American Statistician · doi:10.1198/000313001300339897 · Cited by 2,046 · see citing works
Anxiety Disorders are Associated with Reduced Heart Rate Variability: A Meta-Analysis
Chalmers, J. A., Quintana, D. S., Abbott, M. J.-A., & Kemp, A. H. (2014) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2014.00080 · Cited by 1,067 · see citing works
Self-citationHeart Rate Variability ? A Historical Perspective
Billman, G. E. (2011) · Frontiers in Physiology · doi:10.3389/fphys.2011.00086 · Cited by 786 · see citing works
Alvares, G. A., Quintana, D. S., Hickie, I. B., & Guastella, A. J. (2016) · Journal of Psychiatry and Neuroscience · doi:10.1503/jpn.140217 · Cited by 503 · see citing works
Self-citationPre-post effect sizes should be avoided in meta-analyses
Cuijpers, P., Weitz, E., Cristea, I. A., & Twisk, J. (2017) · Epidemiology and Psychiatric Sciences · doi:10.1017/s2045796016000809 · Cited by 432 · see citing works
Koenig, J., Kemp, A. H., Beauchaine, T. P., Thayer, J. F., & Kaess, M. (2016) · Clinical Psychology Review · doi:10.1016/j.cpr.2016.04.013 · Cited by 285 · see citing works
Clamor, A., Lincoln, T. M., Thayer, J. F., & Koenig, J. (2016) · British Journal of Psychiatry · doi:10.1192/bjp.bp.114.160762 · Cited by 167 · see citing works
Liu, Q., Wang, E. M., Yan, X. J., & Chen, S. L. (2013) · Journal of Digestive Diseases · doi:10.1111/1751-2980.12092 · Cited by 69 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Laborde, S., Mosley, E., & Thayer, J. F. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00213 · Cited by 2,336 · see citing works
“This focus on cases vs. control participants was also found in Quintana (2016), regarding the statistical considerations about reporting and planning HRV studies.”
BackgroundInfluentialEffect Size Guidelines, Sample Size Calculations, and Statistical Power in Gerontology
Brydges, C. R. (2019) · Innovation in Aging · doi:10.1093/geroni/igz036 · Cited by 826 · see citing works
Heart rate variability alterations in late life depression: A meta-analysis
Brown, L., Karmakar, C., Gray, R., Jindal, R., Lim, T., & Bryant, C. (2018) · Journal of Affective Disorders · doi:10.1016/j.jad.2018.04.071 · Cited by 108 · see citing works
Brown, L., Rando, A. A., Eichel, K., Van Dam, N. T., Celano, C. M., Huffman, J. C., & Morris, M. E. (2021) · Psychosomatic Medicine · doi:10.1097/psy.0000000000000900 · Cited by 93 · see citing works
Sustained attention and heart rate variability in children and adolescents with ADHD
Griffiths, K. R., Quintana, D. S., Hermens, D. F., Spooner, C., Tsang, T. W., Clarke, S., & Kohn, M. R. (2017) · Biological Psychology · doi:10.1016/j.biopsycho.2017.01.004 · Cited by 79 · see citing works
Self-citationHeart rate variability is associated with disease severity in psychosis spectrum disorders
Benjamin, B. R., Valstad, M., Elvsåshagen, T., Jönsson, E. G., Moberget, T., Winterton, A., Haram, M., Høegh, M. C., Lagerberg, T. V., Steen, N. E., et al. (2021) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2020.110108 · Cited by 52 · see citing works
Self-citationQuintana, D. S., Dieset, I., Elvsåshagen, T., Westlye, L. T., & Andreassen, O. A. (2017) · Frontiers in Neuroendocrinology · doi:10.1016/j.yfrne.2016.12.004 · Cited by 39 · see citing works
Self-citationReduced heart rate variability in a treatment-seeking early psychosis sample
Cacciotti-Saija, C., Quintana, D. S., Alvares, G. A., Hickie, I. B., & Guastella, A. J. (2018) · Psychiatry Research · doi:10.1016/j.psychres.2018.08.068 · Cited by 18 · see citing works
Self-citationInvolvement of Cardiorespiratory Capacity on the Acute Effects of Caffeine on Autonomic Recovery
Gonzaga, L. A., Vanderlei, L. C. M., Gomes, R. L., Garner, D. M., & Valenti, V. E. (2019) · Medicina · doi:10.3390/medicina55050196 · Cited by 13 · see citing works
- Authors
- Quintana, D. S.
- Affiliations
- University of Oslo (NO)
- Corresponding author
- Quintana, Daniel S.
- ORCID
- Quintana, Daniel S.
- Venue
- Psychophysiology
- Publisher
- Wiley
- Published
- 2016-12-03
- Volume
- 54
- Issue
- 3
- Pages
- 344-349
- ISSN
- 0048-5772 (ISSN-L)
- 1469-8986
- OpenAlex
- W2560234115
- Cited by
- 192
- References
- 13
- Citations by year
- 2026: 21 · 2025: 18 · 2024: 24 · 2023: 14 · 2022: 12 · 2021: 18 · 2020: 31 · 2019: 26 · 2018: 18 · 2017: 10
- FWCI
- 14.87
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- Psychophysiology (publishedVersion)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Funders
- Novo Nordisk Foundation (NNF16OC0019856)
- Topics
- Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › Heart Rate Variability and Autonomic Control
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- heart rate variability, effect size distribution, sample size determination, statistical power, cardiac autonomic control, case-control study, replication, effect size interpretation
- MeSH terms
- Autonomic Nervous System, Data Interpretation, Statistical, Heart Rate, Humans, Psychophysics, Research Design, Reproducibility of Results, Case-Control Studies, Sample Size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
