Testing the significance of a correlation with nonnormal data: Comparison of Pearson, Spearman, transformation, and resampling approaches.
The DOI matches its Crossref record; 99 of the 99 references it cites were found in a registry or an index.
Abstract
It is well known that when data are nonnormally distributed, a test of the significance of Pearson's r may inflate Type I error rates and reduce power. Statistics textbooks and the simulation literature provide several alternatives to Pearson's correlation. However, the relative performance of these alternatives has been unclear. Two simulation studies were conducted to compare 12 methods, including Pearson, Spearman's rank-order, transformation, and resampling approaches. With most sample sizes (n ≥ 20), Type I and Type II error rates were minimized by transforming the data to a normal shape prior to assessing the Pearson correlation. Among transformation approaches, a general purpose rank-based inverse normal transformation (i.e., transformation to rankit scores) was most beneficial. However, when samples were both small (n ≤ 10) and extremely nonnormal, the permutation test often outperformed other alternatives, including various bootstrap tests.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Deviations from normality usually do not bias regression coefficients (Ramsey & Schafer, 2013; Williams et al., 2013) or impair hypothesis testing (no inflated type I error rate, e.g., Bishara & Hittner, 2012; Ives, 2015; Puth et al., 2014; Ramsey & Schafer, 2013; Szöcs & Schäfer, 2015; Warton et al., 2016) even at relatively small sample sizes.”
From Violating the normality assumption may be the lesser of two evils (2021) · cites this paper as Bishara & Hittner, 2012
In the citing paper
“We focus on fairly drastic non-normality because only little bias can be expected from minor violations (Bishara & Hittner, 2012; Glass et al., 1972; Hack, 1958; Puth et al., 2014).”
From Violating the normality assumption may be the lesser of two evils (2021) · cites this paper as Bishara & Hittner, 2012
In the citing paper
“it may be most advisable to apply a rank-based inverse normal (RIN) transformation (aka rankit scores, Bliss, 1967) to the data, which can approximately normalize most distributional shapes and which effectively minimizes type I errors and maximizes statistical power (Bishara & Hittner, 2012; Puth et al., 2014).”
From Violating the normality assumption may be the lesser of two evils (2021) · cites this paper as Bishara & Hittner, 2012
In the citing paper
“All these deviations are generally thought to be of little concern (e.g., Bishara & Hittner, 2012), even if they are far from fitting to the bell-shaped curve, such as binomial data (Cochran, 1950).”
From Violating the normality assumption may be the lesser of two evils (2021) · cites this paper as Bishara & Hittner, 2012
Citations
Cites
Tabachnick, B. G., & Fidell, L. S. (1983) · Cited by 80,032
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences
Cohen, P., Cohen, P., West, S. G., & Aiken, L. S. (2014) · Psychology Press eBooks · doi:10.4324/9781410606266 · Cited by 20,737
Bootstrap methods: another look at the jackknife
Efron, B. (1979) · The Annals of Statistics · doi:10.1214/aos/1176344552 · Cited by 17,602
An Analysis of Transformations
Box, G. E. P., & Cox, D. R. (1964) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1964.tb00553.x · Cited by 15,052
Statistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
(2007) · Cited by 10,092
Educational Research: Competencies for Analysis and Application
Mills, G. E., & Airasian, P. W. (1995) · Medical Entomology and Zoology · Cited by 5,395
Fundamentals of Biostatistics.
(1969) · Annals of Internal Medicine · doi:10.7326/0003-4819-70-4-873 · Cited by 4,575
Cited by
Items 901–947 of 947
Wang, W., Zhu, B., Huang, Y., Liang, L., Li, Z., Diao, Y., Guo, M., Zhang, H., Zhou, J., Wu, F., et al. (2026) · Interdisciplinary medicine · doi:10.1002/inmd.70165
Nerius, J. (2026) · Scholars Crossing (Liberty University)
Schammler, S., Persike, M., & Weber, F. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1918459
Oliveira, P. D., Meunier, H., Breton, A., & Dambrun, M. (2026) · Neuroscience of Consciousness · doi:10.1093/nc/niag022
Wiratmo, C. Y., Sudirman, P. L., & Sidiartha, I. G. A. F. N. (2023) · Bali Dental Journal · doi:10.51559/bdj.v7i1.227
Eliason, H. A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7358596
Zhang, H., Srinivasan, R. S., Chen, C., Yang, X., Zhi, 黄. H., Ganesan, V., & Zhou, R. (2026) · Atmospheric Environment · doi:10.1016/j.atmosenv.2026.122350
Abdolhosseini, M., & Ogunsanya, I. G. (2026) · Construction and Building Materials · doi:10.1016/j.conbuildmat.2026.148142
Fuzzy logic-based efficiency index for water supply systems in Northeastern Brazil
da Silva, J. T. G., & Bezerra, S. D. T. M. (2026) · Revista Brasileira de Ciências Ambientais · doi:10.67790/rbciamb.026.2820
Hydrochemical regimes and nitrate–fluoride surveillance in groundwater of Odisha, Eastern India
Patra, T. R. (2026) · Environmental Monitoring and Assessment · doi:10.1007/s10661-026-15901-1
Baldoviso, N. F. A., Trabelsi, N., Abu-alburak, A., batal, N. E., & Tashjian, H. (2026) · NURSE EDUCATORS AND PRACTITIONERS JOURNAL · doi:10.64397/nepj.v02i01.2026.a37
Hubungan Intensitas Paparan Konten YouTube Traveling dengan Sikap Followers
Najwa, A., & Rochim, M. (2026) · Bandung Conference Series Public Relations · doi:10.29313/bcspr.v6i2.26146
Khánh, H. D., Tam, D. T., Thanh, H. V., & Dang, H. N. N. (2026) · Tourism and Hospitality · doi:10.3390/tourhosp7090315
Influence of Flake Properties and Dropping Position on Hall Thruster Spark Discharge
Yu, Q., Li, Y., Hu, S., Tang, J., Ding, Y., Wei, L., & Yu, D. (2026) · Journal of Propulsion and Power · doi:10.2514/1.b40445
Ellwart, T., Birke, J., & Paul, J. T. (2026) · JMIR Human Factors · doi:10.2196/87735
Makai, A. L., & Dőry, T. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1968268
Engaging Teenagers in Cyberbullying Prevention: Insights from a Digital Toolkit
Bonetti, F., Aprosio, A. P., Piras, E. M., & Tonelli, S. (2026) · Technology Knowledge and Learning · doi:10.1007/s10758-026-10008-5
Multi-objective intake strategy optimization for elliptical rotor engines with unique intake system
裴毅强, Wang, Y., Qin, J., Deng, X., Yao, D., & Lei, J. (2026) · Applied Thermal Engineering · doi:10.1016/j.applthermaleng.2026.133464
Plaza, F., Aránguiz, M., Catasti, V., Barbieri, M. Á., Poca, O. C., & Eleuterio Yáñez, R. (2026) · Deep Sea Research Part II Topical Studies in Oceanography · doi:10.1016/j.dsr2.2026.105711
Nugroho, A. B., Alhadid, B., & Samoria, Y. (2025) · Proceeding International Conference on Religion Science and Education
IMPACT OF SHARED LEADERSHIP ON PROJECT SUCCESS: MEDIATING ROLE OF KNOWLEDGE SHARING
Batool, L., Qureshi, M. S., & Chaudhry, A. A. (2026) · Policy Journal of Social Science Review
Intelligent cross-scale modeling of wildland fire risk and development
Li, Y. (n.d.) · PolyU Institutional Research Archive (Hong Kong Polytechnic University)
Stalking: Perceptions of victim responsibility, attributions of blame and interventions
Tarant, E. R. A. (2025) · Nottingham ePrints (University of Nottingham)
Changing lifestyles through generations
Špindler, T., Gorenak, M., & Lesjak, M. (2025) · University of Maribor digital library (University of Maribor)
Acceptance of Machine Translation and Machine Translation Post-Editing by Croatian Translators
Omazić, M., Ručević, S., & Rajh, I. (2026) · Springer eBooks · doi:10.1007/978-3-032-24009-5_4
Zhou, A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7573078
Picón-Rentería, P., Herrera, H. W., AVENDAÑO, J. M., Tocora-Protz, M. C., Sánchez‐Restrepo, A. F., Fernández, F., Heraty, J. M., Bui, A. Y., Peck, S. B., & Sarmiento, C. E. (2026) · ZooKeys · doi:10.3897/zookeys.1294.200001
Pain in Young Survivors of Childhood Cancer: Examining the Role of Child and Parent Mental Health
Benedictson, J. (2026) · Spectrum Research Repository (Concordia University)
Ηγεσία, οργανωσιακή κουλτούρα, επικοινωνία και δυναμική ομάδων σε εργαζόμενους ξενοδοχειακών μονάδων
Νταλάκος, Ά. Γ. (2026) · University of Thessaly Institutional Repository (University of Thessaly)
Campoverde, W. G. S., & Sandoval, P. A. C. (2026) · F1000Research · doi:10.12688/f1000research.191356.1
Lesher, T. R. (2026) · Digital Commons (Liberty University)
Jones, C. (2024) · Adelaide Research & Scholarship (AR&S) (University of Adelaide)
Hubungan Persepsi Demonstrasi dengan Respons Belajar Fiqih Fardu Kifayah pada Mahasiswa PAI
Rayana, A., Suhardiman, S., & Haryanto, U. F. (2025) · Jurnal Generasi Ceria Indonesia · doi:10.47709/geci.v3i2.9767
Kauttonen, J., Tuomi, A., Vuori, J., Saaranen, P., & Lahtinen, A. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2750593
Influence of anatomical characteristics on the physical properties of wood from Cerrado species
Monteiro, R. D. O., de Carvalho, D. C., Wimmer, P., Bambil, D. R., Guilhon, D. S., & Sonsin‐Oliveira, J. (2026) · IAWA Journal - KU Leuven/IAWA Journal · doi:10.1163/22941932-bja10230
Measuring Deviation from Complete Spatial Randomness in Two-Dimensional Point Clouds
Benavides, D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-43217-9_7
Seebohm, R. (n.d.) · Murdoch University Research Portal
Lee, Y.-C., Chou, W., Shun, S., Lin, Y.-K., Yang, C.-Y., & Liao, Y. (2026) · Asia-Pacific Journal of Oncology Nursing · doi:10.1016/j.apjon.2026.101058
Zhou, Y., Xiao, F., Song, Y., Gao, D., Liu, X., Gui, X., Song, F., Tian, S., & 桂小杰 (2026) · Animals · doi:10.3390/ani16213519
Johnson, E., Rebecca Fellowes, and, K. C., & Hunt, S. (2022) · doi:10.3389/fgwh.2021.778183.s001
Ning, J., Wang, F., Zhu, K., Li, B., Shu, Q., & Liu, W. (2021) · doi:10.3389/fgene.2021.779155.s001
Shitu, K., Adugna, A., Kassie, A., & Handebo, S. (2022) · doi:10.1371/journal.pone.0263568.r007
Adult attachment and health-related quality of life after acquired brain injury
Deakins, J. (2014) · Royal Holloway Digital Repository (Royal Holloway University of London)
Metadata
- Authors
- Bishara, A. J., & Hittner, J. B.
- Affiliations
- College of Charleston (US)
- Venue
- Psychological Methods
- Published
- 2012-09
- Volume
- 17
- Issue
- 3
- Pages
- 399-417
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2111495793
- Cited by
- 766
- References
- 98
- Citations by year
- 2026: 80 · 2025: 113 · 2024: 88 · 2023: 77 · 2022: 86 · 2021: 53 · 2020: 64 · 2019: 52 · 2018: 34 · 2017: 40 · 2016: 23 · 2015: 25 · 2014: 23 · 2013: 5 · 2012: 3
- FWCI
- 20.74
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Keywords
- non-normal data, Pearson correlation, data transformation, permutation tests, bootstrap tests, type I error, statistical power, small sample size
- MeSH terms
- Data Interpretation, Statistical, Monte Carlo Method, Statistics as Topic, Statistical Distributions, Linear Models, Nonlinear Dynamics, Sample Size, Statistics, Nonparametric
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.