Testing the significance of a correlation with nonnormal data: Comparison of Pearson, Spearman, transformation, and resampling approaches.
The DOI matches its Crossref record; 99 of the 99 references it cites were found in a registry or an index.
Abstract
It is well known that when data are nonnormally distributed, a test of the significance of Pearson's r may inflate Type I error rates and reduce power. Statistics textbooks and the simulation literature provide several alternatives to Pearson's correlation. However, the relative performance of these alternatives has been unclear. Two simulation studies were conducted to compare 12 methods, including Pearson, Spearman's rank-order, transformation, and resampling approaches. With most sample sizes (n ≥ 20), Type I and Type II error rates were minimized by transforming the data to a normal shape prior to assessing the Pearson correlation. Among transformation approaches, a general purpose rank-based inverse normal transformation (i.e., transformation to rankit scores) was most beneficial. However, when samples were both small (n ≤ 10) and extremely nonnormal, the permutation test often outperformed other alternatives, including various bootstrap tests.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Deviations from normality usually do not bias regression coefficients (Ramsey & Schafer, 2013; Williams et al., 2013) or impair hypothesis testing (no inflated type I error rate, e.g., Bishara & Hittner, 2012; Ives, 2015; Puth et al., 2014; Ramsey & Schafer, 2013; Szöcs & Schäfer, 2015; Warton et al., 2016) even at relatively small sample sizes.”
From Violating the normality assumption may be the lesser of two evils (2021) · cites this paper as Bishara & Hittner, 2012
In the citing paper
“We focus on fairly drastic non-normality because only little bias can be expected from minor violations (Bishara & Hittner, 2012; Glass et al., 1972; Hack, 1958; Puth et al., 2014).”
From Violating the normality assumption may be the lesser of two evils (2021) · cites this paper as Bishara & Hittner, 2012
In the citing paper
“it may be most advisable to apply a rank-based inverse normal (RIN) transformation (aka rankit scores, Bliss, 1967) to the data, which can approximately normalize most distributional shapes and which effectively minimizes type I errors and maximizes statistical power (Bishara & Hittner, 2012; Puth et al., 2014).”
From Violating the normality assumption may be the lesser of two evils (2021) · cites this paper as Bishara & Hittner, 2012
In the citing paper
“All these deviations are generally thought to be of little concern (e.g., Bishara & Hittner, 2012), even if they are far from fitting to the bell-shaped curve, such as binomial data (Cochran, 1950).”
From Violating the normality assumption may be the lesser of two evils (2021) · cites this paper as Bishara & Hittner, 2012
Citations
Cites
Tabachnick, B. G., & Fidell, L. S. (1983) · Cited by 80,032
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences
Cohen, P., Cohen, P., West, S. G., & Aiken, L. S. (2014) · Psychology Press eBooks · doi:10.4324/9781410606266 · Cited by 20,737
Bootstrap methods: another look at the jackknife
Efron, B. (1979) · The Annals of Statistics · doi:10.1214/aos/1176344552 · Cited by 17,602
An Analysis of Transformations
Box, G. E. P., & Cox, D. R. (1964) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1964.tb00553.x · Cited by 15,052
Statistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
(2007) · Cited by 10,092
Educational Research: Competencies for Analysis and Application
Mills, G. E., & Airasian, P. W. (1995) · Medical Entomology and Zoology · Cited by 5,395
Fundamentals of Biostatistics.
(1969) · Annals of Internal Medicine · doi:10.7326/0003-4819-70-4-873 · Cited by 4,575
Cited by
Items 701–800 of 947
Generacijski vidik motivov turistov v gorah
Špindler, T. (2023) · Repository of the University of Primorsk (University of Primorska) · Cited by 1
Completing and studentising Spearman's correlation in the presence of tiesPreprint
Hurley, L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.23993 · Cited by 1
How Uncertain Is the Grade? A Benchmark of Uncertainty Metrics for LLM-Based Automatic AssessmentPreprint
Li, H., Yang, K., Long, X., Filippov, F., Chu, Y., Copur‐Gencturk, Y., He, P., Miller, C. S., Shin, N., Krajcik, J., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.16039 · Cited by 1
Babo-Rebelo, M. (2017) · theses.fr (ABES) · doi:10.70675/1811aa63zf6c7z4922zb08dz05dd2cee0e27 · Cited by 1
Individual differences in intelligence and personality guide human social learning
Burtenshaw, T., Walker, B., Gignac, G., Grueter, C. C., & Fay, N. (2026) · Personality and Individual Differences · doi:10.1016/j.paid.2026.113639
Hoeber, S. J. (2015) · Utrecht University Repository (Utrecht University)
Analyzing Effects of Public Communication onto Player Behavior in Massively Multiplayer Online Games
Lakkaraju, K., Bernstein, J., & Whetzel, J. H. (2014) · Springer proceedings in complexity · doi:10.1007/978-3-319-07142-8_3
Studies in Human Autobiographical Memory and Rodent Spatial Navigation
Gardner, R. S. (2015) · doi:10.13021/mars/5091
A NEW ROBUST APPROACH TO TESTING SIGNIFICANCE OF CORRELATION
Karahasan, M. (2014) · Erzincan Üniversitesi Fen Bilimleri Enstitüsü Dergisi · doi:10.18185/eufbed.16673
The Financial Accelerator and Countercyclical Interest Margins: Evidence from Norwegian Data
Johannessen, D. W. (2014) · NORA - Norwegian Open Research Archives
Brain-to-Brain (mind-to-mind) Interaction at distance: a pilot study v.2Preprint
Tressoldi, P. E., Pederzoli, L., Bilucaglia, M., Caini, P., Fedele, P., Ferrini, A., & Melloni, S. (2014) · SSRN Electronic Journal
Karamians, R. (2015) · Pepperdine Digital Commons (Pepperdine University)
Moore, K. A. (2016) · California State University ScholarWorks (system-wide DSpace)
Tollefson, D. (2016) · University of Minnesota Digital Conservancy (University of Minnesota)
Kruger, R. (2016) · Boloka Institutional Repository (North-west University)
Wulf, D., & Bertsch, V. (2016) · Munich Personal RePEc Archive (Ludwig Maximilian University of Munich)
Hemingway, A. (2017) · ScholarWorks (Boise State University) · doi:10.18122/b2dd8h
Spatial and temporal variability of carbon stocks within the River Colne Estuary
Tavallali, L. (2018) · Open Access at Essex (University of Essex)
Value-based decision making and alcohol use disorder
Nebe, S. (2018) · Qucosa (Saxon State and University Library Dresden) · doi:10.5167/uzh-172634
Ng, W. W., Panu, U. S., & Rasmussen, P. F. (2018) · World Environmental and Water Resources Congress (EWRI) · doi:10.1061/9780784481417.017
Physical Activity and Student Health: A Canadian University Perspective
Aktary, W. M. (2015) · ERA: Education and Research Archive (University of Alberta) · doi:10.7939/r3h708b6m
Murray, A. L. (2016) · ERA
The Stylometric Processing of Sensory Open Source Data
Kernot, D. (2018) · ANU Open Research (Australian National University) · doi:10.25911/5d611fc2972df
Chakraverty, V., & Zimmer, H. (2018) · Zeitschrift für Neuropsychologie · doi:10.1024/1016-264x/a000233
Analysis of the Federal District’s Legislative Chamber in the Information Age
Dias, W. L., Reyes, E., & Reis, A. L. N. (2019) · Journal of Legislative Studies · doi:10.1080/13572334.2019.1570610
Motivation and consumption substitution by South African football fans
Moosa, Y. (2017) · UpSpace Institutional Repository (University of Pretoria)
The Effect of service quality on customer loyalty: a case of the Laundromatt Company in Nairobi.
Wanyoike, S. (2018) · SU+ Digital Repository (Strathmore University)
Viraraghavan, V. R., & Seth, A. (2019) · Clinical Nutrition ESPEN · doi:10.1016/j.clnesp.2019.03.010
Crosby, H. (2018) · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York)
Kibanya, W. (2018) · SU+ Digital Repository (Strathmore University)
Tranter, C. L. (2019) · WestminsterResearch (University of Westminster) · doi:10.34737/qv980
A Rebalancing Chinese Economy: Drivers and ChallengesPreprint
Ma, G., Roberts, I., & Kelly, G. (2017) · RePEc: Research Papers in Economics
Donati, G. (2019) · Birkbeck Institutional Research Online (BiRO) - Theses · doi:10.18743/pub.00040401
Detecting Differentially Co-Expressed Gene Modules Via The Edge-Count Test
Lin, A. G. (2019) · Journal of the Arkansas Academy of Science
Putting Workplace Demography in Context : A Cross-Level Perspective
Reinwald, M. (2020) · KOPS (University of Konstanz)
Facundo, M. (2020) · National University System Repository (National University System)
Teaching Kids to Care: A Needs-Based Intervention to Increase Ethical Sensitivity in Schools
Friedman, R. (2018) · W&M Publish (College of William & Mary)
Joint Dynamics of Investor Networks and Stock PricesPreprint
Le, V. H. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3575580
Host communities' social impact perceptions and involvement in tourism: Mopani District Municipality
Mokgope, T. (2020) · Boloka Institutional Repository (North-west University)
Wallace, E. S. (2020) · VCU Scholars Compass (Virginia Commonwealth University) · doi:10.25772/tmyt-3h67
Jiménez-Gutiérrez, S. V. (2020) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas)
Ebute, E. V. (2020) · University Of Wales Trinity Saint David Research Repository (University of Wales Trinity Saint David) · doi:10.82227/repository.uwtsd.ac.uk.00001525
Wang, X. (2021) · Tuwhera (Auckland University of Technology)
The acute and habitual effects of alcohol consumption on sleep in late adolescents
Gourlay, C. G. (2020) · Minerva Access (University of Melbourne)
Lee, B. (2020) · Digital Commons-TMC (Texas Medical Center)
Handwashing : population-tailored promotion and the problem of self-reporting
Contzen, N. (2015) · DORA Eawag (Swiss Federal Institute of Aquatic Science and Technology (Eawag)) · doi:10.5167/uzh-111318
Frost, N. J. (2021) · UWA Profiles and Research Repository (UWA) · doi:10.26182/yr6c-jk30
Wishful Thinking in Macroeconomic ExpectationsPreprint
Li, K. K., & Huang, B. (2021) · RePEc: Research Papers in Economics
Karasiński, J., & Zduńczak, P. (2021) · Journal of Economics and Management · doi:10.22367/jem.2021.43.10
From Simple to Sophisticated: The Organization of Terrorist Groups
Logan, M. K. (2020) · DigitalCommons@UNO
Design Your TrustworthinessPreprint
Geng, S., & Guan, M. (2020) · RePEc: Research Papers in Economics
Ree, M. (2019) · ScholarWorks - UA (University of Alaska System)
Pluck, G. C., & Haro, K. I. (2021) · Revista Latinoamericana de Psicología · doi:10.14349/rlp.2021.v53.17
(2021) · Elsevier eBooks · doi:10.1016/b978-0-12-820098-8.00019-1
Kramer, J., Carrasco, M. Á. L., & Kümmerli, R. (2020) · FEMS Microbiology Ecology · doi:10.1093/femsec/fiaa250
Liu, J., & liu, H. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4067313
Calle, J., Guzmán, E., Lima, J.-F., Patiño-León, A., Orellana, M., & Cedillo, P. (2022) · Communications in computer and information science · doi:10.1007/978-3-030-99170-8_23
(2017) · Elsevier eBooks · doi:10.1016/b978-0-12-804733-0.00018-4
Coping With Racial Microaggressions: The Moderating Effects of Parenting Style
Talley, Q., & Dunn, M. G. (2014) · PsycEXTRA Dataset · doi:10.1037/e540762014-001
Jurnal Psikologi Indonesia Vol 12 No 1 2017 (JPI Himpsi)Preprint
Abraham, J. (2017) · INA-Rxiv (OSF Preprints) · doi:10.31227/osf.io/2nvkr
Brain-to-Brain (mind-to-mind) interaction at distance: a confirmatory study
Tressoldi, P. E., Pederzoli, L., Bilucaglia, M., Caini, P., Fedele, P., Ferrini, A., Melloni, S., Richeldi, D., Richeldi, F., & Accardo, A. (2014) · F1000Research · doi:10.12688/f1000research.4336.1
Bodratti, A. M., He, Z., Tsianou, M., Cheng, C., & Alexandridis, P. (2017) · IGI Global eBooks · doi:10.4018/978-1-5225-1798-6.ch021
Advances in Multi-Sensor Information Fusion: Theory and Applications 2017
Jin, X., Sun, S., Wei, H., & Yang, F. (2018) · MDPI eBooks · doi:10.3390/books978-3-03842-934-0
Genelleştirilmiş Toplamsal Modellerin Zooteknide Kullanımı
KURDAL, N., & Önder, H. (2022) · Black Sea Journal of Engineering and Science · doi:10.34248/bsengineering.1075808
A Monte Carlo Simulation of Rat Choice Behavior with Interdependent Outcomes
Frankot, M. A. (2022) · The Research Repository @ WVU (West Virginia University) · doi:10.33915/etd.11446
Raznatovska, О. М., & Myronchuk, Y. V. (2022) · Current issues in pharmacy and medicine science and practice · doi:10.14739/2409-2932.2022.2.255746
Sural nerve conduction study: Reference values in the Algerian population
Zamoum, M., & Athmane, C. (2022) · Journal of Neuroscience and Neurological Disorders · doi:10.29328/journal.jnnd.1001067
(2022) · Natural hazards and earth system sciences · doi:10.5194/nhess-2022-244-rc1
(2022) · Natural hazards and earth system sciences · doi:10.5194/nhess-2022-244-rc2
(2022) · Natural hazards and earth system sciences · doi:10.5194/nhess-2022-244-rc3
Zhang, M. (2022) · Natural hazards and earth system sciences · doi:10.5194/nhess-2022-244-ac2
Zhang, M. (2022) · Natural hazards and earth system sciences · doi:10.5194/nhess-2022-244-ac1
Zhang, M. (2022) · Natural hazards and earth system sciences · doi:10.5194/nhess-2022-244-ac3
(2022) · Natural hazards and earth system sciences · doi:10.5194/nhess-2022-244-rc4
Zhang, M. (2023) · Natural hazards and earth system sciences · doi:10.5194/nhess-2022-244-ac4
The feasibility of immunocorrective therapy in the treatment of children with new tuberculosis cases
Myronchuk, Y. V., & Raznatovska, О. М. (2022) · Zaporozhye Medical Journal · doi:10.14739/2310-1210.2022.5.256227
Wilcox, R. R. (2023) · Springer eBooks · doi:10.1007/978-3-031-41713-9_9
Currency Exchange Rate Forecasting Using Support Vector Classifiers
Koutsellis, T., Makarouni, I., Choumas, S., Ververidis, C. N., Papapolyzos, T., Bitsikas, A., & Doukas, H. (2023) · International Conference on Information, Intelligence, Systems & Applications (IISA) · doi:10.1109/iisa59645.2023.10345946
Paantjens, M. A. (2024) · Utrecht University Repository (Utrecht University) · doi:10.33540/2161
DeepTQ: Predictive TSN Switch Queue Length Based on Deep Learning
Jia, Y., Long, X., Xiong, W., Wang, X., Shang, Z., & Yuan, Z. (2024) · Lecture notes in electrical engineering · doi:10.1007/978-981-97-3336-1_1
Bone quality and incidence of urolithiasis in male broiler breeders fed a male or female ration.
Dube, S. (2024) · ResearchSpace (University of KwaZulu-Natal) · doi:10.29086/10413/23098
Analysis Of Pyrolysis Products of Cotton Straw Based on Reaction Kinetics and Random Forests
Yu, L., Cai, Y., Bao, S., Cha, J., & Lv, R. (2024) · Highlights in Science Engineering and Technology · doi:10.54097/bs5ba928
Balogh, S. J., Sprouse, G., Beckman, K. B., Watson, R. H. B., Johnson, D. M., Pinkerton, L. D., Nollet, Y. H., Sealock, A. W., Atkins, W. S. C., Selenke, L. M., et al. (2024) · medRxiv · doi:10.1101/2024.08.21.24311866
Eating Behaviors Associated With Suicidal Behaviors and Overall Risk
Sutphin, T. M. S., Hicks, A. D., & McCord, D. (2024) · Journal of Psychiatric Practice · doi:10.1097/pra.0000000000000809
A Comparative Analysis of Recent Tests for Multivariate Normality
Chukwudozie, C. E., & Umeh, E. U. (2024) · European Journal of Theoretical and Applied Sciences · doi:10.59324/ejtas.2024.2(5).32
Hortegal, R. A. (2023) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/t.98.2023.tde-22102024-154342
A Novel Prognostic Model for DLBCL Patients Based on Cuproptosis-Related Genes
li, F., Cai, J., Li, J., Rao, J., Dong, S., Lin, S., Xiang, X., Zhang, X., & Gao, L. (2024) · Blood and genomics discovery. · doi:10.70322/bg20240110002
Kerman, M., & Özdemir, M. (2024) · Research Square · doi:10.21203/rs.3.rs-5611398/v1
Gao, Z., Han, J., Chen, C., Zhang, Q., & Wang, W. (2025) · Postgraduate Medical Journal · doi:10.1093/postmj/qgaf009
Integrating Mechanism-Driven and Data-Driven Approaches forAlgal Growth Modeling in Small WatershedsPreprint
Liu, Y., Ren, Z., Chen, Q., Jing, Z., Wang, Y., & Chen, Y. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5134087
A Statistical Method Assessing the Influence of Cobots' Technical Parameters on Performance Indices
Zhao, Z., & Xie, B. (2024) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio64047.2024.10907444
(2025) · Frontiers research topics · doi:10.3389/978-2-8325-6092-1
Metadata
- Authors
- Bishara, A. J., & Hittner, J. B.
- Affiliations
- College of Charleston (US)
- Venue
- Psychological Methods
- Published
- 2012-09
- Volume
- 17
- Issue
- 3
- Pages
- 399-417
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2111495793
- Cited by
- 766
- References
- 98
- Citations by year
- 2026: 80 · 2025: 113 · 2024: 88 · 2023: 77 · 2022: 86 · 2021: 53 · 2020: 64 · 2019: 52 · 2018: 34 · 2017: 40 · 2016: 23 · 2015: 25 · 2014: 23 · 2013: 5 · 2012: 3
- FWCI
- 20.74
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Keywords
- non-normal data, Pearson correlation, data transformation, permutation tests, bootstrap tests, type I error, statistical power, small sample size
- MeSH terms
- Data Interpretation, Statistical, Monte Carlo Method, Statistics as Topic, Statistical Distributions, Linear Models, Nonlinear Dynamics, Sample Size, Statistics, Nonparametric
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.