Testing the significance of a correlation with nonnormal data: Comparison of Pearson, Spearman, transformation, and resampling approaches.
The DOI matches its Crossref record; 99 of the 99 references it cites were found in a registry or an index.
Abstract
It is well known that when data are nonnormally distributed, a test of the significance of Pearson's r may inflate Type I error rates and reduce power. Statistics textbooks and the simulation literature provide several alternatives to Pearson's correlation. However, the relative performance of these alternatives has been unclear. Two simulation studies were conducted to compare 12 methods, including Pearson, Spearman's rank-order, transformation, and resampling approaches. With most sample sizes (n ≥ 20), Type I and Type II error rates were minimized by transforming the data to a normal shape prior to assessing the Pearson correlation. Among transformation approaches, a general purpose rank-based inverse normal transformation (i.e., transformation to rankit scores) was most beneficial. However, when samples were both small (n ≤ 10) and extremely nonnormal, the permutation test often outperformed other alternatives, including various bootstrap tests.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Deviations from normality usually do not bias regression coefficients (Ramsey & Schafer, 2013; Williams et al., 2013) or impair hypothesis testing (no inflated type I error rate, e.g., Bishara & Hittner, 2012; Ives, 2015; Puth et al., 2014; Ramsey & Schafer, 2013; Szöcs & Schäfer, 2015; Warton et al., 2016) even at relatively small sample sizes.”
From Violating the normality assumption may be the lesser of two evils (2021) · cites this paper as Bishara & Hittner, 2012
In the citing paper
“We focus on fairly drastic non-normality because only little bias can be expected from minor violations (Bishara & Hittner, 2012; Glass et al., 1972; Hack, 1958; Puth et al., 2014).”
From Violating the normality assumption may be the lesser of two evils (2021) · cites this paper as Bishara & Hittner, 2012
In the citing paper
“it may be most advisable to apply a rank-based inverse normal (RIN) transformation (aka rankit scores, Bliss, 1967) to the data, which can approximately normalize most distributional shapes and which effectively minimizes type I errors and maximizes statistical power (Bishara & Hittner, 2012; Puth et al., 2014).”
From Violating the normality assumption may be the lesser of two evils (2021) · cites this paper as Bishara & Hittner, 2012
In the citing paper
“All these deviations are generally thought to be of little concern (e.g., Bishara & Hittner, 2012), even if they are far from fitting to the bell-shaped curve, such as binomial data (Cochran, 1950).”
From Violating the normality assumption may be the lesser of two evils (2021) · cites this paper as Bishara & Hittner, 2012
Citations
Cites
Tabachnick, B. G., & Fidell, L. S. (1983) · Cited by 80,032
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences
Cohen, P., Cohen, P., West, S. G., & Aiken, L. S. (2014) · Psychology Press eBooks · doi:10.4324/9781410606266 · Cited by 20,737
Bootstrap methods: another look at the jackknife
Efron, B. (1979) · The Annals of Statistics · doi:10.1214/aos/1176344552 · Cited by 17,602
An Analysis of Transformations
Box, G. E. P., & Cox, D. R. (1964) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1964.tb00553.x · Cited by 15,052
Statistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
(2007) · Cited by 10,092
Educational Research: Competencies for Analysis and Application
Mills, G. E., & Airasian, P. W. (1995) · Medical Entomology and Zoology · Cited by 5,395
Fundamentals of Biostatistics.
(1969) · Annals of Internal Medicine · doi:10.7326/0003-4819-70-4-873 · Cited by 4,575
Cited by
Items 801–900 of 947
Jurnal Psikologi Indonesia Vol 12 No 1 2017 (JPI Himpsi)Preprint
Abraham, J. (2017) · INA-Rxiv (OSF Preprints) · doi:10.31227/osf.io/2nvkr_v1
Locus de control y optimismo atributivo en emprendedores universitarios en contexto de post Covid-19
Viladegut Salazar, M. D. P. (2024) · Universidad Peruana de Ciencias Aplicadas (UPC) · doi:10.19083/tesis/676196
What do we know about highly-cited papers in medical informatics? Reasons or attributes
Fahimifar, S., Amoozgar, A., Soleimanfar, M. H., Mozaffari, F., Ausloos, M., Gholampour, B., & Gholampour, S. (2025) · Quality & Quantity · doi:10.1007/s11135-025-02218-3
Terrain music spectrum: An effective model for landform interpretation and recognition
Sun, Z., Chen, N., Wang, C., & Li, W. (2025) · Progress in Physical Geography Earth and Environment · doi:10.1177/03091333251349991
Koster, E., & Stuhlmacher, M. (2025) · American Journal of Undergraduate Research · doi:10.33697/ajur.2025.142
An Experimental Comparison of Three Smaa Methods Representing Distinct Preference RationalitiesPreprint
Pereira, J., Contreras, P., Figueroa, R., & Gacitúa, R. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5367297
Hailu, K. T. (2025) · African Journal of Sports and Physical Sciences · doi:10.62154/ajsps.2025.05.01011
Sukbumrung, Y., Wong, N. H., Budhraja, S., & Pluck, G. C. (2025) · International STEM Education Conference (iSTEM-Ed) · doi:10.1109/istem-ed65612.2025.11129263
Johnson, L. D. (2025) · University of North Texas Digital Library (University of North Texas) · doi:10.12794/metadc2482400
Merdiwijaya, B., Sarma, M., & Hidayatulloh, F. S. (2025) · Dinasti International Journal of Economics Finance & Accounting · doi:10.38035/dijefa.v6i5.5526
Zhang, C., Xiao, X., Wang, Y., Hu, W., & Jiang, S. (2025) · IEEE Power Engineering Society General Meeting · doi:10.1109/pesgm52009.2025.11225532
Transfer Effects of Long-Term and High-Frequency Web Service Use on Target-Pointing Performance
Yamanaka, S., & Taguchi, H. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3773066
Setiawan, A. N., Pan, H.-H., Griffiths, S., McQueen, D., Exton, M., Khan, J. R., & Herbert, N. A. (2025) · Aquaculture Research · doi:10.1155/are/8898144
Cîrstian, R., Forde, N. J., Bučková, B. R., Dell’Acqua, F., Charman, T., Loth, E., Oakley, B. F. M., Colomar, L., Yorke, I., Stones, R., et al. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.11.25.690402
Yu, T., Zhang, Y., Yuan, J., Zhang, Y., Li, J., & Huang, Z. (2024) · Figshare · doi:10.6084/m9.figshare.26713960
Wagland, Z. (2024) · Edinburgh Napier Research Repository (Edinburgh Napier University) · doi:10.17869/enu.2024.3789763
Morlat, P. V., & Chassy, P. (2024) · Urban Crime. An international Journal · doi:10.26250/heal.panteion.uc.v5i2.359
Walker, D. L. (2023) · UWA Profiles and Research Repository (UWA) · doi:10.26182/mr20-t308
Smirnov, P., Smith, I. C. P., Safikhani, Z., Ba-alawi, W., Khodakarami, F., Lin, E., Yu, Y., Martin, S. E., Ortmann, J., Aittokallio, T. A., et al. (2022) · Figshare · doi:10.6084/m9.figshare.19793292
Wayessa, D. I., Chala, M. B., Demissie, S. F., Abebe, A. B., Janakiraman, B., Deme, S., & Gashaw, M. (2023) · Figshare · doi:10.6084/m9.figshare.22599623
Hölgyesi, Á., Zrubka, Z., Gulácsi, L., Baji, P., Haidegger, T. P., Kozlovszky, M., Weszl, M., Kovács, L., & Péntek, M. (2024) · Figshare · doi:10.6084/m9.figshare.25511856
Hölgyesi, Á., Zrubka, Z., Gulácsi, L., Baji, P., Haidegger, T. P., Kozlovszky, M., Weszl, M., Kovács, L., & Péntek, M. (2024) · Figshare · doi:10.6084/m9.figshare.25511889
SARS-CoV-2 Seroprevalence and Vaccine Correlate of Protection Standardization
Rosin, S. P. (2023) · Carolina Digital Repository (University of North Carolina at Chapel Hill) · doi:10.17615/wesd-wq90
General trust in middle childhood: Measurement, age-related differences, and psychosocial correlates
Wheeler, A. R. (2025) · UWA Profiles and Research Repository (University of Western Australia) · doi:10.26182/4jpc-2887
den Houting, J. (2018) · Griffith Research Online · doi:10.25904/1912/2941
The Socio-Technical Dynamics of Renewable Energy Policies in Germany
Dehler-Holland, J. (2023) · Repository KITopen (Karlsruhe Institute of Technology) · doi:10.5445/ir/1000158922
Wayessa, D. I., Chala, M. B., Demissie, S. F., Abebe, A. B., Janakiraman, B., Deme, S., & Gashaw, M. (2023) · Figshare · doi:10.6084/m9.figshare.22599626
Hölgyesi, Á., Zrubka, Z., Gulácsi, L., Baji, P., Haidegger, T. P., Kozlovszky, M., Weszl, M., Kovács, L., & Péntek, M. (2024) · Figshare · doi:10.6084/m9.figshare.26699621
Chala, M. B., Donnelly, C., Wondie, Y., Ghahari, S., & Miller, J. (2021) · Figshare · doi:10.6084/m9.figshare.13642138
CEO Compensation, Financial Performance, and Donor Contributions in Nonprofit Hospitals
Beal, C. P. (2024) · ScholarWorks (Walden University)
do Amaral, F. P. D. M. (2012) · Americanae (AECID Library)
Abramović, V. (2024) · Repositório do ISCTE-IUL
Comparative analysis of methods for microbiome study
Iyer, M. V. (2020) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign)
The Relationship between Morphological Awareness and Literacy Skills in German
Haase, A. (2020) · Common Library Network (Der Gemeinsame Bibliotheksverbund)
Social Disorder and Level of Engagement of Small Business Owners in the Virgin Islands
Santiago, S. (2015) · ScholarWorks (Walden University)
Gysi, D. M. (2019) · Qucosa (Saxon State and University Library Dresden)
Relationship between gluten strength and free asparagine content of Canadian wheat varieties
Trevisan, S. (2022) · Mspace (University of Manitoba)
The development of perceptual priors
Kiryakova, R. (2021) · Durham e-Theses (Durham University)
Cox, F. E. (2020) · Repository@Hull (Worktribe) (University of Hull)
Georgsson Westergren, L. (2024) · DiVA (Digitala Vetenskapliga Arkivet)
Calderón, D. G. (2019) · Communities in DSpace (Pontifical Catholic University of Peru)
Exploring the effects of homework on secondary school adolescents’ leisure
Blaszko, A. (2023) · Nottingham Trent University's Institutional Repository (Nottingham Trent Repository)
La precisión del recuerdo de patrones de juego en jugadores infantiles (sub-13 años) de balonmano
del Campo, V. L., González, C., & Morenas, J. (2020) · Institutional Repository University of Extremadura (University of Extremadura)
On Monte Carlo simulation algorithms for research in psychometrics
Astivia, O. L. O. (2017) · cIRcle (University of British Columbia)
Mlomo, Z. W. (2025) · Open University of Cape Town (University of Cape Town)
Economic Impact of ESG-initiatives on Company Results
Hanserot, A., Dahlström, A., & Lilja, W. (2024) · DiVA (Digitala Vetenskapliga Arkivet)
Parker, Z. (2025) · Open University of Cape Town (University of Cape Town)
Vivian, K. (2024) · University of Malaya Students Repository
SILVANA NOOR HARAHAP (2025) · Repository UIN Sultan Syarif Kasim Riau (Universitas Islam Negeri Sultan Syarif Kasim Riau)
A Mechanism-Data Fusion Model for Algal Growth in Hydrologically Constrained River SegmentsPreprint
Liu, Y., Ren, Z., Jing, Z., Liu, J., Chen, Q., Wang, Y., He, L., & Chen, Y. (2025) · Research Square · doi:10.21203/rs.3.rs-8517267/v1
Stress and the Farmer’s Profile: Demographic and Structural Divides
Vveinhardt, J., & Žukovskis, J. (2025) · European Countryside · doi:10.2478/euco-2025-0035
The Impact of Psychological Mindedness on Executive Coaching Effectiveness
Orban, J. (2026) · Springer eBooks · doi:10.1007/978-3-032-11842-4_8
Ferreira, A. C. S. (2023) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas)
Duarte, B. L. (2024) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas)
Retrospective study of scorpionism in the State of Goiás (2003-2019)
de Alcantara, R. L. (2023) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas)
Validation of the Ethnic Identity Scale With Americans of Arab Heritage
Atari-Khan, R., Azmat, A., Elhedk, F. R., & Han, S. (2026) · The Counseling Psychologist · doi:10.1177/00110000251414169
Manan, A. A., Shifa, M., Mahrizal, & Wahyu, A. (2025) · Jurnal Riset Ilmu Ekonomi dan Bisnis · doi:10.29313/jrieb.v5i2.7899
Jayathilakan, A., Ngo, T., Tsui, K. W. H., Redmayne, N. B., Balli, F., & Fu, X. (2026) · Case Studies on Transport Policy · doi:10.1016/j.cstp.2026.101736
Speech Perception and Language Development in Individuals with Special Educational Needs
(2026) · Frontiers research topics · doi:10.3389/978-2-8325-7515-4
Prior preferences interfere with the associative learning of food values
Rich, A., Kapadia, S., Henry, R., Dan, O., & Levy, I. (2026) · Communications Psychology · doi:10.1038/s44271-026-00533-5
What information could teleconnections provide on flow variability in Spain?
Gaitán, E. (2023) · e_Buah
Rosensteel, R. E. (2026) · ScholarWorks (Walden University)
Chen, P., Wu, S., Zhang, Y., & Gao, L. (2026) · Sustainability · doi:10.3390/su18052608
Apego entre iguales y su relación con la percepción del sexismo en adolescentes
Espinoza, A. N. P., Parillo, J. R. C., Collado, R. A. C., Soto-Yana, R., Jaramillo-Saavedra, E., & Sierra-Delgado, M. S. (2026) · Revista Estudios Psicológicos · doi:10.35622/j.rep.2026.01.001
Soutelo, A. P. D. S. (2025) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT)
Impacto da carga física e da rotação de tarefas na perceção de esforço, força de preensão e dor
Silva, G. D. S. J. M. (2025) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT)
Zember, S., Bodulić, K., Balent, N. C., Markotić, A., & Rode, O. Đ. (2026) · Vaccines · doi:10.3390/vaccines14040305
Cougoul, A. (2019) · theses.fr (ABES) · doi:10.70675/838c1161zf826z44eaz885fz96dc69cdb923
Waldeck, J. B. (2026) · Electronic Theses of LMU Munich (Ludwig-Maximilians-Universität München) · doi:10.5282/edoc.36769
METHODOLOGYCAL APPROACH TO EMPIRICAL ANALYSIS OF URBAN INFRASTRUCTURE SUSTAINABILITY DETERMINANTS
Нєшева, А., & Smolennikov, D. (2026) · Економіка та суспільство · doi:10.32782/2524-0072/2026-85-29
Yang, Y., Conde, S., & Bhullar, R. (2026) · Journal of Applied Research in Intellectual Disabilities · doi:10.1111/jar.70234
Rho, K., Kataria, A., Birch, S., & LeMoult, J. (2026) · Behavioral Sciences · doi:10.3390/bs16050705
Target Gene Prediction From Microarray Data Using Data Mining Methods
Çavdar, Z. Y., Sertbaş, A., Ensarı, T., & Akçaman, M. N. (2026) · GAZI UNIVERSITY JOURNAL OF SCIENCE · doi:10.35378/gujs.1710419
Mosalla, S., Jeon, S. Y., Kim, J., Lee, T., & Lee, J.-B. (2026) · Applied Sciences · doi:10.3390/app16115382
Sibilia, F., Fotouhi, M., Lei, X., Cabeen, R. P., Cen, S., Choupan, J., Hazany, S., Sheikh‐Bahaei, N., & Initiative, A. D. N. (2026) · Journal of Alzheimer s Disease · doi:10.1177/13872877261450973
Gare, G., Galeotti, J., Mozer, M., Ramanan, D., & Ke, N. R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.03251
Ni, Y., Cao, Y., Lin, H., Guo, T., & Wen, X. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1809532
Fadliyah, L., Tyas, A. P. M., Sulpat, E., Mardhika, A., Laili, A. N., & Rua, Y. M. (2026) · Journal of Vocational Nursing · doi:10.20473/jovin.v7i1.88000
A SMAA-based framework for robustness analysis in multi-criteria facility location under uncertainty
Pereira, J., Contreras, P., Taramasco, C., Figueroa, R., & Gacitúa, R. (2026) · Transportation Research Part E Logistics and Transportation Review · doi:10.1016/j.tre.2026.105002
Aliyev, R., Backhaus, J., Westphale, S., & Koenig, S. (2026) · JMIR Medical Education · doi:10.2196/75516
VMDSK-TransNeXt enhanced U-Net for in-situ stress prediction of well logging data
Liu, N., Mei, K., Liu, Y., Wu, H., & Gao, J. (2026) · Journal of Applied Geophysics · doi:10.1016/j.jappgeo.2026.106361
Dealing with partial missing correlations in multivariate and surrogate meta-analysesPreprint
Santis, R. D., & Guolo, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.25670
Promoting breastfeeding through digital communication: a longitudinal randomized controlled trial
Guidetti, M., Scaglioni, G., Cavazza, N., Reggiani, R., Infante, R., Molinazzi, M. T., & Neri, I. (2026) · Health Psychology and Behavioral Medicine · doi:10.1080/21642850.2026.2685914
Phonkaphon, S., Songkohb, E., Sirichunchuen, K., & Unahalekhaka, P. (2026) · Clean Energy Science and Technology · doi:10.18686/cest546
Guzmán-Castro, V., Razo-Zárate, R., Sandoval, O. A. A., Rodríguez-Laguna, R., Capulín-Grande, J., & Rodríguez, M. Á. R. (2026) · Revista Internacional de Contaminación Ambiental · doi:10.20937/rica.55649
Tagle, F. (2026) · Psychology and Education A Multidisciplinary Journal · doi:10.70838/pemj.580708
Basali, M. A. Q. (2022) · Victoria University Research Repository (Victoria University)
Khoirunnisa, A., Adiwinata, A. H., & Budiman, N. (2026) · Berkala Ilmiah Pendidikan · doi:10.51214/bip.v6i2.1930
Tracking Air Pollution in Peri-Urban Hyderabad, India Using Remote Sensing Data
Bhushan, R., Bharadwaj, R., Gupta, M., Suresh, L., & Clune, T. S. (2025) · IEEE India Geoscience and Remote Sensing Symposium (InGARSS) · doi:10.1109/ingarss67683.2025.11583972
Khatri, P. H., Newton, M. A., Newton, M. A., & Dinh, H. Q. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.07.06.736800
A Distributed CatBoost Approach with Weighted Aggregation for STI/HIV Risk PredictionPreprint
Tang, Z., Li, Y., Şengür, A., & Acharya, U. R. (2026) · Research Square · doi:10.21203/rs.3.rs-10354103/v1
Creation and Validation of the Madrid Loneliness Questionnaire in Spanish Population
Esteban-Rodríguez, L., González-Espejito, F., Zubiaur, E. D., Sevilla-Martínez, J., León, A. H., Lillo-López, E., Blasco-Novalbos, G., & Pérez, E. J. P. (2026) · The Spanish Journal of Psychology · doi:10.1017/sjp.2026.10043
Development of a Thai Lexical Decision Task as a Tool for Premorbid Estimation of Cognitive Function
Chonwattanagul, S., Nugoolsuksiri, N., Jan, R., Rodríguez, D. R., Chantavarin, S., & Pluck, G. C. (2026) · Journal of the Medical Association of Thailand · doi:10.35755/jmedassocthai.2026.7.03092
Ayan, M. A. (2026) · Sayıştay Dergisi · doi:10.52836/sayistay.1935827
Calubag, L. V. (2026) · Journal of Research in Education and Pedagogy. · doi:10.70232/jrep.v3i3.197
Haas, C. N. (2026) · Wiley eBooks · doi:10.1002/9781394253418.ch13
Metadata
- Authors
- Bishara, A. J., & Hittner, J. B.
- Affiliations
- College of Charleston (US)
- Venue
- Psychological Methods
- Published
- 2012-09
- Volume
- 17
- Issue
- 3
- Pages
- 399-417
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2111495793
- Cited by
- 766
- References
- 98
- Citations by year
- 2026: 80 · 2025: 113 · 2024: 88 · 2023: 77 · 2022: 86 · 2021: 53 · 2020: 64 · 2019: 52 · 2018: 34 · 2017: 40 · 2016: 23 · 2015: 25 · 2014: 23 · 2013: 5 · 2012: 3
- FWCI
- 20.74
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Keywords
- non-normal data, Pearson correlation, data transformation, permutation tests, bootstrap tests, type I error, statistical power, small sample size
- MeSH terms
- Data Interpretation, Statistical, Monte Carlo Method, Statistics as Topic, Statistical Distributions, Linear Models, Nonlinear Dynamics, Sample Size, Statistics, Nonparametric
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.