Choice of test for association in small sample unordered r × c tables
Open accessAbstract
Pearson's chi-squared, the likelihood-ratio, and Fisher-Freeman-Halton's test statistics are often used to test the association of unordered r x c tables. Asymptotical, exact conditional, or exact conditional with mid-p adjustment methods are commonly used to compute the p-value. We have compared test power and significance level for these test statistics and p-value calculations in small sample r x c tables, mostly 3 x 2 and some with both r and c are greater than 2. After extensive simulations, in general we recommend using an exact conditional mid-p test with Pearson's chi-squared or Fisher-Freeman-Halton's statistic, which usually is the most powerful test yet preserve the approximate significance level. Moreover, we recommend that the asymptotic Pearson's chi-squared or other asymptotic tests not be used for small sample r x c tables.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 10 cited sources checked · 0 supported · 0 flagged
- Findingown data
In most cases, the mid-p test outperforms the asymptotic test.
“In most cases, the mid-p test outperforms the asymptotic test.”
Quote in the paperSubstantial claim - Findingown data
Asymptotic tests are unreliable for small samples.
“Moreover, the asymptotic tests are unreliable for small samples.”
- Also stated in: § 7. DISCUSSION AND CONCLUSIONS · plain
“Moreover, we found that asymptotic tests are less reliable in small samples, as at times, their significance level may fluctuate substantially.”
Quote in the paperSubstantial claim - Also stated in: § 7. DISCUSSION AND CONCLUSIONS · plain
- Findingown data
Among asymptotic, exact, and exact mid-p tests, the exact mid-p version usually has a significance level closer to the nominal level.
“Among the asymptotic, exact and exact mid-p tests, we find that the exact mid-p version usually has a significance level that is closer to this ideal, as shown in Figures 12345and in Table IV.”
Quote in the paperSubstantial claim - Findingown data
For small samples, the significance level of an asymptotic test may be substantially less than or greater than the nominal level.
“On the other hand, for small samples, the significance level of an asymptotic test may be substantially less than or greater than .”
Quote in the paperSubstantial claim - Findingnone given
The power of the mid-p test tends to be close to the power of the randomized tests.
“Typically, the power of the mid-p test tends to be close to the power of the randomized tests.”
Quote in the paperSubstantial claim
- Findingcitation
The choice of test statistic for exact or mid-p tests has little influence on the significance level.
“The choice of test statistic for the exact or mid-p tests has little influence on the significance level (results are not shown here).”
- Also stated in: § 7. DISCUSSION AND CONCLUSIONS · plain
“The choice of test statistic in exact or mid-p tests does not substantially influence the results, as earlier pointed out by Lydersen et al. [6].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [6] 6. Lydersen S, Pradhan V, Senchaudhuri P, Laake P. Comparison of exact tests for association in unordered contingency tables using standard, mid-p, and randomized test versions. Journal of Statistical Computation and Simulation 2005; 75(6):447-458. source
- Also stated in: § 7. DISCUSSION AND CONCLUSIONS · plain
- Findingown data
In relatively few cases, the asymptotic test slightly outperforms the exact mid-p test.
“In relatively few cases, the asymptotic test slightly outperforms the exact mid-p test (see Figure 3).”
Quote in the paperSubstantial claim - Findingown data
In asymptotic versions of the tests, Fisher's asymptotic test tends to be more conservative than Pearson's asymptotic test.
“However, in the asymptotic versions of the tests, Fisher's asymptotic test tends to be more conservative than Pearson's asymptotic test (Figure 1 versus Figure 2).”
Quote in the paperSubstantial claim - Interpretationreasoning
With increasing numbers of rows or columns, the number of possible outcomes conditional on the margin sums increases and diminishes the difference between the exact and mid-p tests.
“With increasing numbers of rows or columns, the number of possible outcomes conditional on the margin sums increases and diminishes the difference between the exact and mid-p tests.”
Quote in the paperNot a substantial claim - Premisecitation
In most practical designs, Pearson's chi-squared statistic and Fisher's statistic perform slightly better than the LR statistic in exact conditional, exact conditional mid-p, and randomized tests in small to moderate samples.
“Lydersen et al. [6] have compared the test statistics for exact conditional, exact conditional mid-p, and randomized tests in small to moderate samples and have concluded that in most practical designs, Pearson's chi-squared statistic and Fisher's statistic perform slightly better than does the LR statistic.”
- Also stated in: § in Table IV(b). · plain
“But Pearson's and Fisher's statistics tend to perform slightly better than the LR statistic, as noted by Lydersen et al. [6].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [6] 6. Lydersen S, Pradhan V, Senchaudhuri P, Laake P. Comparison of exact tests for association in unordered contingency tables using standard, mid-p, and randomized test versions. Journal of Statistical Computation and Simulation 2005; 75(6):447-458. source
- Also stated in: § in Table IV(b). · plain
- Interpretationreasoning
The conclusions from simulations on 3 × 2, 4 × 3, and 3 × 9 tables extend to other r × c tables such as 3 × 3 and 4 × 2.
“Our simulations were performed for 3 × 2 tables, a 4 × 3 table and a 3 × 9 table.The above conclusions all imply similar trends, so we extend the recommendations to other r × c tables, such as 3 × 3, 4 × 2 and so forth.”
Quote in the paperSubstantial claim
- May combine two claims. 10
- The quote only partly states this claim. 4
- Classified as probably not a substantial claim. 9
May combine two claims.
Findingown dataFor Pearson's test in Simulation 1, the true power of asymptotic and mid-p tests tends to be higher than the asymptotically computed power when the power is above 0.5, and lower when the power is less than 0.5.
“We observe that the true power of the asymptotic and mid-p tests tends to be higher than the asymptotic power computed if the power is above 0.5 and the other way round if the power is less than 0.5.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe significance levels of exact tests can be substantially lower than the nominal level, demonstrating that exact tests are conservative.
“In general, we observe that the significance levels of the exact tests can be substantially lower than , which demonstrates that exact tests are conservative.”
- Also stated in: § 7. DISCUSSION AND CONCLUSIONS · boosted
“We have confirmed that exact tests are conservative for small samples.”
Quote in the paperSubstantial claim- Also stated in: § 7. DISCUSSION AND CONCLUSIONS · boosted
The quote only partly states this claim.
Findingown dataFor the renal allograft biopsy data in Table I, the exact mid-p test with Pearson's chi-squared or Fisher's statistic gives p = 0.0112 for the test of association between change in patient management and biopsy indication.
“In conclusion, we recommend using exact mid-p tests, with Pearson's chi-squared or Fisher's test statistics, which in Table II gives p = 0.0112 for the test of association between the change in patient management and biopsy indication.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [1] 1. Pascual M, Vallhonrat H, Cosimi AB, Tolkoff-Rubin N, Colvin RB, Delmonico FL, Ko DSC, Schoenfeld DA, Williams WW. The clinical usefulness of the renal allograft biopsy in the cyclosporine era-a prospective study. Transplantation 1999; 67(5):737-741. source
The quote only partly states this claim.
FindingcitationExact tests are less conservative for the 9 × 3 table than for smaller tables.
“But the exact tests are less conservative for the 9 × 3 table (Table IV(b)).”
- Also stated in: § 7. DISCUSSION AND CONCLUSIONS · hedged
“As seen for the 3 × 9 table in Table IV(b), exact tests tend to be less conservative as r or c increase, since the increasing number of possible values for the test statistic tends to increase [5; p. 98].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [5] 5. Andres AM, Quevedo MJS, Garcia JMT, Silva-Mato A. On the validity condition of the chi-squared test in 2 × 2 tables. Test 2005; 14(1):99-128. source
- Also stated in: § 7. DISCUSSION AND CONCLUSIONS · hedged
May combine two claims.
Findingown dataThe nominal level of a mid-p test is seldom exceeded, and if so, only slightly.
“We also observe that the nominal level of a mid-p test is seldom exceeded, and if so, only slightly.”
Quote in the paperSubstantial claimThe quote only partly states this claim.
Findingnone givenThe exact tests have a lower power than the mid-p tests.
“The exact tests have a lower power, as expected.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe asymptotic tests have a power close to that of the mid-p tests (Pearson) or somewhat lower (Fisher).
“The asymptotic tests have a power close to that of the mid-p tests (Pearson) or somewhat lower (Fisher), but as noted earlier, the asymptotic tests preserve test size to a lesser extent.”
Quote in the paperSubstantial claimMay combine two claims. The quote only partly states this claim.
Findingown dataIn some cases, asymptotic tests are extremely conservative, even more so than the mid-p tests, and the asymptotic version of Fisher's test seems very conservative.
“In some cases, the asymptotic tests are the most conservative of all.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationFor r × c tables, the mid-p type I error typically is approximately equal to or slightly less than the nominal level.
“For r × c tables, Lydersen et al. [6] found that the mid-p type I error typically is approximately equal to or slightly less than the nominal level.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [6] 6. Lydersen S, Pradhan V, Senchaudhuri P, Laake P. Comparison of exact tests for association in unordered contingency tables using standard, mid-p, and randomized test versions. Journal of Statistical Computation and Simulation 2005; 75(6):447-458. source
May combine two claims.
Interpretationown dataAsymptotic power calculations are conservative and useful for powers in the range of practical interest (typically 0.8 or 0.9).
“This implies that the asymptotic powers are conservative and consequently useful for the powers in the range of practical interest.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationFor Hardy-Weinberg equilibrium, approximate unconditional tests performed less well than conditional tests.
“Maiste and Weir [10] studied various tests for Hardy-Weinberg equilibrium and found that for them, approximate unconditional tests performed less well than conditional tests.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [10] 10. Maiste PJ, Weir BS. Optimal testing strategies for large, sparse multinomial models. Computational Statistics and Data Analysis 2004; 46(3):605-620. source
Classified as probably not a substantial claim.
Interpretationown dataAsymptotic Pearson's chi-squared or other asymptotic tests should not be used for small sample r × c tables.
“Moreover, we recommend that the asymptotic Pearson's chi-squared or other asymptotic tests not be used for small sample r × c tables.”
- Also stated in: § 7. DISCUSSION AND CONCLUSIONS · plain
“Hence, asymptotic tests should not be used for small sample r × c tables, not even in attempts to reduce the conservatism of exact tests.”
Quote in the paperSubstantial claim- Also stated in: § 7. DISCUSSION AND CONCLUSIONS · plain
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationSimilar results were reported for multinomial goodness-of-fit tests.
“Similar results were reported by Tang [19] for multinomial goodness-of-fit tests.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [19] 19. Tang ML. Small-sample study of the use of mid-P power divergence goodness-of-fit tests. Journal of Statistical Computation and Simulation 1998; 62(1-2):137-180. source
Probably not a substantial claim: may not pass S4 (based).
Premisenone givenThe mid-p-value may be computed using the p-value and the point probability.
“The mid-p-value may be computed using the p-value and the point probability.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationIn small sample r × c tables, the Pearson statistic and the Read and Cressie statistic give conservative tests, while the LR statistic is quite liberal.
“Parshall et al. [17] studied various test statistics for asymptotic tests in small sample r × c tables.They found that the Pearson statistic and the Read and Cressie statistic give conservative tests, while the LR statistic is quite liberal.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [17] 17. Parshall CG, Kromrey JD, Dailey R. Comparative performance of three statistical tests of homogeneity for sparse I × J contingency tables. Communications in Statistics-Simulation and Computation 1999; 28(1):275-289. source
Classified as probably not a substantial claim.
Premisenone givenThere is no study of Cochran's criterion for r × c tables.
“but we have found no study of Cochran's criterion for r × c tables.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataIn small sample unordered r × c tables, an exact conditional mid-p test using Pearson's chi-squared or Fisher-Freeman-Halton's statistic usually has the highest power among the compared tests while approximately preserving the significance level.
“After extensive simulations, in general we recommend using an exact conditional mid-p test with Pearson's chi-squared or Fisher-Freeman-Halton's statistic, which usually is the most powerful test yet preserve the approximate significance level.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
PremisecitationCochran's criterion states that Pearson's asymptotic chi-square test is acceptable when all expected counts m_ij > 1 and no more than 20% of cells have m_ij < 5; otherwise Fisher's exact test is recommended.
“Cochran's [4] criterion for choice of test in r × c tables is extensively cited (more than 1800 times by 2006).It maintains that Pearson's asymptotic chi-square test is acceptable when all m i j >1 and no more than 20 per cent of the cells have m i j <5.Otherwise, Fisher's exact test is recommended.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [4] 4. Cochran WG. Some methods for strengthening the common chi squared tests. Biometrics 1954; 10(4):417-451. source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationFor ordered r × c tables, the mid-p score test was the most powerful, with the actual significance level close to the nominal level.
“Tang [18] compared the asymptotic, conditional, and conditional mid-p score test for ordered r × c tables and found that the mid-p score test was the most powerful, with the actual significance level close to the nominal level.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [18] 18. Tang ML. Exact and asymptotic power determination for dose-response studies with ordinal response. Biometrical Journal 2002; 44(3):273-287. source
The paper relies on this without support.
The asymptotic Pearson's chi-squared test is not recommended in tables with few counts because the difference between the true significance level and the nominal level may be substantial.
“The Pearson's chi-squared test is not recommended in tables with few counts, because the difference between the true significance level (the type I error rate) and the nominal significance levels (usually = 0.05) may be substantial.”
The paper relies on this without support.
Lancaster's mid-p adjustment may be applied to make an exact test less conservative.
“However, Lancaster's mid-p adjustment may be applied to make the test less conservative.”
The paper relies on this without support.
Among methods of computing a p-value, only exact conditional and exact unconditional methods guarantee preservation of the test size.
“Among the methods of computing a p-value, only the exact conditional and exact unconditional methods guarantee preservation of the test size.”
The paper relies on this without support.
A mid-p test does not always preserve the test size.
“A mid-p test does not always preserve the test size.”
The paper relies on this without support.
Computing time may be excessive in r × c tables when r > 2 or c > 2, and there is no efficient algorithm for exact unconditional computation.
“Computing time may be excessive in r × c tables when r >2 or c>2.There is no efficient algorithm for this computation, so we will not pursue it further.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Simulation 8 table dimensions are reversed
MinorThe text calls the table in Table IV(b) a 9 × 3 table, but the table caption identifies Simulation 8 as using a 3 × 9 table.
“But the exact tests are less conservative for the 9 × 3 table (Table IV(b)).”
“Table IV.(a) Significance level for the 3 × 2 table, full multinomial model, N = 30, (Simulation 7) and (b) significance level for a small sample 3 × 9 table (Simulation 8).”
Cites
Agresti, A. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/0471249688 · Cited by 6,746 · see citing works
Casella, G., & Berger, R. L. (1993) · Chapman & Hall/CRC eBooks · doi:10.1201/9781003456285 · Cited by 5,835 · see citing works
Testing Statistical Hypotheses
Niewiadomska-Bugaj, M., & Bartoszynski, R. (2021) · Wiley series in probability and statistics · doi:10.1002/9781119243830.ch12 · Cited by 5,188 · see citing works
Some Methods for Strengthening the Common χ 2 Tests
Cochran, W. G. (1954) · Biometrics · doi:10.2307/3001616 · Cited by 3,718 · see citing works
Testing Statistical Hypotheses
Casella, G., Olkin, I., & Fienberg, S. (2005) · Springer texts in statistics · doi:10.1007/0-387-27605-x · Cited by 3,209 · see citing works
Ripley, B. D. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316726 · Cited by 2,241 · see citing works
Suárez, E. L., Pérez, C. M., Nogueras, G. M., & Moreno-Gorrín, C. (2016) · CRC Press eBooks · doi:10.1201/b19547-8 · Cited by 1,052 · see citing works
Bedford, T., & Cooke, R. (2001) · Cambridge University Press eBooks · doi:10.1017/cbo9780511813597.005 · Cited by 724 · see citing works
Exact inference for categorical data: recent advances and continuing controversies
Agresti, A. (2001) · Statistics in Medicine · doi:10.1002/sim.738 · Cited by 162 · see citing works
Exact Power of Conditional and Unconditional Tests: Going Beyond the 2 × 2 Contingency Table
Mehta, C. R., & Hilton, J. F. (1993) · The American Statistician · doi:10.1080/00031305.1993.10475946 · Cited by 87 · see citing works
Exact Power of Conditional and Unconditional Tests: Going Beyond the 2 × 2 Contingency Table
Mehta, C. R., & Hilton, J. F. (1993) · The American Statistician · doi:10.2307/2685184 · Cited by 87 · see citing works
Exact Analysis of Discrete Data
(2008) · Technometrics · doi:10.1198/tech.2008.s908 · Cited by 84 · see citing works
Exact Analysis of Discrete Data
Hirji, K. F. (2005) · Chapman & Hall/CRC eBooks · doi:10.1201/9781420036190 · Cited by 78 · see citing works
THE CLINICAL USEFULNESS OF THE RENAL ALLOGRAFT BIOPSY IN THE CYCLOSPORINE ERA: A Prospective Study1
Pascual, M., Vallhonrat, H., Cosimi, A. B., Tolkoff-Rubin, N., Colvin, R. B., Delmonico, F. L., Ko, D. S. C., Schoenfeld, D. A., & Williams, W. W. (1999) · Transplantation · doi:10.1097/00007890-199903150-00016 · Cited by 58 · see citing works
Exact Analysis of Discrete Data
Booth, J. G. (2007) · Journal of the American Statistical Association · doi:10.1198/jasa.2007.s193 · Cited by 42 · see citing works
Lydersen, S., & Laake, P. (2003) · Statistics in Medicine · doi:10.1002/sim.1671 · Cited by 24 · see citing works
Self-citationOn the validity condition of the chi-squared test in 2×2 tables
Andrés, A. M., Sánchez Quevedo, M. J., Tapia García, J. M., & Silva-Mato, A. (2005) · Test · doi:10.1007/bf02595399 · Cited by 18 · see citing works
Optimal testing strategies for large, sparse multinomial models
Maiste, P. J., & Weir, B. S. (2004) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2003.08.002 · Cited by 8 · see citing works
Small-sample study of the use of mid-<i>p</i>power divergence goodness-of-fit tests
Tang, M.-L. (1998) · Journal of Statistical Computation and Simulation · doi:10.1080/00949659808811929 · Cited by 6 · see citing works
Lydersen, S., Pradhan, V., Senchaudhuri, P., & Laake, P. (2005) · Journal of Statistical Computation and Simulation · doi:10.1080/00949650410001729454 · Cited by 5 · see citing works
Self-citationDuchateau, L., & Janssen, P. (1999) · Biometrical Journal · doi:10.1002/(sici)1521-4036(199909)41:5 · Cited by 4 · see citing works
Parshall, C. G., Kromrey, J. D., & Dailey, R. (1999) · Communications in Statistics - Simulation and Computation · doi:10.1080/03610919908813548 · Cited by 3 · see citing works
Parshall, C. G., Kromrey, J. D., & Dailey, R. J. (1999) · Communications in Algebra · doi:10.1080/00927879908826840
THE CLINICAL USEFULNESS OF THE ALLOGRAFT BIOPSY IN THE CYCLOSPORINE ERA: A PROSPECTIVE STUDY
PASCUAL, M., Vallhonrat, H., TOLKOFF-RUBIN, N. E., et al. (1998) · Transplantation · doi:10.1097/00007890-199810270-00243
Exact and Asymptotic Power Determination For Dose-Response Studies With Ordinal Response
Tang, M.-L. (2002) · Biometrical Journal · doi:10.1002/1521-4036(200204)44:3
Flagged first, then by citations
Research works printed but not linked: 2
- SAS Institute Inc, 1999, SAS Language Reference [8]
- Cytel Inc, 2004, StatXact 6 PROCs for SAS Users. A Software Package for Exact Nonparametric Inferences
Cited by
Sorted by citations
Evaniew, N., Files, C., Smith, C., Bhandari, M., Ghert, M. A., Walsh, M. W., Devereaux, P. J., & Guyatt, G. H. (2015) · The Spine Journal · doi:10.1016/j.spinee.2015.06.004 · Cited by 189 · see citing works
“The Fisher exact test is an alternative to the Pearson chisquare test for comparing proportions in a two-by-two contingency table and is more conservative than the Pearson chi-square test for most situations [18,19].”
Good practice in testing for an association in contingency tables
Ruxton, G. D., & Neuhäuser, M. (2010) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-010-1014-0 · Cited by 81 · see citing works
“In particular, the mid-P version of this Fisher–Freeman–Halton test appears to have the best properties (Agresti 2001; Lydersen et al. 2005, 2007).”
Schäfer, S. K., Supke, M., Kausmann, C., Schaubruch, L. M., Lieb, K., & Cohrdes, C. (2024) · Communications Psychology · doi:10.1038/s44271-024-00138-w · Cited by 106 · see citing works
Sensory and repetitive behaviors among children with autism spectrum disorder at home
Kirby, A. V., Boyd, B. A., Williams, K. L., Faldowski, R. A., & Baranek, G. T. (2016) · Autism · doi:10.1177/1362361316632710 · Cited by 71 · see citing works
“Fisher’s exact test (FET), which is equivalent to an exact test of the weighted Pearson chi-squared for 2×2 tables (Agresti, 1992, 2012; Lydersen et al., 2007; Mehta and Patel, 1983; SAS Institute, 2013), was used to compare the results in 2×2 Table 1.”
MethodMajdán, M., Mauritz, W., Wilbacher, I., et al. (2012) · European Journal of Public Health · doi:10.1093/eurpub/cks074 · Cited by 55 · see citing works
Ekhtiari, S., Gazendam, A., Nucci, N. W., et al. (2020) · The Journal of Arthroplasty · doi:10.1016/j.arth.2020.12.015 · Cited by 43 · see citing works
Baena, E., Cunha, S. R., Maravić, T., Comba, A., Paganelli, F., Bonetti, G. A., Ceballos, L., Tay, F. R., Breschi, L., & Mazzoni, A. (2020) · Marine Drugs · doi:10.3390/md18050263 · Cited by 40 · see citing works
Yin, G., & Jin, H. (2020) · JMIR Public Health and Surveillance · doi:10.2196/19464 · Cited by 42 · see citing works
No-waiting dentine self-etch concept—Merit or hype
Huang, X.-Q., Pucci, C. R., Luo, T., et al. (2017) · Journal of Dentistry · doi:10.1016/j.jdent.2017.05.007 · Cited by 34 · see citing works
The fragility of statistically significant findings from randomized trials in head and neck surgery
Noel, C. W., McMullen, C. P., Yao, C. M. K. L., Monteiro, E., Goldstein, D. P., Eskander, A. E., & de Almeida, J. R. (2018) · The Laryngoscope · doi:10.1002/lary.27183 · Cited by 32 · see citing works
“That said, it remains the preferred approach in cases of small sample sizes and/or outcomes events, as was the case for the majority of RCTs in our study.(53,54) Despite its many benefits, the FI does have two key limitations.”
ResultMight, K. R., Bachelez, A., Martinez, S. A., et al. (2012) · Veterinary Surgery · doi:10.1111/j.1532-950x.2012.01064.x · Cited by 24 · see citing works
The Fragility of Statistically Significant Randomized Controlled Trials in Plastic Surgery
Chin, B. H., Copeland, A. E., Gallo, L., Wakeham, S., Coroneos, C. J., Walsh, M. W., Thoma, A., & Voineskos, S. H. (2019) · Plastic & Reconstructive Surgery · doi:10.1097/prs.0000000000006102 · Cited by 22 · see citing works
Vermeirsch, J., Verhaeghe, L., Casaer, A., Faes, F., Oostra, A., & Roeyers, H. (2020) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-020-04573-6 · Cited by 21 · see citing works
The fragility of statistically significant results in otolaryngology randomized trials
Skinner, M., Tritz, D., Farahani, C., Ross, A., Hamilton, T., & Vassar, M. B. (2018) · American Journal of Otolaryngology · doi:10.1016/j.amjoto.2018.10.011 · Cited by 18 · see citing works
Risk of pathogen spillover to bighorn sheep from domestic sheep and goat flocks on private land
Heinse, L., Hardesty, L. H., & Harris, R. B. (2016) · Wildlife Society Bulletin · doi:10.1002/wsb.718 · Cited by 18 · see citing works
Kumar, S., Marchina, S., Langmore, S. E., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-14390-9 · Cited by 17 · see citing works
Yin, G., & Jin, H. (2020) · medRxiv · doi:10.1101/2020.04.02.20050740 · Cited by 16 · see citing works
Zanolli, M. B., Streit, D. S., Maciel, D. T., Muraguchi, E. M. O., Martins, M. A., & Tiberio, I. F. L. C. (2020) · BMC Medical Education · doi:10.1186/s12909-020-02193-3 · Cited by 15 · see citing works
Seyed-Safi, A. G., & Daniels, J. T. (2020) · Experimental Eye Research · doi:10.1016/j.exer.2020.108063 · Cited by 14 · see citing works
Vivekanantha, P., Shah, A., Hoit, G., Ayeni, O. R., & Whelan, D. B. (2023) · World Journal of Surgery · doi:10.1007/s00268-023-06928-3 · Cited by 13 · see citing works
Crivelli, M., Wyss, K., Grize, L., Matthys, B., Aebi, T., & Zemp, E. (2018) · International Journal of Public Health · doi:10.1007/s00038-018-1088-4 · Cited by 13 · see citing works
“The last was applied when the count size was small (not all expected frequencies were C 2, or if more than 20% of the cells had an expected frequency of B 5) (Lydersen et al. 2007; McDonald 2009).”
MethodSekiguchi, Y., HONDA, K., Owaki, D., & Izumi, S. (2021) · Brain Sciences · doi:10.3390/brainsci11111512 · Cited by 12 · see citing works
Nair, S. J., Sujatha, C., P, S. C. K., et al. (2021) · Journal of Forensic and Legal Medicine · doi:10.1016/j.jflm.2021.102247 · Cited by 12 · see citing works
The mechanics behind gait problems in patients with Dravet Syndrome
Wyers, L., Verheyen, K., Ceulemans, B., Schoonjans, A., Desloovere, K., de Walle, P. V., & Hallemans, A. (2020) · Gait & Posture · doi:10.1016/j.gaitpost.2020.12.029 · Cited by 11 · see citing works
Moloney, E., O’Donovan, M. R., Sezgin, D., et al. (2023) · Journal of Emergency Nursing · doi:10.1016/j.jen.2023.08.008 · Cited by 10 · see citing works
Sharkawy, R. M. E., Zaki, A. M., Kamel, A. A. E. F., et al. (2014) · Alexandria Journal of Medicine · doi:10.1016/j.ajme.2013.10.002 · Cited by 10 · see citing works
Split-door criterion: Identification of causal effects through auxiliary outcomes
Sharma, A. K., Hofman, J. M., & Watts, D. J. (2018) · The Annals of Applied Statistics · doi:10.1214/18-aoas1179 · Cited by 9 · see citing works
“Thus, when working with small time intervals τ we recommend the use of exact independence tests and randomization inference (Agresti, 1992, 2001; Lydersen et al., 2007).”
MethodBrückner, S., & Zlatkin‐Troitschanskaia, O. (2018) · Methodology of educational measurement and assessment · doi:10.1007/978-3-319-74338-7_6 · Cited by 9 · see citing works
Accidental flexicurity or workfare? Navigating ride-share work and Australia’s welfare system
Veen, A., Barratt, T., Goods, C., & Baird, M. (2023) · Economic and Industrial Democracy · doi:10.1177/0143831x231197057 · Cited by 9 · see citing works
“Due to the relatively small sample and varying levels at which the three cohorts were present as well as the distribution of the data, nonparametric Mann–Whitney U tests (see e.g. McKnight and Najab, 2010) and the Fisher–Freeman–Halton Exact Test (Lydersen et al., 2007) were used to explore whether there were statisti…”
InfluentialSarcher, A., Brochard, S., Perrouin-Verbe, B., et al. (2019) · Annals of Physical and Rehabilitation Medicine · doi:10.1016/j.rehab.2019.08.001 · Cited by 7 · see citing works
Cunha, D. P. M., Santos, A. L., Matias, A., & Sianto, L. (2017) · Antropologia Portuguesa · doi:10.14195/2182-7982_34_6 · Cited by 8 · see citing works
Brückner, S., & Pellegrino, J. W. (2017) · Social indicators research series · doi:10.1007/978-3-319-56129-5_3 · Cited by 6 · see citing works
Watcharapalakorn, A., Poyomtip, T., Srisurattanamethakul, N., et al. (2025) · Ophthalmic and Physiological Optics · doi:10.1111/opo.13543 · Cited by 4 · see citing works
Thomas, D., Gomez, M., Muis, A., Trezise, N., Williamson, E., Choo, J. X. M., Case, J., Murray, E. L., Grigos, M. I., Iuzzini-Seigel, J., et al. (2026) · American Journal of Speech-Language Pathology · doi:10.1044/2025_ajslp-25-00358 · Cited by 3 · see citing works
Clark, C., Dorland, A., Jaworska, N., McQuaid, R. J., Langlois, M., & Dzierszinski, F. S. (2023) · Health Science Reports · doi:10.1002/hsr2.1114 · Cited by 2 · see citing works
Thirkettle, C., Oduola, S., McEntegart, L., & Beazley, P. I. (2025) · Mental Health Science · doi:10.1002/mhs2.70026 · Cited by 1 · see citing works
Anaspure, O. S., Patel, S., Baumann, A. N., et al. (2024) · Foot & Ankle Orthopaedics · doi:10.1177/24730114241300160 · Cited by 1 · see citing works
Yin, G., & Jin, H. (2020) · doi:10.2196/preprints.19464 · Cited by 1 · see citing works
Rudge, L. A. (2019) · The journal of magazine media · doi:10.1353/jmm.2019.0003 · Cited by 1 · see citing works
Ryan, T. P. (2013) · Wiley series in probability and statistics · doi:10.1002/9781118439241.ch10 · Cited by 2 · see citing works
Iacoban, C. G., Alvansazyazdi, S., Parodi, C. B., Kenari, B. S., Ramaglia, A., Mercuri, C., Gaggero, G., Piatelli, G., Milanaccio, C., Cangelosi, D., et al. (2026) · European Radiology Experimental · doi:10.1186/s41747-026-00810-2
Devery, C., Sweetman, L., & Cody, D. (2026) · Physica Medica · doi:10.1016/j.ejmp.2026.105907
Wejak, G. L., Nuari, D. A., Destiani, D. P., et al. (2026) · Jurnal Ilmiah Farmako Bahari · doi:10.52434/jifb.v17i2.43716
Evaluating Effects of the PAINAD Scale on Pain Management in Patients With Dementia
King, M., & Kwekkeboom, K. L. (2026) · Pain Management Nursing · doi:10.1016/j.pmn.2026.05.006
Friedrich, L., Walgenbach, M., O`Connor, M., et al. (2026) · Journal of Interventional Cardiac Electrophysiology · doi:10.1007/s10840-026-02283-0
The Savings Finance and Public Trust: Empirical Analysis
Yorulmaz, Ö., & Sağın, A. (2025) · Springer eBooks · doi:10.1007/978-3-032-01765-9_5
Amorim, D., Ramos, J. P., Barbosa, P., et al. (2024) · Jornal Brasileiro de Pneumologia · doi:10.36416/1806-3756/e20240357
“(15) Statistical computations were performed with IBM SPSS Statistics software package, version 27.0 (IBM Corporation, Armonk, NY, USA).”
InfluentialCopeland, R. S., & Razowski, J. (2019) · Proceedings of the Entomological Society of Washington · doi:10.4289/0013-8797.121.3.382
Mecanismos de atención materno infantiles en dos contextos comparativos: México-Beijing
Saul, E. G. O., & Sánchez, E. Y. P. (2017) · Antropologia Portuguesa · doi:10.14195/2182-7982_34_4
- Authors
- Lydersen, S., Pradhan, V., Senchaudhuri, P., & Laake, P.
- Affiliations
- Norwegian University of Science and Technology (NO)
- Cytel (United States) (US)
- University of Oslo (NO)
- Corresponding author
- Lydersen, S.
- Venue
- Statistics in Medicine
- Publisher
- Wiley
- Published
- 2007-02-20
- Volume
- 26
- Issue
- 23
- Pages
- 4328-4343
- ISSN
- 0277-6715 (ISSN-L)
- 1097-0258
- OpenAlex
- W2149108580
- Semantic Scholar
- 7d6ccaf49e61f198b7b59685c97deb50f9db44d6
- Cited by
- 48
- References
- 23
- Citations by year
- 2026: 6 · 2025: 3 · 2024: 3 · 2023: 4 · 2022: 1 · 2021: 2 · 2020: 7 · 2019: 4 · 2018: 5 · 2017: 5 · 2016: 2 · 2015: 1 · 2014: 1 · 2013: 1 · 2012: 2
- FWCI
- 0.31
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Statistics in Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- association testing, small sample testing, Pearson chi-square test, likelihood ratio test, statistical power, significance level
- MeSH terms
- Choice Behavior, Norway, Transplantation, Homologous, United States, Models, Statistical, Likelihood Functions, Sample Size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-11.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-11.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
