The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Exact p-values for pairwise comparison of Friedman rank sums, with application to comparing classifiers
Open accessAbstract
BACKGROUND: The Friedman rank sum test is a widely-used nonparametric method in computational biology. In addition to examining the overall null hypothesis of no significant difference among any of the rank sums, it is typically of interest to conduct pairwise comparison tests. Current approaches to such tests rely on large-sample approximations, due to the numerical complexity of computing the exact distribution. These approximate methods lead to inaccurate estimates in the tail of the distribution, which is most relevant for p-value calculation. RESULTS: We propose an efficient, combinatorial exact approach for calculating the probability mass distribution of the rank sum difference statistic for pairwise comparison of Friedman rank sums, and compare exact results with recommended asymptotic approximations. Whereas the chi-squared approximation performs inferiorly to exact computation overall, others, particularly the normal, perform well, except for the extreme tail. Hence exact calculation offers an improvement when small p-values occur following multiple testing correction. Exact inference also enhances the identification of significant differences whenever the observed values are close to the approximate critical value. We illustrate the proposed method in the context of biological machine learning, were Friedman rank sum difference tests are commonly used for the comparison of classifiers over multiple datasets. CONCLUSIONS: We provide a computationally fast method to determine the exact p-value of the absolute rank sum difference of a pair of Friedman rank sums, making asymptotic tests obsolete. Calculation of exact p-values is easy to implement in statistical software and the implementation in R is provided in one of the Additional files and is also available at http://www.ru.nl/publish/pages/726696/friedmanrsd.zip .
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Agresti, A. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/0471249688 · Cited by 6,746
Approximations of the critical region of the fbietkan statistic
Iman, R. L., & Davenport, J. M. (1980) · Communications in Statistics - Theory and Methods · doi:10.1080/03610928008827904 · Cited by 1,125
Cited by
Carrasco, J., García, S., Rueda, M. M., Das, S., & Herrera, F. (2020) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2020.100665 · Cited by 513
A Review of Android Malware Detection Approaches Based on Machine Learning
Liu, K., Xu, S., Xu, G., Zhang, M., Sun, D., & Liu, H. (2020) · IEEE Access · doi:10.1109/access.2020.3006143 · Cited by 374
Badeja, D., Schier, F., McGinity, M., & Hoyer, J. (2026) · Psychotherapy Research · doi:10.1080/10503307.2025.2534978 · Cited by 2
“To test whether the present non-normally distributed residuals biased the above results, Friedman’s test followed by post-hoc comparisons according to Eisinga et al. (2017) were conducted as a non-parametric alternative to the applied LMM analysis.”
Metadata
- Authors
- Eisinga, R., Heskes, T., Pelzer, B., & Te Grotenhuis, M.
- Venue
- BMC Bioinformatics
- Published
- 2017-01-25
- Volume
- 18
- Issue
- 1
- Pages
- 68-68
- ISSN
- 1471-2105 (ISSN-L)
- OpenAlex
- W2581151820
- Cited by
- 181
- References
- 2
- FWCI
- 5.28
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Exact p-values for pairwise comparison of Friedman rank sums, with application to comparing classifiers. https://app.cochat.ai/papers/10.1186/s12859-017-1486-2
- The paper
- Eisinga, R., Heskes, T., Pelzer, B., & Te Grotenhuis, M. (2017). Exact p-values for pairwise comparison of Friedman rank sums, with application to comparing classifiers. BMC Bioinformatics. https://doi.org/10.1186/s12859-017-1486-2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.