Note on the Sampling Error of the Difference Between Correlated Proportions or Percentages
The DOI matches its Crossref record.
Abstract
Two formulas are presented for judging the significance of the difference between correlated proportions. The chi square equivalent of one of the developed formulas is pointed out.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“we used McNemar’s test62 adopting the approach outlined by Gillick and Cox63.”
From Text classification models for assessing the completeness of randomized controlled trial publications based on CONSORT reporting guidelines (2024) · cites this paper as [62]
In the citing paper
“For GPT-4, the trend reverses: LINC underperforms CoT.However, we perform a McNemar's test (McNemar, 1947) to get the pvalue on this GPT-4 LINC vs. CoT comparison, and we find that the difference is not significant (p = 0.58).”
From LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers (2023) · cites this paper as McNemar, 1947
Citations
Cites
Still being checked.
Cited by
Items 1,001–1,026 of 1,026
Does producing semantically related words aid word retrieval in people with aphasia?
Lyalka, O., Howard, D., Morris, J., & Nickels, L. (2019) · Aphasiology · doi:10.1080/02687038.2018.1562149 · Cited by 36
ISSUES IN BIOMEDICAL STATISTICS: ANALYSING 2 × 2 TABLES OF FREQUENCIES
Ludbrook, J., & Dudley, H. A. F. (1994) · Australian and New Zealand Journal of Surgery · doi:10.1111/j.1445-2197.1994.tb04539.x · Cited by 36
Upadhyaya, S. G., Murphree, D. H., Ngufor, C. G., Knight, A. M., Cronk, D. J., Cima, R. R., Curry, T. B., Pathak, J., Carter, R. E., & Kor, D. J. (2017) · Mayo Clinic Proceedings Innovations Quality & Outcomes · doi:10.1016/j.mayocpiqo.2017.04.005 · Cited by 44
Self Hyper-Parameter Tuning for Data Streams
Veloso, B., Gama, J., & Malheiro, B. (2018) · IFIP Working Conference on Database Semantics · doi:10.1007/978-3-030-01771-2_16 · Cited by 38
Fisher, E., Brzezinski, R. Y., Ehrenwald, M., Shapira, I., Zeltser, D., Berliner, S. A., Marcus, Y., Shefer, G., Stern, N., Rogowski, O., et al. (2019) · International Journal of Obesity · doi:10.1038/s41366-018-0312-x · Cited by 44
Do Advanced Language Models Eliminate the Need for Prompt Engineering in Software Engineering?
Wang, G., Sun, Z., Ye, S., Gong, Z., Chen, Y., Zhao, Y., Liang, Q., & Hao, D. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3771933 · Cited by 26
An End-to-End Approach to Automatic Speech Assessment for Cantonese-speaking People with Aphasia
Qin, Y., Wu, Y., Lee, T., & Kong, A. P. H. (2020) · Journal of Signal Processing Systems · doi:10.1007/s11265-019-01511-3 · Cited by 25
Pennica, C., Hanna, G., Islam, S. A., Sternberg, M. J. E., & David, A. (2023) · Journal of Molecular Biology · doi:10.1016/j.jmb.2023.168060 · Cited by 25
Portillo, F. (2002) · The European Journal of Public Health · doi:10.1093/eurpub/12.4.295 · Cited by 22
(2024) · doi:10.1101/2024.03.31.24305138 · Cited by 18
Hadjistavropoulos, H. D., Peynenburg, V., Thiessen, D., Schneider, L. H., Nugent, M., Wilhelms, A., Karin, E., Titov, N., & Dear, B. F. (2022) · European Journal of Pain · doi:10.1002/ejp.1866 · Cited by 16
“…2 (1, n = 108) = 0, p = 1.0 Psychologist/counsellor 43 28.7 36 33.3 2 (1, n = 108) = 0.11, p = 0.75 aIncludes data from patients enrolled from January 2018 onward. bMcNemar test (Mcnemar, 1947) was used to examine differences in the proportion of patients responding ‘yes’ from pre- treatment to 3-…”
MethodInfluentialJiang, L., Lan, M., Menke, J. D., Vorland, C. J., & Kilicoglu, H. (2024) · Scientific Reports · doi:10.1038/s41598-024-72130-7 · Cited by 18
“To observe whether different input representations and data augmentation approaches led to statistically significant differences in model performances compared to the baseline, we used McNemar’s test 62 adopting the approach outlined by Gillick and Cox 63 .”
MethodHe, T., Kong, R., Holmes, A. J., Nguyen, M., Sabuncu, M. R., Eickhoff, S. B., Bzdok, D., Feng, J., & Yeo, B. T. T. (2018) · doi:10.1101/473603 · Cited by 16
Vemu, S., Denaro, K., Sato, B. K., & Williams, A. E. (2022) · CBE—Life Sciences Education · doi:10.1187/cbe.21-08-0216 · Cited by 18
“1947).”
Insights for variant clinical interpretation based on a benchmark of 65 variant effect predictors
Radjasandirane, R., Diharce, J., Gelly, J.-C., & de Brevern, A. G. (2025) · Genomics · doi:10.1016/j.ygeno.2025.111036 · Cited by 15
Ben Ahmed, K., Hall, L. O., Goldgof, D. B., & Fogarty, R. (2022) · IEEE Access · doi:10.1109/access.2022.3193700 · Cited by 12
Improving Instruction-Following in Language Models through Activation Steering
Stolfo, A., Balachandran, V., Yousefi, S., Horvitz, E., & Nushi, B. (2024) · arXiv · doi:10.48550/arxiv.2410.12877 · Cited by 10
“To assess whether the score improvement from steering is significantly different from standard inference, we carry out McNemar’s test (McNemar, 1947).”
Teuho, J., Schultz, J., Klén, R., Juarez-Orozco, L. E., Knuuti, J., Saraste, A., Ono, N., & Kanaya, S. (2024) · Journal of Nuclear Cardiology · doi:10.1016/j.nuclcard.2024.101889 · Cited by 7
Abdaoui, H., Barki, C., Dergaa, I., Tlili, K., Ceylan, H. İ., Bragazzi, N. L., de Giorgio, A., Ben Salah, R., & Rahmouni, H. B. (2025) · Bioengineering · doi:10.3390/bioengineering13010030 · Cited by 6
“McNemar’s test evaluated the prediction discordance between model pairs [38].”
Logical Concepts of (Im)possibility Guide Young Children's Decision‐Making
Cesana‐Arlotti, N., Jáuregui, S., Mazalik, P., Nichols, S., & Halberda, J. (2025) · Developmental Science · doi:10.1111/desc.70044 · Cited by 6
From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents
Ahn, J., & Kim, M. (2026) · arXiv · doi:10.48550/arxiv.2607.08028 · Cited by 1
“Because the conditions are paired by scenario and repeat, this contrast is significant under McNemar’s test [69] ( 𝑝 0 .”
SE-Blackboard: A Shared-State Architecture for Multi-Agent Software Engineering Pipelines
Liu, E., Zhu, Q., & Dong, X. (2026) · IEEE Access · doi:10.1109/access.2026.3695435
“Given the paired design (each issue under all three configurations), we use McNemar’s exact test [26] for resolve rate comparisons, Wilcoxon signed-rank test [27] for continuous measurements (tokens, latency, IFS), Cohen’s h for effect size, and odds ratio with 95% CI.”
ReaComp: Compiling LLM Reasoning into Symbolic Solvers for Efficient Program Synthesis
Naik, A., Mathur, Y., Prakam, Rose, C., & Mortensen, D. (2026) · arXiv · doi:10.48550/arxiv.2605.05485
Plausible Is Not Enough: Trust and Evidence in AI-Authored Pull Request Review
Rahman, M. S., Alam, K., Roy, B., & Roy, C. K. (2026) · arXiv · doi:10.48550/arxiv.2610.21144
Tokenization is Sensitive to Language Variation
Wegmann, A., Nguyen, D., & Jurgens, D. (2025) · arXiv · doi:10.48550/arxiv.2502.15343
The Hidden Cost of Readability: How Code Formatting Silently Consumes Your LLM Budget
Pan, D., Sun, Z., Zhang, C., Lo, D., & Du, X. (2026) · Proceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering · doi:10.1145/3744916.3764556
“• Statistical Validation: We employ McNemar’s test [26], a non-parametric method particularly suited for comparing the performance of two models on the same test instances.”
Metadata
- Authors
- McNemar, Q.
- Affiliations
- Stanford University (US)
- Corresponding author
- McNemar, Quinn
- Venue
- Psychometrika
- Publisher
- Cambridge University Press (CUP)
- Published
- 1947-06
- Volume
- 12
- Issue
- 2
- Pages
- 153-157
- ISSN
- 0033-3123 (ISSN-L)
- 1860-0980
- OpenAlex
- W2084413241
- Cited by
- 4,754
- References
- 0
- Citations by year
- 2026: 753 · 2025: 274 · 2024: 242 · 2023: 262 · 2022: 242 · 2021: 288 · 2020: 257 · 2019: 178 · 2018: 194 · 2017: 148 · 2016: 164 · 2015: 165 · 2014: 165 · 2013: 132 · 2012: 131
- FWCI
- 3.42
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychometrika (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Survey Sampling and Estimation Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- sampling error, significance testing, chi-square test
- MeSH terms
- Humans, Selection Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.