Does McNemar’s test compare the sensitivities and specificities of two diagnostic tests?
Abstract
McNemar’s test is often used in practice to compare the sensitivities and specificities for the evaluation of two diagnostic tests. For correct evaluation of accuracy, an intuitive recommendation is to test the diseased and the non-diseased groups separately so that the sensitivities can be compared among the diseased, and specificities can be compared among the healthy group of people. This paper provides a rigorous theoretical framework for this argument and study the validity of McNemar’s test regardless of the conditional independence assumption. We derive McNemar’s test statistic under the null hypothesis considering both assumptions of conditional independence and conditional dependence. We then perform power analyses to show how the result is affected by the amount of the conditional dependence under alternative hypothesis.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Across all discrepancies, GPT-4o was significantly more effective in criteria filtering compared to the medical student (McNemar test: χ21=5.985; P=.015).
“Across all discrepancies, GPT-4o was significantly more effective in criteria filtering compared to the medical student (McNemar test for paired proportions [30]: χ21=5.985; P=.015).”
From Identifying Deprescribing Opportunities With Large Language Models in Older Adults: Retrospective Cohort Study (2025) · cites this paper as [30]
Cites
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736 · see citing works
Note on the Sampling Error of the Difference Between Correlated Proportions or Percentages
McNemar, Q. (1947) · Psychometrika · doi:10.1007/bf02295996 · Cited by 4,754 · see citing works
Contingency Tables Involving Small Numbers and the χ2 Test
Yates, F. (1934) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.2307/2983604 · Cited by 1,283 · see citing works
Bayesian Approaches to Modeling the Conditional Dependence Between Multiple Diagnostic Tests
Dendukuri, N., & Joseph, L. (2001) · Biometrics · doi:10.1111/j.0006-341x.2001.00158.x · Cited by 543 · see citing works
Epidemiology and outcome of invasive fungal infections in solid organ transplant recipients
Neofytos, D., Fishman, J. A., Horn, D., Anaissie, E. J., Chang, C., Olyaei, A. J., Pfaller, M. A., Steinbach, W. J., Webster, K. M., & Marr, K. A. (2010) · Transplant Infectious Disease · doi:10.1111/j.1399-3062.2010.00492.x · Cited by 455 · see citing works
Pulmonary Aspergillosis: Early Diagnosis Improves Survival
von Eiff, M., Roos, N., Schulten, R., Hesse, M., Zühlsdorf, M., & van de Loo, J. (2009) · Respiration · doi:10.1159/000196477 · Cited by 351 · see citing works
Conditional dependence between tests affects the diagnosis and surveillance of animal diseases
Gardner, I. A., Stryhn, H., Lind, P., & Collins, M. T. (2000) · Preventive Veterinary Medicine · doi:10.1016/s0167-5877(00)00119-7 · Cited by 340 · see citing works
The Effect of Conditional Dependence on the Evaluation of Diagnostic Tests
Vacek, P. M. (1985) · Biometrics · doi:10.2307/2530967 · Cited by 318 · see citing works
Hawass, N. E. (1997) · The British Journal of Radiology · doi:10.1259/bjr.70.832.9166071 · Cited by 230 · see citing works
McNemar χ2test revisited: comparing sensitivity and specificity of diagnostic examinations
Trajman, A., & Luiz, R. R. (2008) · Scandinavian Journal of Clinical and Laboratory Investigation · doi:10.1080/00365510701666031 · Cited by 219 · see citing works
Effect of High Temperature on Infectivity of Toxoplasma gondii Tissue Cysts in Pork
Dubey, J. P., KOTULA, A. W., Sharar, A. K., Andrews, C. D., & Lindsay, D. S. (1990) · Journal of Parasitology · doi:10.2307/3283016 · Cited by 217 · see citing works
Arabia, S. (1997) · Cited by 171 · see citing works
Methodology for the assessment of new dichotomous diagnostic tests
Staquet, M. J., Rozencweig, M., Lee, Y. J., & Muggia, F. M. (1981) · Journal of Chronic Diseases · doi:10.1016/0021-9681(81)90059-x · Cited by 163 · see citing works
Aspergillus infection in lung transplant patients: incidence and prognosis
Iversen, M., Burton, C. M., Vand, S., Skovfoged, L., Carlsen, J., Milman, N. T., Andersen, C. B., Rasmussen, M., & Tvede, M. (2007) · European Journal of Clinical Microbiology & Infectious Diseases · doi:10.1007/s10096-007-0376-3 · Cited by 148 · see citing works
Luong, M., Clancy, C. J., Vadnerkar, A., Kwak, E. J., Silveira, F. P., Wissel, M., Grantham, K. J., Shields, R. K., Crespo, M. M., Pilewski, J. M., et al. (2011) · Clinical Infectious Diseases · doi:10.1093/cid/cir185 · Cited by 144 · see citing works
Matchmaking and McNemar in the comparison of diagnostic modalities.
Dwyer, A. J. (1991) · Radiology · doi:10.1148/radiology.178.2.1987587 · Cited by 135 · see citing works
Fosgate, G. T., Adesiyun, A. A., Hird, D. W., Johnson, W. O., Hietala, S. K., Schurig, G. G., & Ryan, J. (2002) · American Journal of Veterinary Research · doi:10.2460/ajvr.2002.63.1598 · Cited by 66 · see citing works
Newcombe, R. G. (2001) · Statistics in Medicine · doi:10.1002/sim.906 · Cited by 51 · see citing works
Gardner, I. A., Greiner, M., & Dubey, J. P. (2009) · Zoonoses and Public Health · doi:10.1111/j.1863-2378.2009.01281.x · Cited by 48 · see citing works
McNemar Test Is Preferred for Comparison of Diagnostic Techniques
Liao, Y., & Lin, Y. (2008) · American Journal of Roentgenology · doi:10.2214/ajr.08.1090 · Cited by 10 · see citing works
Inappropriate use of a chi 2 test in modality comparison study of related samples.
Mann, F. A., & Hildebolt, C. F. (1991) · Radiology · doi:10.1148/radiology.178.2.1987632 · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Evaluation metrics and statistical tests for machine learning
Rainio, O., Teuho, J., & Klén, R. (2024) · Scientific Reports · doi:10.1038/s41598-024-56706-x · Cited by 1,338 · see citing works
Tapeh, A. T. G., & Naser, M. Z. (2023) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-022-09793-w · Cited by 320 · see citing works
Socrates, V., Wright, D. S., Huang, T., Fereydooni, S., Dien, C., Chi, L., Albano, J., Patterson, B., Sasidhar Kanaparthy, N., Wright, C. X., et al. (2025) · JMIR Aging · doi:10.2196/69504 · Cited by 17 · see citing works
- Authors
- Kim, S., & Lee, W.
- Affiliations
- The University of Texas Health Science Center at Houston (US)
- Inha University (KR)
- Corresponding author
- Lee, Woojoo
- Venue
- Statistical Methods in Medical Research
- Publisher
- SAGE Publications
- Published
- 2016-09-30
- Volume
- 26
- Issue
- 1
- Pages
- 142-154
- ISSN
- 0962-2802 (ISSN-L)
- 1477-0334
- OpenAlex
- W2092146823
- Cited by
- 120
- References
- 21
- Citations by year
- 2026: 10 · 2025: 17 · 2024: 12 · 2023: 13 · 2022: 15 · 2021: 23 · 2020: 10 · 2019: 3 · 2018: 7 · 2017: 8 · 2016: 2
- FWCI
- 3.52
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Statistical Methods in Medical Research (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- McNemar test, diagnostic accuracy, sensitivity, specificity, diagnostic test comparison, conditional independence, conditional dependence, statistical power
- MeSH terms
- Diagnostic Tests, Routine, Humans, Research Design, Sensitivity and Specificity, Toxoplasmosis, Lung Transplantation, Pulmonary Aspergillosis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
