Negative Controls: A Tool for Detecting Confounding and Bias in Observational Studies
Open accessThe DOI matches its Crossref record; 24 of the 25 references it cites were found in a registry or an index.
Abstract
Noncausal associations between exposures and outcomes are a threat to validity of causal inference in observational studies. Many techniques have been developed for study design and analysis to identify and eliminate such errors. Such problems are not expected to compromise experimental studies, where careful standardization of conditions (for laboratory work) and randomization (for population studies) should, if applied properly, eliminate most such noncausal associations. We argue, however, that a routine precaution taken in the design of biologic laboratory experiments--the use of "negative controls"--is designed to detect both suspected and unsuspected sources of spurious causal inference. In epidemiology, analogous negative controls help to identify and resolve confounding as well as other sources of error, including recall bias or analytic flaws. We distinguish 2 types of negative controls (exposure controls and outcome controls), describe examples of each type from the epidemiologic literature, and identify the conditions for the use of such negative controls to detect confounding. We conclude that negative controls should be more commonly employed in observational studies, and that additional work is needed to specify the conditions under which negative controls will be sensitive detectors of other sources of error in observational studies.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Finally, in order to evaluate the specificity of the association to ADHD medication, the tenth used selective serotonin reuptake inhibitors (SSRIs) as a negative control medication exposure (27, 30).”
From ADHD Medication and Substance-Related Problems (2017) · cites this paper as [30]
In the citing paper
“The use of ‘negative controls’ for outcomes or exposures can be helpful to assess the likely presence of unmeasured or residual confounding in observational research [51].”
From COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology (2019) · cites this paper as [51]
In the citing paper
“In a third approach–negative control studies–we would expect an unbiased negative control study to be null and the ‘real’ study not-null if there is a true causal effect and the assumptions of this approach are met (see Table 1).39,40”
From Triangulation in aetiological epidemiology (2017) · cites this paper as [39]
Citations
Cites
(1979) · Medical Entomology and Zoology · Cited by 11,252
The environment and disease: association or causation?
Hill, A. B. (2015) · Journal of the Royal Society of Medicine · doi:10.1177/0141076814562718 · Cited by 7,285
The Environment and Disease: Association or Causation?
Hill, A. B. (1965) · Proceedings of the Royal Society of Medicine · doi:10.1177/003591576505800503 · Cited by 6,974 · cited 1×
“19 Hill argued that causal inferences were more credible if the exposure (in his example, nickel mining) was associated with only certain types of outcomes (death from lung and nose cancer but not death from other cancers), and if the outcome was associated with one kind of exposure (nickel mining)…”
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
A structural approach to selection bias
Hernán, M. A., Hernández-Díaz, S., & Robins, J. M. (2004) · Epidemiology · doi:10.1097/01.ede.0000135174.63482.43 · Cited by 2,844
An introduction to instrumental variables for epidemiologists
Greenland, S. (2000) · International Journal of Epidemiology · doi:10.1093/ije/29.4.722 · Cited by 1,156
Instruments for Causal Inference: An Epidemiologist's Dream?
Hernán, M. A., & Robins, J. M. (2006) · Epidemiology · doi:10.1097/01.ede.0000222409.00878.37 · Cited by 961
Vaccines for preventing influenza in the elderly
Demicheli, V., Jefferson, T., Di Pietrantonj, C., Ferroni, E., Thorning, S., Thomas, R. E., & Rivetti, A. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004876.pub4 · Cited by 695
Impact of Influenza Vaccination on Seasonal Mortality in the US Elderly Population
(2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.3.265 · Cited by 543 · cited 1×
“8 To test this hypothesis, Jackson et al 7 reproduced earlier estimates of the protective effect of influenza vaccination, but then repeated the analysis for two sets of negative control outcomes, and showed that the protective effect was observed even in circumstances where the vaccine could not…”
Mortality benefits of influenza vaccination in elderly people: an ongoing controversy
Simonsen, L., Taylor, R. J., Viboud, C., Miller, M. A., & Jackson, L. A. (2007) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(07)70236-0 · Cited by 520 · cited 1×
“However, older age is associated with a less robust immune response to influenza vaccination, and ecological data suggest that the benefits measured in observational studies far exceed the corresponding benefits expected at the population level when influenza vaccination rates have increased among the elderly.(6) Impo…”
Cited by
Items 601–700 of 1,002
Medical facilities in the neighborhood and incidence of sudden cardiac arrest
Goh, C. E., Mooney, S. J., Siscovick, D. S., Lemaitre, R. N., Hurvitz, P., Sotoodehnia, N., Kaufman, T. K., Zulaika, G., & Lovasi, G. S. (2018) · Resuscitation · doi:10.1016/j.resuscitation.2018.07.005 · Cited by 18
“Sensitivity analyses Because confounding by unmeasured aspects of the residential neighborhood environment might bias our findings, we considered local financial institutions as a negative control [27] in order to distinguish the availability of medical facilities from total commercial density [28].”
BackgroundLi, B., Cheung, K. S., Wong, Y., Leung, W. K., & Law, S. Y.-K. (2020) · International Journal of Cancer · doi:10.1002/ijc.33379 · Cited by 18
“We also evaluated the effect of other classes of hypertensive medications (ACEIs, ARBs, beta blockers and diuretics) as negative control exposure.(31) If any positive association between any other classes of antihypertensive drugs and development of GC was detected, this would suggest the presence of residual/unmeasur…”
Latour, C. D., Delgado, M., Su, I., Wiener, C., Acheampong, C. O., Poole, C., Edwards, J. K., Quinto, K., Stürmer, T., Lund, J. L., et al. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae234 · Cited by 21
Best practices in plant fluorescence imaging and reporting: A primer
Czymmek, K. J., Benitez‐Alfonso, Y., Burch‐Smith, T. M., Costanzo, L. D., Drakakaki, G., Facette, M. R., Kierzkowski, D., Klebanovych, A., Radin, I., Roychoudhry, S., et al. (2025) · The Plant Cell · doi:10.1093/plcell/koaf143 · Cited by 18
“Controls are required for the proper interpretation of any scientific data (Baker 1984; Lipsitch et al. 2010; Torday and Baluška 2019), and microscopy is no exception.”
Shale gas activity and increased rates of sexually transmitted infections in Ohio, 2000–2016
Deziel, N. C., Humeau, Z., Elliott, E. G., Warren, J. L., & Niccolai, L. M. (2018) · PLoS ONE · doi:10.1371/journal.pone.0194203 · Cited by 17
Cho, E. J., Chung, G. E., Yoo, J., Cho, Y., Shin, D. W., Kim, Y. J., Yoon, J., Han, K., & Yu, S. J. (2023) · PLoS Medicine · doi:10.1371/journal.pmed.1004244 · Cited by 18
“A negative control outcome shares the same potential sources of bias as the primary outcome but cannot plausibly be related to the conditions of interest [25].”
BackgroundCardiovascular Outcomes of α-Blockers vs 5-α Reductase Inhibitors for Benign Prostatic Hyperplasia
Zhang, J., Latour, C. D., Olawore, O., Pate, V., Friedlander, D. F., Stürmer, T., Funk, M. J., & Jensen, B. C. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.43299 · Cited by 17
Lawal, O. D., Aronow, H. D., Hume, A. L., Shobayo, F. O., Matson, K. L., Barbour, M. M., Zhang, Y., & Wen, X. (2023) · Research and Practice in Thrombosis and Haemostasis · doi:10.1016/j.rpth.2023.102293 · Cited by 17
“Third, to exclude the presence of substantial residual confounding, we performed a negative control analysis using pneumonia as the outcome measure [45], an outcome with no known relationship that differs between DOACs and warfarin.”
MethodChen, Y., Chang, H., Lin, Y., Chien, K., Hsieh, Y., Chung, F., Lin, C., Lip, G. Y. H., & Chen, S. (2024) · Diabetes/Metabolism Research and Reviews · doi:10.1002/dmrr.3775 · Cited by 20
Hamer, M., Ding, D., Chau, J. Y., Duncan, M. J., & Stamatakis, E. (2020) · Journal of Epidemiology & Community Health · doi:10.1136/jech-2019-212739 · Cited by 17
“Negative control comparisons are increasingly recognised as a means of strengthening causal inference in observational study settings.(11) A”
BackgroundGeneslaw, A. S., Lu, Y., Miles, C. H., Hua, M. S., Cappell, J., Smerling, A. J., Olfson, M., Edwards, J. D., & Ing, C. H. (2021) · Pediatric Critical Care Medicine · doi:10.1097/pcc.0000000000002790 · Cited by 17
“7–13 255 (19) 1,139 (17) 1,762 (23) 8,639 (22)”
InfluentialDevelopment of Open-Angle Glaucoma in Adults With Seropositive Rheumatoid Arthritis in Korea
Kim, S. H., Jeong, S. H., Kim, H., Park, E., & Jang, S. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.3345 · Cited by 19
Wang, S. V., Sreedhara, S. K., Bessette, L. G., & Schneeweiß, S. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.08.012 · Cited by 17
Shen, C., Lee, S. Y., Lam, T. H., & Schooling, C. M. (2015) · American Journal of Epidemiology · doi:10.1093/aje/kwv142 · Cited by 17
Casey, J. A., & Schwartz, B. S. (2016) · Epidemiology · doi:10.1097/ede.0000000000000537 · Cited by 17
Detection Bias in EHR-Based Research on Clinical Exposures and Dementia
Wang, J., Choi, M., Buto, P. T., Kelly, J. D., Joie, R. L., Kornak, J., Zimmerman, S. C., Chen, R., Raphael, E., Schaefer, C. A., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.6637 · Cited by 19
Using negative controls to adjust for unmeasured confounding bias in time series studies
Hu, J. K., Tchetgen, E. J. T., & Dominici, F. (2023) · Nature Reviews Methods Primers · doi:10.1038/s43586-023-00249-4 · Cited by 19
Guo, F., McGee, E. E., Chiu, Y., Giovannucci, E. L., Mucci, L. A., & Dickerman, B. A. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae184 · Cited by 19
Silva, T. C., Barral‐Netto, M., & Boaventura, V. S. (2025) · The Lancet Global Health · doi:10.1016/s2214-109x(25)00279-7 · Cited by 22
Lai, C., Kuo, R. N.-C., Chen, H., Chen, M.-F., Chan, K. A., & Lai, M. (2015) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.12671 · Cited by 16
Ambient Particle Components and Newborn Blood Pressure in Project Viva
Zanobetti, A., Coull, B. A., Luttmann‐Gibson, H., van Rossem, L., Rifas‐Shiman, S. L., Kloog, I., Schwartz, J. D., Oken, E., Bobb, J. F., Koutrakis, P., et al. (2020) · Journal of the American Heart Association · doi:10.1161/jaha.120.016935 · Cited by 18
Melgaard, L., Overvad, T. F., Jensen, M., Christensen, T. D., Lip, G. Y. H., Larsen, T. B., & Nielsen, P. B. (2021) · Journal of the American Heart Association · doi:10.1161/jaha.121.022628 · Cited by 16
“35 In detail, we emulated an individual target trial with pneumonia as the outcome and an individual target trial with cancer as the outcome using the described features from the ITT analyses.”
Casey, J. A., Tartof, S. Y., Davis, M. F., Nachman, K. E., Price, L. B., Liu, C. H., Yu, K. C., Gupta, V., Innes, G. K., Tseng, H. F., et al. (2023) · Environmental Health Perspectives · doi:10.1289/ehp11221 · Cited by 17
Català, M., Burn, E., Rathod‐Mistry, T., Xie, J., Delmestri, A., Prieto‐Alhambra, D., & Jödicke, A. M. (2023) · International Journal of Epidemiology · doi:10.1093/ije/dyad138 · Cited by 16
“i) Although negative control outcomes (NCO) are outcomes not causally associated with the exposure of interest, 25 here COVID-19 vaccination, their association with vaccination should be affected by the same type of unmeasured confounding, e.g. health care-seeking behaviour, as the…”
BackgroundA population-based case–control study on statin exposure and risk of acute diverticular disease
Sköldberg, F., Svensson, T., Olén, O., Hjern, F., Schmidt, P. T., & Ljung, R. (2015) · Scandinavian Journal of Gastroenterology · doi:10.3109/00365521.2015.1081274 · Cited by 16
“In separate analyses, statin exposure was substituted with exposure for anti-glaucoma preparations and miotics or vitamin B12 (Appendix 1), with ‘‘current use’’ and ‘‘former use’’ defined as above, to serve as negative control exposures [25].”
BackgroundNegative Control Exposures in Epidemiologic Studies
Lipsitch, M., Tchetgen, E. T., & Cohen, T. (2012) · Epidemiology · doi:10.1097/ede.0b013e3182460c23 · Cited by 16
Self-citationShahzade, C., Chun, J., DeLisi, L. E., & Manschreck, T. C. (2018) · Schizophrenia Research · doi:10.1016/j.schres.2018.01.008 · Cited by 17
“Second, the statistical inclusion of a healthy control cohort greatly reduces confounding and reporting bias (Lipsitch et al., 2010).”
Kipp, R. T., Askari, M., Fan, J., Field, M. E., & Turakhia, M. P. (2018) · JACC. Clinical electrophysiology · doi:10.1016/j.jacep.2018.08.025 · Cited by 17
“To investigate for the presence of residual confounding when evaluating patients prescribed class IC versus class III AAD, we performed a falsification analysis by applying the full multivariable adjusted analysis cohort, the inverse weighting for the probability of treatment full cohort, and the CHADS2 scores 0 or 1…”
MethodElser, H. C., Ben‐Michael, E., Rehkopf, D. H., Modrek, S., Eisen, E. A., & Cullen, M. R. (2019) · Journal of Epidemiology & Community Health · doi:10.1136/jech-2018-211774 · Cited by 16
Nation, M. L., Moss, R. G., Spittal, M. J., Kotsimbos, T. C., Kelly, P., & Cheng, A. C. (2020) · Clinical Infectious Diseases · doi:10.1093/cid/ciz1238 · Cited by 20
Miao, W., & Tchetgen, E. T. (2018) · arXiv (Cornell University) · Cited by 26
Selection bias and multiple inclusion criteria in observational studies
Zetterstrom, S., & Waernbaum, I. (2022) · Epidemiologic Methods · doi:10.1515/em-2022-0108 · Cited by 17
“Examples are using negative controls (Lipsitch, Tchetgen, and Cohen 2010), calculation of the E-value (VanderWeele and Ding 2017), Bayesian sensitivity analysis (McCandless, Gustafson, and Levy 2007) and a class of methods resulting in the construction of bounds, see for example Manski (1990) for…”
Long-term exposure to ambient ozone and adult-onset asthma: A prospective cohort study
Zhang, J., Ai, B., Guo, Y., Chen, G., Chen, G., Li, H., Lin, H., & Zhang, Z. (2024) · Environmental Research · doi:10.1016/j.envres.2024.118962 · Cited by 15
Penicillin allergy as an instrumental variable for estimating antibiotic effects on resistance
Saciuk, Y., Nevo, D., Chowers, M., & Obolski, U. (2025) · Nature Communications · doi:10.1038/s41467-025-56287-x · Cited by 15
“Gentamicin, an aminoglycoside antibiotic, served as an NCO for the IV independence assumption 15,16 (explained below).”
InfluentialIoannou, G. N., Berry, K., Rajeevan, N., Li, Y., Yan, L., Huang, Y., Lin, H., Bui, D. P., Hynes, D. M., Rowneki, M., et al. (2025) · Annals of Internal Medicine · doi:10.7326/annals-24-01015 · Cited by 17
“Wecompared as a negative control outcome (26) the incidence of SARS-CoV-2 infection during days 0 to 9 after the index date, which should not be affected by vaccination, to assess for residual confounding or outcome ascertainment bias.”
Characterization of post-acute multi-organ sequelae following dengue infection
Lim, J. T., Wee, L. E., Tan, W. Z., Chiew, C. J., Kurupatham, L., Poh, C., Suhaimi, N.-A. M., Chua, H. Z., Ng, L. C., Chia, P. Y., et al. (2025) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2025.06.012 · Cited by 16
Interventions to control nosocomial infections: study designs and statistical issues
Wolkewitz, M., Barnett, A. G., Palomar, M., Frank, U., & Schumacher, M. C. (2013) · Journal of Hospital Infection · doi:10.1016/j.jhin.2013.09.015 · Cited by 15
Effect of Uterine Rupture on a Hospital's Future Rate of Vaginal Birth After Cesarean Delivery
Riddell, C. A., Kaufman, J. S., Hutcheon, J. A., Strumpf, E., Teunissen, P. W., & Abenhaim, H. A. (2014) · Obstetrics and Gynecology · doi:10.1097/aog.0000000000000545 · Cited by 17
Sinnott, S., Smeeth, L. L., Williamson, E., Perel, P., Nitsch, D., Tomlinson, L. A., & Douglas, I. (2019) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4808 · Cited by 16
“We used incidence of Herpes Zoster as a negative control outcome to explore unmeasured confounding between fourth‐ lineanti‐hypertensive groups (see the Supporting Information for codes used to identify all outcomes).(23)”
MethodWei, X., Huang, P., Gao, C., Shen, S., Tu, S., Guo, Y., Zhang, L., Lu, M., Lu, J., Wang, C. C., et al. (2021) · Pediatric Allergy and Immunology · doi:10.1111/pai.13703 · Cited by 38
Rudan, I., Millington, T., Antal, K., Grange, Z., Fenton, L., Sullivan, C. B., Buelo, A., Wood, R., Woolford, L., Swann, O. V., et al. (2022) · The Lancet Regional Health - Europe · doi:10.1016/j.lanepe.2022.100513 · Cited by 19
Association Between Hospital-Documented Atrial Fibrillation and Central Retinal Artery Occlusion
Lusk, J. B., Song, A., Unnithan, S., Al‐Khalidi, H. R., Delic, A., de Havenon, A., Biousse, V., Schrag, M., Poli, S., Piccini, J. P., et al. (2023) · Stroke · doi:10.1161/strokeaha.122.042292 · Cited by 15
Sun, Y., Wu, C. S., & Olsen, J. (2014) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.3691 · Cited by 15
Child maltreatment hospitalisations in Hong Kong: incidence rate and seasonal pattern
Ip, P., Ho, F. K. W., Chan, K. L., Yip, P. S.-F., Lau, J. T. F., Wong, W. H. S., Chow, C. B., & Jiang, F. (2016) · Archives of Disease in Childhood · doi:10.1136/archdischild-2015-310151 · Cited by 15
Use of Social Psychology to Improve Adherence to National Bronchiolitis Guidelines
Sprecher, E., Chi, G., Ozonoff, A., Cox, J. E., Patel, N., & Conroy, K. (2018) · PEDIATRICS · doi:10.1542/peds.2017-4156 · Cited by 15
Reduced risk of cancer among low‐dose aspirin users: Data from French health care databases
AJROUCHE, A., Rycke, Y. D., Dalichampt, M., Messika–Zeitoun, D., Hulot, J., Estellat, C., & Tubach, F. (2019) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4870 · Cited by 15
Kwon, S., Lee, S., Choi, E., Choe, W.-S., Lee, E., Jung, J., Han, K., Oh, S., & Lip, G. Y. H. (2020) · American Heart Journal · doi:10.1016/j.ahj.2020.08.006 · Cited by 17
Greene, S. K., Levin‐Rector, A., McGibbon, E., Baumgartner, J., Devinney, K., Ternier, A., Sell, J. L., Kahn, R. J., & Kishore, N. (2021) · Vaccine X · doi:10.1016/j.jvacx.2021.100134 · Cited by 15
“Findings from the regression discontinuity design would have been further strengthened with a negative control outcome [31,40], i.”
Marion, J. W., Lee, C., Lee, C. S., Wang, Q., Lemeshow, S. A., Buckley, T. J., Saif, L. J., & Lee, J. (2014) · PLoS ONE · doi:10.1371/journal.pone.0112029 · Cited by 15
“observational studies [31] and done by Colford et al.”
MethodDujon, A. M., Boutry, J., Tissot, S., Meliani, J., Guimard, L., Rieu, O., Újvári, B., & Thomas, F. (2022) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13938 · Cited by 15
“…control is also of key importance to estimate the background rate at which a species may naturally develop tumours (e.g. Hartung, 1942; Stephan, 1962), a parameter that is important to take into account when establishing dose– response relationships (Lipsitch et al., 2010; Zeise et al., 1987).”
BackgroundDeSilva, M. B., Mitchell, P. K., Klein, N. P., Dixon, B. E., Tenforde, M. W., Thompson, M. G., Naleway, A. L., Grannis, S. J., Ong, T. C., Natarajan, K., et al. (2022) · The Journal of Infectious Diseases · doi:10.1093/infdis/jiac458 · Cited by 14
Kennedy‐Shaffer, L. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae019 · Cited by 17
“22,23 The functional form of the parallel trends assumption can also be changed by transforming the outcome variable.”
BackgroundChen, L., Sha, S., Brenner, H., & Schöttker, B. (2024) · Maturitas · doi:10.1016/j.maturitas.2024.107998 · Cited by 18
Park, C., Richardson, D. B., & Tchetgen, E. J. T. (2024) · Biometrics · doi:10.1093/biomtc/ujae027 · Cited by 20
“An approach that is sometimes used in practice to assess residual confounding bias, is to check whether known null effects can be recovered free of bias, by evaluating whether the exposure or treatment of interest is found to be associated with a so-called negative control outcome (NCO), upon adjusting for measured co…”
Bi, Q., Dickerman, B. A., Nguyen, H. Q., Martin, E. T., Gaglani, M., Wernli, K. J., Balasubramani, G. K., Flannery, B. L., Lipsitch, M., Cobey, S. E., et al. (2024) · The Journal of Infectious Diseases · doi:10.1093/infdis/jiae220 · Cited by 19
Self-citationAntoine, A., Pérol, D., Robain, M., Bachelot, T., Choquet, R., Jacot, W., Yahia, B. B. H., Grinda, T., Delaloge, S., Lasset, C., et al. (2024) · European Journal of Cancer · doi:10.1016/j.ejca.2024.115072 · Cited by 15
Huang, S., Otsuka, R., Nishita, Y., Meng, L., Hsiao, F., Shimokata, H., Chen, L., & Arai, H. (2024) · Journal of Cachexia Sarcopenia and Muscle · doi:10.1002/jcsm.13660 · Cited by 17
Conover, M. M., Ryan, P. B., Chen, Y., Suchard, M. A., Hripcsak, G. M., & Schuemie, M. J. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae317 · Cited by 17
Four targets: an enhanced framework for guiding causal inference from observational data
Lu, H., Li, F., Lesko, C. R., Fink, D. S., Rudolph, K. E., Harhay, M. O., Rentsch, C. T., Fiellin, D. A., & Gonsalves, G. (2024) · International Journal of Epidemiology · doi:10.1093/ije/dyaf003 · Cited by 14
Peng, Z., Hua, Y., Huang, W., Wu, J., & Ou, H. (2025) · BMC Medicine · doi:10.1186/s12916-025-03955-w · Cited by 14
“Third, considering the potential of unmeasured confounders (i.e., health awareness), negative control analyses were conducted [31], where influenza events that occurred 7 or 13 days following vaccination, influenza events in the sampled cohort with 50,000 subjects obtained from a vaccine-mismatched…”
Abiodun, A., Ju, C., Welch, C. A., Lai, J. H., Tyrer, F., Chambers, P., Paley, L., Vernon, S., Goodwin, A., Ludman, P., et al. (2025) · JACC CardioOncology · doi:10.1016/j.jaccao.2025.01.019 · Cited by 14
Dillon, P., Smith, S. M., Gallagher, P., & Cousins, G. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-022927 · Cited by 15
“Negative controls are a tool for detecting confounding bias in observational studies to help identify potential non-causal associations.(41) In negative control tests, conditions are reproduced that cannot involve the hypothesised causal mechanism, but likely involve the same sources of bias, such as the healthy adher…”
BackgroundMethodInfluentialLin, C. J., Richardson, D. B., Hilborn, E. D., Weinberg, H. S., Engel, L. S., & Wade, T. J. (2019) · Epidemiology · doi:10.1097/ede.0000000000001083 · Cited by 14
“Children (≤5 years) 1,189 (21) 1,156 (23)”
Comment: Reflections on the Deconfounder
D’Amour, A. (2019) · Journal of the American Statistical Association · doi:10.1080/01621459.2019.1689138 · Cited by 14
de Tristan, M.-A., Marchand‐Martin, L., Roué, J., Anand, K. J. S., Pierrat, V., Tourneux, P., Kuhn, P., Milési, C., Benhammou, V., Ancel, P., et al. (2021) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2020.12.069 · Cited by 15
Visontay, R., Sunderland, M., Slade, T. N., Wilson, J., & Mewton, L. R. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-043985 · Cited by 14
Radin, D. I., Wahbeh, H., Michel, L., & Arnaud Delorme, A. (2021) · Physics Essays · doi:10.4006/0836-1398-34.1.79 · Cited by 14
Kern, D. M., Lovestone, S., & Cepeda, M. S. (2021) · Alzheimer s & Dementia Translational Research & Clinical Interventions · doi:10.1002/trc2.12163 · Cited by 14
Foo, D., Sarna, M. K., Pereira, G. F., Moore, H. C., & Regan, A. K. (2022) · Vaccine · doi:10.1016/j.vaccine.2021.11.084 · Cited by 15
Bann, D., Wright, L., Goisis, A., Hardy, R., Johnson, W., Maddock, J., McElroy, E., Moulton, V., Patalay, P., Scholes, S., et al. (2022) · Discover Social Science and Health · doi:10.1007/s44155-022-00021-1 · Cited by 18
“, via statistical adjustment (and sensitivity analysis to note importance of residual confounding [100]), use of fixed effects analysis [101], negative controls [102], instrumental variable analysis [103], and/or use of genetically informed designs to aid inference [104]).”
MethodLv, S., Liu, X., Li, Z., Lu, F., Guo, M., Liu, M., Wei, J., Wu, Z., Yu, S., Li, S., et al. (2022) · Environmental Research · doi:10.1016/j.envres.2022.114746 · Cited by 14
Mehta, H. B., An, H., Ardeshir‐Rouhani‐Fard, S., Raji, M. A., Alexander, G. C., & Segal, J. B. (2022) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.122.008951 · Cited by 15
The Effect of Inhaled Corticosteroids on Pneumonia Risk in Patients With COPD-Bronchiectasis Overlap
Ritchie, A. I., Singanayagam, A., Mitchell, S., Wedzicha, J. A., Shah, A., & Bloom, C. I. (2023) · CHEST Journal · doi:10.1016/j.chest.2023.06.007 · Cited by 14
Negative controls: Concepts and caveats
de Vries, B. B. L. P., & Groenwold, R. H. H. (2023) · Statistical Methods in Medical Research · doi:10.1177/09622802231181230 · Cited by 18
Hansen, C. H. (2024) · Eurosurveillance · doi:10.2807/1560-7917.es.2024.29.7.2300259 · Cited by 14
“Supplementary analyses with negative control exposures or outcomes are recommended in observational VE studies as a way to assess such bias [14,15].”
Backgroundde Groot, M., Klungel, O. H., Leufkens, H. G. M., van Dijk, L., Grobbee, D. E., & van de Garde, E. M. W. (2014) · European Journal of Epidemiology · doi:10.1007/s10654-014-9941-0 · Cited by 17
“In other words, this approach increases the likelihood of detecting methodology and data origin as source of biases leading to the heterogeneity in the case–control study findings [28].”
BackgroundEgami, N., & Tchetgen, E. J. T. (2023) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1093/jrsssb/qkad132 · Cited by 26
“Likewise, an NCE (also known as a placebo treatment) is a treatment variable that does not causally affect the outcome of interest (Lipsitch et al., 2010).”
Identification of causal diffusion effects using placebo outcomes under structural stationarity
Egami, N. (2024) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1093/jrsssa/qnae014 · Cited by 14
“This paper also draws upon emerging literature of negative controls (Lipsitch et al., 2010).”
Tsujimoto, T., & Kajio, H. (2019) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2018.11.032 · Cited by 15
Patil, S. R., Arnold, B. F., Salvatore, A. L., Briceño, B., Colford, J. M., & Gertler, P. (2013) · World Bank, Washington, DC eBooks · doi:10.1596/1813-9450-6702 · Cited by 23
“We additionally measured respiratory symptoms (cough, congestion, difficulty breathing) and child health symptoms not plausibly influenced by the TSC program (bruising/abrasions, itchy skin/scalp) to serve as negative control outcomes [26] because we were concerned that caregiverreported symptoms could be subject to d…”
BackgroundDetection bias and the role of negative control outcomes
Núñez, I., & Matthews, A. (2025) · BMJ Medicine · doi:10.1136/bmjmed-2025-001336 · Cited by 14
“13 18 The directed acyclic graph in figure 3 illustrates the structure of an optimally (figure 3A), suboptimally (figure 3B), and incorrectly (figure 3C) selected negative control outcome to assess detection bias.”
Miles, A. C., Vojicic, J., Peyrani, P., Rosenstock, S., Li, H., Vietri, J., Qi, Y., Randall, A. E., Zhao, X., Zhu, W., et al. (2026) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(26)00115-5 · Cited by 14
Adjustment for unmeasured confounding through informative priors for the confounder-outcome relation
Groenwold, R. H. H., Shofty, I., Miočević, M., van Smeden, M., & Klugkist, I. G. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0634-3 · Cited by 14
Sanglier, T., Fabi, A., Flores, C., Flahavan, E. M., Peña-Murillo, C., Meyer, A., & Montemurro, F. (2022) · Cancers · doi:10.3390/cancers14102468 · Cited by 13
“a negative exposure control [23] to assess whether selection bias would occur irrespective of the drug defining cohort entry.”
BackgroundIncidence of Statin-Associated Adverse Events in Kidney Transplant Recipients
Bae, S., Ahn, J. B., Joseph, C., Whisler, R., Schnitzler, M. A., Lentine, K. L., KADOSH, B. S., Segev, D. L., & McAdams‐DeMarco, M. (2023) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.0000000000000124 · Cited by 13
Chen, T., Wang, T., Yiu, Z. Z. N., Lee, M., Chen, L., Chan, K. A., Griffiths, C. E. M., Ashcroft, D. M., & (GPA), T. G. P. A. (2023) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.19466 · Cited by 13
Muntner, P., Hernandez, R. K., Kent, S. T., Browning, J. E., Gilbertson, D. T., Hurwitz, K. E., Jick, S. S., Lai, E. C., Lash, T. L., Monda, K. L., et al. (2024) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.5770 · Cited by 15
“29 After controlling for a set of confounders, through adjustment, a propensity score or an alternate approach, any observed association between the treatment and a negative control outcome suggests uncontrolled confounding.”
BackgroundNakafero, G., Grainge, M. J., Card, T. R., Mallen, C. D., Nguyen‐Van‐Tam, J. S., & Abhishek, A. (2024) · The Lancet Rheumatology · doi:10.1016/s2665-9913(24)00128-0 · Cited by 15
Graham, S., Walker, J. L., Andrews, N. J., Hulme, W. J., Nitsch, D., Parker, E. P. K., & McDonald, H. I. (2024) · Open Forum Infectious Diseases · doi:10.1093/ofid/ofae598 · Cited by 13
“Negative control exposures assume no causal mechanism between the negative control exposure and outcome, and they include confounding structures that reflect those of the primary exposure [23].”
Universal Health Checkups and Risk of Incident Diabetes and Hypertension
Takeuchi, M., Shinozaki, T., & Kawakami, K. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.51813 · Cited by 13
Beware of counter-intuitive levels of false discoveries in datasets with strong intra-correlations
Kanduri, C., Mamica, M., Olstad, E. W., Zucknick, M., Li, J. J., & Sandve, G. K. (2025) · Genome biology · doi:10.1186/s13059-025-03734-z · Cited by 17
“…a distribution of the number of significant findings based on repetitions of negative control data (similar to “negative controls” used by experimental biologists [37]) can reveal if a similar counter-intuitively high number of false findings can occur given the properties of the original dataset.”
Haroon, S. M., Subramanian, A., Cooper, J. M., Anand, A., Gokhale, K. M., Byne, N., Dhalla, S., Acosta, D., Taverner, T., Okoth, K., et al. (2021) · BMC Infectious Diseases · doi:10.1186/s12879-021-05951-w · Cited by 13
Does comprehensive education reduce health inequalities?
Popham, F., & Iannelli, C. (2021) · SSM - Population Health · doi:10.1016/j.ssmph.2021.100834 · Cited by 16
“BMI at baseline will be used as a negative control in sensitivity analysis (Lipsitch et al., 2010).”
MethodLevintow, S. N., Orroth, K. K., Breskin, A., Park, A. S., Flores‐Arredondo, J. H., Dluzniewski, P. J., Návar, A. M., Sørensen, H. T., & Brookhart, M. A. (2021) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.5396 · Cited by 13
“Negative control methods are increasingly used to detect uncontrolled confounding between treatment groups in observational studies.(11,12) Negative control outcomes have no plausible mechanism by which they can be caused by the treatment of interest but should share a confounding structure with the treatment-outcome…”
MethodYu, G., Kim, D., Sung, J., Jang, E., Yu, H. T., Kim, T., Pak, H., Lee, M., Lip, G. Y. H., Yang, P., et al. (2023) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.1050744 · Cited by 13
Chen, S., Lin, C., Chan, Y., Wu, V. C., Cheng, Y., Tung, Y., Hsiao, F., Chen, D., Hung, K., Chou, A., et al. (2023) · Circulation Journal · doi:10.1253/circj.cj-22-0682 · Cited by 13
Htoo, P. T., Paik, J. M., Alt, E. M., Kim, D. H., Wexler, D. J., Kim, S. C., & Patorno, E. (2023) · The Journals of Gerontology Series A · doi:10.1093/gerona/glad075 · Cited by 14
Baclofen and the risk of fall and fracture in older adults: A real‐world cohort study
Hwang, Y. J., Chang, A. R., Brotman, D. J., Inker, L. A., Grams, M. E., & Shin, J. (2023) · Journal of the American Geriatrics Society · doi:10.1111/jgs.18665 · Cited by 14
“32 We selected nephrolithiasis as the negative control outcome a priori because there was no known association with the study muscle relaxants.”
Jackson, M. L., Nelson, J. C., & Jackson, L. A. (2011) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.2160 · Cited by 13
“Further research into other exposure/outcome pairs is needed, particularly in situations where true causal relationships are known or can be inferred through the use of control periods, control exposures, or control outcomes.(22)”
BackgroundGottlieb, A., Yanover, C., Cahan, A., & Goldschmidt, Y. (2017) · BMJ Open Diabetes Research & Care · doi:10.1136/bmjdrc-2017-000435 · Cited by 17
Metadata
- Authors
- Lipsitch, M., Tchetgen Tchetgen, E., & Cohen, T.
- Affiliations
- Harvard University (US)
- Brigham and Women's Hospital (US)
- Corresponding author
- Lipsitch, Marc
- Venue
- Epidemiology
- Published
- 2010-05
- Volume
- 21
- Issue
- 3
- Pages
- 383-388
- ISSN
- 1044-3983 (ISSN-L)
- PMC
- PMC3053408
- OpenAlex
- W2009428399
- Cited by
- 1,583
- References
- 25
- Citations by year
- 2026: 278 · 2025: 215 · 2024: 183 · 2023: 138 · 2022: 153 · 2021: 132 · 2020: 107 · 2019: 75 · 2018: 72 · 2017: 71 · 2016: 56 · 2015: 32 · 2014: 26 · 2013: 25 · 2012: 11
- FWCI
- 11.64
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- Funders
- National Institutes of Health (5u01gm076497-03, 1dp2od006663-01)
- National Institute of General Medical Sciences (U54GM088558, U01GM076497, 5U01GM076497)
- National Institute of Environmental Health Sciences (T32ES007142)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- negative control, confounding, spurious association, exposure control, outcome control, observational study validity, recall bias, experimental design, causal inference
- MeSH terms
- Humans, Bias, Causality, Confounding Factors, Epidemiologic, Observation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.