Negative Controls: A Tool for Detecting Confounding and Bias in Observational Studies
Open accessThe DOI matches its Crossref record; 24 of the 25 references it cites were found in a registry or an index.
Abstract
Noncausal associations between exposures and outcomes are a threat to validity of causal inference in observational studies. Many techniques have been developed for study design and analysis to identify and eliminate such errors. Such problems are not expected to compromise experimental studies, where careful standardization of conditions (for laboratory work) and randomization (for population studies) should, if applied properly, eliminate most such noncausal associations. We argue, however, that a routine precaution taken in the design of biologic laboratory experiments--the use of "negative controls"--is designed to detect both suspected and unsuspected sources of spurious causal inference. In epidemiology, analogous negative controls help to identify and resolve confounding as well as other sources of error, including recall bias or analytic flaws. We distinguish 2 types of negative controls (exposure controls and outcome controls), describe examples of each type from the epidemiologic literature, and identify the conditions for the use of such negative controls to detect confounding. We conclude that negative controls should be more commonly employed in observational studies, and that additional work is needed to specify the conditions under which negative controls will be sensitive detectors of other sources of error in observational studies.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Finally, in order to evaluate the specificity of the association to ADHD medication, the tenth used selective serotonin reuptake inhibitors (SSRIs) as a negative control medication exposure (27, 30).”
From ADHD Medication and Substance-Related Problems (2017) · cites this paper as [30]
In the citing paper
“The use of ‘negative controls’ for outcomes or exposures can be helpful to assess the likely presence of unmeasured or residual confounding in observational research [51].”
From COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology (2019) · cites this paper as [51]
In the citing paper
“In a third approach–negative control studies–we would expect an unbiased negative control study to be null and the ‘real’ study not-null if there is a true causal effect and the assumptions of this approach are met (see Table 1).39,40”
From Triangulation in aetiological epidemiology (2017) · cites this paper as [39]
Citations
Cites
(1979) · Medical Entomology and Zoology · Cited by 11,252
The environment and disease: association or causation?
Hill, A. B. (2015) · Journal of the Royal Society of Medicine · doi:10.1177/0141076814562718 · Cited by 7,285
The Environment and Disease: Association or Causation?
Hill, A. B. (1965) · Proceedings of the Royal Society of Medicine · doi:10.1177/003591576505800503 · Cited by 6,974 · cited 1×
“19 Hill argued that causal inferences were more credible if the exposure (in his example, nickel mining) was associated with only certain types of outcomes (death from lung and nose cancer but not death from other cancers), and if the outcome was associated with one kind of exposure (nickel mining)…”
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
A structural approach to selection bias
Hernán, M. A., Hernández-Díaz, S., & Robins, J. M. (2004) · Epidemiology · doi:10.1097/01.ede.0000135174.63482.43 · Cited by 2,844
An introduction to instrumental variables for epidemiologists
Greenland, S. (2000) · International Journal of Epidemiology · doi:10.1093/ije/29.4.722 · Cited by 1,156
Instruments for Causal Inference: An Epidemiologist's Dream?
Hernán, M. A., & Robins, J. M. (2006) · Epidemiology · doi:10.1097/01.ede.0000222409.00878.37 · Cited by 961
Vaccines for preventing influenza in the elderly
Demicheli, V., Jefferson, T., Di Pietrantonj, C., Ferroni, E., Thorning, S., Thomas, R. E., & Rivetti, A. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004876.pub4 · Cited by 695
Impact of Influenza Vaccination on Seasonal Mortality in the US Elderly Population
(2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.3.265 · Cited by 543 · cited 1×
“8 To test this hypothesis, Jackson et al 7 reproduced earlier estimates of the protective effect of influenza vaccination, but then repeated the analysis for two sets of negative control outcomes, and showed that the protective effect was observed even in circumstances where the vaccine could not…”
Mortality benefits of influenza vaccination in elderly people: an ongoing controversy
Simonsen, L., Taylor, R. J., Viboud, C., Miller, M. A., & Jackson, L. A. (2007) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(07)70236-0 · Cited by 520 · cited 1×
“However, older age is associated with a less robust immune response to influenza vaccination, and ecological data suggest that the benefits measured in observational studies far exceed the corresponding benefits expected at the population level when influenza vaccination rates have increased among the elderly.(6) Impo…”
Cited by
Items 701–800 of 1,002
Keys, M. T., Serra‐Burriel, M., Martínez-Lizaga, N., Pellisé, M., Balaguer, F., Sánchez, A., Bernal‐Delgado, E., & Castells, A. (2020) · International Journal of Epidemiology · doi:10.1093/ije/dyaa166 · Cited by 16
“These results suggest that our findings are unlikely to be the result of unmeasured secular trends in mortality in the target population, or CRC-specific trends in mortality shared across all age bands.(52) Our results also captured a delayed and increasing impact of screening over time, consistent with results from r…”
BackgroundResultCredible Mendelian Randomization Studies in the Presence of Selection Bias Using Control Exposures
Yang, Z., Schooling, C. M., & Kwok, M. K. (2021) · Frontiers in Genetics · doi:10.3389/fgene.2021.729326 · Cited by 15
“In epidemiological studies, a formal approach has been described in detail and suggested as a means of detecting residual confounding, selection bias, and measurement bias (Lipsitch et al., 2010; Arnold et al., 2016).”
BackgroundInterstate Variation in Modifiable Risk Factors and Cardiovascular Mortality in the United States
Patel, S. A., Narayan, K. M. V., Ali, M. K., & Mehta, N. K. (2014) · PLoS ONE · doi:10.1371/journal.pone.0101531 · Cited by 13
“Second, we examined transport accident mortality (ICD-10 codes V01-V99) and allcause mortality among infants (under 1 year of age; mortality is defined as number of deaths per 1000 live births) as ‘‘negative controls’’ [16] based on the assumption that these deaths should not be causally associated with lifestyle-rela…”
BackgroundLee, W. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-18 · Cited by 13
“This is in contrast to other approaches dealing with unmeasured confounding, such as the methods of negative control [10,11], the instrumental variable [12,13], and the latent variable [14], where only one or a few variables are considered.”
ResultSköldberg, F., Olén, O., Ekbom, A., & Schmidt, P. T. (2018) · Diseases of the Colon & Rectum · doi:10.1097/dcr.0000000000001086 · Cited by 13
Zhang, H. T., McGrath, L. J., Ellis, A. R., Wyss, R., Lund, J. L., & Stürmer, T. (2019) · American Journal of Epidemiology · doi:10.1093/aje/kwz083 · Cited by 13
Associations Between Structural Stigma and Psychopathology Among Early Adolescents
Martino, R. M., Weissman, D. G., McLaughlin, K. A., & Hatzenbuehler, M. L. (2023) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2023.2272936 · Cited by 14
“This analysis serves as a negative control approach (Lipsitch et al., 2010); inferences are strengthened if there is no association among groups where we would not expect it.”
Wu, C., Shah, B. R., Sharma, A., Sheng, Y., Liu, P. P., Kopp, A., Saskin, R., Edwards, J. D., & Swardfager, W. (2024) · European Journal of Heart Failure · doi:10.1002/ejhf.3464 · Cited by 13
Shin, A., Shin, J., & Kang, E. H. (2024) · Diabetes Care · doi:10.2337/dc24-1652 · Cited by 13
Tan, S. T., Rodríguez-Barraquer, I., Kwan, A. T., Blumberg, S., Park, H. J., Hutchinson, J., Leidner, D., Lewnard, J. A., Sears, D., & Lo, N. C. (2025) · Nature Communications · doi:10.1038/s41467-024-55029-9 · Cited by 14
“We performed a negative control analysis by testing a negative control exposure of in fl uenza vaccination in roommates (instead of COVID-19 vaccination) 31 .”
Corsi, D. J., Morris, T., Reed, Z. E., & Smith, G. D. (2025) · European Journal of Epidemiology · doi:10.1007/s10654-025-01233-w · Cited by 13
“Specifically, we use paternal cannabis use during pregnancy, which is not available in the Ontario registry as a negative control exposure for maternal cannabis use [17, 18].”
Robinson, C., Fan, E., Grandi, S. M., Urner, M., & Parekh, R. S. (2025) · The Lancet Child & Adolescent Health · doi:10.1016/s2352-4642(25)00131-2 · Cited by 13
Antoon, J. W., Williams, D. J., Bruce, J., Sekmen, M., Zhu, Y., & Grijalva, C. G. (2025) · JAMA Neurology · doi:10.1001/jamaneurol.2025.1995 · Cited by 13
Mendelian Randomization Methods for Causal Inference: Estimands, Identification and Inference
Yao, M., Wang, A., Li, X., & Liu, Z. (2026) · Statistics in Medicine · doi:10.1002/sim.70394 · Cited by 15
“To mitigate these biases, researchers often apply methods such as adjusting for principal components, matching, or the use of negative controls [67, 135–137].”
Weinstein, R. B., Ryan, P. B., Berlin, J. A., Schuemie, M. J., Swerdel, J., & Fife, D. (2020) · Drug Safety · doi:10.1007/s40264-020-00950-3 · Cited by 12
Giuseppe, D. D., Scognamiglio, V., Malferrari, D., Nodari, L., Pasquali, L., Gualtieri, M. L., Scarfì, S., Mirata, S., Tessari, U., Hanuskovà, M., et al. (2021) · Minerals · doi:10.3390/min11121378 · Cited by 13
“Bioassays invariably require positive and negative controls in their experimental procedures [41,42].”
BackgroundBakker, S., Burggraaf, J. L. I., Kruip, M. J. H. A., van der Meer, F. J. M., Lijfering, W. M., & van Rein, N. (2022) · Thrombosis and Haemostasis · doi:10.1055/a-1957-6305 · Cited by 12
Comparative effectiveness of BNT162b2 and ChAdOx1 nCoV-19 vaccines against COVID-19
Wei, J., Zhang, W., Doherty, M., Wallace, Z. S., Sparks, J. A., Lu, N., Li, X., Zeng, C., Lei, G., & Zhang, Y. (2023) · BMC Medicine · doi:10.1186/s12916-023-02795-w · Cited by 14
“, SARSCoV-2 infection occurred over the 14 days after receiving the first dose of the COVID-19 vaccine and death from cardiovascular disease [CVD] defined as a death within 30 days before or after documentation of myocardial infarction, stroke or heart failure) to evaluate for potential residual confounding [31].”
BackgroundWu, C., Wang, C., Saskin, R., Shah, B. R., Kapral, M. K., Lanctôt, K. L., Herrmann, N., Cogo‐Moreira, H., MacIntosh, B. J., Edwards, J. D., et al. (2023) · Journal of Internal Medicine · doi:10.1111/joim.13723 · Cited by 12
“To assess healthy user effects, the rate of fecal occult blood testing (FOBT) prior to censoring or events was selected as a negative outcome [38], as metformin should have no effect on cancer screening rates, but individuals with poorer health-seeking behaviors are less likely to have cancer…”
BackgroundSwanson, S. A., & Miller, M. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae036 · Cited by 12
Risk of Myocardial Infarction and Ischemic Stroke after Dental Treatments
Chen, T. T., D’Aiuto, F., Yeh, Y. C., Lai, M., Chien, K., & Tu, Y. (2018) · Journal of Dental Research · doi:10.1177/0022034518805745 · Cited by 13
Begum, M., Pilkington, R. M., Chittleborough, C., Lynch, J. W., Penno, M. A. S., & Smithers, L. G. (2020) · Diabetologia · doi:10.1007/s00125-020-05111-w · Cited by 13
Effect of previous coronary stenting on subsequent coronary artery bypass grafting outcomes
Cheng, Y., Chen, D., Wu, V. C., Chou, A., Chang, S., Chu, P., & Chen, S. (2020) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2020.09.068 · Cited by 12
Effect of Permanent Pacemaker Implantation After Valve Surgery on Long-Term Outcomes
Chi, M.-C., Hung, K., Chang, S., Wu, V. C., Chou, A., Chan, Y., Lin, C., Chang, C., Chu, P., & Chen, S. (2021) · Circulation Journal · doi:10.1253/circj.cj-20-0905 · Cited by 12
Improving Assessments of Population-level Vaccine Impact
Bruhn, C., Schuck‐Paim, C., Kürüm, E., Taylor, R. J., Simonsen, L., & Weinberger, D. M. (2016) · Epidemiology · doi:10.1097/ede.0000000000000598 · Cited by 12
Shikamura, M., Takayama, A., Takeuchi, M., & Kawakami, K. (2024) · Diabetes Obesity and Metabolism · doi:10.1111/dom.15632 · Cited by 12
Wang, S. V., Russo, M., Glynn, R. J., Bradley, M. C., He, J., Concato, J., & Schneeweiss, S. (2025) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.3621 · Cited by 11
“The BenchExCal approach, as with quantitative bias analysis 16,17 and negative con-trol methods, 18,19 attempts to quantify the direction, magnitude, and uncertainty around systematic sources of error.”
Periodontitis and Systemic Disease: The Impact of Covariate Selection
Bashir, N. Z., Woolf, B. A. R., Burgess, S., & Bernabé, E. (2025) · Journal of Dental Research · doi:10.1177/00220345251356469 · Cited by 12
“From an analytic perspective, several techniques have been described in the causal inference literature, such as the use of instrumental variables and negative control exposures (Lipsitch et al. 2010; Baiocchi et al. 2014).”
From prediction to intervention: causal digital twins for personalized clinical decision support
Vallée, A. (2026) · Journal of Translational Medicine · doi:10.1186/s12967-026-07895-8 · Cited by 17
“…safeguards include the incorporation of negative and positive control outcomes or exposures, formal sensitivity analyses to quantify the magnitude of unmeasured confounding required to overturn conclusions, and partial identification approaches that yield bounded rather than point estimates [45].”
One-Step Weighting to Generalize and Transport Treatment Effect Estimates to a Target Population
Chattopadhyay, A., Cohn, E. R., & Zubizarreta, J. R. (2023) · The American Statistician · doi:10.1080/00031305.2023.2267598 · Cited by 23
“Heuristically, we can use this in a sensitivity analysis for the other estimates by assuming that any unmeasured effect modification for the other target populations is no larger than that for the white population, in the spirit of negative controls (Lipsitch et al. 2010).”
Flanders, W. D., Waller, L. A., Zhang, Q., Getahun, D., Silverberg, M. J., & Goodman, M. G. (2022) · Epidemiology · doi:10.1097/ede.0000000000001528 · Cited by 12
Gao, Y. N., & Olfson, M. (2022) · Schizophrenia Research · doi:10.1016/j.schres.2022.09.023 · Cited by 13
The timing of parental unemployment, insurance and children's education
Mari, G., Keizer, R., & van Gaalen, R. I. (2023) · European Societies · doi:10.1080/14616696.2023.2188550 · Cited by 11
Jock, J., Kobayashi, L. C., Chakraborty, R., Chen, X., Wing, C., Berkman, L. F., Canning, D., Kabudula, C. W., Tollman, S. M., & Rosenberg, M. S. (2023) · Journal of Aging & Social Policy · doi:10.1080/08959420.2023.2195785 · Cited by 14
“This type of negative control analysis serves to check the robustness of findings using samples where the effect would not be expected to be observed (Lipsitch et al., 2010).”
Dang, L. E., Fong, E., Tarp, J. M., Clemmensen, K. K. B., Ravn, H., Kvist, K., Buse, J. B., van der Laan, M. J., & Petersen, M. L. (2023) · Journal of Clinical and Translational Science · doi:10.1017/cts.2023.656 · Cited by 14
“NCOs are sometimes used to detect bias in observational studies [29,30].”
BackgroundReich, K. M., Gill, S. S., Eckenhoff, R. G., Berger, M., Austin, P. C., Rochon, P. A., Nguyen, P., Goodarzi, Z., & Seitz, D. P. (2024) · Journal of the American Geriatrics Society · doi:10.1111/jgs.18736 · Cited by 14
“25 Individuals were followed for up to a maximum of 5 years from their index date.”
BackgroundTarget trial emulation in psychiatry: a call for more rigorous observational analyses
Szmulewicz, A. (2024) · The Lancet Psychiatry · doi:10.1016/s2215-0366(24)00104-4 · Cited by 11
Fang, Y., Mishra-Kalyani, P. S., Zhang, X., Gruber, S., Yang, S., Ding, P., Shan, M., Lee, J.-Y., van der Laan, M. J., Faries, D. E., et al. (2025) · Statistics in Biopharmaceutical Research · doi:10.1080/19466315.2025.2546360 · Cited by 13
Nelson, J. C., Marsh, T. L., Lumley, T., Larson, E. B., Jackson, L. A., & Jackson, M. L. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.01.015 · Cited by 11
“The association between influenza vaccination and risk of all-cause mortality is a useful example for studying problems of confounding in treatment effectiveness studies that rely on administrative databases [3], as strong confounding is present, and there is a natural control period before influenza season that can b…”
BackgroundAssociations between Apgar scores and children’s educational outcomes at eight years of age
Yisma, E., Mol, B. W., Lynch, J. W., Mittinty, M., & Smithers, L. G. (2020) · Australian and New Zealand Journal of Obstetrics and Gynaecology · doi:10.1111/ajo.13220 · Cited by 12
Prenatal alcohol exposure and offspring subsequent alcohol use: A systematic review
Duko, B., Pereira, G. F., Tait, R. J., Bedaso, A., Newnham, J. P., Betts, K. S., & Alati, R. (2022) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2022.109324 · Cited by 14
Wang, H.-J., Tan, E. C.-H., Chiang, T.-Y., Chen, W.-C., Shen, C., & Ueng, Y. (2022) · Journal of Food and Drug Analysis · doi:10.38212/2224-6614.3401 · Cited by 12
“A negative control outcome, bone fracture due to a vehicle accident, was used to detect and adjust for residual systematic bias [34].”
MethodÖztürk, B., Ehrenstein, V., Sørensen, H. T., Laugesen, K., & Pedersen, L. H. (2022) · Epidemiology · doi:10.1097/ede.0000000000001494 · Cited by 11
Bijlsma, M. J., Vansteelandt, S., Janssen, F., & Hak, E. (2016) · BMC Public Health · doi:10.1186/s12889-016-2986-0 · Cited by 11
“In particular, falsification endpoints (also known as negative controls) may provide a useful indicator of bias [8].”
BackgroundYeung, S. L. A., Jiang, C., Cheng, K. K., Xu, L., Zhang, W., Lam, T. H., Leung, G. M., & Schooling, C. M. (2017) · Psychiatry Research · doi:10.1016/j.psychres.2017.09.040 · Cited by 11
“Given later age at menarche is associated with taller height (Onland-Moret et al., 2005), we considered height as a positive control outcome (Lipsitch et al., 2010).”
BackgroundMedenwald, D., Vordermark, D., & Dietzel, C. T. (2018) · Clinical Epidemiology · doi:10.2147/clep.s156764 · Cited by 11
“The key idea of the negative controls approach is to select an alternative outcome that is not causally affected by the exposure of interest, which in our case is the number of RTMs, but has the same sources of bias 18 as the actual outcome of interest.”
Quinn, P. D., Chang, Z., Bair, M. J., Rickert, M. E., Gibbons, R. D., Kroenke, K., & D'Onofrio, B. M. (2021) · Pain · doi:10.1097/j.pain.0000000000002415 · Cited by 11
Lesko, C. R., & Fox, M. P. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae367 · Cited by 11
Neumiller, J. J., Herrin, J. S., SWARNA, K. S., Polley, E. C., Galindo, R. J., Umpierrez, G. E., Deng, Y., Ross, J. S., Mickelson, M. M., & McCoy, R. G. (2024) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.0000000587 · Cited by 13
“45,46 Two additional sensitivity analyses were conducted us-ing an as-treated approach, whereby patients were censored upon ( 1 ) discontinuation of the assigned treatment (no fi lls after 30 days beyond the dispensed pill count, starting from index date), death, or disenrollment from the health…”
Summit, A. G., Moseley, M. C., Chaku, N., Elam, K. K., Jacobs, W., Lederer, A. M., Vaughan, E. L., & Quinn, P. D. (2024) · Addiction · doi:10.1111/add.16716 · Cited by 12
A Guide to Causal Inference in Life-Event Studies
Lawes, M., West, S. G., & Eid, M. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459241302015 · Cited by 12
“These different comparison groups are sometimes termed negative control exposures (Lipsitch et al., 2010).”
Al-Moussally, F., Khan, S., Katukuri, V. K., Kinaan, M., & Mansi, I. A. (2025) · Drugs · doi:10.1007/s40265-025-02177-x · Cited by 12
“To ensure that we did not have residual confounders, we introduced a negative control outcome, that cannot be different between the two groups due to pharmacologic effect of medications in the two comparison groups [40].”
SGLT2 inhibitors and the risk of hepatocellular carcinoma in patients with MASLD and Type 2 diabetes
Soong, R., Chen, W., Jao, A.-T., Lee, M., Wu, S., & Lin, C. (2025) · Diabetes & Metabolism · doi:10.1016/j.diabet.2025.101698 · Cited by 13
Tu, T.-Y., Chang, R., Lai, J., Tseng, C., Chen, M., Yip, H., Hung, Y., & Wei, J. C. (2021) · Journal of Autoimmunity · doi:10.1016/j.jaut.2021.102618 · Cited by 10
Rantala, A. K., Tapia, G., Magnus, M. C., Stene, L. C., Jaakkola, J. J. K., Størdal, K., Karlstad, Ø., & Nystad, W. (2022) · European Journal of Epidemiology · doi:10.1007/s10654-022-00897-y · Cited by 10
Trager, R. J., Cupler, Z. A., Srinivasan, R., Casselberry, R. M., Perez, J. A., & Dusek, J. A. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-073258 · Cited by 10
Shin, J., & Grams, M. E. (2023) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2023.07.024 · Cited by 10
Hossin, M. Z., de la Cruz, L. F., McKay, K. A., Oberlander, T. F., Sandström, A., & Razaz, N. (2023) · International Journal of Epidemiology · doi:10.1093/ije/dyad184 · Cited by 10
Causal fairness assessment of treatment allocation with electronic health records
Zhang, L., Richter, L. R., Wang, Z., Ostropolets, A., Elhadad, N., Blei, D. M., & Hripcsak, G. M. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104656 · Cited by 13
“Negative controls have also been used to empirically assess residual bias [54,55].”
MethodInterrupted time series for assessing the causality of intervention effects
Mathes, T., Röding, D., Stegbauer, C., Laxy, M., & Pieper, D. (2024) · Deutsches Ärzteblatt international · doi:10.3238/arztebl.m2024.0150 · Cited by 13
Using negative controls to identify causal effects with invalid instrumental variables
Dukes, O., Richardson, D. B., Shahn, Z., Robins, J. M., & Tchetgen, E. J. T. (2024) · Biometrika · doi:10.1093/biomet/asae064 · Cited by 15
Andersen, K. M., Allen, K. E., Nepal, R. M., Mateus, J. S., Yu, T., Zhou, A., Porter, T. M., Lopez, S. M. C., Puzniak, L., McLaughlin, J. M., et al. (2025) · Vaccine · doi:10.1016/j.vaccine.2025.126881 · Cited by 10
Bierrenbach, A. L., Minamisava, R., Alencar, A. P., Alencar, G. P., & Andrade, A. L. (2017) · Human Vaccines & Immunotherapeutics · doi:10.1080/21645515.2017.1391431 · Cited by 14
“erwise be affected by the same set of causal relations as the main outcome of interest.(39) These deaths were defined as those”
BackgroundMethodJakubowski, A., Mai, D., Asch, S. M., & Bendavid, E. (2019) · American Journal of Public Health · doi:10.2105/ajph.2019.305084 · Cited by 10
DAG With Omitted Objects Displayed (DAGWOOD): a framework for revealing causal assumptions in DAGs
Haber, N., Wood, M. E., Wieten, S. E., & Breskin, A. (2022) · Annals of Epidemiology · doi:10.1016/j.annepidem.2022.01.001 · Cited by 12
“In others, negative controls (32), sensitivity analyses, and triangulation with other methods (33) are appropriate.”
MethodEstimands and inference in cluster‐randomized vaccine trials
Kilpatrick, K. W., Hudgens, M. G., & Halloran, M. E. (2020) · Pharmaceutical Statistics · doi:10.1002/pst.2026 · Cited by 10
Proton pump inhibitors and odds of cholangiocarcinoma: A retrospective case‐control study
Xiong, J., Wang, Y., Xu, W., Liu, Z., Wang, H., Zhang, Z., Han, Y., Yin, C., Cao, S., Yang, Z., et al. (2020) · Liver International · doi:10.1111/liv.14663 · Cited by 12
Gao, L., Man, K. K. C., Chan, E. W., Chui, C. S. L., Li, X., Coghill, D., Hon, K. E., Tse, M. L., Lum, T. Y. S., Wong, K. H. T. W., et al. (2021) · CNS Drugs · doi:10.1007/s40263-021-00824-x · Cited by 10
“If significant results are found, then unmeasured confounders might have been present and any significant findings in the main analysis would be questionable [34].”
BackgroundDebost, J.-C., Thorsteinsson, E., Trabjerg, B. B., Benros, M. E., Albiñana, C., Vilhjálmsson, B. J., Børglum, A. D., Mors, O. N., Werge, T. M., Mortensen, P. B., et al. (2022) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.13491 · Cited by 11
Jiang, L., Sun, Y., Brumpton, B. M., Langhammer, A., Chen, Y., & Mai, X. (2022) · BMC Cancer · doi:10.1186/s12885-022-10215-0 · Cited by 12
“4) To further address residual confounding by smoking, we performed negative control exposure analysis by using migraine as an alternative exposure [29] (details described in Supplementary Text 1).”
MethodEvaluation of Causal Effects and Local Structure Learning of Causal Networks
Geng, Z., Liu, Y., Liu, C., & Miao, W. (2018) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-030718-105312 · Cited by 11
Kim, G. L., Yi, Y. H., Hwang, H. R., Kim, J., Park, Y., Kim, Y. J., Lee, J. G., Tak, Y. J., Lee, S. H., Lee, S. Y., et al. (2021) · Journal of Clinical Medicine · doi:10.3390/jcm10092044 · Cited by 10
“Negative control outcomes were applied for quantification of systematic errors [23].”
MethodReal-world evidence for regulatory purposes: The example of DARWIN EU®
Raventós, B., & Prieto‐Alhambra, D. (2025) · Farmacia Hospitalaria · doi:10.1016/j.farma.2025.02.011 · Cited by 10
Guo, J., Wang, T., Cao, H. Q., Ma, Q., Tang, Y., Li, T., Wang, L., Xu, Y., & Zhan, S. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111737 · Cited by 10
Halladay, J. E., Kershaw, S., Devine, E. K., Grummitt, L. R., Visontay, R., Lynch, S. J., Ji, C., Scott, L., Bower, M., Mewton, L. R., et al. (2025) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2025.112665 · Cited by 10
Sathishkumar, N., Valenzuela, J. G. C., Le, N. H., Yong, A. K. C., Rossotti, M. A., Dahmer, J., Sklavounos, A. A., Plante, M., Brassard, D., Malic, L., et al. (2025) · Lab on a Chip · doi:10.1039/d5lc00308c · Cited by 9
Regression-based Proximal Causal Inference for Right-censored Time-to-event Data
Li, K. Q., Linderman, G. C., Shi, X., & Tchetgen, E. J. T. (2025) · Epidemiology · doi:10.1097/ede.0000000000001884 · Cited by 13
“Such approaches include negative control methods 10 , sensitivity analysis 11 , instrumental variable methods 12 , and difference-in-differences 13 .”
Xiao, Z., Li, Q., Guo, X., Long, G., Meng, Z., & Xu, C. Y. (2025) · Environmental Research · doi:10.1016/j.envres.2025.122197 · Cited by 9
Fancourt, D. E., & Bone, J. K. (2025) · American Journal of Epidemiology · doi:10.1093/aje/kwaf149 · Cited by 10
Sklepinski, S. M., Deng, Y., SWARNA, K. S., Herrin, J. S., Polley, E. C., Neumiller, J. J., Galindo, R. J., Umpierrez, G. E., Ross, J. S., Borah, B. J., et al. (2025) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2025.112910 · Cited by 9
Cagiza, C., Faustino, M., Cagiza, I., & Cajiza, A. (2025) · Sustainability · doi:10.3390/su17209336 · Cited by 14
“• We conduct diagnostics (residual plots; VIF < 3) and negative-control checks (e.g., regressing outcomes on a non-used module) to screen for spurious associations [49].”
Étiévant, L., Sampson, J. N., & Gail, M. H. (2022) · Biometrics · doi:10.1111/biom.13663 · Cited by 9
“If relative risks are homogeneous across strata of 𝑾, MH is asymptotically efficient with a few large strata (Nurminen, 1981) and consistent with sparse strata (Greenland and Robins, 1985).”
BackgroundMale-origin microchimerism and endometrial cancer: A prospective case-cohort study
Hallum, S., Petersen, G. L., Jakobsen, M. A., Pinborg, A. B., Kuhlmann, C., Tjønneland, A. M., & Kamper‐Jørgensen, M. (2022) · Cancer Epidemiology · doi:10.1016/j.canep.2022.102169 · Cited by 9
Association between prescribed opioid dose and risk of motor vehicle crashes
Quinn, P. D., Chang, Z., Pujol, T. A., Bair, M. J., Gibbons, R. D., Kroenke, K., & D'Onofrio, B. M. (2022) · Pain · doi:10.1097/j.pain.0000000000002790 · Cited by 9
“We conducted a negative control analysis to evaluate the specificity of the within-individual results to prescription opioids rather than to pain or its management more generally [45,46,59].”
Mullachery, P. H., Quistberg, D. A., Lazo, M., Indvik, K., Ferrer, C. P., López‐Olmedo, N., Colchero, M. A., & Bilal, U. (2022) · Injury Epidemiology · doi:10.1186/s40621-022-00407-4 · Cited by 9
Examining neighborhood effects on mental health utilizing a novel two-stage modeling approach
Boderie, N. W., van Kippersluis, H., Been, J. V., van Lenthe, F. J., & Groeniger, J. O. (2023) · Annals of Epidemiology · doi:10.1016/j.annepidem.2023.04.012 · Cited by 10
Big data approach in the field of gastric and colorectal cancer research
Cheung, K. S. (2024) · Journal of Gastroenterology and Hepatology · doi:10.1111/jgh.16527 · Cited by 9
“…and diuretics (loop diuretics, thiazide diuretics, aldosterone antagonists, and potassium-sparing diuretics), used as negative control exposures, 94 were not shown to have signi fi cant association with PCCRC-3y risk, hence ruling out signi fi cant residual/unmeasured confounding.”
BackgroundShimonovich, M., Thomson, H., Pearce, A., & Katikireddi, S. V. (2024) · Research Synthesis Methods · doi:10.1002/jrsm.1730 · Cited by 11
“49,50 Data extraction to identify studies evaluating residual confounding (e.g., using negative controls' 49 ).”
Bove, F., Greek, A. A., Gatiba, R., Boehm, R., & Mohnsen, M. M. (2024) · Environmental Health · doi:10.1186/s12940-024-01099-7 · Cited by 11
“Because information on smoking and alcohol consumption was unavailable, negative control diseases [16] were used to estimate prevalence differences between Camp Lejeune and Camp Pendleton.”
MethodAssociation of statin use with risk of depression and anxiety: A prospective large cohort study
Yang, Q., Yang, Z., Zeng, B., Jia, J., & Sun, F. (2024) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2024.07.015 · Cited by 11
Dietary quality and chemotherapy‐induced peripheral neuropathy in colon cancer
Compton, S. L. E., Yang, S., Madere, J. D., Weltzien, E. K., Caan, B. J., Meyerhardt, J. A., Schmitz, K. H., & Brown, J. C. (2024) · Cancer · doi:10.1002/cncr.35599 · Cited by 9
The effect of influenza vaccination on the rate of dementia amongst older adults
Appel, A. M., Janbek, J., Jensen‐Dahm, C., Laursen, T. M., & Waldemar, G. (2024) · European Journal of Neurology · doi:10.1111/ene.16489 · Cited by 9
Zhang, Y., Fan, M., Tsie, N. T. Y., Lee, E. H. M., Chang, W. C., Chen, E. Y. H., Chan, E. W., Wong, I. C. K., Chui, C. S. L., & Wong, A. Y. S. (2024) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.70036 · Cited by 11
“We selected amoxicillin/clavulanic acid as it shares similar indications (e.g., respiratory tract infections) as fluoroquinolones [6, 15] and has a relatively good neuropsychiatric safety profile, which is the recommended selection criteria for an active comparator [16, 17].”
On the Causal Interpretation of Rate-Change Methods: The Prior Event Rate Ratio and Rate Difference
van Aalst, R., Thommes, E. W., Postma, M. J., Chit, A., & Dahabreh, I. J. (2020) · American Journal of Epidemiology · doi:10.1093/aje/kwaa122 · Cited by 9
Ten Rules for Conducting Retrospective Pharmacoepidemiological Analyses: Example COVID-19 Study
Powell, M. A., Koenecke, A., Byrd, J. B., Nishimura, A., Konig, M. F., Xiong, R., Mahmood, S., Mucaj, V., Bettegowda, C., Rose, L., et al. (2021) · Frontiers in Pharmacology · doi:10.3389/fphar.2021.700776 · Cited by 10
Estimating the effect of cesarean delivery on long-term childhood health across two countries
Keshet, A., Rossman, H., Shilo, S., Barbash‐Hazan, S., Amit, G., Bivas‐Benita, M., Yanover, C., Girshovitz, I., Akiva, P., Ben‐Haroush, A., et al. (2022) · PLoS ONE · doi:10.1371/journal.pone.0268103 · Cited by 9
“One tool to detect unmeasured confounding is the use of negative controls [64].”
Costantini, I., Sallis, H., Tilling, K. M., Major‐Smith, D., Pearson, R. M., & Kounali, D. (2023) · JCPP Advances · doi:10.1002/jcv2.12141 · Cited by 10
Baclofen and the Risk of Encephalopathy: A Real-World, Active-Comparator Cohort Study
Hwang, Y. J., Chang, A. R., Brotman, D. J., Inker, L. A., Grams, M. E., & Shin, J. (2023) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2022.11.004 · Cited by 9
Distinguishing Immunologic and Behavioral Effects of Vaccination
Stensrud, M. J., Nevo, D., & Obolski, U. (2024) · Epidemiology · doi:10.1097/ede.0000000000001699 · Cited by 12
Feng, X., Liu, Y., Astell‐Burt, T., Yin, P., Page, A., Liu, S., Liu, J., Wang, L., & Zhou, M. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-009370 · Cited by 9
Metadata
- Authors
- Lipsitch, M., Tchetgen Tchetgen, E., & Cohen, T.
- Affiliations
- Harvard University (US)
- Brigham and Women's Hospital (US)
- Corresponding author
- Lipsitch, Marc
- Venue
- Epidemiology
- Published
- 2010-05
- Volume
- 21
- Issue
- 3
- Pages
- 383-388
- ISSN
- 1044-3983 (ISSN-L)
- PMC
- PMC3053408
- OpenAlex
- W2009428399
- Cited by
- 1,583
- References
- 25
- Citations by year
- 2026: 278 · 2025: 215 · 2024: 183 · 2023: 138 · 2022: 153 · 2021: 132 · 2020: 107 · 2019: 75 · 2018: 72 · 2017: 71 · 2016: 56 · 2015: 32 · 2014: 26 · 2013: 25 · 2012: 11
- FWCI
- 11.64
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- Funders
- National Institutes of Health (5u01gm076497-03, 1dp2od006663-01)
- National Institute of General Medical Sciences (U54GM088558, U01GM076497, 5U01GM076497)
- National Institute of Environmental Health Sciences (T32ES007142)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- negative control, confounding, spurious association, exposure control, outcome control, observational study validity, recall bias, experimental design, causal inference
- MeSH terms
- Humans, Bias, Causality, Confounding Factors, Epidemiologic, Observation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.