Negative Controls: A Tool for Detecting Confounding and Bias in Observational Studies
Open accessThe DOI matches its Crossref record; 24 of the 25 references it cites were found in a registry or an index.
Abstract
Noncausal associations between exposures and outcomes are a threat to validity of causal inference in observational studies. Many techniques have been developed for study design and analysis to identify and eliminate such errors. Such problems are not expected to compromise experimental studies, where careful standardization of conditions (for laboratory work) and randomization (for population studies) should, if applied properly, eliminate most such noncausal associations. We argue, however, that a routine precaution taken in the design of biologic laboratory experiments--the use of "negative controls"--is designed to detect both suspected and unsuspected sources of spurious causal inference. In epidemiology, analogous negative controls help to identify and resolve confounding as well as other sources of error, including recall bias or analytic flaws. We distinguish 2 types of negative controls (exposure controls and outcome controls), describe examples of each type from the epidemiologic literature, and identify the conditions for the use of such negative controls to detect confounding. We conclude that negative controls should be more commonly employed in observational studies, and that additional work is needed to specify the conditions under which negative controls will be sensitive detectors of other sources of error in observational studies.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Finally, in order to evaluate the specificity of the association to ADHD medication, the tenth used selective serotonin reuptake inhibitors (SSRIs) as a negative control medication exposure (27, 30).”
From ADHD Medication and Substance-Related Problems (2017) · cites this paper as [30]
In the citing paper
“The use of ‘negative controls’ for outcomes or exposures can be helpful to assess the likely presence of unmeasured or residual confounding in observational research [51].”
From COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology (2019) · cites this paper as [51]
In the citing paper
“In a third approach–negative control studies–we would expect an unbiased negative control study to be null and the ‘real’ study not-null if there is a true causal effect and the assumptions of this approach are met (see Table 1).39,40”
From Triangulation in aetiological epidemiology (2017) · cites this paper as [39]
Citations
Cites
(1979) · Medical Entomology and Zoology · Cited by 11,252
The environment and disease: association or causation?
Hill, A. B. (2015) · Journal of the Royal Society of Medicine · doi:10.1177/0141076814562718 · Cited by 7,285
The Environment and Disease: Association or Causation?
Hill, A. B. (1965) · Proceedings of the Royal Society of Medicine · doi:10.1177/003591576505800503 · Cited by 6,974 · cited 1×
“19 Hill argued that causal inferences were more credible if the exposure (in his example, nickel mining) was associated with only certain types of outcomes (death from lung and nose cancer but not death from other cancers), and if the outcome was associated with one kind of exposure (nickel mining)…”
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
A structural approach to selection bias
Hernán, M. A., Hernández-Díaz, S., & Robins, J. M. (2004) · Epidemiology · doi:10.1097/01.ede.0000135174.63482.43 · Cited by 2,844
An introduction to instrumental variables for epidemiologists
Greenland, S. (2000) · International Journal of Epidemiology · doi:10.1093/ije/29.4.722 · Cited by 1,156
Instruments for Causal Inference: An Epidemiologist's Dream?
Hernán, M. A., & Robins, J. M. (2006) · Epidemiology · doi:10.1097/01.ede.0000222409.00878.37 · Cited by 961
Vaccines for preventing influenza in the elderly
Demicheli, V., Jefferson, T., Di Pietrantonj, C., Ferroni, E., Thorning, S., Thomas, R. E., & Rivetti, A. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004876.pub4 · Cited by 695
Impact of Influenza Vaccination on Seasonal Mortality in the US Elderly Population
(2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.3.265 · Cited by 543 · cited 1×
“8 To test this hypothesis, Jackson et al 7 reproduced earlier estimates of the protective effect of influenza vaccination, but then repeated the analysis for two sets of negative control outcomes, and showed that the protective effect was observed even in circumstances where the vaccine could not…”
Mortality benefits of influenza vaccination in elderly people: an ongoing controversy
Simonsen, L., Taylor, R. J., Viboud, C., Miller, M. A., & Jackson, L. A. (2007) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(07)70236-0 · Cited by 520 · cited 1×
“However, older age is associated with a less robust immune response to influenza vaccination, and ecological data suggest that the benefits measured in observational studies far exceed the corresponding benefits expected at the population level when influenza vaccination rates have increased among the elderly.(6) Impo…”
Cited by
Items 901–1,000 of 1,002
Ng, E. S. W., Klungel, O. H., Groenwold, R. H. H., & van Staa, T. P. (2015) · Pharmaceutical Statistics · doi:10.1002/pst.1697 · Cited by 7
“The investigators opted for the combination of a new-user (or inception) cohort study design [28] with negative control outcomes [29], which had recently been shown to perform well in an extensive simulation study of some 400 data scenarios by Ryan et al.”
MethodCharles, H., Heron, J., Hickman, M., Brown, J., & Hines, L. A. (2021) · Addiction · doi:10.1111/add.15371 · Cited by 6
“tions that generally affected the level of reported use of both legal and illicit substances, or whether differences may be attributable to participant report bias resulting from the illicit nature of the drug use [12].”
BackgroundSoohoo, M., Hashemi, L., Hsiung, J., Moradi, H., Budoff, M. J., Kovesdy, C. P., Kalantar‐Zadeh, K., & Streja, E. (2021) · Journal of the American Heart Association · doi:10.1161/jaha.121.022988 · Cited by 7
Practical data considerations for the modern epidemiology student
Tran, N. K., Lash, T. L., & Goldstein, N. D. (2021) · Global Epidemiology · doi:10.1016/j.gloepi.2021.100066 · Cited by 7
“For the epidemiology student, practical approaches that are relatively simple to implement and impose fewer assumptions such as estimation of the E-value (the minimum strength of association of an unmeasured confounder to fully explain away an exposure-outcome relation as measured on the risk ratio scale) [41] or inco…”
Doubly robust proximal causal inference under confounded outcome-dependent sampling
Li, K. Q., Shi, X., Miao, W., & Tchetgen, E. T. (2026) · Biometrics · doi:10.1093/biomtc/ujag157 · Cited by 6
Elahy, V., Jiang, L., Lee, S., & Odegaard, A. (2023) · Obesity · doi:10.1002/oby.23688 · Cited by 6
Dixon, P., Martin, R. M., & Harrison, S. (2024) · Medical Decision Making · doi:10.1177/0272989x241232607 · Cited by 6
“We estimated sex-stratified linear regressions of each economic outcome against the breast cancer PRS (for men) and the prostate cancer PRS (for women) controlling for age, 40 genetic principal components, and indicators for UK Biobank recruitment center.”
MethodInfluentialLane, N. E., Bai, L., Seitz, D. P., Juurlink, D. N., Paterson, J. M., Guan, J., & Stukel, T. A. (2024) · Journal of the American Geriatrics Society · doi:10.1111/jgs.18930 · Cited by 7
“29 We also assessed the relationship between initiating bupropion, a hyponatremia-independent antidepressant, and the primary outcome to assess for pro-topathic bias: potential confounding by examining antidepressant initiation in individuals presenting with depressive symptoms linked to an…”
BackgroundThern, E., Carslake, D., Smith, G. D., Tynelius, P., & Rasmussen, F. E. (2017) · Alcohol and Alcoholism · doi:10.1093/alcalc/agx069 · Cited by 6
“Furthermore we were able to compare our estimates from the potentially exposed cohorts to a negative control cohort, which gave us the opportunity to detect any potential bias or confounding in relation to the regions in which the alcohol policy change was implemented (Lipsitch et al., 2010; Gage et al., 2016).”
BackgroundResultInfluentialAnalysis of counts for cluster randomized trials: Negative controls and test‐negative designs
Dufault, S. M., & Jewell, N. P. (2020) · Statistics in Medicine · doi:10.1002/sim.8488 · Cited by 6
Yang, Z., Schooling, C. M., & Kwok, M. K. (2021) · COPD Journal of Chronic Obstructive Pulmonary Disease · doi:10.1080/15412555.2021.1955848 · Cited by 7
Causation and causal inference
Hoggatt, K. J., VanderWeele, T. J., & Greenland, S. (2021) · doi:10.1093/med/9780198816805.003.0037 · Cited by 6
Association Between Statins and Mortality
Jackson, L. A., & Nelson, J. C. (2012) · The Journal of Infectious Diseases · doi:10.1093/infdis/jis344 · Cited by 6
Interpregnancy interval and adverse perinatal outcomes: A within‐individual comparative method
Sevoyan, M. K., Geraci, M., Frongillo, E. A., Liu, J., & Boghossian, N. S. (2024) · Health Science Reports · doi:10.1002/hsr2.2313 · Cited by 5
“(2) We used post ‐ birth intervals 9,13 in a negative controls 24 analysis to detect the potential influence of unmeasured confounding on the association between post ‐ birth IPI (IPI between second and third birth) and odds of perinatal outcomes of the second birth for our sample of individuals…”
MethodLe, T., Huynh, P., Sharma, A., Setty, A. R., Blanchard, M., Ng, H. K. T., Luo, Y., & Le, T. Q. (2024) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2024.102010 · Cited by 7
Weaver, J. C., Voss, E. A., Cafri, G., Beyrau, K. K., Nashleanas, M., & Suruki, R. (2024) · BMC Medical Research Methodology · doi:10.1186/s12874-024-02428-7 · Cited by 6
Pérez, T. V., Øvergaard, K. R., Frigessi, A., & Biele, G. (2025) · International Journal of Epidemiology · doi:10.1093/ije/dyaf010 · Cited by 6
“Additional sensitivity analyses, including negative controls [37] and proximal causal inference [38, 39] are available in Supplementary Material, Section E. Due to apparently conflicting results with previous studies, we conducted a post hoc informal replication of the analytic strategy of [19] on…”
Rotem, R. S., Hernández–Díaz, S., Hauser, R. B., & Weisskopf, M. G. (2025) · Current Environmental Health Reports · doi:10.1007/s40572-025-00488-5 · Cited by 6
“…negative control exposures —defined as exposures expected to be unrelated to the outcome, where any observed statistical association may indicate residual confounding or bias [395]—can be problematic in many contexts, since preconception exposures may be etiologically relevant through the father.”
Lawal, O. D., Aronow, H. D., Shobayo, F. O., Zhang, Y., Hume, A. L., Taveira, T. H., Matson, K. L., Gold, J. L., & Wen, X. (2025) · Pharmacotherapy The Journal of Human Pharmacology and Drug Therapy · doi:10.1002/phar.70055 · Cited by 6
Review of tools to support Target Trial Emulation
van Hal, C. A., Bernstam, E. V., & Johnson, T. R. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104897 · Cited by 5
Chowers, M., Atias, D., Gottesman, B., Low, M., & Obolski, U. (2025) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2025.11.013 · Cited by 5
Duke, A. E. E., Harri, B. I., Iyamu, I. O., Obi‐Jeff, C., Dahiru, A. M. C., Ojo, T. M.-F., Musami, U. B., Said, J. M., Agyapong, V. I. O., Orji, R., et al. (2025) · Journal of Primary Care & Community Health · doi:10.1177/21501319251406840 · Cited by 6
“19,20 E -values quantify how strong an unmeasured confounder would need to be to nullify an observed association.”
Ioannou, G. N., Berry, K., Yan, L. G., Huang, Y., Lin, H., Bui, D. P., Hynes, D. M., Boyko, E. J., Ferguson, J. M., Aslan, M., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-67796-0 · Cited by 5
Taran, S., Citerio, G., Taccone, F. S., Rezoagli, E., Liu, K., McCREDIE, V. A., Laffey, J. G., Calabrò, L. A., Patroniti, N., Graziano, F., et al. (2026) · Intensive Care Medicine · doi:10.1007/s00134-025-08241-0 · Cited by 5
Niven, C. G., Clark, B., Floess, E., Chirwa, B., Matekenya, M., Budden, E., Cadono, S., Chavula, J., Chisamanga, V., Dzinkambani, A., et al. (2026) · Environmental Science & Technology · doi:10.1021/acs.est.5c08820 · Cited by 7
Iyer, H. S., Perlstein, M. M., Joseph, S. A., Roscoe, C., Zeinomar, N., Cole, A. P., Schumacher, F. R., Beebe‐Dimmer, J., Ruterbusch, J. J., Wiggins, C. L., et al. (2026) · Environmental Research · doi:10.1016/j.envres.2026.123909 · Cited by 6
Luo, S., Gong, Q., Ai, Y., Zhang, J., Chan, L., Wong, W. C. W., Ip, P. C. T., Chan, E. W., Tanuseputro, P., Wong, I. C. K., et al. (2026) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2026.2215 · Cited by 11
Gustavson, K., Smith, G. D., & Eilertsen, E. M. (2022) · European Journal of Epidemiology · doi:10.1007/s10654-022-00857-6 · Cited by 7
Specificity of association in epidemiology
Blanchard, T. (2022) · Synthese · doi:10.1007/s11229-022-03944-z · Cited by 6
“For instance, Petitti et al. (1986) reject the hypothesis that hormone replacement therapy (HRT) prevents cardiovascular disease on the ground that HRT is associated with a lesser risk for many other pathologies as well.”
BackgroundMethodWhat should be done and what should be avoided when comparing two treatments?
Bouvier, F. B., & Porcher, R. (2023) · Best Practice & Research Clinical Haematology · doi:10.1016/j.beha.2023.101473 · Cited by 5
Carter, A. R., Clayton, G., Borges, M. C., Howe, L. D., Hughes, R. A., Smith, G. D., Lawlor, D. A., Tilling, K. M., & Griffith, G. J. (2023) · BMC Public Health · doi:10.1186/s12889-023-16767-5 · Cited by 5
“We used positive and negative control exposures to investigate potential selection bias [32, 33].”
BackgroundMethodShen, B., Ren, H., Shardell, M. D., Falvey, J. R., & Chen, C. (2023) · Statistics in Medicine · doi:10.1002/sim.9982 · Cited by 5
“MNAR typically requires stronger assumptions for parameter identification that may not be testable in practice 17 .”
BackgroundBennett, J. C., Luiten, K. G., O’Hanlon, J., Han, P. D., McDonald, D., Wright, T., Wolf, C. R., Lo, N. K., Acker, Z., Regelbrugge, L., et al. (2024) · Vaccine · doi:10.1016/j.vaccine.2024.01.080 · Cited by 5
Application of the target trial emulation framework to external comparator studies
Arnold, K. F., Antunes, L., Coles, B., & Lee, H. (2024) · Frontiers in Drug Safety and Regulation · doi:10.3389/fdsfr.2024.1380568 · Cited by 8
“…to assess the sensitivity of results to known or suspected confounders, or indirect approaches (e.g., negative controls or instrumental variables) may be used to assess sensitivity to unmeasured confounding (Lin et al., 1998; Martens et al., 2006; Lipsitch et al., 2010; Kutcher et al., 2021).”
Tan, E. H., Rathod‐Mistry, T., Strauss, V. Y., O’Kelly, J., Giorgianni, F., Baxter, R., Brunetti, V. C., Pedersen, A. B., Ehrenstein, V., & Prieto‐Alhambra, D. (2024) · Journal of Bone and Mineral Research · doi:10.1093/jbmr/zjae059 · Cited by 5
Milder, C. M., Bellamy, M. B., Howard, S. C., Ellis, E. D., Golden, A. P., Cohen, S. S., Mumma, M. T., French, B. C., Zablotska, L. B., & Boice, J. D. (2024) · Occupational and Environmental Medicine · doi:10.1136/oemed-2023-109192 · Cited by 6
Liu, S. A., Brown, T. R., King, A. A., Lin, L. A., Rehman, S. S. U., Grucza, R. A., & Xu, K. Y. (2024) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2024.09.013 · Cited by 5
Trager, R. J., Cupler, Z. A., Srinivasan, R., Harper, E., & Perez, J. A. (2025) · PLoS ONE · doi:10.1371/journal.pone.0317663 · Cited by 6
Jiang, C., Talbot, D., Carazo, S., & Schnitzer, M. E. (2023) · medRxiv · doi:10.1002/sim.70025 · Cited by 6
Causal Inference with Unobserved Confounding: Leveraging Negative Control Outcomes Using Lavaan
Loh, W. W. (2025) · Multivariate Behavioral Research · doi:10.1080/00273171.2025.2507742 · Cited by 7
“An outcome variable such as Z endowed with these two causal characteristics is a negative control outcome (Lipsitch et al., 2010; Richardson et al., 2015).”
Knapp, E. A., Bennett, W. L., Wilson, R. F., Zhang, A. W., Tseng, E., Cheskin, L. J., Bass, E. B., Kharrazi, H. H. K., & Stuart, E. A. (2019) · Obesity · doi:10.1002/oby.22645 · Cited by 5
“served confounder can be conducted (49), and negative controls can also be used to identify and address sources of confounding (50).”
BackgroundWong, R. S. M., Ho, F. K. W., Tung, K. T. S., Fu, K., & Ip, P. (2020) · JMIR Serious Games · doi:10.2196/17112 · Cited by 5
Park, J., You, S. C., Cho, J., Park, C. H., Shin, W. G., Park, R. W., & Seo, S. I. (2022) · BMC Pharmacology and Toxicology · doi:10.1186/s40360-022-00549-7 · Cited by 5
“To estimate the systematic error in the models, we employed 123 negative control outcomes to estimate systematic error in the models (Supplementary Table 1) [20, 21].”
MethodA Long Way from Steubenville: Environmental Epidemiology in a Rapidly Changing World
Ritz, B. R. (2022) · American Journal of Epidemiology · doi:10.1093/aje/kwac031 · Cited by 5
US shelter in place policies and child abuse Google search volume during the COVID-19 pandemic
Riddell, C. A., Farkas, K., Neumann, K., Santaularia, N. J., Ahern, J., & Mason, S. M. (2022) · Preventive Medicine · doi:10.1016/j.ypmed.2022.107215 · Cited by 5
Oliveros, H., & Buitrago, G. (2022) · Pharmaceuticals · doi:10.3390/ph15111409 · Cited by 5
“However, since this study included a large number of comorbidities in the Charlson index (validated version for Colombia) [27], and it took into account patient diagnoses when admitted to intensive care, as well as negative outcomes, such as pneumonia, we were better able to control for and diagnose residual confoundi…”
BackgroundMethodHoff, R., Maltzahn, N., Hasting, R. L., Merkus, S. L., Undem, K., Kristensen, P., Mehlum, I. S., & Gran, J. M. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-062558 · Cited by 6
McCabe, R., Katikireddi, S. V., Dundas, R., & Craig, P. (2023) · The Lancet Public Health · doi:10.1016/s2468-2667(23)00121-4 · Cited by 5
Huang, Y.-H., Shen, C.-W., Chen, C.-Y., & Bair, M. (2023) · Frontiers in Pharmacology · doi:10.3389/fphar.2023.1301120 · Cited by 5
Chen, S., Tikhonovsky, N., Dhanji, N., & Ramagopalan, S. V. (2023) · Value in Health · doi:10.1016/j.jval.2023.11.010 · Cited by 5
Estimate an exposure response function with negative controls: a Bayesian nonparametric approach
Hu, J. K., Zorzetto, D., & Dominici, F. (2026) · American Journal of Epidemiology · doi:10.1093/aje/kwag073 · Cited by 5
Journal Club: Depression before and after diagnosis with amyotrophic lateral sclerosis
Cragg, J. J., Seals, R. M., Cashman, N. R., & Weisskopf, M. G. (2016) · Neurology · doi:10.1212/wnl.0000000000003363 · Cited by 5
Sedgh, R. C., Moon, J., & Jackevicius, C. A. (2021) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.120.007476 · Cited by 6
Alfageh, B. H., Besag, F., Gao, L., Ma, T., Man, K. K. C., Wong, I. C. K., & Brauer, R. (2021) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-021-05208-0 · Cited by 6
“An additional SCCS analysis was carried using a negative control outcome: a negative control outcome is a tool that is commonly applied in observational studies to examine the validity of the causal inferences (Lipsitch et al., 2010).”
MethodHtoo, P. T., Measer, G. T., Orr, R., Bohn, J. M., Sorbello, A. F., Francis, H., Dutcher, S. K., Cosgrove, A., Carruth, A., Toh, S., et al. (2022) · American Journal of Epidemiology · doi:10.1093/aje/kwac020 · Cited by 5
What Can Canada Learn From Accountable Care Organizations: A Comparative Policy Analysis
Peckham, A., Rudoler, D., Bhatia, D., Allin, S. M., Abdelhalim, R., & Marchildon, G. P. (2022) · International Journal of Integrated Care · doi:10.5334/ijic.5677 · Cited by 6
“Test the robustness of estimates by determining whether the statistical models find an effect in places they should not (e.g., outcomes not affected by policy change, time-periods before policy change) [21, 28].”
Pennington, A. F., Vaidyanathan, A., Ahmed, F. S., Manangan, A. P., Mirabelli, M. C., Sircar, K. D., Yip, F. Y., & Flanders, W. D. (2023) · Journal of Exposure Science & Environmental Epidemiology · doi:10.1038/s41370-023-00531-3 · Cited by 5
The ups and downs of observational vaccine research
Barda, N. (2023) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(23)00119-6 · Cited by 6
Doubly robust proximal synthetic controls
Qiu, H., Shi, X., Miao, W., Dobriban, E., & Tchetgen, E. T. (2024) · Biometrics · doi:10.1093/biomtc/ujae055 · Cited by 11
“In this paper, we relax this requirement by leveraging the proximal causal inference framework as in Shi et al. (2021). We develop two novel methods to estimate and draw inferences about the average treatment effect.”
BackgroundMethodModelling paternal exposure as a negative control
Brew, B. K., & Gong, T. (2020) · International Journal of Epidemiology · doi:10.1093/ije/dyaa054 · Cited by 5
Psychophysical effects in double-slit interference patterns: Response to a critiquePreprint
Radin, D. I., Wahbeh, H., Michel, L., & Arnaud Delorme, A. (2019) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/9csgu · Cited by 5
Characteristics of Vaccine Safety Observational Studies and Authors’ Attitudes: A Systematic Review
Barosa, M., & Prasad, V. K. (2024) · The American Journal of Medicine · doi:10.1016/j.amjmed.2024.10.007 · Cited by 4
Denz, R., van de Sand, H., Basten, J., Meiszl, K., Tokic, M. C., Oganowski, T., Grüter, T., Stock, S., Simić, D., Shukri, A., et al. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1583434 · Cited by 5
“This outcome has been used in other vaccine effectiveness studies for influenza-(73, 74), herpes zoster-(75), and pneumococcal vaccines (76).”
Ji, J. S., Wang, H., Song, J., Iyer, H. S., Xu, M., Zhang, Y., Farré, X., Zeng, H., & Ping, H. (2025) · Environment International · doi:10.1016/j.envint.2025.109812 · Cited by 4
Lee, C., Chen, P., Jhou, H., Chang, W.-C., Chen, H., Kao, L., Hsieh, T. Y. J., & Dai, M. (2025) · The Breast · doi:10.1016/j.breast.2025.104597 · Cited by 4
“The use of negative control outcomes (kidney stones, burns) that showed no between-group differences validates our study design ’ s ability to control confounding [22].”
Causal Inference With Regression: Leveraging Graphical Models in Tourism and Hospitality Context
Yhee, Y., & Park, J. (2025) · Journal of Smart Tourism · doi:10.1177/27652157251390601 · Cited by 4
“Nevertheless, in practical and ethical terms, it is often impossible to conduct RCTs, calling for alternate strategies to identify causal treatment in observational data (Goldstein et al., 2018; Lipsitch et al., 2010).”
Wang, Y., Li, W., Wang, L., Wei, Y., Zhao, Y., Feng, G., & Huang, L. (2026) · The Innovation Medicine · doi:10.59717/j.xinn-med.2026.100187 · Cited by 6
Mental Health Outcomes in Children after Parental Firearm Injury
Karandinos, G., Giuriato, M., Zubizarreta, J. R., Koh, K. A., Masiakos, P. T., Song, Z., & Sacks, C. A. (2026) · New England Journal of Medicine · doi:10.1056/nejmsa2502702 · Cited by 4
Hickman, R. C., Lin, J. J. Y., McAlaine, K., Punshon, T., Jackson, B. P., Bidlack, F. B., Germine, L. T., Bartell, S. M., Mangano, J. J., Farmer, J. G., et al. (2026) · Neurology · doi:10.1212/wnl.0000000000214616 · Cited by 4
Practical elements to consider when emulating a target trial
Tran, V.-T., Porcher, R., Perrodeau, É., Yaacoub, S. G., Boutron, I., & Ravaud, P. (2026) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2026.112205 · Cited by 5
Necessary conditions for valid causal inference from observational data
Cox, L. A. (2025) · Critical Reviews in Toxicology · doi:10.1080/10408444.2025.2611827 · Cited by 5
Autoimmune Disease Risk With GLP ‐ 1RA , DPP ‐4i, and SGLT2i Treatment in Patients With Diabetes
Mahajan, A., Bates, D. W., Doria, A., Foer, D., LaChance, A., & Sparks, J. A. (2026) · ACR Open Rheumatology · doi:10.1002/acr2.90046 · Cited by 5
“28 Change in body mass index (BMI) and hemoglobin A1c (HbA1c) over the study period was evaluated as a positive control analysis to con fi rm medication exposure and alignment with expected treatment effects.”
The impact of MPOWER tobacco control policies on tobacco use in African countries
Ogbodo, S. C. (2023) · Journal of Public Health · doi:10.1007/s10389-023-01961-x · Cited by 4
“…here is this: if a significant association is found between MPOWER components and unemployment rate (which should ordinarily be unrelated), then any previously observed association between MPOWER and tobacco use may have been a product of significant unmeas-ured confounding (Lipsitch et al. 2010).”
Sloot, R., Breskin, A., Colantonio, L. D., Allmon, A. G., Yu, Y., Sakhuja, S., Chen, L., Muntner, P., Brookhart, M. A., & Dhalwani, N. N. (2024) · Epidemiology · doi:10.1097/ede.0000000000001730 · Cited by 4
“9 Therefore, in the absence of residual confounding, there should be no association between the treatment and the negative control outcomes.”
BackgroundMethodChoi, Y., Kim, B. K., Won, J., Yoo, J. W., Choi, W., Jung, S., Kim, J. Y., Choi, I. Y., Chung, N., Lee, J. W., et al. (2024) · Clinical Epidemiology · doi:10.2147/clep.s454263 · Cited by 4
Zhu, G., Zhao, L., Lin, T., Yu, X., Sun, H., Zhang, Z., & Wang, T. (2024) · BMC Public Health · doi:10.1186/s12889-024-18750-0 · Cited by 5
“Furthermore, our study employs negative control meth-ods (NCMs) [32].”
MethodWang, H., Wei, Y., Lin, G., Boyer, C., Jia, K. M., Hung, C. T., Jiang, X., Li, C., Yam, C. H. K., Chow, T. Y., et al. (2024) · Emerging Microbes & Infections · doi:10.1080/22221751.2024.2421397 · Cited by 5
“To assess the robustness of the confounding control method, we also employed an external-cause re-hospitalization as a negative con-trol outcome [24], defined as a hospitalization due to injury and poisoning (ICD-9 800-999 and E000-E999).”
Hormone therapy and venous thromboembolism risk in women of menopausal age: a target trial emulation
Yeh, Y.-C., Yin, C., Chang, C. Y., & Chen, P. (2024) · European Journal of Epidemiology · doi:10.1007/s10654-024-01181-x · Cited by 4
“To strengthen our findings, we conducted an analysis using negative control outcomes, an approach applied to detect bias due to unobserved con - founders [20].”
Michalik, F., Xie, M., Eyting, M., Heß, S., Chung, S., & Geldsetzer, P. (2024) · Alzheimer s & Dementia · doi:10.1002/alz.088262 · Cited by 5
Conover, M. M., Albogami, Y., Hardin, J., Reich, C., Ostropolets, A., & Ryan, P. B. (2025) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.70087 · Cited by 4
“Violating the exchangeability assumption may limit the ability of the negative control distribution to calibrate estimates and identify residual systematic error [11, 28].”
Zhang, J., Andersen, Z. J., Napolitano, G. M., & Lim, Y.-H. (2025) · Environmental Epidemiology · doi:10.1097/ee9.0000000000000394 · Cited by 4
“For example, a negative control design 37 offers a useful tool to detect the presence of unmeasured confounding or selection bias.”
Jiesisibieke, Z. L., Cronjé, H. T., Schooling, C. M., & Burgess, S. (2025) · European Journal of Epidemiology · doi:10.1007/s10654-025-01246-5 · Cited by 4
Ashby, E., Janes, H., Follmann, D. A., Gilbert, P. B., Zhou, H., Wang, X., Girard, B., Priddy, F. H., Kublin, J. G., Corey, L., et al. (2025) · American Journal of Epidemiology · doi:10.1093/aje/kwaf176 · Cited by 6
Skapetze, L., Koller, D., Zwergal, A., Feuerriegel, S., Rubinski, A., & Grill, E. (2025) · Nature Communications · doi:10.1038/s41467-025-64192-6 · Cited by 6
“Fourth, to test for unmeasured confounding, we conducted a negative control outcome analysis 55 , which involves selecting an outcome that should not be affected by the intervention, given proper adjustment for confounders.”
Long-term prognosis in early-onset atrial fibrillation without overt cardiovascular disease
Holt, A., Strange, J. E., Gislason, G., Schou, M., Lamberts, M., Hansen, M. L., & Rasmussen, P. V. (2025) · Heart Rhythm · doi:10.1016/j.hrthm.2025.09.044 · Cited by 6
Trimethoprim-Sulfamethoxazole and Acute Respiratory Failure in Adolescents and Young Adults
Ahmadi, F., McArthur, E., García‐Bournissen, F., Rieder, M. J., & Muanda, F. T. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.45251 · Cited by 4
Antidepressant Use During Pregnancy and Birth Outcomes: Analysis From the UK , Norway, and Sweden
Martin, F. Z., Ahlqvist, V. H., Madley‐Dowd, P., Lundberg, M., Cohen, J. M., Furu, K., Rai, D., Forbes, H. J., Easey, K., Håberg, S. E., et al. (2026) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.70446 · Cited by 6
Bots, S. H., Onland‐Moret, N. C., & den Ruijter, H. M. (2023) · Frontiers in Global Women s Health · doi:10.3389/fgwh.2022.1006425 · Cited by 5
Thomsen, M. K., Nicolaisen, S. K., Pedersen, L. H., Lash, T. L., Erichsen, R., Sørensen, H. T., & Mikkelsen, E. M. (2023) · American Journal of Epidemiology · doi:10.1093/aje/kwad096 · Cited by 4
Qiu, X., Robert, A. L., McAlaine, K., Quan, L., Mangano, J. J., & Weisskopf, M. G. (2023) · Psychological Medicine · doi:10.1017/s0033291723002702 · Cited by 4
Gregori, M., de Jong, M. G., & Pieters, R. G. M. (2023) · Journal of Marketing Research · doi:10.1177/00222437231205252 · Cited by 4
Adult Ossabaw Pigs Prefer Fermented Sorghum Tea over Isocaloric Sweetened Water
Nelson, C. E., Aramouni, F., Goering, M. J., Bortoluzzi, E. M., Knapp, L. A., Herrera-Ibatá, D. M., Li, K. W., Jermoumi, R., Hooker, J., Sturek, J. T., et al. (2023) · Animals · doi:10.3390/ani13203253 · Cited by 4
“The non-oral behaviors associated with the roasted sumac tea were the same as the non-roasted sumac tea, even though gallic acid was reduced by the roasting process [66,67].”
BackgroundAbdullah, S. S., Rostamzadeh, N., Muanda, F. T., McArthur, E., Weir, M. A., Sontrop, J. M., Kim, R. B., Sedig, K., & Garg, A. X. (2024) · Canadian Journal of Kidney Health and Disease · doi:10.1177/20543581231221891 · Cited by 5
“35 To allow for a greater buffer between their initial illness/reason for prescription and this new cohort entry date, a 90-day lookback period will be used for negative-control exposure analysis.”
MethodArora, P., Gupta, A., Mackay, E., Heeg, B. M. S., & Thorlund, K. (2024) · Value in Health · doi:10.1016/j.jval.2024.04.007 · Cited by 5
Identification and estimation of causal effects in the presence of confounded principal strata
Luo, S., Li, W., Miao, W., & He, Y. (2024) · Statistics in Medicine · doi:10.1002/sim.10175 · Cited by 4
Association between Exclusive Breastfeeding and the Incidence of Childhood Nephrotic Syndrome
Park, P. G., Heo, J. S., Ahn, Y. H., & Kang, H. G. (2024) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2024.114266 · Cited by 6
Sato, S., Sasabuchi, Y., Okada, A., & Yasunaga, H. (2024) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.16243 · Cited by 4
“35 The first, second, fourth and fifth sensitivity analyses used the cumulative incidence function of new fractures, assessed by the Pepe and Mori test, similar to the primary analysis.”
Cai, X., Zeng, L., Fowler, C., Dixon, L. B., Öngür, D., Baker, J. T., Onnela, J.-P. “., & Valeri, L. (2026) · Biostatistics · doi:10.1093/biostatistics/kxag014 · Cited by 5
“Specifically, it would be of interest to leverage the negative controls that are naturally available in time series data to detect violations of the no unmeasured confounding assumption [57, 58].”
BackgroundTrager, R. J., Haering, C., Baumann, A. N., & Wright, D. S. (2025) · Thrombosis Research · doi:10.1016/j.thromres.2025.109279 · Cited by 4
Feng, S., Zhang, Y., Khanam, F., Voysey, M., Pitzer, V. E., Qadri, F., Clemens, J. D., Pollard, A. J., & Liu, X. (2025) · The Lancet Global Health · doi:10.1016/s2214-109x(25)00056-7 · Cited by 5
Metadata
- Authors
- Lipsitch, M., Tchetgen Tchetgen, E., & Cohen, T.
- Affiliations
- Harvard University (US)
- Brigham and Women's Hospital (US)
- Corresponding author
- Lipsitch, Marc
- Venue
- Epidemiology
- Published
- 2010-05
- Volume
- 21
- Issue
- 3
- Pages
- 383-388
- ISSN
- 1044-3983 (ISSN-L)
- PMC
- PMC3053408
- OpenAlex
- W2009428399
- Cited by
- 1,583
- References
- 25
- Citations by year
- 2026: 278 · 2025: 215 · 2024: 183 · 2023: 138 · 2022: 153 · 2021: 132 · 2020: 107 · 2019: 75 · 2018: 72 · 2017: 71 · 2016: 56 · 2015: 32 · 2014: 26 · 2013: 25 · 2012: 11
- FWCI
- 11.64
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- Funders
- National Institutes of Health (5u01gm076497-03, 1dp2od006663-01)
- National Institute of General Medical Sciences (U54GM088558, U01GM076497, 5U01GM076497)
- National Institute of Environmental Health Sciences (T32ES007142)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- negative control, confounding, spurious association, exposure control, outcome control, observational study validity, recall bias, experimental design, causal inference
- MeSH terms
- Humans, Bias, Causality, Confounding Factors, Epidemiologic, Observation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.