Negative Controls: A Tool for Detecting Confounding and Bias in Observational Studies
Open accessThe DOI matches its Crossref record; 24 of the 25 references it cites were found in a registry or an index.
Abstract
Noncausal associations between exposures and outcomes are a threat to validity of causal inference in observational studies. Many techniques have been developed for study design and analysis to identify and eliminate such errors. Such problems are not expected to compromise experimental studies, where careful standardization of conditions (for laboratory work) and randomization (for population studies) should, if applied properly, eliminate most such noncausal associations. We argue, however, that a routine precaution taken in the design of biologic laboratory experiments--the use of "negative controls"--is designed to detect both suspected and unsuspected sources of spurious causal inference. In epidemiology, analogous negative controls help to identify and resolve confounding as well as other sources of error, including recall bias or analytic flaws. We distinguish 2 types of negative controls (exposure controls and outcome controls), describe examples of each type from the epidemiologic literature, and identify the conditions for the use of such negative controls to detect confounding. We conclude that negative controls should be more commonly employed in observational studies, and that additional work is needed to specify the conditions under which negative controls will be sensitive detectors of other sources of error in observational studies.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Finally, in order to evaluate the specificity of the association to ADHD medication, the tenth used selective serotonin reuptake inhibitors (SSRIs) as a negative control medication exposure (27, 30).”
From ADHD Medication and Substance-Related Problems (2017) · cites this paper as [30]
In the citing paper
“The use of ‘negative controls’ for outcomes or exposures can be helpful to assess the likely presence of unmeasured or residual confounding in observational research [51].”
From COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology (2019) · cites this paper as [51]
In the citing paper
“In a third approach–negative control studies–we would expect an unbiased negative control study to be null and the ‘real’ study not-null if there is a true causal effect and the assumptions of this approach are met (see Table 1).39,40”
From Triangulation in aetiological epidemiology (2017) · cites this paper as [39]
Citations
Cites
(1979) · Medical Entomology and Zoology · Cited by 11,252
The environment and disease: association or causation?
Hill, A. B. (2015) · Journal of the Royal Society of Medicine · doi:10.1177/0141076814562718 · Cited by 7,285
The Environment and Disease: Association or Causation?
Hill, A. B. (1965) · Proceedings of the Royal Society of Medicine · doi:10.1177/003591576505800503 · Cited by 6,974 · cited 1×
“19 Hill argued that causal inferences were more credible if the exposure (in his example, nickel mining) was associated with only certain types of outcomes (death from lung and nose cancer but not death from other cancers), and if the outcome was associated with one kind of exposure (nickel mining)…”
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
A structural approach to selection bias
Hernán, M. A., Hernández-Díaz, S., & Robins, J. M. (2004) · Epidemiology · doi:10.1097/01.ede.0000135174.63482.43 · Cited by 2,844
An introduction to instrumental variables for epidemiologists
Greenland, S. (2000) · International Journal of Epidemiology · doi:10.1093/ije/29.4.722 · Cited by 1,156
Instruments for Causal Inference: An Epidemiologist's Dream?
Hernán, M. A., & Robins, J. M. (2006) · Epidemiology · doi:10.1097/01.ede.0000222409.00878.37 · Cited by 961
Vaccines for preventing influenza in the elderly
Demicheli, V., Jefferson, T., Di Pietrantonj, C., Ferroni, E., Thorning, S., Thomas, R. E., & Rivetti, A. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004876.pub4 · Cited by 695
Impact of Influenza Vaccination on Seasonal Mortality in the US Elderly Population
(2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.3.265 · Cited by 543 · cited 1×
“8 To test this hypothesis, Jackson et al 7 reproduced earlier estimates of the protective effect of influenza vaccination, but then repeated the analysis for two sets of negative control outcomes, and showed that the protective effect was observed even in circumstances where the vaccine could not…”
Mortality benefits of influenza vaccination in elderly people: an ongoing controversy
Simonsen, L., Taylor, R. J., Viboud, C., Miller, M. A., & Jackson, L. A. (2007) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(07)70236-0 · Cited by 520 · cited 1×
“However, older age is associated with a less robust immune response to influenza vaccination, and ecological data suggest that the benefits measured in observational studies far exceed the corresponding benefits expected at the population level when influenza vaccination rates have increased among the elderly.(6) Impo…”
Cited by
Items 501–600 of 1,002
Matthay, E. C., Galin, J., Rudolph, K. E., Farkas, K., Wintemute, G. J., & Ahern, J. (2017) · Annals of Internal Medicine · doi:10.7326/m17-1792 · Cited by 26
“arm and nonfirearm injuries confounded our findings (34).”
BackgroundResultMcGrath, L. J., Spangler, L., Curtis, J. R., Ehrenstein, V., Sørensen, H. T., Saul, B., Levintow, S. N., Reams, D., Bradbury, B. D., & Brookhart, M. A. (2020) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.5037 · Cited by 25
Kirby, M. A., Nagel, C., Rosa, G. A., Umupfasoni, M. M., Iyakaremye, L., Thomas, E., & Clasen, T. (2017) · International Journal of Hygiene and Environmental Health · doi:10.1016/j.ijheh.2017.05.013 · Cited by 29
“Toothache was included as a negative control (Lipsitch et al., 2010).”
Franke, M. F., Jerome, J.-G., Matias, W. R., Ternier, R., Hilaire, I. J., Harris, J. B., & Ivers, L. C. (2017) · Vaccine · doi:10.1016/j.vaccine.2017.09.025 · Cited by 24
“We also conducted a bias-indicator analysis to assess potential bias in VE estimates derived using community controls [1,16].”
Negative Control Falsification Tests for Instrumental Variable Designs
Danieli, O., Nevo, D., Walk, I., Weinstein, B., & Zeltzer, D. (2026) · American Economic Review · doi:10.1257/aer.20240636 · Cited by 35
Yuan, J., Zhang, C., Sparks, J. A., Malspeis, S., Tsoi, K. K., Kim, J. H., Fisher, B. A., Gao, F., Sumerlin, T. S., Liu, Y., et al. (2020) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.15834 · Cited by 25
Air Pollution and Autism Spectrum Disorder in Israel: A Negative Control Analysis
Magen-Molho, H., Weisskopf, M. G., Nevo, D., Shtein, A., Chen, S., Broday, D. M., Kloog, I., Levine, H., Pinto, O. Y., & Raz, R. (2021) · Epidemiology · doi:10.1097/ede.0000000000001407 · Cited by 24
Kim, K., Tsai, A. C., Sumner, J. A., & Jung, S. J. (2022) · Journal of Affective Disorders · doi:10.1016/j.jad.2022.09.117 · Cited by 23
Peng, Z., Yang, C., Kuo, S., Wu, C., Lin, W., & Ou, H. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.46928 · Cited by 23
Early Rhythm Control in Patients With Incident Atrial Fibrillation Who Had a Prior Stroke
Lee, S., Choi, E., Lee, S., Han, K., Oh, S., & Lip, G. Y. H. (2023) · JACC. Clinical electrophysiology · doi:10.1016/j.jacep.2022.11.021 · Cited by 25
Li, G., Gerlovin, H., Muñiz, M. J. F., Wise, J. K., Madenci, A. L., Robins, J. M., Aslan, M., Cho, K., Gaziano, J. M., Lipsitch, M., et al. (2023) · Epidemiology · doi:10.1097/ede.0000000000001709 · Cited by 25
“14 Due to the absence of a prespecified testing strategy in the real world, we also checked whether the frequency of testing over follow-up was similar in the vaccinated and unvaccinated groups.”
BackgroundSelf-citationA Historical Survey of Key Epidemiological Studies of Ionizing Radiation Exposure
Little, M. P., Bаzyка, D., de González, A. B., Brenner, A. V., Chumak, V. V., Cullings, H. M., Daniels, R. D., French, B. C., Grant, E. J., Hamada, N., et al. (2024) · Radiation Research · doi:10.1667/rade-24-00021.1 · Cited by 26
Ahmadi, M. N., Coenen, P., Straker, L. M., & Stamatakis, E. (2024) · International Journal of Epidemiology · doi:10.1093/ije/dyae136 · Cited by 27
“36 We conducted sensitivity analyses to minimize bias attributable to reverse causation by: (i) exclusion of participants who were obese (body mass index > 30 kg/m 2 ); (ii) exclusion of participants reporting fair or poor health; or (iii) those with an event within the first 24months of follow-up.”
MethodSundbakk, L. M., Wood, M. E., Gran, J. M., & Nordeng, H. (2019) · PLoS ONE · doi:10.1371/journal.pone.0217830 · Cited by 24
“We also performed a negative control exposure analysis to detect residual confounding [40].”
BackgroundMethodJensen, A., Andersen, P. K., & Stensballe, L. G. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-029794 · Cited by 23
“Finally, the potential presence of residual confounding was assessed by analysing hospitalisation for accidents (injuries) as a negative control outcome.(36) These types of hospitalisations are not expected to be related to previous vaccination history.”
Methodvan Rein, N., Cannegieter, S. C., Rosendaal, F. R., Reitsma, P. H., & Lijfering, W. M. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.05.011 · Cited by 23
“We therefore decided to explore this potential bias further, and repeated the analysis, only this time using exposure to blood group non-O as a ‘‘negative control,’’ meaning an exposure that, although related to the outcome, is not related to increased risk of death or severe illness [7].”
BackgroundEpidemiological methods in transition: Minimizing biases in classical and digital approaches
Mesquita, S., Perfeito, L., Paolotti, D., & Gonçalves-Sá, J. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000670 · Cited by 26
“…involves an external factor influencing both exposure and outcome; temporal bias occurs when the timing between exposure and outcome is misjudged [6,85,86]. with the winning methodology combining “traditional” and “less traditional” data sources, such as weather and online searches [17,18].”
Waning Effectiveness of the BNT162b2 Vaccine Against Infection in Adolescents in Israel
Prunas, O., Weinberger, D. M., Pitzer, V. E., Gazit, S., & Patalon, T. (2022) · Clinical Infectious Diseases · doi:10.1093/cid/ciac315 · Cited by 23
Wyss, R., Yanover, C., El‐Hay, T., Bennett, D., Platt, R. W., Zullo, A. R., Sari, G., Wen, X., Ye, Y., Yuan, H., et al. (2022) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.5500 · Cited by 30
“Negative and positive controls Another approach that has become increasingly popular for evaluating models for confounder adjustment is the use of real negative controls—exposure-outcome pairs that are not, as far as we know, causally related.(77,97) Such controls can be used to detect residual biases, for example, co…”
BackgroundRegression-based proximal causal inference
Liu, J., Park, C., Li, K., & Tchetgen, E. J. T. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae370 · Cited by 32
Brugger, K., Köfer, J., & Rubel, F. (2016) · BMC Veterinary Research · doi:10.1186/s12917-016-0710-z · Cited by 22
“Further improvements may be achieved by operating different trap types for negative control [48].”
BackgroundSelf-Reported Oil Spill Exposure and Pregnancy Complications: The GROWH Study
Harville, E. W., Shankar, A., Pao, L. Z., & Buekens, P. M. (2017) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph14070692 · Cited by 22
“Thus, the pregnancies before the oil spill serve as negative controls [29].”
BackgroundKeele, L. J., & Small, D. S. (2020) · The American Statistician · doi:10.1080/00031305.2020.1867638 · Cited by 22
“For example, we would argue that the skillful use of negative controls or placebo tests (Lipsitch et al. 2010; Rosenbaum 2005) is far more likely to make a result conclusive than whether one used matching or a black box method.”
MethodLund, I. O., Eilertsen, E. M., Gjerde, L. C., Røysamb, E., Wood, M. E., Reichborn‐Kjennerud, T., & Ystrøm, E. (2019) · Addiction · doi:10.1111/add.14573 · Cited by 24
“While this cannot be due to a direct intrauterine effect, it remains associated with these outcomes in sibling designs, where shared genetic risk factors are controlled for [31,32].”
Pan, S., Yen, Y., Feng, J., Chuang, P., Su, V. Y., Kou, Y. R., Su, W., & Chan, Y. (2019) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.14142 · Cited by 22
Challenges to estimating vaccine impact using hospitalization data
Schuck‐Paim, C., Taylor, R. J., Simonsen, L., Lustig, R. L., Kürüm, E., Bruhn, C., & Weinberger, D. M. (2016) · Vaccine · doi:10.1016/j.vaccine.2016.11.030 · Cited by 21
“One way to identify biases resulting from changes in outpatient care is the analysis of outcomes that are affected by factors that also affect the vaccine target, but are themselves unaffected by the vaccine [26].”
BackgroundImpact of drought on crime in California: A synthetic control approach
Goin, D. E., Rudolph, K. E., & Ahern, J. (2017) · PLoS ONE · doi:10.1371/journal.pone.0185629 · Cited by 28
Cohen, G., Steinberg, D. M., Keinan‐Boker, L., Yuval, Levy, I., Chen, S., Shafran-Nathan, R., Levin, N., Shimony, T., Witberg, G., et al. (2020) · European Journal of Preventive Cardiology · doi:10.1177/2047487320921987 · Cited by 22
Effects of statins and aspirin on HCC risk in alcohol-related cirrhosis: nationwide emulated trials
Kraglund, F., Christensen, D. H., Eiset, A. H., Villadsen, G. E., West, J., & Jepsen, P. (2023) · Hepatology Communications · doi:10.1097/hc9.0000000000000013 · Cited by 24
“To assess whether the study design or confounder model had residual bias, we repeated the analyses with the following “negative control” outcomes:[25] non-HCC cancer (ICD-10: Cx [excl.”
MethodResultCausal inference in the medical domain: a survey
Wu, X., Peng, S., Li, J., Zhang, J., Sun, Q., Li, W., Qian, Q., Liu, Y., & Guo, Y. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05338-9 · Cited by 31
“The primary challenge in this process is to eliminate confounding bias [30], which stems from differences in the distribution of confounding variables between the treatment and control groups.”
Tesch, F., Ehm, F., Loser, F., Bechmann, L., Vivirito, A., Wende, D., Batram, M., Buschmann, T., Menzer, S., Ludwig, M., et al. (2024) · BMC Infectious Diseases · doi:10.1186/s12879-024-10059-y · Cited by 21
“Both end-points are assumed not to be caused by a SARS-CoV-2 or Influenza infection, but subject to the same unmeas-ured exposures such as health seeking behavior (detection bias) after an infection or in case of contemporary controls also to lockdown effects [16].”
BackgroundGokhale, M., Girman, C. J., Chen, Y., Pate, V., Funk, M. J., & Stürmer, T. (2016) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.3977 · Cited by 21
The effect of statins on influenza‐like illness morbidity and mortality
Brassard, P., Wu, J. W., Ernst, P., Dell’Aniello, S., Smiechowski, B., & Suissa, S. (2016) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4112 · Cited by 22
Assessing Individual and Disseminated Effects in Network-Randomized Studies
Buchanan, A. L., Vermund, S. H., Friedman, S. R., & Spiegelman, D. L. (2018) · American Journal of Epidemiology · doi:10.1093/aje/kwy149 · Cited by 21
“In a network-randomized trial, there could be residual confounding even after adjusting for covariates Zki, so subtracting these terms accounts for possible unmeasured confounding due to selfselection of index statuswhen estimating individual and composite effects (21, 22).”
BackgroundRaymakers, A., Sin, D. D., Sadatsafavi, M., FitzGerald, J. M., Marra, C. A., & Lynd, L. D. (2020) · Respiratory Research · doi:10.1186/s12931-020-01344-w · Cited by 21
Racial, ethnic and socioeconomic disparities in SARS‐CoV‐2 infection amongst children
Mannheim, J., Konda, S., & Logan, L. K. (2022) · Paediatric and Perinatal Epidemiology · doi:10.1111/ppe.12865 · Cited by 21
Automated causal inference in application to randomized controlled clinical trials
Wu, J., Horeweg, N., de Bruyn, M., Nout, R. A., Jurgenliemk-Schulz, I. M., Lutgens, L. C. H. W., Jobsen, J. J., van der Steen‐Banasik, E. M., Nijman, H. W., Smit, V. T. H. B. M., et al. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00470-y · Cited by 26
Increase in fertility following coal and oil power plant retirements in California
Casey, J. A., Gemmill, A., Karasek, D. A., Ogburn, E. L., Goin, D. E., & Morello-Frosch, R. A. (2018) · Environmental Health · doi:10.1186/s12940-018-0388-8 · Cited by 22
“Third, we implemented a negative exposure control [31]”
MethodFerdinands, J. M., Gaglani, M., Ghamande, S. A., Martin, E. T., Middleton, D. B., Monto, A. S., Silveira, F. P., Talbot, H. K., Zimmerman, R. K., Smith, E. R., et al. (2020) · The Journal of Infectious Diseases · doi:10.1093/infdis/jiaa772 · Cited by 21
“In particular, the use of a negative control outcome as a bias indicator method and the attributes of an adequate negative control outcome should be further investigated [25].”
MethodMiao, W., & Tchetgen, E. T. (2017) · American Journal of Epidemiology · doi:10.1093/aje/kwx012 · Cited by 23
Population phenomena inflate genetic associations of complex social traits
Morris, T. T., Davies, N. M., Hemani, G., & Smith, G. D. (2020) · Science Advances · doi:10.1126/sciadv.aay0328 · Cited by 20
Cai, C. X., Nishimura, A., Bowring, M. G., Westlund, E., Tran, D., Ng, J. H., Nagy, P., Cook, M., McLeggon, J.-A., DuVall, S. L., et al. (2024) · Ophthalmology Retina · doi:10.1016/j.oret.2024.03.014 · Cited by 20
Endogenous sex steroid hormones and colorectal cancer risk: a systematic review and meta-analysis
Bouras, E., Papandreou, C., Tzoulaki, I., & Tsilidis, K. K. (2021) · Discover Oncology · doi:10.1007/s12672-021-00402-z · Cited by 20
Baum, U., Kulathinal, S., & Auranen, K. J. (2021) · Eurosurveillance · doi:10.2807/1560-7917.es.2021.26.36.2100054 · Cited by 20
“By definition, a negative control outcome must not be affected by the exposure but, at the same time, it must be affected by the same factors that influence both the actual outcome and exposure [12].”
BackgroundGilsanz, P., Young, J. G., Glymour, M. M., Tchetgen, E. J. T., Eng, C. W., Koenen, K. C., & Kubzansky, L. D. (2021) · American Journal of Epidemiology · doi:10.1093/aje/kwab253 · Cited by 23
Hart, B., & Colombo, S. M. (2022) · Aquaculture Research · doi:10.1111/are.15758 · Cited by 20
Lintunen, J., Lähteenvuo, M., Tanskanen, A. J., Tiihonen, J., & Taipale, H. T. (2022) · Journal of Affective Disorders · doi:10.1016/j.jad.2022.06.040 · Cited by 22
Nguyen, T. K., Baker, K. E., Lett, E., & Scheim, A. (2022) · Journal of Health Services Research and Policy · doi:10.1177/13558196221123413 · Cited by 13
Pedersen, N. H., Grøntved, A., Brønd, J. C., Møller, N. C., Larsen, K. T., Debrabant, B., Koch, S., Troelsen, J., Brage, S., Schipperijn, J., et al. (2023) · The Lancet Regional Health - Europe · doi:10.1016/j.lanepe.2022.100575 · Cited by 22
Antoine, A., Pérol, D., Robain, M., Delaloge, S., Lasset, C., & Drouet, Y. (2023) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djad092 · Cited by 24
“Residual bias because of unmeasured confounders should also be investigated in the future with other approaches, such as the negative controls, for example, to support these findings (44,45).”
Kong, E. L., & Ji, Y. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.20694 · Cited by 20
Yang, A., Shi, M., Wu, H., Lau, E. S. H., Cheung, J. T. K., Zhang, X., Fan, B., Chen, T., Kong, A. P. S., Luk, A. O. Y., et al. (2024) · EClinicalMedicine · doi:10.1016/j.eclinm.2024.102568 · Cited by 23
Dang, L. E., Tarp, J. M., Abrahamsen, T. J., Kvist, K., Buse, J. B., Petersen, M. L., & van der Laan, M. J. (2025) · Journal of Causal Inference · doi:10.1515/jci-2024-0041 · Cited by 27
“As demonstrated by [34] in the context of confounding bias and [35] in the context of selection bias and measurement error, an NCO is not affected by treatment but should otherwise share the same causal structure as the true outcome with respect to variables related to potential sources of bias.”
InfluentialAstell‐Burt, T., Feng, X., & Kolt, G. S. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-009803 · Cited by 20
“37 Descriptive analyses reported in tables 2 and 3 show that there were virtually no signi fi cant differences in the distribution of any of these outcomes with respect to proximity to any part of the Western Sydney Parklands in the baseline sample.”
BackgroundRichmond, R. C., & Joubert, B. R. (2017) · Epigenomics · doi:10.2217/epi-2016-0135 · Cited by 20
“A negative control approach is one that utilizes an additional exposure or outcome that would be liable to the same sources of confounding, but for which causal associations cannot be plausibly ascribed [72,73] .”
BackgroundKürüm, E., Warren, J. L., Schuck‐Paim, C., Lustig, R. L., Lewnard, J. A., Fuentes, R., Bruhn, C., Taylor, R. J., Simonsen, L., & Weinberger, D. M. (2017) · Epidemiology · doi:10.1097/ede.0000000000000719 · Cited by 20
You, S. C., Jung, S.-J., Swerdel, J. N., Ryan, P. B., Schuemie, M. J., Suchard, M. A., Lee, S., Cho, J., Hripcsak, G. M., Park, R. W., et al. (2019) · Korean Circulation Journal · doi:10.4070/kcj.2019.0173 · Cited by 20
Sironi, M., Ploubidis, G. B., & Grundy, E. M. D. (2020) · Demography · doi:10.1007/s13524-020-00855-x · Cited by 20
“Finally, to detect potential unmeasured confounding and measurement error, we used the negative controls technique designed to detect both suspected and unsuspected sources of spurious causal inference (Lipsitch et al. 2010).”
MethodCasula, M., Olmastroni, E., Galimberti, F., Tragni, E., Corrao, G., Scotti, L., & Catapano, A. L. (2020) · Atherosclerosis · doi:10.1016/j.atherosclerosis.2020.03.021 · Cited by 23
Flanders, W. D., Flanders, W. D., & Goodman, M. G. (2020) · Annals of Epidemiology · doi:10.1016/j.annepidem.2020.06.011 · Cited by 20
“The Online Supplement further discusses “alternative” outcomes, their similarity to and relationship with negative control outcomes that motivates them [29].”
BackgroundResultKlebanoff, M. A., Fried, P. A., Yeates, K. O., Rausch, J. R., Wilkins, D. G., Blei, H., Sullivan, J. A., Phillips, W. L., Wiese, A. M., Jude, A., et al. (2020) · Paediatric and Perinatal Epidemiology · doi:10.1111/ppe.12693 · Cited by 20
da Cunha, S. S., Santorelli, G., Pearce, N. E., Wright, J., Oddie, S., Petherick, E., & Pembrey, L. (2021) · Clinical & Experimental Allergy · doi:10.1111/cea.13999 · Cited by 22
Holt, A., Blanche, P., Zareini, B., Rasmussen, P. V., Strange, J. E., Rajan, D., Jensen, M. H., El‐Sheikh, M., Schjerning, A.-M., Schou, M., et al. (2021) · Heart · doi:10.1136/heartjnl-2021-319503 · Cited by 21
Juang, S., Sheng‐Kai, K., Kao, P., Wei, J. C., Yip, H., Chou, M., Hung, Y., & Chin, N.-C. (2022) · Journal of Personalized Medicine · doi:10.3390/jpm12050699 · Cited by 21
“A negative control exposure was designated to examine potential external confounders [57,58].”
BackgroundSchnake-Mahl, A. S., O’Leary, G., Mullachery, P. H., Skinner, A., Kolker, J., Roux, A. V. D., Raifman, J., & Bilal, U. (2022) · Health Affairs · doi:10.1377/hlthaff.2022.00779 · Cited by 20
Carrigan, G. J., Bradbury, B. D., Brookhart, M. A., Capra, W. B., Chia, V., Rothman, K. J., Sarsour, K., Taylor, M. D., & Brown, J. S. (2022) · Current Epidemiology Reports · doi:10.1007/s40471-022-00305-9 · Cited by 20
Arfè, A., & Corrao, G. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.12.001 · Cited by 24
Greene, S. K., Li, L., Shay, D. K., Fry, A. M., Lee, G. M., Jacobsen, S. J., Baxter, R. P., Irving, S. A., Jackson, M. L., Naleway, A. L., et al. (2012) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.3363 · Cited by 20
Izurieta, H. S., Chillarige, Y., Kelman, J. A., Forshee, R. A., Qiang, Y., Wernecke, M., Ferdinands, J. M., Lu, Y., Wei, Y., Xu, W., et al. (2018) · Clinical Infectious Diseases · doi:10.1093/cid/ciy100 · Cited by 21
Simon‐Tillaux, N., Martin, G. L., Hajage, D., Scheifer, C., Beydon, M., Dechartres, A., & Tubach, F. (2024) · BMJ Open · doi:10.1136/bmjopen-2024-086595 · Cited by 27
Hernandez, R. K., Critchlow, C. W., Dreyer, N. A., Lash, T. L., Reynolds, R. F., Sørensen, H. T., Lange, J. L., Gatto, N. M., Sobel, R. E., Lai, E. C., et al. (2025) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.3563 · Cited by 22
“35,36 When conditional exchangeability is questionable, alternative methods such as instrumental variable methods can provide treatment effect estimates that depend on different assumptions.”
Firearm Storage and Firearm Suicide
Miller, M. R., Wertz, J., Swanson, S. A., Simonetti, J. A., Zhang, Y., & Azrael, D. R. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.19266 · Cited by 21
Martin, F. Z., Fraser, A., & Zuccolo, L. (2022) · Journal of the American Heart Association · doi:10.1161/jaha.121.025102 · Cited by 19
Zheng, D., Shen, L., Wen, W., Ling, F., Miao, Z., Sun, J., & Lin, H. (2023) · Infectious Disease Modelling · doi:10.1016/j.idm.2023.09.001 · Cited by 19
“), and if there was no statistically significant reduction in scarlet fever, it indicated that bYUbUAz0 (Lipsitch et al., 2010; Shi et al., 2020), then the unmeasured confounding (economic factors and healthcare policies) directly associated with scarlet fever was not exist, which effectively weakens the influence of…”
BackgroundIsmail, S., Funk, M. J., & Flythe, J. E. (2024) · Journal of the American Society of Nephrology · doi:10.1681/asn.0000000000000336 · Cited by 19
“37,38 Taking both the inconsistent fi ndings and the limitations into consideration, we fi nd the negative control outcome of hospitalized fracture inconclusive with regard to the potential for residual confounding in the association of ondansetron with sudden cardiac death.”
BackgroundLong COVID and financial outcomes: evidence from four longitudinal population surveys
Rhead, R. D., Wels, J., Moltrecht, B., Shaw, R. J., Silverwood, R. J., Zhu, J., Hughes, A. D., Chaturvedi, N., Demou, E., Katikireddi, S. V., et al. (2024) · Journal of Epidemiology & Community Health · doi:10.1136/jech-2023-221059 · Cited by 20
“This is concerning given that we found that those with post- COVID-19 syndrome had the worst financial outcomes compared with those who had not contracted COVID-19.”
van Dijk, A. E., Dawe, K., Deanfield, J. E., Stronks, K., Gemke, R. J. B. J., Vrijkotte, T. G. M., & Lawlor, D. A. (2013) · European Journal of Preventive Cardiology · doi:10.1177/2047487313486039 · Cited by 21
Zhang, H. T., McGrath, L. J., Wyss, R., Ellis, A. R., & Stürmer, T. (2017) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4298 · Cited by 20
Hamer, M., Bauman, A. E., Bell, J. A., & Stamatakis, E. (2018) · International Journal of Epidemiology · doi:10.1093/ije/dyy272 · Cited by 19
“Although rarely used in the physical activity field, negative control comparisons are increasingly recognised as a means of strengthening causal inference.(8) The purpose of a negative control is to reproduce a condition that cannot involve the hypothesized causal mechanism, but does involve the same sources of bias a…”
BackgroundWang, Z., Ho, P. W. H., Choy, M. T. H., Wong, I. C. K., Brauer, R., & Man, K. K. C. (2018) · Drug Safety · doi:10.1007/s40264-018-0755-y · Cited by 21
Chen, J., Ho, S. Y., Leung, L. T., Wang, M. P., & Lam, T. H. (2019) · Scientific Reports · doi:10.1038/s41598-018-38266-z · Cited by 19
Marlow, R. D., Finn, A. H. R., & Henderson, J. (2019) · Thorax · doi:10.1136/thoraxjnl-2018-212203 · Cited by 19
Cohen, J. M., Wood, M. E., Hernández–Díaz, S., Ystrøm, E., & Nordeng, H. (2019) · International Journal of Epidemiology · doi:10.1093/ije/dyz170 · Cited by 19
“Unfortunately, we cannot determine the magnitude of the residual confounding bias for maternal exposure estimates, but instead provide a qualitative assessment of confounding.(9) However, a recent study in MoBa found that genetic transmission plays an important role in the association between maternal prenatal depress…”
BackgroundResultBennett, D. L., Webb, C., Mason, K., Schlüter, D. K., Fahy, K., Alexiou, A., Wickham, S. L., Barr, B., & Taylor-Robinson, D. C. (2021) · Children and Youth Services Review · doi:10.1016/j.childyouth.2021.106289 · Cited by 19
“First, to test whether associations identified in our main analysis were likely due to unmeasured confounding, we conducted negative control analyses (Lipsitch et al., 2010; Mason et al., 2021).”
BackgroundINTRODUCING PROXIMAL CAUSAL INFERENCE FOR EPIDEMIOLOGISTS
Zivich, P. N., Cole, S. R., Edwards, J. K., Mulholland, G. E., Shook‐Sa, B. E., & Tchetgen, E. J. T. (2023) · American Journal of Epidemiology · doi:10.1093/aje/kwad077 · Cited by 19
Causal versus spurious spatial exposure–response associations in health risk analysis
Cox, L. A., Popken, D. A., & Berman, D. W. (2013) · Critical Reviews in Toxicology · doi:10.3109/10408444.2013.777689 · Cited by 19
“Negative controls (for exposures or for effects) (Lipsitch et al., 2010) Do exposures predict health effects better than they predict effects that cannot be caused by exposures? Example: Does being vaccinated against influenza have a stronger association with reduction in influenza hospitalizations than with reduction…”
BackgroundInfluentialJaksa, A. A., Gibbs, L. R., Kent, S., Rowark, S., Duffield, S. J., Sharma, M., Kincaid, L., Ali, A. K., Patrick, A. R., Govil, P., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-064662 · Cited by 21
Butler, A. M., Burcu, M., Christian, J. B., Tian, F., Andersen, K. M., Blumentals, W. A., Maddox, K. E. J., & Alexander, G. C. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.11.011 · Cited by 20
Income determines the impact of cash transfers on HIV/AIDS: cohort study of 22.7 million Brazilians
da Silva, A. F., Dourado, I. C., Lua, I., Jesus, G. S., Guimarães, N. S., Morais, G. A. D. S., Anderle, R. V. R., Pescarini, J. M., Machado, D. B., Santos, C. A. D. S. T., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-44975-z · Cited by 18
“More-over, the extended sensitivity analyses showed that the inclusion of other independent variables related to socioeconomic inequalities, healthcare services, and municipal infrastructure, did not affect the results of the PBF effect estimates (appendix, pp. 22-24).”
Using Negative Control Populations to Assess Unmeasured Confounding and Direct Effects
Piccininni, M., & Stensrud, M. J. (2024) · Epidemiology · doi:10.1097/ede.0000000000001724 · Cited by 19
Xu, H., Eriksdotter, M., Ptacek, S. G., Ferreira, D., Ji, D., Bruchfeld, A. N., Xu, Y., & Carrero, J. J. (2024) · Neurology · doi:10.1212/wnl.0000000000209751 · Cited by 18
Ou, S., Chen, H., Kuo, S., Chen, T., Shih, C.-J., & Chen, Y. (2016) · Heart · doi:10.1136/heartjnl-2016-309687 · Cited by 19
Cohen, J. M., Wood, M. E., Hernández–Dı́az, S., & Nordeng, H. (2018) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4411 · Cited by 18
“antidepressants, N06A; and antipsychotics, N05A); (2) prescription medication used episodically (systemic anti‐infectives, J; opioids, N02A; anxiolytics, N05B; and hypnotics/sedatives, N05C); and (3) OTC/prescription medications where some formulations and package sizes require a prescription (paracetamol, N02BE01; no…”
BackgroundKey considerations in the design of real-world studies
Fang, Y., He, W., Wang, H., & Wu, M. (2020) · Contemporary Clinical Trials · doi:10.1016/j.cct.2020.106091 · Cited by 18
Population Estimates of GFR and Risk Factors for CKD in Guatemala
Miller, A. C., Tuiz, E., Shaw, L. C., Flood, D., García, P., Dhaenens, E., Thomson, D. R., Barnoya, J., Montano, C. M., & Rohloff, P. J. (2020) · Kidney International Reports · doi:10.1016/j.ekir.2020.12.015 · Cited by 18
“We included a question about childhood orthopedic fractures as a negative control to help assess presence of recall and selection biases.(39) All surveys were conducted within the subject’s home in a private location.”
BackgroundNewsome, S., Daniel, R., Carr, S. B., Bilton, D., & Keogh, R. H. (2021) · American Journal of Epidemiology · doi:10.1093/aje/kwab263 · Cited by 19
“As in the description of Lipsitch et al. (15), this NCE uses the treatment X and assesses its association with the NCO.”
Lone, N., Simpson, C. R., Kavanagh, K., Robertson, C., McMenamin, J., Ritchie, L. D., & Sheikh, A. (2012) · BMJ Open · doi:10.1136/bmjopen-2012-001019 · Cited by 19
“This approach of using an alternative outcome as a negative control has been shown to be a useful method for detecting residual bias.(22)”
Modifiable Factors Associated with Tuberculosis Disease in Children
Franke, M. F., Castillo, H. D., Pereda, Y., Lecca, L. W., Cárdenas, L. M. C., Fuertes, J., Murray, M. B., Bayona, J. N., & Becerra, M. C. (2013) · The Pediatric Infectious Disease Journal · doi:10.1097/inf.0000000000000000 · Cited by 18
Benjamin‐Chung, J., Sultana, S., Halder, A. K., Ahsan, M. A., Arnold, B. F., Hubbard, A., UNICOMB, L. E., Luby, S. P., & Colford, J. M. (2017) · American Journal of Public Health · doi:10.2105/ajph.2017.303686 · Cited by 19
Estimating population effects of vaccination using large, routinely collected data
Halloran, M. E., & Hudgens, M. G. (2017) · Statistics in Medicine · doi:10.1002/sim.7392 · Cited by 18
“Negative controls have been used as a way to detect and adjust for biases due to unmeasured confounding in observational studies [25].”
Method
Metadata
- Authors
- Lipsitch, M., Tchetgen Tchetgen, E., & Cohen, T.
- Affiliations
- Harvard University (US)
- Brigham and Women's Hospital (US)
- Corresponding author
- Lipsitch, Marc
- Venue
- Epidemiology
- Published
- 2010-05
- Volume
- 21
- Issue
- 3
- Pages
- 383-388
- ISSN
- 1044-3983 (ISSN-L)
- PMC
- PMC3053408
- OpenAlex
- W2009428399
- Cited by
- 1,583
- References
- 25
- Citations by year
- 2026: 278 · 2025: 215 · 2024: 183 · 2023: 138 · 2022: 153 · 2021: 132 · 2020: 107 · 2019: 75 · 2018: 72 · 2017: 71 · 2016: 56 · 2015: 32 · 2014: 26 · 2013: 25 · 2012: 11
- FWCI
- 11.64
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- Funders
- National Institutes of Health (5u01gm076497-03, 1dp2od006663-01)
- National Institute of General Medical Sciences (U54GM088558, U01GM076497, 5U01GM076497)
- National Institute of Environmental Health Sciences (T32ES007142)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- negative control, confounding, spurious association, exposure control, outcome control, observational study validity, recall bias, experimental design, causal inference
- MeSH terms
- Humans, Bias, Causality, Confounding Factors, Epidemiologic, Observation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.