Negative Controls: A Tool for Detecting Confounding and Bias in Observational Studies
Open accessThe DOI matches its Crossref record; 24 of the 25 references it cites were found in a registry or an index.
Abstract
Noncausal associations between exposures and outcomes are a threat to validity of causal inference in observational studies. Many techniques have been developed for study design and analysis to identify and eliminate such errors. Such problems are not expected to compromise experimental studies, where careful standardization of conditions (for laboratory work) and randomization (for population studies) should, if applied properly, eliminate most such noncausal associations. We argue, however, that a routine precaution taken in the design of biologic laboratory experiments--the use of "negative controls"--is designed to detect both suspected and unsuspected sources of spurious causal inference. In epidemiology, analogous negative controls help to identify and resolve confounding as well as other sources of error, including recall bias or analytic flaws. We distinguish 2 types of negative controls (exposure controls and outcome controls), describe examples of each type from the epidemiologic literature, and identify the conditions for the use of such negative controls to detect confounding. We conclude that negative controls should be more commonly employed in observational studies, and that additional work is needed to specify the conditions under which negative controls will be sensitive detectors of other sources of error in observational studies.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Finally, in order to evaluate the specificity of the association to ADHD medication, the tenth used selective serotonin reuptake inhibitors (SSRIs) as a negative control medication exposure (27, 30).”
From ADHD Medication and Substance-Related Problems (2017) · cites this paper as [30]
In the citing paper
“The use of ‘negative controls’ for outcomes or exposures can be helpful to assess the likely presence of unmeasured or residual confounding in observational research [51].”
From COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology (2019) · cites this paper as [51]
In the citing paper
“In a third approach–negative control studies–we would expect an unbiased negative control study to be null and the ‘real’ study not-null if there is a true causal effect and the assumptions of this approach are met (see Table 1).39,40”
From Triangulation in aetiological epidemiology (2017) · cites this paper as [39]
Citations
Cites
(1979) · Medical Entomology and Zoology · Cited by 11,252
The environment and disease: association or causation?
Hill, A. B. (2015) · Journal of the Royal Society of Medicine · doi:10.1177/0141076814562718 · Cited by 7,285
The Environment and Disease: Association or Causation?
Hill, A. B. (1965) · Proceedings of the Royal Society of Medicine · doi:10.1177/003591576505800503 · Cited by 6,974 · cited 1×
“19 Hill argued that causal inferences were more credible if the exposure (in his example, nickel mining) was associated with only certain types of outcomes (death from lung and nose cancer but not death from other cancers), and if the outcome was associated with one kind of exposure (nickel mining)…”
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
A structural approach to selection bias
Hernán, M. A., Hernández-Díaz, S., & Robins, J. M. (2004) · Epidemiology · doi:10.1097/01.ede.0000135174.63482.43 · Cited by 2,844
An introduction to instrumental variables for epidemiologists
Greenland, S. (2000) · International Journal of Epidemiology · doi:10.1093/ije/29.4.722 · Cited by 1,156
Instruments for Causal Inference: An Epidemiologist's Dream?
Hernán, M. A., & Robins, J. M. (2006) · Epidemiology · doi:10.1097/01.ede.0000222409.00878.37 · Cited by 961
Vaccines for preventing influenza in the elderly
Demicheli, V., Jefferson, T., Di Pietrantonj, C., Ferroni, E., Thorning, S., Thomas, R. E., & Rivetti, A. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004876.pub4 · Cited by 695
Impact of Influenza Vaccination on Seasonal Mortality in the US Elderly Population
(2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.3.265 · Cited by 543 · cited 1×
“8 To test this hypothesis, Jackson et al 7 reproduced earlier estimates of the protective effect of influenza vaccination, but then repeated the analysis for two sets of negative control outcomes, and showed that the protective effect was observed even in circumstances where the vaccine could not…”
Mortality benefits of influenza vaccination in elderly people: an ongoing controversy
Simonsen, L., Taylor, R. J., Viboud, C., Miller, M. A., & Jackson, L. A. (2007) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(07)70236-0 · Cited by 520 · cited 1×
“However, older age is associated with a less robust immune response to influenza vaccination, and ecological data suggest that the benefits measured in observational studies far exceed the corresponding benefits expected at the population level when influenza vaccination rates have increased among the elderly.(6) Impo…”
Cited by
Items 401–500 of 1,002
Wang, Y., Yazdi, M. D., Wei, Y., & Schwartz, J. D. (2024) · Nature Communications · doi:10.1038/s41467-024-52117-8 · Cited by 39
“Negative controls have been suggested as a useful tool to enhancecausal inference independently of covariate distributions and to tackle unmeasured confounding bias 16 .”
Phillips, C. V. (2015) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph120505439 · Cited by 32
“” An example of this is observing that the association of better health and getting an influenza vaccination in an elderly population is not stronger during flu season, which is not what we would expect if the association were caused by the vaccine working rather than by confounding [21].”
BackgroundLiew, Z., & Ernst, A. (2021) · Current Environmental Health Reports · doi:10.1007/s40572-020-00301-5 · Cited by 32
Asthma and asthma symptom control in relation to incidence of lung cancer in the HUNT study
Jiang, L., Sun, Y., Langhammer, A., Brumpton, B. M., Chen, Y., Nilsen, T. I., Leivseth, L., Wahl, S. G. F., & Mai, X. (2021) · Scientific Reports · doi:10.1038/s41598-021-84012-3 · Cited by 37
Berstein, F. M., Biessy, C., Viallon, V., Soares, A. G., Casagrande, C., Hémon, B., Kliemann, N., Cairat, M., Blanco, J., Nahas, A. A., et al. (2023) · European Journal of Nutrition · doi:10.1007/s00394-023-03270-1 · Cited by 36
“Finally, we performed a negative control outcome analysis (i.e. where the outcome is not plausibly linked to the exposure of interest) to help identify any residual confounding that could be biasing our results [51].”
Nagel, C., Kirby, M. A., Zambrano, L. D., Rosa, G., Barstow, C. K., Thomas, E., & Clasen, T. (2016) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2016.07.003 · Cited by 36
Advances in methodologies of negative controls: a scoping review
Yang, Q., Yang, Z., Cai, X., Zhao, H., Jia, J., & Sun, F. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.111228 · Cited by 36
Reading and conducting instrumental variable studies: guide, glossary, and checklist
Walker, V. M., Sanderson, E. C. M., Levin, M. G., Damraurer, S. M., Feeney, T., & Davies, N. M. (2024) · BMJ · doi:10.1136/bmj-2023-078093 · Cited by 39
“44 The exclusion restriction is falsifiable by demonstrating that other variables are affected by the instrument, which also affects the outcome.”
Brew, B. K., Toelle, B. G., Webb, K., Almqvist, C., Marks, G. B., & for the CAPS investigators (2014) · European Journal of Clinical Nutrition · doi:10.1038/ejcn.2014.155 · Cited by 31
Fell, D. B., Bhutta, Z. A., Hutcheon, J. A., Karron, R. A., Knight, M., Kramer, M. S., Monto, A. S., Swamy, G. K., Ortiz, J. R., & Savitz, D. A. (2017) · Vaccine · doi:10.1016/j.vaccine.2017.03.056 · Cited by 35
“Even when nested within an RCT, analyses of sub-groups defined post hoc are prone to type 1 errors [91].”
BackgroundHarrison, P. J., Luciano, S., & Colbourne, L. (2020) · Journal of Psychopharmacology · doi:10.1177/0269881120936501 · Cited by 37
“Background An7hypertensive drugs (AHTs), especially calcium channel blockers (CCBs), have been associated with differen7al rates of a number of neuropsychiatric outcomes.”
MethodResultInfluentialMuanda, F. T., Weir, M. A., Ahmadi, F., Sontrop, J. M., Cowan, A., Fleet, J. L., Blake, P. G., & Garg, A. X. (2021) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2021.11.007 · Cited by 34
Tang, E. H. M., Mak, I. L., Tse, E. T. Y., Wan, E. Y. F., Yu, E. Y. T., Chen, J. Y., Chin, W. Y., Chao, D. V. K., Tsui, W. W. S., Ha, T. K. H., et al. (2022) · Diabetes Care · doi:10.2337/dc22-0387 · Cited by 34
Talmor‐Barkan, Y., Yacovzada, N., Rossman, H., Witberg, G., Kalka, I. N., Kornowski, R., & Segal, E. (2022) · European Heart Journal - Cardiovascular Pharmacotherapy · doi:10.1093/ehjcvp/pvac063 · Cited by 33
Shi, X., Li, K. Q., Yu, M., Miao, W., Kuchibhotla, A. K., Hu, M., & Tchetgen, E. T. (2026) · Journal of the American Statistical Association · doi:10.1080/01621459.2026.2639734 · Cited by 38
Improving Causal Inferences in Risk Analysis
Cox, L. A. (2013) · Risk Analysis · doi:10.1111/risa.12072 · Cited by 36
McCoy, R. G., Herrin, J., Swarna, K. S., Deng, Y., Kent, D. M., Ross, J. S., Umpierrez, G. E., Galindo, R. J., Crown, W. H., Borah, B. J., et al. (2024) · Nature Cardiovascular Research · doi:10.1038/s44161-024-00453-9 · Cited by 30
Familial confounding in the associations between maternal health and autism
Khachadourian, V., Arildskov, E. S., Grove, J., O’Reilly, P. F., Buxbaum, J. D., Reichenberg, A., Sandin, S., Croen, L., Schendel, D., Hansen, S. N., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03479-5 · Cited by 33
“Additionally, the use of negative controls (for example, paternal exposures during pregnancy) can identify unmeasured familial confounding linked to maternal exposures and other types of biases 14 .”
Wang, C.-A., Lai, H.-W., Chen, J., Wang, W.-J., Lin, L.-C., Chiu, Y., Cheng, C., & Wu, V. (2025) · Nature Communications · doi:10.1038/s41467-025-64640-3 · Cited by 30
“A negative control outcome analysis was performed by examining the association between treatments and overall cancer incidence, for which no association was expected 50 – 53 .”
Association of Primary Care Consultation Patterns With Early Signs and Symptoms of Psychosis
Sullivan, S. A., Hamilton, W. T., Tilling, K. M., Redaniel, M. T., Moran, P., & Lewis, G. (2018) · JAMA Network Open · doi:10.1001/jamanetworkopen.2018.5174 · Cited by 30
Gondek, D., Ning, K., Ploubidis, G. B., Nasim, B., & Goodman, A. T. (2018) · PLoS ONE · doi:10.1371/journal.pone.0209659 · Cited by 30
Whitlock, R. H., Hougen, I., Komenda, P. V. J., Rigatto, C., Clemens, K. K., & Tangri, N. (2020) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2019.07.017 · Cited by 33
Butler, A. M., Brown, D. S., Newland, J. G., Nickel, K. B., Sahrmann, J. M., O’Neil, C. A., Olsen, M. A. A., Zetts, R. M., Hyun, D. Y., & Durkin, M. J. (2022) · Clinical Infectious Diseases · doi:10.1093/cid/ciac879 · Cited by 31
Lazarevic, N., Dobson, A. J., Barnett, A. G., & Knibbs, L. D. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-008714 · Cited by 33
“We used iron de fi ciency as a negative control 16 as it is not believed to be associated with ambient air pollution, but is likely to be affected by the same unobserved confounders as the outcome – exposure relationships of interest.”
MethodYang, S.-W., Kernic, M. A., Mueller, B. A., Simon, G. E., Chan, K. C. G., & Stoep, A. V. (2020) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2020.1749 · Cited by 31
Sun, J. W., Hernández–Díaz, S., Haneuse, S., Bourgeois, F. T., Vine, S. M., Olfson, M., Bateman, B. T., & Huybrechts, K. F. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.2762 · Cited by 32
Cultural sexism moderates efficacy of psychotherapy: Results from a spatial meta-analysis.
Price, M. A., McKetta, S., Weisz, J. R., Ford, J. V., Lattanner, M. R., Skov, H., Wolock, E., & Hatzenbuehler, M. L. (2021) · Clinical Psychology: Science and Practice · doi:10.1037/cps0000031 · Cited by 29
An Outbreak of Parvovirus B19 in Israel
Patalon, T., Saciuk, Y., Trotzky, D., Pachys, G., Ben‐Tov, A., Segal, Y., & Gazit, S. (2023) · Viruses · doi:10.3390/v15112261 · Cited by 31
“The second supplementary analysis included an examination of cytomegalovirus (CMV) as a negative outcome control [33], comparing incidence in 2023 to the previous study years (2015–2022), implementing a logistic regression analysis, adjusting for the same variables as the primary analysis.”
MethodFirst Trimester Use of Buprenorphine or Methadone and the Risk of Congenital Malformations
Suarez, E. A., Bateman, B. T., Straub, L., Hernández–Díaz, S., Jones, H. E., Gray, K. J., Connery, H. S., Davis, J. M., Lester, B. M., Terplan, M., et al. (2024) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.6986 · Cited by 30
Stevenson, A. C., Clemens, T. L., Pairo‐Castineira, E., Webb, D. J., Weller, R., & Dibben, C. J. L. (2024) · Health & Place · doi:10.1016/j.healthplace.2024.103328 · Cited by 32
Bell, W., Jennings, A., Thompson, A. S., Bondonno, N. P., Tresserra‐Rimbau, A., Kuhn, T. E., & Cassidy, A. (2024) · American Journal of Clinical Nutrition · doi:10.1016/j.ajcnut.2024.09.022 · Cited by 31
Davis, K. A. S., Farooq, S., Hayes, J., John, A. M., Lee, W. E., MacCabe, J. H., McIntosh, A. M., Osborn, D., Stewart, R. J., & Woelbert, E. (2019) · The Lancet Psychiatry · doi:10.1016/s2215-0366(19)30298-6 · Cited by 32
“Studies may also contain an extra arm that is a “negative control”, defined by an exposure that is known to be unrelated to the outcome under study, in order to detect bias and confounding in the data source and model being used.(51) This was done when evaluating in utero exposure to SSRIs and foetal abnormalities by…”
BackgroundDukes, O., Shpitser, I., & Tchetgen, E. J. T. (2023) · Biometrika · doi:10.1093/biomet/asad015 · Cited by 49
“Negative control are examples of such proxies (Lipsitch et al., 2010; Shi et al., 2020b); we refer to Shi et al.”
MethodHitchings, M., Lewnard, J., Dean, N., Ko, A. I., Ranzani, O., Andrews, J. R., & Cummings, D. A. T. (2021) · medRxiv · doi:10.1097/ede.0000000000001484 · Cited by 31
Vaccination nudges: A study of pre-booked COVID-19 vaccinations in Sweden
Bonander, C., Ekman, M., & Jakobsson, N. (2022) · Social Science & Medicine · doi:10.1016/j.socscimed.2022.115248 · Cited by 30
Ward, D., Andersen, N. N., Gørtz, S., Iversen, A. T., Allin, K. H., Beaugerie, L., Kirchgesner, J., & Jess, T. (2021) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2023.06.025 · Cited by 29
Some Approximate Evidence Factors in Observational Studies
Rosenbaum, P. R. (2011) · Journal of the American Statistical Association · doi:10.1198/jasa.2011.tm10422 · Cited by 37
“…al. (1959), Weiss (1981), Rosenbaum and Rubin (1983), Yanagawa (1984), Gastwirth (1992), Marcus (1997), Lin, Psaty, and Kronmal (1998), Robins, Rotnitzky, and Scharfstein (1999), Copas and Eguchi (2001), Imbens (2003), Wang and Krieger (2006), Small (2007), and Lipsitch, Tchetgen, and Cohen (2010).”
BackgroundMcGrath, L. J., Ellis, A. R., & Brookhart, M. A. (2015) · American Journal of Epidemiology · doi:10.1093/aje/kwu485 · Cited by 29
“(38) discuss how to identify and use other kinds of negative controls in observational studies.”
BackgroundIn vivo antimalarial activity of the endophytic actinobacteria, Streptomyces SUK 10
Baba, M. S., Zin, N. M., Hassan, Z. A. A., Latip, J., Pethick, F. E., Hunter, I. S., Edrada‐Ebel, R., & Herron, P. R. (2015) · The Journal of Microbiology · doi:10.1007/s12275-015-5076-6 · Cited by 29
“This was also described in previous in vivo antimalarial studies (Zainal-Abidin et al., 1985; Kurosawa et al., 2000; Isaka et al., 2002; Jochen et al., 2002; Abdulelah and Zainal-Abidin, 2007; Prudhomme et al., 2008; Lipsitch et al., 2010).”
Dibben, C. J. L., Playford, C., & Mitchell, R. (2016) · Journal of Epidemiology & Community Health · doi:10.1136/jech-2016-207898 · Cited by 57
Firearm Storage Practices in Households of Adolescents With and Without Mental Illness
Simonetti, J. A., Theis, M. K., Rowhani‐Rahbar, A., Ludman, E. J., & Grossman, D. C. (2017) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2017.05.017 · Cited by 30
“, whether the household had a functioning smoke detector [yes/no]) [23].”
Falsification Tests for Instrumental Variable Designs With an Application to Tendency to Operate
Keele, L. J., Zhao, Q., Kelz, R. R., & Small, D. S. (2018) · Medical Care · doi:10.1097/mlr.0000000000001040 · Cited by 29
Yao, Q., Gao, Y., Zhang, Y., Qin, K., Liew, Z., & Tian, Y. (2021) · Chemosphere · doi:10.1016/j.chemosphere.2021.131521 · Cited by 29
Rozé, J., Morel, B., Lapillonne, A., Marret, S., Guellec, I., Darmaun, D., Bednarek, N., Moyon, T., Marchand‐Martin, L., Benhammou, V., et al. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.35452 · Cited by 28
Huang, H.-K., Liu, P. P.-S., Lin, S.-M., Hsu, J.-Y., Yeh, J.-I., Lai, E. C.-C., Peng, C. C., Munir, K. M., Loh, C., & Tu, Y.-K. (2022) · Annals of Internal Medicine · doi:10.7326/m21-3498 · Cited by 31
Clarifying the causes of consistent and inconsistent findings in genetics
Dattani, S., Howard, D. M., Lewis, C. M., & Sham, P. C. (2022) · Genetic Epidemiology · doi:10.1002/gepi.22459 · Cited by 29
Daley, M. F., Reifler, L. M., Glanz, J. M., Hambidge, S. J., Getahun, D., Irving, S. A., Nordin, J. D., McClure, D. L., Klein, N. P., Jackson, M. L., et al. (2022) · Academic Pediatrics · doi:10.1016/j.acap.2022.08.006 · Cited by 33
“An ideal negative control outcome is one which shares the potential biases of the primary outcome but cannot plausibly be related to the exposure.(29,30) Here, vaccine-associated aluminum cannot reasonably associate with future injuries in children.”
Venkataramani, A. S., Cook, E. E., O’Brien, R., Kawachi, I. I., Jena, A. B., & Tsai, A. C. (2019) · PLoS Medicine · doi:10.1371/journal.pmed.1002821 · Cited by 31
War-Related Traumas and Mental Health Across Generations
Betancourt, T. S., Thomson, D., & VanderWeele, T. J. (2017) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2017.3530 · Cited by 28
Parental Education, Own Education, and Cognitive Function in Middle-Aged and Older Adults
Luo, S., Chen, W., Hu, W., Wang, H., Li, J., Guo, V. Y., & Rehkopf, D. H. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.13036 · Cited by 31
COVID-19 mRNA Vaccination and 4-Year All-Cause Mortality Among Adults Aged 18 to 59 Years in France
Semenzato, L., Vu, S. L., Botton, J., Bertrand, M., Jabagi, M.-J., Drouin, J., Cuenot, F., Olié, V., Dray‐Spira, R., Weill, A., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.46822 · Cited by 34
Madley‐Dowd, P., Rai, D., Zammit, S., & Heron, J. E. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.10.008 · Cited by 29
“In biological research the negative control design is implemented to test whether factors other than the treatment of interest have led to a causal interpretation of experimental results (see [1] for examples).”
BackgroundBlood Transfusion and Adverse Graft-related Events in Kidney Transplant Patients
Massicotte‐Azarniouch, D., Sood, M. M., Fergusson, D. A., Chassé, M., Tinmouth, A. T., & Knoll, G. A. (2021) · Kidney International Reports · doi:10.1016/j.ekir.2021.01.015 · Cited by 27
Associations Between Prenatal Exposure to Air Pollution and Congenital Hypothyroidism
Harari‐Kremer, R., Calderon‐Margalit, R., Korevaar, T. I. M., Nevo, D., Broday, D. M., Kloog, I., Grotto, I., Karakis, I., Shtein, A., Haim, A., et al. (2021) · American Journal of Epidemiology · doi:10.1093/aje/kwab187 · Cited by 27
Everett, B. G., Limburg, A., McKetta, S. C., & Hatzenbuehler, M. L. (2022) · Psychosomatic Medicine · doi:10.1097/psy.0000000000001092 · Cited by 27
“Documenting result specificity through this negative control analysis (29) will improve inferences that the results are due to LGB policy protections and not to factors that might be associated with it (e.”
BackgroundDouros, A., Ante, Z., Fallone, C. A., Azoulay, L., Renoux, C., Suissa, S., & Brassard, P. (2023) · Alzheimer s & Dementia · doi:10.1002/alz.13561 · Cited by 30
Andersson, N. W., Thiesson, E. M., Pihlström, N., Perälä, J., Faksová, K., Gram, M. A., Poukka, E., Leino, T. M., Ljung, R., & Hviid, A. P. (2024) · BMJ Medicine · doi:10.1136/bmjmed-2024-001074 · Cited by 27
Limitations of empirical calibration of p‐values using observational data
Gruber, S., & Tchetgen, E. T. (2016) · Statistics in Medicine · doi:10.1002/sim.6936 · Cited by 30
Association between Caesarean Delivery and Childhood Asthma in India and Vietnam
Lavin, T., Franklin, P., & Preen, D. B. (2016) · Paediatric and Perinatal Epidemiology · doi:10.1111/ppe.12324 · Cited by 27
Sørup, S., Benn, C. S., Poulsen, A., Krause, T. G., Aaby, P., & Ravn, H. (2016) · Vaccine · doi:10.1016/j.vaccine.2016.11.005 · Cited by 27
“Therefore, we performed an analysis with emergency room visits due to unintentional accidents as negative control outcome [22] to examine if differential health seeking behaviour could explain any association between vaccination status and infectious disease admissions.”
MethodYeung, S. L. A., Jiang, C., Cheng, K. K., Xu, L., Zhang, W., Lam, T. H., Leung, G. M., & Schooling, C. M. (2017) · Preventive Medicine · doi:10.1016/j.ypmed.2017.06.006 · Cited by 27
“Given later age at menarche is consistently associated with taller height (Onland-Moret et al., 2005), we considered height as a positive control to rule out the possibility of underpowered analyses (Lipsitch et al., 2010).”
BackgroundWu, Q., Dalman, C., Karlsson, H., Lewis, G., Osborn, D., Gardner, R. M., & Hayes, J. (2018) · Schizophrenia Bulletin · doi:10.1093/schbul/sby023 · Cited by 30
Unconventional Natural Gas Development and Hospitalization for Heart Failure in Pennsylvania
McAlexander, T. P., Bandeen‐Roche, K., Buckley, J. P., Pollak, J., Michos, E. D., McEvoy, J. W., & Schwartz, B. S. (2020) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2020.10.023 · Cited by 27
Guidelines for estimating causal effects in pragmatic randomized trialsPreprint
Murray, E. J., Swanson, S. A., & Hernán, M. Á. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.06030 · Cited by 29
“A good practice for any per-protocol analysis is to include an outcome that can serve as a negative control (62,63).”
BackgroundGrembi, J. A., Nguyen, A., Rivière, M., Heitmann, G. B., Patil, A., Athni, T. S., Djajadi, S., Ercümen, A., Lin, A., Crider, Y. S., et al. (2024) · PLoS neglected tropical diseases · doi:10.1371/journal.pntd.0012157 · Cited by 28
“To detect potential unmeasured confounding or outcome measurement bias in the diarrhea analysis, we repeated the analysis using caregiver-reported child bruising in the past 7 days as a negative control outcome [35,36].”
Pavúk, M., Adgate, J. L., Bartell, S. M., Bell, E. M., Brown, L. M., Laumbach, R. J., Schaider, L. A., Erve, T. J. V. ‘., Bailey, J. M., Botelho, J. C., et al. (2025) · Environment International · doi:10.1016/j.envint.2025.109589 · Cited by 30
Desai, J. R., Hyde, C. L., Kabadi, S., Louis, M. S., Bonato, V., Loomis, A. K., Galaznik, A., & Berger, M. L. (2016) · Medical Care · doi:10.1097/mlr.0000000000000640 · Cited by 27
“To consistently address these biases, some methodologists have advocated for routine use of negative controls in observational studies to detect the presence, extent, and direction of uncontrolled bias and systematic error.(3,4) These recommendations were prompted by recent studies that showed nearly half of significa…”
BackgroundPorter, M. A. S. M., Maravilla, J. C., Betts, K. S., & Alati, R. (2019) · Archives of Gynecology and Obstetrics · doi:10.1007/s00404-019-05204-x · Cited by 31
Fluoroquinolone Antibiotics and Tendon Injury in Adolescents
Ross, R. K., Kinlaw, A. C., Herzog, M. M., Funk, M. J., & Gerber, J. S. (2021) · PEDIATRICS · doi:10.1542/peds.2020-033316 · Cited by 27
Gomes, M., Latimer, N. R., Soares, M. O., Dias, S., Baio, G., Freemantle, N., Dawoud, D., Wailoo, A., & Grieve, R. (2022) · PharmacoEconomics · doi:10.1007/s40273-022-01141-x · Cited by 31
“In addition, widely used epidemiological methods to detect potential residual confounding, such as outcome controls [36] and E-value approaches [35], are yet to permeate practice in HTA.”
BackgroundInfluentialPark, D., Seo, S. I., Lee, K. J., Kim, J., Kim, Y., Seo, W., Lee, H., Shin, W. G., & Yoo, J. J. (2022) · Journal of Gastroenterology and Hepatology · doi:10.1111/jgh.15879 · Cited by 28
The State of Use and Utility of Negative Controls in Pharmacoepidemiologic Studies
Zafarí, Z., Park, J.-E., Shah, C. H., dosReis, S., Gorman, E. F., Hua, W., Ma, Y., & Tian, F. (2023) · American Journal of Epidemiology · doi:10.1093/aje/kwad201 · Cited by 28
Lewis, S. J., Relton, C. L., Zammit, S., & Smith, G. D. (2013) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12127 · Cited by 28
Lin, H., Ho, Y., & Lin, F. (2018) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.13626 · Cited by 26
Glymour, M. M., & Whitmer, R. A. (2019) · Journal of Alzheimer s Disease · doi:10.3233/jad-190304 · Cited by 26
Metformin reduces radiation-induced cardiac toxicity risk in patients having breast cancer.
Yu, J., Hsieh, M., Qin, L., Zhang, J., & Wu, S. (2019) · PubMed · Cited by 33
Hossin, M. Z., Kazamia, K., Faxén, J., Rudolph, A., Johansson, K., Sandström, A., & Razaz, N. (2024) · European Heart Journal · doi:10.1093/eurheartj/ehae547 · Cited by 25
“12,13 Given that paternal CVD shares the same sources of unmeasured confounding as maternal CVD and does not directly influence the intrauterine environment ( Figure 1 ), paternal CVD as a negative-control exposure can exploit the similarity of confounding structure and thereby serve as an…”
BackgroundNesbitt, R., Asilaza, K. V., Alvarez, C., Gitahi, P., Nkemenang, P., Duncker, J., Haile, M., Gakima, P., Wamala, J. F., Loro, F. B., et al. (2025) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(24)00657-1 · Cited by 25
Instrumental Variable Analyses in Pharmacoepidemiology: What Target Trials Do We Emulate?
Swanson, S. A. (2017) · Current Epidemiology Reports · doi:10.1007/s40471-017-0120-1 · Cited by 26
Wu, P., Lin, Y., Liu, J., Tsai, Y., Kuo, M., Chiu, Y., Hwang, S., & Carrero, J. J. (2020) · Clinical Kidney Journal · doi:10.1093/ckj/sfaa248 · Cited by 25
Zinnia, M. A., & Islam, A. B. M. M. K. (2021) · International Journal of Biological Macromolecules · doi:10.1016/j.ijbiomac.2021.06.197 · Cited by 28
Specialized Versus Generic Allied Health Therapy and the Risk of Parkinson's Disease Complications
Talebi, A. H., Ypinga, J. H. L., de Vries, N. M., Nonnekes, J. H., Munneke, M., Bloem, B. R., Heskes, T., Ben‐Shlomo, Y., & Darweesh, S. K. L. (2022) · Movement Disorders · doi:10.1002/mds.29274 · Cited by 29
“Similarly, observing no association between S< and the rate of orthopedic injuries strengthens the causality of our findings, because this specific allied health intervention should not have an effect on this unrelated outcome.(15) Second, we found that after the last session of AHT, the inverse associations of spec…”
ResultKuo, C., Chen, Y., Hsu, C., Chang, C. C., Chou, R., Li, S., Kuo, S., Huang, P., Chen, J., & Lin, S. (2017) · BMJ Open · doi:10.1136/bmjopen-2017-016179 · Cited by 27
“However, we performed a sensitivity analysis that included positive and negative control outcomes to provide further support for our findings.(31) Fourth, this observational study provided clinically relevant risk estimates without speculating on causation.”
Cocoros, N. M., Arlett, P. R., Dreyer, N. A., Ishiguro, C., Iyasu, S., Sturkenboom, M., Zhou, W., & Toh, S. (2020) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.2045 · Cited by 25
“CLINICAL PHARMACOLOGY & THERAPEUTICS | VOLUME 109 NUMBER 5 | May 2021 1193 or negative control outcomes and exposures can be used to assess the adequacy of confounding control in observational studies.(26,27)”
MethodAbeysekera, K. W. M., Orr, J. G., Madley‐Dowd, P., Fernandes, G., Zuccolo, L., Gordon, F. H., Lawlor, D. A., Heron, J. E., & Hickman, M. (2021) · The Lancet Regional Health - Europe · doi:10.1016/j.lanepe.2021.100206 · Cited by 29
“Negative control studies in epidemiology compare results from the hypothesized (real) association and a negative control association where either the exposure is changed to an exposure that does not plausibly cause the outcome, or vice versa [32-34].”
BackgroundYang, M., He, Q., Gao, F., Nirantharakumar, K., Veenith, T., Qin, X. S., Page, A. T., Wong, M. C. S., Huang, J., Kuo, Z. C., et al. (2021) · BMC Medicine · doi:10.1186/s12916-021-02180-5 · Cited by 32
“To verify if potential biases could have modified the association between PPI use and stroke, we used falsification analyses for negative control outcomes (malignant melanoma cancer and transportation-related death) with the method described by Lipsitch M [29, 30].”
MethodSimpson, C. R., Lone, N., Kavanagh, K., Ritchie, L., Robertson, C., Sheikh, A., & McMenamin, J. (2013) · Health Services and Delivery Research · doi:10.3310/hsdr01100 · Cited by 28
Hill, D. N., Moakler, R., Hubbard, A., Tsemekhman, V., Provost, F. J., & Tsemekhman, K. (2015) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2783258.2788622 · Cited by 31
“Negative controls can help identify bias in causal studies as they provide a test where the analysis should produce an estimate of zero effect [10].”
BackgroundCore Concepts: Self‐Controlled Designs in Pharmacoepidemiology
Bots, S. H., Brown, J. P., Wong, A. Y. S., Martin, I., Douglas, I., Klungel, O. H., & Schultze, A. (2025) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.70071 · Cited by 28
“Negative controls [63] are typically used to detect unmeasured confounding—which in the self-controlled setting is time-varying.”
Tang, H., Lu, Y., Zhang, B., Zhou, T., Zhang, D., Chen, J., Chen, Y., Asch, D. A., & Chen, Y. (2025) · Diabetes Obesity and Metabolism · doi:10.1111/dom.70054 · Cited by 28
Lousdal, M. L., Lash, T. L., Flanders, W. D., Brookhart, M. A., Kristiansen, I. S., Kalager, M., & Støvring, H. (2020) · International Journal of Epidemiology · doi:10.1093/ije/dyaa029 · Cited by 25
“Likewise, an ideal negative-control exposure and the outcome of interest share the same set of common causes as the exposure and outcome of interest.(6) We hypothesized that screening participants are healthier than non-participants and, therefore, already at baseline have a lower risk of breast-cancer death.”
BackgroundInfluentialBlood pressure and risk of cancer: a Mendelian randomization study
Chan, I. I., Kwok, M. K., & Schooling, C. M. (2021) · BMC Cancer · doi:10.1186/s12885-021-09067-x · Cited by 25
“Genetic associations with asthma Genetic associations with asthma (31169 cases and 379656 non-cases) used as a negative control outcome were obtained from the UK Biobank (phenocode: 495), given BP is not known to cause asthma [41], but both share similar confounders [42].”
MethodChang, K., Chen, C., Wang, C., Tu, H., Huang, Y., Wu, H., Chang, C.-H., & Chang, S. (2022) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2022.791285 · Cited by 29
Kim, D., Yang, P., You, S. C., Jang, E., Yu, H. T., Kim, T., Pak, H., Lee, M., Lip, G. Y. H., Sung, J., et al. (2022) · Annals of Internal Medicine · doi:10.7326/m21-4798 · Cited by 25
Torabi, F., Bedston, S., Lowthian, E., Akbari, A., Owen, R. K., Bradley, D., Agrawal, U., Collins, P. W., Fry, R. J., Griffiths, L. J., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-20118-6 · Cited by 28
Cardiorenal Outcomes Among Patients With Atrial Fibrillation Treated With Oral Anticoagulants
Trevisan, M., Hjemdahl, P., Clase, C. M., de Jong, Y., Evans, M., Bellocco, R., Fu, E. L., & Carrero, J. J. (2022) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2022.07.017 · Cited by 24
Elser, H. C., Rowland, S. T., Marek, M. S., Kiang, M. V., Shea, B., Do, V., Benmarhnia, T., Schneider, A. L. C., & Casey, J. A. (2023) · Headache The Journal of Head and Face Pain · doi:10.1111/head.14442 · Cited by 28
Association of Proton Pump Inhibitor Use With Risk of Acquiring Drug-Resistant Enterobacterales
Willems, R. P. J., Schut, M. C., Kaiser, A. M., Groot, T., Abu–Hanna, A., Twisk, J. W. R., van Dijk, K., & Vandenbroucke‐Grauls, C. M. J. E. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.0470 · Cited by 27
Ranitidine Use and Incident Cancer in a Multinational Cohort
You, S. C., Seo, S. I., Falconer, T., Yanover, C., Duarte‐Salles, T., Seager, S. L., Posada, J., Shah, N. H., Nguyen, P., Kim, Y.-S., et al. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.33495 · Cited by 27
Rogne, T., Wang, R., Pin, W., Deziel, N. C., Metayer, C., Wiemels, J. L., Chen, K., Warren, J. L., & Ma, X. (2024) · The Lancet Planetary Health · doi:10.1016/s2542-5196(24)00121-9 · Cited by 28
“26 As embryonic lymphopoiesis starts around gestational week 8, 9 we hypothesised that we would see no association between ambient temperature and acute lymphoblastic leukaemia in the weeks preceding pregnancy.”
Metadata
- Authors
- Lipsitch, M., Tchetgen Tchetgen, E., & Cohen, T.
- Affiliations
- Harvard University (US)
- Brigham and Women's Hospital (US)
- Corresponding author
- Lipsitch, Marc
- Venue
- Epidemiology
- Published
- 2010-05
- Volume
- 21
- Issue
- 3
- Pages
- 383-388
- ISSN
- 1044-3983 (ISSN-L)
- PMC
- PMC3053408
- OpenAlex
- W2009428399
- Cited by
- 1,583
- References
- 25
- Citations by year
- 2026: 278 · 2025: 215 · 2024: 183 · 2023: 138 · 2022: 153 · 2021: 132 · 2020: 107 · 2019: 75 · 2018: 72 · 2017: 71 · 2016: 56 · 2015: 32 · 2014: 26 · 2013: 25 · 2012: 11
- FWCI
- 11.64
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- Funders
- National Institutes of Health (5u01gm076497-03, 1dp2od006663-01)
- National Institute of General Medical Sciences (U54GM088558, U01GM076497, 5U01GM076497)
- National Institute of Environmental Health Sciences (T32ES007142)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- negative control, confounding, spurious association, exposure control, outcome control, observational study validity, recall bias, experimental design, causal inference
- MeSH terms
- Humans, Bias, Causality, Confounding Factors, Epidemiologic, Observation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.