Negative Controls: A Tool for Detecting Confounding and Bias in Observational Studies
Open accessThe DOI matches its Crossref record; 24 of the 25 references it cites were found in a registry or an index.
Abstract
Noncausal associations between exposures and outcomes are a threat to validity of causal inference in observational studies. Many techniques have been developed for study design and analysis to identify and eliminate such errors. Such problems are not expected to compromise experimental studies, where careful standardization of conditions (for laboratory work) and randomization (for population studies) should, if applied properly, eliminate most such noncausal associations. We argue, however, that a routine precaution taken in the design of biologic laboratory experiments--the use of "negative controls"--is designed to detect both suspected and unsuspected sources of spurious causal inference. In epidemiology, analogous negative controls help to identify and resolve confounding as well as other sources of error, including recall bias or analytic flaws. We distinguish 2 types of negative controls (exposure controls and outcome controls), describe examples of each type from the epidemiologic literature, and identify the conditions for the use of such negative controls to detect confounding. We conclude that negative controls should be more commonly employed in observational studies, and that additional work is needed to specify the conditions under which negative controls will be sensitive detectors of other sources of error in observational studies.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Finally, in order to evaluate the specificity of the association to ADHD medication, the tenth used selective serotonin reuptake inhibitors (SSRIs) as a negative control medication exposure (27, 30).”
From ADHD Medication and Substance-Related Problems (2017) · cites this paper as [30]
In the citing paper
“The use of ‘negative controls’ for outcomes or exposures can be helpful to assess the likely presence of unmeasured or residual confounding in observational research [51].”
From COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology (2019) · cites this paper as [51]
In the citing paper
“In a third approach–negative control studies–we would expect an unbiased negative control study to be null and the ‘real’ study not-null if there is a true causal effect and the assumptions of this approach are met (see Table 1).39,40”
From Triangulation in aetiological epidemiology (2017) · cites this paper as [39]
Citations
Cites
(1979) · Medical Entomology and Zoology · Cited by 11,252
The environment and disease: association or causation?
Hill, A. B. (2015) · Journal of the Royal Society of Medicine · doi:10.1177/0141076814562718 · Cited by 7,285
The Environment and Disease: Association or Causation?
Hill, A. B. (1965) · Proceedings of the Royal Society of Medicine · doi:10.1177/003591576505800503 · Cited by 6,974 · cited 1×
“19 Hill argued that causal inferences were more credible if the exposure (in his example, nickel mining) was associated with only certain types of outcomes (death from lung and nose cancer but not death from other cancers), and if the outcome was associated with one kind of exposure (nickel mining)…”
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
A structural approach to selection bias
Hernán, M. A., Hernández-Díaz, S., & Robins, J. M. (2004) · Epidemiology · doi:10.1097/01.ede.0000135174.63482.43 · Cited by 2,844
An introduction to instrumental variables for epidemiologists
Greenland, S. (2000) · International Journal of Epidemiology · doi:10.1093/ije/29.4.722 · Cited by 1,156
Instruments for Causal Inference: An Epidemiologist's Dream?
Hernán, M. A., & Robins, J. M. (2006) · Epidemiology · doi:10.1097/01.ede.0000222409.00878.37 · Cited by 961
Vaccines for preventing influenza in the elderly
Demicheli, V., Jefferson, T., Di Pietrantonj, C., Ferroni, E., Thorning, S., Thomas, R. E., & Rivetti, A. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004876.pub4 · Cited by 695
Impact of Influenza Vaccination on Seasonal Mortality in the US Elderly Population
(2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.3.265 · Cited by 543 · cited 1×
“8 To test this hypothesis, Jackson et al 7 reproduced earlier estimates of the protective effect of influenza vaccination, but then repeated the analysis for two sets of negative control outcomes, and showed that the protective effect was observed even in circumstances where the vaccine could not…”
Mortality benefits of influenza vaccination in elderly people: an ongoing controversy
Simonsen, L., Taylor, R. J., Viboud, C., Miller, M. A., & Jackson, L. A. (2007) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(07)70236-0 · Cited by 520 · cited 1×
“However, older age is associated with a less robust immune response to influenza vaccination, and ecological data suggest that the benefits measured in observational studies far exceed the corresponding benefits expected at the population level when influenza vaccination rates have increased among the elderly.(6) Impo…”
Cited by
Items 801–900 of 1,002
Epidemiology beyond its limits
McCullough, L. E., Maliniak, M. L., Amin, A. B., Baker, J. M., Baliashvili, D., Barberio, J., Barrera, C. M., Brown, C., Collin, L. J., Freedman, A. A., et al. (2022) · Science Advances · doi:10.1126/sciadv.abn3328 · Cited by 10
Yim, G., Roberts, A., Ascherio, A., Wypij, D., Kioumourtzoglou, M.-A., & Weisskopf, A. M. G. (2022) · Epidemiology · doi:10.1097/ede.0000000000001467 · Cited by 9
“Middle 16,729 (47) 5,961 (45) 2,365 (47)”
Arnold, B. F., Mäusezahl, D., Schmidt, W., Christen, A., & Colford, J. M. (2012) · Environmental Science & Technology · doi:10.1021/es204193j · Cited by 9
Minimax Kernel Machine Learning for a Class of Doubly Robust Functionals.Preprint
Ghassami, A., Ying, A., Shpitser, I., & Tchetgen, E. T. (2021) · arXiv (Cornell University) · Cited by 13
McCoy, R. G., SWARNA, K. S., Neumiller, J. J., Polley, E. C., Deng, Y., Mickelson, M. M., & Herrin, J. S. (2024) · Clinical Diabetes · doi:10.2337/cd24-0007 · Cited by 8
Hurwitz, K. E., Rathnayaka, N. S., Hendrickson, K., & Brookhart, M. A. (2024) · Clinical Infectious Diseases · doi:10.1093/cid/ciae516 · Cited by 12
Kang, S., Kim, J., Yang, J. S., Jeon, Y. J., Lee, H., Suglia, S. F., Tsai, A. C., Kang, J. I., & Jung, S. J. (2024) · BMC Medicine · doi:10.1186/s12916-024-03704-5 · Cited by 8
“For a negative control analysis [49], we investigated the association between the use of other antihypertensive class (CCB, beta-blocker, and thiazide-related diuretics) and probable PTSD.”
BackgroundSchwarze, J.-E., Tennant, P. W. G., Barnhart, K. T., Platt, R. W., Gupta, S. K., Venetis, C. A., D’Hooghe, T., & Schisterman, E. F. (2025) · Fertility and Sterility · doi:10.1016/j.fertnstert.2025.05.168 · Cited by 9
Medications and Risk for Microscopic Colitis: A Nationwide Study of Older Adults in Sweden
Khalili, H., McGee, E. E., Challa, P. K., Roelstraete, B., Johnell, K., Schneeweiß, S., Wastesson, J. W., & Ludvigsson, J. F. (2025) · Annals of Internal Medicine · doi:10.7326/annals-25-00268 · Cited by 8
Wu, C., Su, C., Chang, Y., Lee, P., & Su, Y. (2025) · Diabetes & Metabolism · doi:10.1016/j.diabet.2025.101695 · Cited by 10
Ahn, H. M., Kaelber, D. C., Hoell, N. G., Ambani, R. N., Quatromoni, J. G., Khalifeh, A., Kirksey, L., Hanak, C. R., Cameron, S. J., Lyden, S. P., et al. (2025) · Journal of Vascular Surgery · doi:10.1016/j.jvs.2025.11.021 · Cited by 11
Kotecha, P., Smith, S. M., Donahoo, W. T., DeKosky, S. T., Bian, J., & Guo, J. S. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.41939 · Cited by 12
Using routinely collected data for research purposes: challenges and mitigation strategies
Hoffmann, S., Morris, T. P., Herrmann, M., Heinze, G., Wynants, L., CALSTER, B. V., Bischl, B., Schmid, M., Shaw, P. A., Mathes, T., et al. (2026) · BMJ · doi:10.1136/bmj-2025-087812 · Cited by 12
Farhat, N., Birkett, N., Haddad, N., Fortin, Y., Momoli, F., Wen, S. W., Wielgosz, A. T., McNair, D., Mattison, D. R., & Krewski, D. R. (2020) · Drugs - Real World Outcomes · doi:10.1007/s40801-020-00204-9 · Cited by 9
“[26] stressed that negative controls should be applied more extensively in epidemiological studies to detect confounding from unmeasured variables.”
BackgroundWhere to begin? Thirty must‐read papers for newcomers to pharmacoepidemiology
Pottegård, A., Morin, L., Hallas, J., Gerhard, T., Winterstein, A. G., Perez‐Gutthann, S., & Tadrous, M. (2021) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.5382 · Cited by 8
Wander, P. L., Lowy, E., Beste, L. A., Tulloch-Palomino, L. G., Korpak, A. M., Peterson, A. C., Kahn, S. E., Danaei, G., & Boyko, E. J. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-058363 · Cited by 8
Do greenness and landscape indices for greenspace correlate with suicide ratio?
Lee, H., Chang, H.-T., Herianto, S., Wu, C., Liu, W., Yu, C.-P., Pan, W., & Wu, C. (2023) · Landscape and Urban Planning · doi:10.1016/j.landurbplan.2023.104935 · Cited by 8
Yum, Y., Kim, J. H., Joo, H. J., Kim, Y. H., & Kim, E. J. (2024) · American Journal of Hypertension · doi:10.1093/ajh/hpae012 · Cited by 9
Bove, F., Greek, A. A., Gatiba, R., Kohler, B., Sherman, R., Shin, G. T., & Bernstein, A. S. (2024) · Environmental Health Perspectives · doi:10.1289/ehp14966 · Cited by 9
“For civilian workers, the QBA assumed about half smoked and one-third consumed ≥ 1 drink/d. Negative control diseases 26 were used to estimate smoking and alcohol consumption prevalence di ff erences between Camp Lejeune and Camp Pendleton.”
MethodZhuo, H., Ritz, B. R., Warren, J. L., Pollitt, K. J. G., & Liew, Z. (2025) · Environmental Health Perspectives · doi:10.1289/ehp14742 · Cited by 8
“24 To limit multiple testing, we focused on mixtures of TAC exposure in the NCE analysis.”
Low-Tidal-Volume Ventilation and Mortality in Patients With Acute Brain Injury
Daza, J. F., Hamad, D. M., Urner, M., Liu, K., Wahlster, S., Robba, C., Stevens, R. D., McCREDIE, V. A., Cinotti, R., & Taran, S. (2025) · CHEST Journal · doi:10.1016/j.chest.2025.06.042 · Cited by 9
“36 For this analysis, we re fi t the models using ventilator-associated pneumonia (VAP) as our end point, anticipating a null association if the impact of un-measured confounding was minimal.”
Is the Smog Lifting? Causal Inference in Environmental Epidemiology
Flanders, W. D., & Garber, M. D. (2019) · Epidemiology · doi:10.1097/ede.0000000000000986 · Cited by 8
Harrison, P. J., Colbourne, L., & Luciano, S. (2021) · The British Journal of Psychiatry · doi:10.1192/bjp.2020.249 · Cited by 8
Regression‐based negative control of homophily in dyadic peer effect analysis
Liu, L., & Tchetgen, E. T. (2021) · Biometrics · doi:10.1111/biom.13483 · Cited by 9
“Negative control variables have primarily been used in epidemiological applications to detect and sometimes correct for unmeasured confounding (Lipsitch et al., 2010; Tchetgen Tchetgen, 2013; Sofer et al., 2016;Miao et al., 2018; Shi et al., 2020).”
BackgroundMethodSaeed, O., Zhang, S., Patel, S. R., Jorde, U. P., Garcia, M. J., Bulcha, N., Gupta, T., Xian, Y., Matsouaka, R., Shah, S. D., et al. (2021) · Journal of Cardiac Failure · doi:10.1016/j.cardfail.2021.02.017 · Cited by 8
Harville, E. W., Shankar, A., Buekens, P. M., Wickliffe, J. K., & Lichtveld, M. Y. (2021) · International Journal of Hygiene and Environmental Health · doi:10.1016/j.ijheh.2021.113829 · Cited by 8
Foo, D., Sarna, M. K., Pereira, G. F., Moore, H. C., & Regan, A. K. (2023) · Archives of Disease in Childhood · doi:10.1136/archdischild-2022-324269 · Cited by 8
Sawyer, G., Heron, J. E., & Joinson, C. J. (2023) · Journal of Affective Disorders · doi:10.1016/j.jad.2023.05.003 · Cited by 11
“Alternatively, if both mother and partner exposures are associated with offspring outcomes, confounding by shared genetic or environmental factors, such as family adversity and increased stress, are the more likely explanations (Lipsitch et al., 2010).”
BackgroundEklund, E., Holmberg, R., Svensson, M., Gustavsson, J., & Bonander, C. (2023) · Injury Prevention · doi:10.1136/ip-2022-044808 · Cited by 11
“In control municipalities, we defined 65+ years as the eligible age group and 50–64 years as internal controls, as this is the most common age of eligibility for ice cleat distribution used by Swedish municipalities.”
BackgroundMethodInfluentialOh, M., Kim, M., So, M. W., Lim, D., Choi, S. J., Lee, J. H., Her, M., Kim, S.-H., & Lee, S. (2023) · The Korean Journal of Internal Medicine · doi:10.3904/kjim.2023.222 · Cited by 8
“…be more precise estimates of the total risk of knee OA for mortality in Koreans. knee OA on all-cause and cause-specific mortality in three sequential propensity score-matched cohorts of asymptomatic ROA, symptomatic ROA, and a negative exposure con-trol group with knee pain but without ROA [15].”
BackgroundKweon, T., Kim, Y., Lee, K. J., Seo, W., Seo, S. I., Shin, W. G., & Shin, D. H. (2023) · Scientific Reports · doi:10.1038/s41598-023-48430-9 · Cited by 9
“Empirical calibration of the P values was performed by fitting an empirical null distribution to the point estimates of the negative control outcomes, which were assumed not to be associated with the target or comparative cohorts 27,28 .”
MethodSensitivity analysis for causality in observational studies for regulatory science
Diaz, I. M., Lee, H., Kıcıman, E., Schenck, E. J., Akacha, M., Follman, D., & Ghosh, D. (2023) · Journal of Clinical and Translational Science · doi:10.1017/cts.2023.688 · Cited by 9
“Additional methods such as negative control treatments and outcomes may be used to rule out the possibility that observed adjusted associations are due to unobserved confounding [43].”
BackgroundUthman, O. A., Court, R. A., Anjorin, S., Enderby, J., Al-Khudairy, L., Nduka, C. U., Mistry, H., Meléndez‐Torres, G. J., Taylor‐Phillips, S., & Clarke, A. (2023) · Health Technology Assessment · doi:10.3310/nmfg0214 · Cited by 10
Horne, E. M. F., Hulme, W. J., Parker, E. P. K., Keogh, R. H., Williamson, E., Walker, V. M., Palmer, T., Denholm, R., Knight, R., Curtis, H. J., et al. (2024) · Epidemiology · doi:10.1097/ede.0000000000001747 · Cited by 9
A novel method for controlling unobserved confounding using double confounders
Liu, L., Hou, L., Yu, Y., Liu, X., Sun, X., Yang, F., Wáng, Q., Jing, M., Xu, Y., Li, H., et al. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01049-0 · Cited by 8
Girardi, P., Lanfranchi, S., Mastromatteo, L. Y., Stafoggia, M., & Scrimin, S. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph182111648 · Cited by 8
“, adaptive and socio-emotional domains) as reported in the literature [52], constituting a negative control [53].”
BackgroundBramante, C. T., Johnson, S., Garcia, V. L., Evans, M. D., Harper, J. R., Wilkins, K. J., Huling, J. D., Mehta, H. B., Alexander, C., Tronieri, J. S., et al. (2022) · PLoS ONE · doi:10.1371/journal.pone.0271574 · Cited by 10
“Additionally, back-pain was assessed as a negative control outcome [28].”
BackgroundInstrumental variables: Don't throw the baby out with the bathwater
Keele, L. J., & Small, D. S. (2019) · Health Services Research · doi:10.1111/1475-6773.13130 · Cited by 8
School suspension predicts trichomoniasis five years later in a matched sample
Rosenbaum, J. E. (2020) · BMC Public Health · doi:10.1186/s12889-020-8197-8 · Cited by 9
Migrant status and childhood hospitalizations for asthma and other wheezing disorders
Leung, J. Y. Y., Leung, G. M., & Schooling, C. M. (2017) · Clinical & Experimental Allergy · doi:10.1111/cea.12896 · Cited by 8
Guzman, M. C. D., Chua, P. A. P., & Sedano, F. S. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.09.16.299438 · Cited by 8
“532 533 Negative controls are important since they are used to detect confounding variables [73] 534 and serve as a basis of comparison for different test groups.”
BackgroundLi, K., Emerman, I., Cook, A. J., FIREMAN, B. H., Sundaram, M. E., Tseng, H. F., Weintraub, E., Yu, O., Nelson, J. C., & Shi, X. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae439 · Cited by 7
Martino, R. M., Hollinsaid, N. L., Colich, N. L., McLaughlin, K. A., & Hatzenbuehler, M. L. (2025) · Scientific Reports · doi:10.1038/s41598-025-00378-8 · Cited by 10
McGee, E. E., Hernán, M. Á., Giovannucci, E. L., Mucci, L. A., Chiu, Y., Eliassen, A. H., & Dickerman, B. A. (2025) · Epidemiology · doi:10.1097/ede.0000000000001889 · Cited by 10
Metoyer, G. T., Whiteson, H. Z., Chen, Y., Li, Y., Gao, C., Menon, G., Bae, S., Lentine, K. L., Segev, D. L., McAdams‐DeMarco, M., et al. (2025) · Clinical Transplantation · doi:10.1111/ctr.70354 · Cited by 7
McCoy, R. G., SWARNA, K. S., Polley, E. C., Deng, Y., Chawla, S., Neumiller, J. J., Galindo, R. J., Umpierrez, G. E., Ross, J. S., Mickelson, M. M., et al. (2026) · Endocrine Practice · doi:10.1016/j.eprac.2026.01.012 · Cited by 7
Li, Z., Liu, S., Ho, J. C., Narayan, K. M. V., Ali, M. K., & Varghese, J. S. (2026) · Diabetologia · doi:10.1007/s00125-026-06687-5 · Cited by 8
Kim, M. G., Lin, T., Arora, T., Zhao, H., Balasubramanian, A., Stad, R. K., O’Kelly, J., Spangler, L., Bradbury, B. D., & Curtis, J. R. (2020) · Journal of Bone and Mineral Research · doi:10.1002/jbmr.4817 · Cited by 7
“NCOs share the same potential sources of bias with the primary outcome, but cannot plausibly be related to the treatment of interest.((27,28)) For example, receipt of influenza vaccine can serve as an NCO because it should have no causal relationship with osteoporosis treatments.”
BackgroundMethodInfluentialYim, G., Roberts, A. L., Lyall, K., Ascherio, A., & Weisskopf, M. G. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae038 · Cited by 8
Lu, Y., Matuska, K., McEvoy, R., Izurieta, H. S., Hervol, J. R., Menis, M. D., Lindaas, A., Steele, W. R., Chillarige, Y., Wernecke, M., et al. (2024) · The Journal of Infectious Diseases · doi:10.1093/infdis/jiae503 · Cited by 9
Enhancing causal inference in population-based neuroimaging data in children and adolescents
Visontay, R., Squeglia, L. M., Sunderland, M., Devine, E. K., Byrne, H. R., & Mewton, L. R. (2024) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2024.101465 · Cited by 9
“…tests to quantify robustness to possible unmeasured confounding, similar to those described for causal mediation analysis, are also gaining popularity (e.g., ‘E-values analysis ’ ; VanderWeele and Ding, 2017), and can be combined with others such as negative control testing (Lipsitch et al., 2010).”
Khan, S. U., Akbar, U. A., Khan, M. S., Patel, K. V., Nadeem, A., Thakkar, S., Arshad, H. B., Virani, S. S., Nasir, K., Goel, S. S., et al. (2025) · JACC Advances · doi:10.1016/j.jacadv.2024.101566 · Cited by 7
Arzika, A. M., Amza, A., Maliki, R., Aichatou, B., Bello, I. M., Beidi, D., Galo, N., Harouna, N., Karamba, A. M., Mahamadou, S., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.19693 · Cited by 8
Gebreegziabher, E., Ouattara, M., Bountogo, M., Coulibaly, B., Boudo, V., Ouédraogo, T., Lebas, E., Hu, H., Glidden, D. V., Arnold, B. F., et al. (2025) · Malaria Journal · doi:10.1186/s12936-025-05597-y · Cited by 7
“Additionally, since all communities were treated and there was no exposure control group, a negative control outcome was used to assess potential unmeasured confounding from other factors that may change over time [24, 25].”
Thomas, T., Golin, C. E., Kinlaw, A. C., Kirkman, M. S., Golden, S. D., Lightfoot, A. F., & Samuel‐Hodge, C. D. (2021) · Journal of General Internal Medicine · doi:10.1007/s11606-021-06749-x · Cited by 7
City-wide school-located influenza vaccination: A retrospective cohort study
Benjamin‐Chung, J., Arnold, B. F., Mishra, K., Kennedy, C. J., Nguyen, A., Pokpongkiat, N. N., Djajadi, S., Seth, A., Klein, N. P., Hubbard, A., et al. (2021) · Vaccine · doi:10.1016/j.vaccine.2021.08.099 · Cited by 7
“171 We pre-specified two negative control analyses to detect residual confounding or 172 selection bias [33,34].”
Influenza Vaccine Uptake in the Year After Concurrent vs Separate Influenza and Zoster Immunization
Rome, B. N., Feldman, W. B., Fischer, M. A., Desai, R. J., & Avorn, J. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.35362 · Cited by 7
Assessing the effectiveness of empirical calibration under different bias scenarios
Hwang, H., Quiroz, J. C., & Gallego, B. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01687-6 · Cited by 8
Holmberg, R., Gustavsson, J., Svensson, M., & Bonander, C. (2022) · Injury Prevention · doi:10.1136/ip-2022-044681 · Cited by 9
“21 We found no association between programme exposure and ice cleat use among the negative controls, implying that the results are not driven by age-independent confounders that set programme municipalities apart from the other munic-ipalities in the data.”
Chen, Y.-J., Lin, I., Chuang, J.-H., Huang, H., & Chan, T. (2022) · Emerging Microbes & Infections · doi:10.1080/22221751.2022.2155584 · Cited by 9
Gu, K., Zhou, M., Luo, W. P., Liu, Y., Dou, P., Huang, C., & Di, Q. (2023) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2023.162656 · Cited by 7
Huang, H., Liu, P. P., Lin, S., Yeh, J., Hsu, J., Peng, C. C., Munir, K. M., Loh, C., & Tu, Y. (2023) · European Heart Journal - Cardiovascular Pharmacotherapy · doi:10.1093/ehjcvp/pvad038 · Cited by 7
Atrial Fibrillation and Retinal Stroke
Lusk, J. B., Nalawade, V. V., Wilson, L. E., Song, A., Schrag, M., Biousse, V., Dumitrașcu, O., Poli, S., Piccini, J. P., Hammill, B. G., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.53819 · Cited by 9
Proximal causal inference for conditional separable effects
Park, C., Stensrud, M. J., & Tchetgen, E. J. T. (2026) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1093/jrsssb/qkag051 · Cited by 7
Deconfounding confounding part 1: traditional explanations
Attia, J., Jones, M. P., & Hure, A. J. (2017) · The Medical Journal of Australia · doi:10.5694/mja16.00491 · Cited by 7
Comment: Will Competition-Winning Methods for Causal Inference Also Succeed in Practice?
Zhao, Q., Keele, L. J., & Small, D. S. (2019) · Statistical Science · doi:10.1214/18-sts680 · Cited by 7
Gaulton, T. G., Deshpande, S. K., Small, D. S., & Neuman, M. D. (2019) · American Journal of Epidemiology · doi:10.1093/aje/kwz260 · Cited by 8
Demolition Activity and Elevated Blood Lead Levels among Children in Detroit, Michigan, 2014–2018
Bezold, C. P., Bauer, S., Buckley, J. P., Batterman, S., Haroon, H., & Fink, L. E. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17176018 · Cited by 7
Signorovitch, J., Moshyk, A., Zhao, J., Le, T. K., Burns, L. C., Gooden, K., & Hamilton, M. (2022) · Future Oncology · doi:10.2217/fon-2021-1054 · Cited by 8
Titus, A. R., Mijanovich, T., Terlizzi, K., Ellen, I. G., Anastasiou, E., Shelley, D., Wyka, K., Elbel, B. D., & Thorpe, L. E. (2022) · American Journal of Epidemiology · doi:10.1093/aje/kwac089 · Cited by 7
Kim, T., Seo, S. I., Lee, K. J., Park, C. H., Kim, T. J., Kim, J., & Shin, W. G. (2022) · Antibiotics · doi:10.3390/antibiotics11081052 · Cited by 8
“We performed empirical calibration of the p-values by fitting an empirical null distribution to point estimates of the negative control outcomes, which were assumed to not be associated with the target or comparator cohorts [33].”
MethodUnderstanding and Mitigating the Replication Crisis, for Environmental Epidemiologists
Bartell, S. M. (2019) · Current Environmental Health Reports · doi:10.1007/s40572-019-0225-4 · Cited by 8
Markovic, M., Swanson, S. A., Stricker, B. H. C., Jaddoe, V. W. V., Verhulst, F. C., Tiemeier, H. W., & Marroun, H. E. (2019) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4625 · Cited by 7
“To explore possible systematic biases to our estimates and prior studies' estimates, we compared our estimates from maternally reported outcomes to a teacher's report and further used NSAID use before pregnancy and somatic problems as a “negative” exposure and outcome, respectively.(20)”
MethodResultThe role of 5α‐reductase inhibitors in gastro‐oesophageal cancer risk: A nested case‐control study
Busby, J., Karasneh, R. A., Murchie, P., McMenamin, Ú. C., Gadalla, S. M., Camargo, M. C., Iversen, L., Lee, A. J., Spence, A. D., & Cardwell, C. R. (2019) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4909 · Cited by 7
Duko, B., Pereira, G. F., Betts, K. S., Tait, R. J., Newnham, J. P., & Alati, R. (2021) · Addictive Behaviors · doi:10.1016/j.addbeh.2021.106871 · Cited by 7
Fine needle aspiration cytology of epidermal inclusion cysts in the male breast.
Kapila, K., & Verma, K. (2003) · PubMed · Cited by 9
Cultural Sexism Moderates Efficacy of Psychotherapy: Results from a Spatial Meta-AnalysisPreprint
Price, M. A., Ford, J. V., Lattanner, M. R., Skov, H. E., & Wolock, E. R. (2020) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/df8yx · Cited by 7
Faksová, K., Thiesson, E. M., Pihlström, N., Baum, U., Biering‐Sørensen, T., Poukka, E., Leino, T. M., Ljung, R., & Hviid, A. P. (2025) · The Lancet Regional Health - Europe · doi:10.1016/j.lanepe.2025.101518 · Cited by 6
Zhang, Y., Sun, Q., Wang, Y., Sun, Y., Fernández, M. F., Messerlian, C., & Mustieles, V. (2025) · The Lancet Planetary Health · doi:10.1016/j.lanplh.2025.101394 · Cited by 7
Xie, M., Geldsetzer, P., Blakeman, T., James, M. T., Scale, T., Tan, Z., & Sawhney, S. (2026) · Kidney International · doi:10.1016/j.kint.2026.01.019 · Cited by 6
Amoxicillin-Clavulanate vs Amoxicillin for Acute Sinusitis in Adults
Savage, T. J., Butler, A. M., Kronman, M. P., Durkin, M. J., Sreedhara, S. K., Kabbani, S., Hicks, L. A., & Huybrechts, K. F. (2026) · JAMA · doi:10.1001/jama.2025.26902 · Cited by 8
Bots, S. H., Belitser, S. V., Groenwold, R. H. H., Durán, C. E., Riera‐Arnau, J., Schultze, A., Messina, D., Segundo, E., Douglas, I., Carreras, J. J., et al. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae172 · Cited by 6
“8,10 Under the assumption that unmeasured confounding is absent, running the main analysis with the negative control should give a null result, and deviation from this may signal unmeasured confounding for that particular relationship (and by extension the relationship of interest).”
Butler, A. M., Nickel, K. B., Olsen, M. A. A., Sahrmann, J. M., Colvin, R., Neuner, E. A., O’Neil, C. A., Fraser, V. J., & Durkin, M. J. (2024) · Clinical Infectious Diseases · doi:10.1093/cid/ciae519 · Cited by 6
Li, X., Singh, S., Rasouli, B. S., Lyons, J. G., Cocoros, N. M., Platt, R. G., Abi‐Elias, I. H., & Gurwitz, J. H. (2025) · Alzheimer s & Dementia Translational Research & Clinical Interventions · doi:10.1002/trc2.70080 · Cited by 7
“42,43 Quantitative bias analysis uses methods such as bias formulas, bounding methods, and probabilistic bias analysis to quantify and assess the sensitivity of study results to potential residual bias by specifying one value, a range of values, or a probability distribution of the unknown…”
Ballin, M., Nordström, A., Nordström, P., & Ahlqvist, V. H. (2025) · European Journal of Preventive Cardiology · doi:10.1093/eurjpc/zwaf267 · Cited by 6
“One such approach is negative control outcome analysis, 5 where the association between fitness and an outcome not expected to be causally affected is examined (e.g. random accidents).”
Ochoa, L. B., García‐Gómez, P., Steegers, E. A. P., Ourti, T. G. M. V., Bertens, L. C. M., & Been, J. V. (2025) · Public Health · doi:10.1016/j.puhe.2025.105819 · Cited by 7
Sieger, M. L., Godoy, L. D. C., Moore, T. E., Nichols, C., Goldsborough, E. J., Chen, S., Terplan, M., Griffin, B. A., & Patrick, S. W. (2025) · Health Affairs · doi:10.1377/hlthaff.2024.01160 · Cited by 6
Outcomes with cesarean delivery vs vaginal birth in extremely preterm breech singletons
Wang, Y., Santaguida, P. L., Parpia, S., Bacchini, F., Shah, P. S., Murphy, K. E., Shivananda, S. P., McDonald, S. D., Beltempo, M., Shah, P. S., et al. (2025) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2025.07.016 · Cited by 7
Aroke, H. A., Buchanan, A. L., Katenka, N., Crawford, F. W., Lee, T., Halloran, M. E., & Latkin, C. A. (2022) · AIDS and Behavior · doi:10.1007/s10461-022-03792-5 · Cited by 6
“The absence of a relationship between negative control and positive control terms or exposures and our outcome of interest when analyzed using the same model as for the main exposure-outcome possibly suggests the absence of significant unmeasured confounding [47].”
ResultFlanders, W. D. (2023) · Epidemiology · doi:10.1097/ede.0000000000001650 · Cited by 6
Young‐Xu, Y., Korves, C., Zwain, G., Satram, S., Drysdale, M., Reyes, C. M., Cheng, M. M., Bonomo, R. A., Epstein, L. H., Marconi, V. C., et al. (2023) · Open Forum Infectious Diseases · doi:10.1093/ofid/ofad605 · Cited by 6
Zhang, F., Zhu, S., Zhao, D., Tang, H., Ruan, L., & Zhu, W. (2023) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2023.169474 · Cited by 6
Chang, H., Jock, J., Rosenberg, M. S., Li, C., Cho, T., Gaziano, T. A., Lisabeth, L. D., & Kobayashi, L. C. (2024) · Innovation in Aging · doi:10.1093/geroni/igae010 · Cited by 6
“We re-ran our analyses among women born between 1928 and 1954 ( n = 1 346) taking consideration that women were not eligible for the OPG expansion but would presumably have been subjected to similar environmental exposures that affect hypertension risk as men (32).”
BackgroundMethodLöffler, L., Gögenür, I., & Gögenur, M. (2024) · International Journal of Colorectal Disease · doi:10.1007/s00384-024-04631-w · Cited by 7
Emulating target trials of postexposure vaccines using observational data
Boyer, C., & Lipsitch, M. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae350 · Cited by 10
Self-citationGuo, Y., Strauss, V. Y., Català, M., Jödicke, A. M., Khalid, S., & Prieto‐Alhambra, D. (2024) · Frontiers in Pharmacology · doi:10.3389/fphar.2024.1395707 · Cited by 6
Jeon, J., Ryoo, S., Oh, S., Hong, S. J., Yu, C. W., Kim, Y. H., Kim, E. J., & Joo, H. J. (2024) · American Journal of Hypertension · doi:10.1093/ajh/hpae147 · Cited by 6
Assessing the Risk for Falls in Older Adults After Initiating Gabapentin Versus Duloxetine
Chaitoff, A., Desai, R. J., Choudhry, N. K., Jungo, K. T., Haff, N., & Lauffenburger, J. C. (2025) · Annals of Internal Medicine · doi:10.7326/annals-24-00636 · Cited by 6
“…would not be expected to differ between incident users of gabapentin or dulox-etine and thus can provide additional insight into any meaningful underlying unmeasured differences in the sample that could be driving any associations between incident use of medication and the primary outcomes (51).”
Bach, D. R., Rigdon, E. E., & Sarstedt, M. (2025) · Journal of Business Research · doi:10.1016/j.jbusres.2025.115352 · Cited by 6
“This, however, is a classic problem in experimental design (Lipsitch et al., 2010).”
Role of placebo samples in observational studies
Ye, T., He, Q., Chen, S., & Zhang, B. (2025) · Journal of Causal Inference · doi:10.1515/jci-2023-0020 · Cited by 7
Metadata
- Authors
- Lipsitch, M., Tchetgen Tchetgen, E., & Cohen, T.
- Affiliations
- Harvard University (US)
- Brigham and Women's Hospital (US)
- Corresponding author
- Lipsitch, Marc
- Venue
- Epidemiology
- Published
- 2010-05
- Volume
- 21
- Issue
- 3
- Pages
- 383-388
- ISSN
- 1044-3983 (ISSN-L)
- PMC
- PMC3053408
- OpenAlex
- W2009428399
- Cited by
- 1,583
- References
- 25
- Citations by year
- 2026: 278 · 2025: 215 · 2024: 183 · 2023: 138 · 2022: 153 · 2021: 132 · 2020: 107 · 2019: 75 · 2018: 72 · 2017: 71 · 2016: 56 · 2015: 32 · 2014: 26 · 2013: 25 · 2012: 11
- FWCI
- 11.64
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- Funders
- National Institutes of Health (5u01gm076497-03, 1dp2od006663-01)
- National Institute of General Medical Sciences (U54GM088558, U01GM076497, 5U01GM076497)
- National Institute of Environmental Health Sciences (T32ES007142)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- negative control, confounding, spurious association, exposure control, outcome control, observational study validity, recall bias, experimental design, causal inference
- MeSH terms
- Humans, Bias, Causality, Confounding Factors, Epidemiologic, Observation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.