Methods to assess intended effects of drug treatment in observational studies are reviewed
The DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241 · cited 1×
“An alternative way of dealing with confounding caused by nonrandomized assignment of treatments in cohort studies, is the use of propensity scores, a method developed by Rosenbaum and Rubin [35].”
MethodA simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655 · cited 1×
“It’s obvious that also in these models the number of subjects or the number of events puts a restriction on the number of covariates; a ratio of 10–15 subjects or events per independent variable is mentioned in the literature [33,34].”
BackgroundSchulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634 · cited 1×
“This means that due to differences in quality, size or other characteristics disagreement among RCTs is not uncommon [62,63].”
BackgroundRosenbaum, P. R., & Rubin, D. B. (1985) · The American Statistician · doi:10.2307/2683903 · Cited by 4,774 · cited 1×
“The latter method should be used when the number of untreated patients is much greater than the number of treated patients; an additional gain in bias reduction can be reached when a variable number per pair, instead of a fixed number, is used [39].”
MethodIdentification of Causal Effects Using Instrumental Variables
Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476902 · Cited by 4,230
Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs
Concato, J., Shah, N., & Horwitz, R. I. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422507 · Cited by 3,563 · cited 1×
“Their validity for assessing intended effects of therapies has long been debated and remains controversial [8] [9] [10].”
BackgroundEstimating Causal Effects from Large Data Sets Using Propensity Scores
Rubin, D. B. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-8_part_2-199710151-00064 · Cited by 2,877 · cited 1×
“Advantages are (i) clear interpretation and communication of results, (ii) direct warning when treatment groups do not adequately overlap on used covariates, and (iii) no assumptions about the relation between outcome and covariates (e.g., linearity) [29] [30].”
BackgroundA comparison of observational studies and randomized, controlled trials.
Benson, K., & Hartz, A. J. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422506 · Cited by 2,347
The Case-Crossover Design: A Method for Studying Transient Effects on the Risk of Acute Events
Maclure, M. (1991) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a115853 · Cited by 2,327 · cited 1×
“This study design may avoid bias resulting from differences between exposed and nonexposed patients, but variations in the underlying disease state within individuals could still confound the association between treatment and outcome [25].”
BackgroundFundamentals of Clinical Trials
Friedman, L. M., Furberg, C. D., DeMets, D. L., Reboussin, D. M., & Granger, C. B. (2015) · doi:10.1007/978-3-319-18539-2 · Cited by 2,237
Peduzzi, P., Concato, J., Feinstein, A. R., & Holford, T. R. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(95)00048-8 · Cited by 2,154 · cited 1×
“It’s obvious that also in these models the number of subjects or the number of events puts a restriction on the number of covariates; a ratio of 10–15 subjects or events per independent variable is mentioned in the literature [33,34].”
BackgroundAngrist, J. D., & Imbens, G. W. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476535 · Cited by 1,341
Discrepancies between Meta-Analyses and Subsequent Large Randomized, Controlled Trials
LeLorier, J., Grégoire, G., Benhaddad, A., Lapierre, J., & Derderian, F. (1997) · New England Journal of Medicine · doi:10.1056/nejm199708213370806 · Cited by 1,171 · cited 1×
“This means that due to differences in quality, size or other characteristics disagreement among RCTs is not uncommon [62,63].”
BackgroundThe Risk of Myocardial Infarction Associated With Antihypertensive Drug Therapies
Psaty, B. M. (1995) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1995.03530080036038 · Cited by 1,003 · cited 1×
“This strategy was used in two case–control studies to compare the effects of different antihypertensive drug therapies on the risks of myocardial infarction and ischemic stroke [21,22].”
MethodSelf-citationThe Effectiveness of Adjustment by Subclassification in Removing Bias in Observational Studies
Cochran, W. G. (1968) · Biometrics · doi:10.2307/2528036 · Cited by 961 · cited 1×
“Advantages are (i) clear interpretation and communication of results, (ii) direct warning when treatment groups do not adequately overlap on used covariates, and (iii) no assumptions about the relation between outcome and covariates (e.g., linearity) [29] [30].”
BackgroundAllocation concealment in randomised trials: defending against deciphering
Schulz, K. F., & Grimes, D. A. (2002) · The Lancet · doi:10.1016/s0140-6736(02)07750-4 · Cited by 923 · cited 1×
“Proper randomization should remove all kinds of potential selection bias, such as physician preference for giving the new treatment to selected patients or patient preference for one of the treatments in the trial [2,3].”
BackgroundComparison of Evidence of Treatment Effects in Randomized and Nonrandomized Studies
Ioannidis, J. P. A. (2001) · JAMA · doi:10.1001/jama.286.7.821 · Cited by 919 · cited 1×
“To answer the question whether observational studies produce similar estimates of treatment effects compared to randomized studies, several authors have compared the results of randomized and nonrandomized studies for a number of conditions, sometimes based on meta-analyses [56] [57] [58].”
ResultCepeda, M. S. (2003) · American Journal of Epidemiology · doi:10.1093/aje/kwg115 · Cited by 859 · cited 1×
“Recently, the propensity score method was compared to logistic regression in a simulation study with a low number of events and multiple confounders [42].”
MethodSmoking and Lung Cancer: Recent Evidence and a Discussion of Some Questions
(1959) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/22.1.173 · Cited by 841 · cited 1×
“[59] when they attacked Fisher’s [60] hypothesis that the apparent association between smoking and lung cancer could be explained by an unmeasured genetic confounder related to both smoking and lung cancer.”
BackgroundRandomized versus historical controls for clinical trials
Sacks, H., Chalmers, T. C., & Smith, H. (1982) · The American Journal of Medicine · doi:10.1016/0002-9343(82)90815-4 · Cited by 834 · cited 1×
“Before the introduction and acceptance of the RCT as the gold standard for assessing the intended effect of treatments, it was common to compare the outcome of treated patients with the outcome of historical controls (patients previously untreated or otherwise treated) [16].”
ResultKunz, R., & Oxman, A. D. (1998) · BMJ · doi:10.1136/bmj.317.7167.1185 · Cited by 770 · cited 1×
“To answer the question whether observational studies produce similar estimates of treatment effects compared to randomized studies, several authors have compared the results of randomized and nonrandomized studies for a number of conditions, sometimes based on meta-analyses [56] [57] [58].”
ResultAssessing the sensitivity of regression results to unmeasured confounders in observational studies
Lin, D. Y., Psaty, B. M., & Kronmal, R. A. (1998) · Biometrics · doi:10.2307/2533848 · Cited by 742 · cited 1×
“[61] developed a general approach for assessing the sensitivity of the treatment effect to the confounding effects of unmeasured confounders after adjusting for measured covariates, assuming that the true treatment effect can be represented in a regression model.”
BackgroundSelf-citationECONOMETRICS IN OUTCOMES RESEARCH: The Use of Instrumental Variables
Newhouse, J. P., & McClellan, M. (1998) · Annual Review of Public Health · doi:10.1146/annurev.publhealth.19.1.17 · Cited by 676 · cited 1×
“This method can be used for the estimation of treatment effects (the effect of treatment on the treated) in observational studies [45] as an alternative to making causal inferences in RCTs.”
MethodAgostino, R. B. D. (1998) · Cited by 564
The Exclusion of the Elderly and Women From Clinical Trials in Acute Myocardial Infarction
Gurwitz, J. H. (1992) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1992.03490110055029 · Cited by 532 · cited 1×
“the period that drugs are used in clinical practice [6,7].”
BackgroundEarly Statin Treatment Following Acute Myocardial Infarction and 1-Year Survival
Stenestrand, U. (2001) · JAMA · doi:10.1001/jama.285.4.430 · Cited by 530
Evidence from randomised trials on the long-term effects of hormone replacement therapy
Beral, V., Banks, E., & Reeves, G. (2002) · The Lancet · doi:10.1016/s0140-6736(02)11032-4 · Cited by 487 · cited 1×
“Most observational studies indicated that HRT reduces the risk of cardiovascular disease, whereas RCTs demonstrated that HRT increases cardiovascular risk [12].”
BackgroundMcKee, M., Britton, A., Black, N., McPherson, K., Sanderson, C., & Bain, C. (1999) · BMJ · doi:10.1136/bmj.319.7205.312 · Cited by 399
Assessment and Control for Confounding by Indication in Observational Studies
Psaty, B. M., Koepsell, T. D., Lin, D., Weiss, N. S., Siscovick, D. S., Rosendaal, F. R., Pahor, M., & Furberg, C. D. (1999) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.1999.tb01603.x · Cited by 363 · cited 1×
“Compared to selection, restriction, stratification, or matching, more advanced multivariable statistical techniques have been developed to reduce bias due to differences in prognosis between treatment groups in observational studies [32].”
BackgroundSelf-citationSTRATIFICATION BY A MULTIVARIATE CONFOUNDER SCORE
MIETTINEN, O. S. (1976) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a112339 · Cited by 348 · cited 2×
“Although Miettinen did not specifically propose this method to adjust for confounding in studies of intended effects of treatment, the multivariate confounder score is very similar to the propensity score, except that the propensity score is not conditional on the outcome of interest, whereas the multivariate confound…”
MethodEffects of Misspecification of the Propensity Score on Estimators of Treatment Effect
Drake, C. (1993) · Biometrics · doi:10.2307/2532266 · Cited by 347
MCCLELLAN, M., MCNEIL, B. J., & NEWHOUSE, J. P. (1995) · Survey of Anesthesiology · doi:10.1097/00132586-199506000-00003 · Cited by 331
Understanding the Divergent Data on Postmenopausal Hormone Therapy
Grodstein, F., Clarkson, T. B., & Manson, J. E. (2003) · New England Journal of Medicine · doi:10.1056/nejmsb022365 · Cited by 329 · cited 1×
“The recent example of the potential cardiovascular risk reducing effects of hormone replacement therapy (HRT) illustrates this controversy [11].”
BackgroundAronow, H. D., Topol, E. J., Roe, M. T., Houghtaling, P. L., Wolski, K. E., Lincoff, A. M., Harrington, R. A., Califf, R. M., Ohman, E. M., Kleiman, N. S., et al. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04257-4 · Cited by 313 · cited 1×
“Examples of the these methods can be found in two studies of the effect of early statin treatment on the short-term risk of death [37,38].”
BackgroundSuissa, S. (1995) · Epidemiology · doi:10.1097/00001648-199505000-00010 · Cited by 270 · cited 1×
“With this design and with certain assumptions confounding due to time trends in exposure can be removed, but variations in the severity of disease over time within individuals, although probably correlated with exposure levels, cannot be controlled [26] [27] [28].”
BackgroundSubstantial Gains in Bias Reduction from Matching with a Variable Number of Controls
Ming, K., & Rosenbaum, P. R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00118.x · Cited by 258 · cited 1×
“An optimal method combined with a variable number of controls should be the preferred method [40].”
MethodMacMahon, S., & Collins, R. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04017-4 · Cited by 249 · cited 1×
“Although this may be correct in specific situations [17,55], it does not provide evidence for the validity of observational methods for the evaluation of treatments in general [8].”
Background(2003) · Technometrics · doi:10.1198/tech.2003.s149 · Cited by 242
Developing improved observational methods for evaluating therapeutic effectiveness
Horwitz, R. I., Viscoli, C. M., Clemens, J. D., & Sadock, R. T. (1990) · The American Journal of Medicine · doi:10.1016/0002-9343(90)90182-d · Cited by 204 · cited 1×
“[54] proposed to validate observational studies by constructing a cohort of subjects in clinical practice that is restricted by the inclusion criteria of RCTs.”
BackgroundFundamentals of clinical trials
Sylvester, R. (1982) · Controlled Clinical Trials · doi:10.1016/0197-2456(82)90029-0 · Cited by 191
Hlatky, M. A., Califf, R. M., Harrell, F. E., Lee, K. L., Mark, D. B., & Pryor, D. B. (1988) · Journal of the American College of Cardiology · doi:10.1016/0735-1097(88)90086-1 · Cited by 180 · cited 1×
“Although this may be correct in specific situations [17,55], it does not provide evidence for the validity of observational methods for the evaluation of treatments in general [8].”
BackgroundConfounding and Exposure Trends in Case-Crossover and Case-Time-Control Designs
Greenland, S. (1996) · Epidemiology · doi:10.1097/00001648-199605000-00003 · Cited by 178
McClellan, M. (1994) · JAMA · doi:10.1001/jama.272.11.859 · Cited by 163
Simultaneous-Equation Estimation in a Clinical Trial of the Effect of Smoking on Birth Weight
Permutt, T., & Hebel, J. R. (1989) · Biometrics · doi:10.2307/2531503 · Cited by 159
Fisher, R. A. (1958) · Nature · doi:10.1038/182108a0 · Cited by 150 · cited 1×
“This method was probably first used by Cornfield et al. [59] when they attacked Fisher's [60] hypothesis that the apparent association between smoking and lung cancer could be explained by an unmeasured genetic confounder related to both smoking and lung cancer.”
MethodKalra, L. (2000) · BMJ · doi:10.1136/bmj.320.7244.1236 · Cited by 150 · cited 2×
“An example of this method can be found in Kalra et al. [17].”
BackgroundMethodConfounding and indication for treatment in evaluation of drug treatment for hypertension
Grobbee, D. E., & Hoes, A. W. (1997) · BMJ · doi:10.1136/bmj.315.7116.1151 · Cited by 129 · cited 1×
“The main criticism of observational studies is the absence of a randomized assignment of treatments, with the result that uncontrolled confounding by unknown, unmeasured, or inadequately measured covariates may provide an alternative explanation for the treatment effect [13,14].”
BackgroundSelf-citationCurrent problems and future challenges in randomized clinical trials.
Feinstein, A. R. (1984) · Circulation · doi:10.1161/01.cir.70.5.767 · Cited by 100 · cited 1×
“There are settings where a randomized comparison of treatments may not be feasible due to ethical, economic or other constraints [5].”
BackgroundSuissa, S. (1998) · Epidemiology · doi:10.1097/00001648-199807000-00016 · Cited by 91 · cited 1×
“With this design and with certain assumptions confounding due to time trends in exposure can be removed, but variations in the severity of disease over time within individuals, although probably correlated with exposure levels, cannot be controlled [26] [27] [28].”
BackgroundThe case-time-control design: further assumptions and conditions.
Suissa, S. (1998) · PubMed · Cited by 77
The value of randomization and control in clinical trials
Urbach, P. (1993) · Statistics in Medicine · doi:10.1002/sim.4780121508 · Cited by 67 · cited 1×
“Randomization does not assure equality on all prognostic factors in the treatment groups, especially with small sample sizes, but it assures confidence intervals and P-values to be valid by using probability theory [4].”
MethodAntihypertensive Drug Therapies and the Risk of Ischemic Stroke
Klungel, O. H., Heckbert, S. R., Longstreth, W. T., Furberg, C. D., Kaplan, R. C., Smith, N. L., Lemaître, R. N., Leufkens, H. G. M., de Boer, A., & Psaty, B. M. (2001) · Archives of Internal Medicine · doi:10.1001/archinte.161.1.37 · Cited by 66 · cited 1×
“This strategy was used in two case–control studies to compare the effects of different antihypertensive drug therapies on the risks of myocardial infarction and ischemic stroke [21,22].”
MethodSelf-citationJohnston, S. C., Henneman, T., McCulloch, C. E., & van der Laan, M. (2002) · American Journal of Epidemiology · doi:10.1093/aje/kwf095 · Cited by 60 · cited 1×
“Keeping the analysis at the individual level, the individual treatment variable will be replaced by an ecological or grouped-treatment variable, indicating the percentage of treated persons at the aggregated level [53].”
BackgroundWhy transition from alternation to randomisation in clinical trials was made
Chalmers, I. (1999) · BMJ · doi:10.1136/bmj.319.7221.1372 · Cited by 56 · cited 1×
“Proper randomization should remove all kinds of potential selection bias, such as physician preference for giving the new treatment to selected patients or patient preference for one of the treatments in the trial [2,3].”
BackgroundMesserli, F. H. (1995) · Annals of Internal Medicine · doi:10.7326/0003-4819-123-11-199512010-00015 · Cited by 56
Uses of Ecologic Studies in the Assessment of Intended Treatment Effects
Wen, S. W. (1999) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(98)00136-x · Cited by 52
COOK, E. F., & GOLDMAN, L. (1988) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a114838 · Cited by 49 · cited 1×
“A method found in the literature that is worth mentioning, is asymmetric stratification [31].”
MethodPike, M. C., Anderson, J., & Day, N. (1979) · Epidemiology and Community Health · doi:10.1136/jech.33.1.104 · Cited by 40 · cited 1×
“The multivariate confounder score has been evaluated for validity [44].”
MethodIdentifying Confounding by Indication through Blinded Prospective Review
Johnston, S. C. (2001) · American Journal of Epidemiology · doi:10.1093/aje/154.3.276 · Cited by 37 · cited 1×
“The selection of a cohort of candidates for treatment can also be conducted by a panel of physicians after presenting them the clinical characteristics of the patients in the study [20].”
BackgroundCardiovascular drugs: discrepancies in demographics between pre- and post-registration use
Wieringa, N. F., de Graeff, P. A., van der Werf, G. T., Vos, R., & de Graeff, P. A. (1999) · European Journal of Clinical Pharmacology · doi:10.1007/s002280050670 · Cited by 32 · cited 1×
“the period that drugs are used in clinical practice [6,7].”
BackgroundRandomised trials comparing chemotherapy regimens for advanced non-small cell lung cancer
Ioannidis, J. P. A., Polycarpou, A., Ntais, C., & Pavlidis, N. A. (2003) · European Journal of Cancer · doi:10.1016/s0959-8049(03)00571-9 · Cited by 26 · cited 1×
“Changes of the characteristics of a general population or subgroup over time are not uncommon [18].”
BackgroundKlungel, O. H., Stricker, B. H. C., Breteler, M. M. B., Seidell, J. C., Psaty, B. M., & de Boer, A. (2001) · Epidemiology · doi:10.1097/00001648-200105000-00014 · Cited by 17 · cited 1×
“As a preliminary selection, this method was used in a cohort study to estimate the effect of drug treatment of hypertension on the incidence of stroke in the general population by selecting candidates on the basis of their blood pressure and the presence of other cardiovascular risk factors [19].”
MethodSelf-citationD’Agostino (1998) · doi:10.1002/(sici)1097-0258(19981015)17:19<2265::aid-sim918>3.0.co;2-b · cited 1×
“D’Agostino [36] found that “the propensity score for an individual, defined as the conditional probability of being treated given the individual’s covariates, can be used to balance the covariates in observational studies, and thus reduce bias.”
Background
Cited by
Mendelian Randomization as an Approach to Assess Causality Using Observational Data
Sekula, P., Del Greco M, F., Pattaro, C., & Köttgen, A. (2016) · Journal of the American Society of Nephrology · doi:10.1681/asn.2016010098 · Cited by 2,351
“(1) Genetic variant is strongly associated with”
BackgroundInfluentialInstrumental Variables: Application and Limitations
Martens, E. P., Pestman, W. R., de Boer, A., Belitser, S. V., & Klungel, O. H. (2006) · Epidemiology · doi:10.1097/01.ede.0000215160.88317.cb · Cited by 464
Self-citationEl–Serag, H. B., Johnson, M. L., Hachem, C., & Morgana, R. O. (2009) · Gastroenterology · doi:10.1053/j.gastro.2009.01.053 · Cited by 291
Smoking and suicide: A brief overview
Hughes, J. R. (2008) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2008.06.003 · Cited by 236
“Second, most studies of suicide and smoking have been case-control studies and this type of study almost always suffers from some self-selection or indication bias (Rothman & Greenland 1998; Shiffman et al. 2005; Klungel et al. 2004).”
BackgroundLimited Resection for the Treatment of Patients With Stage IA Lung Cancer
Wisnivesky, J. P., Henschke, C. I., Swanson, S., Yankelevitz, D. F., Zulueta, J., Marcus, S., & Halm, E. A. (2010) · Annals of Surgery · doi:10.1097/sla.0b013e3181c0e5f3 · Cited by 195
“The decision to indicate a treatment for a particular patient is inextricably connected to their clinical status and likelihood of response.(27) Consequently, it can be argued that whenever a rational indication for an intervention exists it tends to constitute a confounder for the study of its intended effect.”
Background
Metadata
- Authors
- Klungel, O. H., Martens, E. P., Psaty, B. M., Grobbee, D. E., Sullivan, S. D., Stricker, B. H. C., Leufkens, H. G. M., & de Boer, A.
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2004-12
- Volume
- 57
- Issue
- 12
- Pages
- 1223-1231
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2086038978
- Cited by
- 197
- References
- 63
- FWCI
- 7.87
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.