How to assess the external validity of therapeutic trials: a conceptual approach
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
BACKGROUND: External validity of study results is an important issue from a clinical point of view. From a methodological point of view, however, the concept of external validity is more complex than it seems to be at first glance. METHODS: Methodological review to address the concept of external validity. RESULTS: External validity refers to the question whether results are generalizable to persons other than the population in the original study. The only formal way to establish the external validity would be to repeat the study for that specific target population. We propose a three-way approach for assessing the external validity for specified target populations. (i) The study population might not be representative for the eligibility criteria that were intended. It should be addressed whether the study population differs from the intended source population with respect to characteristics that influence outcome. (ii) The target population will, by definition, differ from the study population with respect to geographical, temporal and ethnical conditions. Pondering external validity means asking the question whether these differences may influence study results. (iii) It should be assessed whether the study's conclusions can be generalized to target populations that do not meet all the eligibility criteria. CONCLUSION: Judging the external validity of study results cannot be done by applying given eligibility criteria to a single target population. Rather, it is a complex reflection in which prior knowledge, statistical considerations, biological plausibility and eligibility criteria all have place.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“It can reflect a high generalizability, supporting the external validity of this study.16”
From Small Interfering RNA for LDL-Cholesterol Reduction (2026) · cites this paper as [16]
In the citing paper
“it is important to note that confounding and selection bias refer to biases that are internal to the study (‘internal validity’) and not to issues of generalisability or applicability (‘external validity’) [20].”
From COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology (2019) · cites this paper as [20]
Citations
Cites
The Effect of Spironolactone on Morbidity and Mortality in Patients with Severe Heart Failure
Pitt, B., Zannad, F., Remme, W. J., Cody, R., Castaigne, A., Perez, A., Palensky, J., & Wittes, J. (1999) · New England Journal of Medicine · doi:10.1056/nejm199909023411001 · Cited by 9,214
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373
The Effect of Carvedilol on Morbidity and Mortality in Patients with Chronic Heart Failure
Packer, M., Bristow, M. R., Cohn, J. N., Colucci, W. S., Fowler, M. B., Gilbert, E. M., & Shusterman, N. H. (1996) · New England Journal of Medicine · doi:10.1056/nejm199605233342101 · Cited by 4,649
External validity of randomised controlled trials: “To whom do the results of this trial apply?”
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17670-8 · Cited by 2,544
(2005) · The Lancet · doi:10.1016/s0140-6736(05)67660-x · Cited by 1,858
Rates of hyperkalemia after publication of the Randomized Aldactone Evaluation Study
Juurlink, D. N., Mamdani, M. M., Lee, D. S., Kopp, A., Austin, P. C., Laupacis, A., & Redelmeier, D. A. (2004) · New England Journal of Medicine · doi:10.1056/nejmoa040135 · Cited by 1,761
Chen, Z.-M. (1997) · The Lancet · doi:10.1016/s0140-6736(97)04010-5 · Cited by 1,232
Materson, B. J., Reda, D. J., Cushman, W. C., Massie, B. M., Freis, E. D., Kochar, M. S., Hamburger, R. J., Fye, C., Lakshman, R., Gottdiener, J., et al. (1993) · New England Journal of Medicine · doi:10.1056/nejm199304013281303 · Cited by 1,199
Lung Cancer and Other Causes of Death in Relation to Smoking
Doll, R., & Hill, A. B. (1956) · BMJ · doi:10.1136/bmj.2.5001.1071 · Cited by 795
Cited by
The PRECIS-2 tool: designing trials that are fit for purpose
Loudon, K., Treweek, S., Sullivan, F., Donnan, P., Thorpe, K. E., & Zwarenstein, M. (2015) · BMJ · doi:10.1136/bmj.h2147 · Cited by 1,846
Dekkers, O. M., Vandenbroucke, J. P., Cevallos, M., Renehan, A. G., Altman, D. G., & Egger, M. (2019) · PLOS Medicine · doi:10.1371/journal.pmed.1002742 · Cited by 525
Self-citationMultisystem morbidity and mortality in Cushing’s syndrome: a cohort study
Dekkers, O. M., Horváth-Puhó, E., Jørgensen, J. O. L., Cannegieter, S. C., Ehrenstein, V., Vandenbroucke, J. P., Pereira, A. M., & Sørensen, H. T. (2013) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2012-3582 · Cited by 419
Self-citationBig Data in Organ Transplantation: Registries and Administrative Claims
Massie, A. B., Kuricka, L. M., & Segev, D. L. (2014) · American Journal of Transplantation · doi:10.1111/ajt.12777 · Cited by 382
Distinguishing Case Series From Cohort Studies
Dekkers, O. M., Egger, M., Altman, D. G., & Vandenbroucke, J. P. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-156-1-201201030-00006 · Cited by 308
Self-citationA Review of Generalizability and Transportability
Degtiar, I., & Rose, S. (2023) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-042522-103837 · Cited by 295
Use of Mendelian randomisation to assess potential benefit of clinical intervention.
Burgess, S., Butterworth, A., Malarstig, A., & Thompson, S. G. (2012) · BMJ · doi:10.1136/bmj.e7325 · Cited by 292
Generalizing Study Results: A Potential Outcomes Perspective
Lesko, C. R., Buchanan, A. L., Westreich, D., Edwards, J. K., Hudgens, M. G., & Cole, S. R. (2017) · Epidemiology · doi:10.1097/ede.0000000000000664 · Cited by 269
“18,19,48,49 We argue that evaluations of generalizability could be more straightforward if considered quantitatively within the potential outcomes framework or the (logically equivalent) graphical models framework.”
den Exter, P. L., Hooijer, J., Dekkers, O. M., & Huisman, M. V. (2011) · Journal of Clinical Oncology · doi:10.1200/jco.2010.34.0984 · Cited by 264
Self-citationDekkers, O. M., Lagro, J., Burman, P., Jørgensen, J. O., Romijn, J. A., & Pereira, A. M. (2010) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2009-1238 · Cited by 230
Self-citationNieuwenhuis, M. H., Casparie, M., Mathus‐Vliegen, L. M. H., Dekkers, O. M., Hogendoorn, P. C. W., & Vasen, H. F. A. (2011) · International Journal of Cancer · doi:10.1002/ijc.25664 · Cited by 224
Self-citationMethadone at tapered doses for the management of opioid withdrawal
Amato, L., Davoli, M., Minozzi, S., Ferroni, E., Ali, R., & Ferri, M. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003409.pub4 · Cited by 222
Treatment decisions in multiple sclerosis — insights from real-world observational studies
Trojano, M., Tintore, M., Montalban, X., Hillert, J., Kalincik, T., Iaffaldano, P., Spelman, T., Sormani, M. P., & Butzkueven, H. (2017) · Nature Reviews Neurology · doi:10.1038/nrneurol.2016.188 · Cited by 191
Perils and Potentials of Self-Selected Entry to Epidemiological Studies and Surveys
Keiding, N., & Louis, T. A. (2016) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/rssa.12136 · Cited by 184
A review of causal inference for biomedical informatics
Kleinberg, S., & Hripcsak, G. (2011) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2011.07.001 · Cited by 171
Auditory rhythmic cueing in movement rehabilitation: findings and possible mechanisms
Schaefer, R. (2014) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2013.0402 · Cited by 168
de Jongh, R. T., Lips, P., van Schoor, N. M., Rijs, K. J., Deeg, D. J. H., Comijs, H. C., Kramer, M. H. H., Vandenbroucke, J. P., & Dekkers, O. M. (2011) · European Journal of Endocrinology · doi:10.1530/eje-11-0430 · Cited by 157
Self-citationvan Hulsteijn, L. T., Dekkers, O. M., Hes, F. J., Smit, J. W. A., & Corssmit, E. P. M. (2012) · Journal of Medical Genetics · doi:10.1136/jmedgenet-2012-101192 · Cited by 150
Self-citationTransferability of interventions in health education: a review
Cambon, L., Minary, L., Ridde, V., & Alla, F. (2012) · BMC Public Health · doi:10.1186/1471-2458-12-497 · Cited by 148
Naidoo, N., Nguyen, V. T., Ravaud, P., Young, B., Amiel, P., Schanté, D., Clarke, M., & Boutron, I. (2020) · BMC Medicine · doi:10.1186/s12916-019-1476-5 · Cited by 144
Prophylactic corticosteroids for cardiopulmonary bypass in adults
Dieleman, J. M., van Paassen, J., van Dijk, D., Arbous, M. S., Kalkman, C. J., Vandenbroucke, J. P., van der Heijden, G. J. M. G., & Dekkers, O. M. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005566.pub3 · Cited by 111
Self-citationCausality, Probability, and Time
Kleinberg, S. (2012) · Cambridge University Press eBooks · doi:10.1017/cbo9781139207799 · Cited by 107
Sensitivity and attachment interventions in early childhood: A systematic review and meta-analysis
Mountain, G., Cahill, J., & Thorpe, H. (2017) · Infant Behavior and Development · doi:10.1016/j.infbeh.2016.10.006 · Cited by 102
Vaginal misoprostol prior to insertion of an intrauterine device: an RCT
Dijkhuizen, K., Dekkers, O. M., Holleboom, C. A. G., de Groot, C. J. M., Hellebrekers, B. W. J., van Roosmalen, G. J. J., Janssen, C. A. H., & Helmerhorst, F. M. (2010) · Human Reproduction · doi:10.1093/humrep/deq348 · Cited by 99
Self-citationMijan, M. C., de Amorim, R. G., Leal, S. C., Mulder, J., Oliveira, L. B., Creugers, N. H. J., & Frencken, J. E. (2013) · Clinical Oral Investigations · doi:10.1007/s00784-013-1077-1 · Cited by 96
External Validity and Model Validity: A Conceptual Approach for Systematic Review Methodology
Khorsan, R., & Crawford, C. (2014) · Evidence-Based Complementary and Alternative Medicine · doi:10.1155/2014/694804 · Cited by 95
Jordan, S., Watkins, A., Storey, M., Allen, S. J., Brooks, C. J., Garaiova, I., Heaven, M. L., Jones, R., Plummer, S. F., Russell, I. T., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0067912 · Cited by 95
Cai, R., Liu, M., Hu, Y. Q., Melton, B. L., Matheny, M. E., Xu, H., Duan, L., & Waitman, L. R. (2017) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2017.01.004 · Cited by 91
Schatz, J., Schlenz, A. M., McClellan, C. B., Puffer, E. S., Hardy, S., Pfeiffer, M., & Roberts, C. W. (2015) · The Clinical Journal of Pain · doi:10.1097/ajp.0000000000000183 · Cited by 86
Improving quantitative synthesis to achieve generality in ecology
Spake, R., O’Dea, R. E., Nakagawa, S., Doncaster, C. P., Ryo, M., Callaghan, C. T., & Bullock, J. M. (2022) · Nature Ecology & Evolution · doi:10.1038/s41559-022-01891-z · Cited by 83
External validity, generalisability, applicability and directness: a brief primer
Murad, M. H., Katabi, A., Benkhadra, R., & Montori, V. M. (2018) · BMJ Evidence-Based Medicine · doi:10.1136/ebmed-2017-110800 · Cited by 82
“This statistical concept relates to sampling theory and can be evaluated by examining the size and representativeness of the study sample.(2) 3 A census or a large enough randomly selected sample should optimise generalisability.”
BackgroundMoran, M., Nickens, D. J., Adcock, K., Bennetts, M., Desscan, A., Charnley, N., & Fife, K. M. (2019) · Targeted Oncology · doi:10.1007/s11523-019-00653-5 · Cited by 78
What matters most: quantifying an epidemiology of consequence
Keyes, K., & Galea, S. (2015) · Annals of Epidemiology · doi:10.1016/j.annepidem.2015.01.016 · Cited by 75
Zeilstra, D., Younes, J. A., Brummer, R. J., & Kleerebezem, M. (2018) · Advances in Nutrition · doi:10.1093/advances/nmy046 · Cited by 74
External validity of randomized controlled trials in older adults, a systematic review
van Deudekom, F. J., Postmus, I., van der Ham, D. J., Pothof, A. B., Broekhuizen, K., Blauw, G. J., & Mooijaart, S. P. (2017) · PLOS ONE · doi:10.1371/journal.pone.0174053 · Cited by 65
Commentary: External validity of results of randomized trials: disentangling a complex concept
Rothwell, P. M. (2010) · International Journal of Epidemiology · doi:10.1093/ije/dyp305 · Cited by 64
Systematic mapping of checklists for assessing transferability
Munthe‐Kaas, H. M., Nøkleby, H., & Nguyen, L. (2019) · Systematic Reviews · doi:10.1186/s13643-018-0893-4 · Cited by 64
Long-term outcome of kidney function in patients with ANCA-associated vasculitis
Álamo, B. S., Moi, L., Bajema, I. M., Berden, A. E., Floßmann, O., Hrušková, Z., Jayne, D. R. W., Trejo, M. A. C. W., Wallquist, C., & Westman, K. W. A. (2024) · Nephrology Dialysis Transplantation · doi:10.1093/ndt/gfae018 · Cited by 61
Costa, L. J., Hari, P. N., & Kumar, S. K. (2016) · Leukemia & Lymphoma · doi:10.3109/10428194.2016.1170828 · Cited by 57
“the clinical trial population and how closely it resembles the affected population.[8,9] Therefore, we per-”
BackgroundVerloop, H., Dekkers, O. M., Peeters, R. P., Schoones, J. W., & Smit, J. W. A. (2014) · European Journal of Endocrinology · doi:10.1530/eje-14-0302 · Cited by 56
Self-citationHealthcare utilization is a collider: an introduction to collider bias in EHR data reuse
Weiskopf, N. G., Dorr, D. A., Jackson, C., Lehmann, H. P., & Thompson, C. A. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad013 · Cited by 56
Cappuccio, F. P., Buchanan, L. A., Ji, C., Siani, A., & Miller, M. A. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-011716 · Cited by 55
Burkow, T. M., Vognild, L. K., Østengen, G., Johnsen, E., Risberg, M. J., Bratvold, A., Hagen, T., Brattvoll, M., Krogstad, T., & Hjalmarsen, A. (2013) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-13-33 · Cited by 53
“The time elapsed since a study took place may influence the generalisation of the results due to factors such as changes in medical practice [22].”
BackgroundAhmad, N., Boutron, I., Dechartres, A., Durieux, P., & Ravaud, P. (2010) · Trials · doi:10.1186/1745-6215-11-20 · Cited by 53
Case series and descriptive cohort studies in neurosurgery: the confusion and solution
Esene, I. N., Ngu, J. N., Elzoghby, M., Solaroğlu, İ., Sikod, A. M., Kotb, A., Dechambenoit, G., & Husseiny, H. E. (2014) · Child s Nervous System · doi:10.1007/s00381-014-2460-1 · Cited by 53
Chung, V. C., Wong, C. H. L., Zhong, C. C., Tjioe, Y. Y., Leung, T. H., & Griffiths, S. M. (2020) · Integrative Medicine Research · doi:10.1016/j.imr.2020.100469 · Cited by 52
Papageorgiou, S. N., Papadopoulos, M. A., & Athanasiou, A. E. (2014) · The European Journal of Orthodontics · doi:10.1093/ejo/cjt008 · Cited by 51
Pagnoux, C., Carette, S., KHALIDI, N. A., Walsh, M. W., Hiemstra, T. F., Cuthbertson, D. D., Langford, C. A., Hoffman, G. C., Koening, C. L., Monach, P. A., et al. (2015) · University of Groningen research database (University of Groningen / Centre for Information Technology) · Cited by 51
Interventions for mycosis fungoides
Weberschock, T. B., Strametz, R., Lorenz, M., Röllig, C., Bunch, C., Bauer, A., & Schmitt, J. M. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008946.pub2 · Cited by 49
Post, P. N., de Beer, H., & Guyatt, G. H. (2012) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2012.01888.x · Cited by 48
‘Being disabled’ as an exclusion criterion for clinical trials: a scoping review
Camanni, G., Ciccone, O., Lepri, A., Tinarelli, C., Bedetti, C., Cicuttin, S., Murgia, N., & Elisei, S. (2023) · BMJ Global Health · doi:10.1136/bmjgh-2023-013473 · Cited by 47
Expertise-based randomized clinical trial of laparoscopic versus small-incision open cholecystectomy
Rosenmüller, M., Örnberg, M. T., Myrnäs, T., Lundberg, O., Nilsson, E., & Haapamäki, M. M. (2013) · British journal of surgery · doi:10.1002/bjs.9133 · Cited by 47
The Risk GP Model: The standard model of prediction in medicine
Fuller, J., & Flores, L. J. (2015) · Studies in History and Philosophy of Science Part C Studies in History and Philosophy of Biological and Biomedical Sciences · doi:10.1016/j.shpsc.2015.06.006 · Cited by 47
Target Validity: Bringing Treatment of External Validity in Line with Internal Validity
Lesko, C. R., Ackerman, B., Webster‐Clark, M., & Edwards, J. K. (2020) · Current Epidemiology Reports · doi:10.1007/s40471-020-00239-0 · Cited by 46
The risk for breast cancer is not evidently increased in women with hyperprolactinemia
Dekkers, O. M., Romijn, J. A., de Boer, A., & Vandenbroucke, J. P. (2009) · Pituitary · doi:10.1007/s11102-009-0214-y · Cited by 46
Self-citationCoknaz, D., Mirzeoglu, A. D., Atasoy, H. I., Alkoy, S., Coknaz, H., & Goral, K. (2019) · European Journal of Pediatrics · doi:10.1007/s00431-019-03457-x · Cited by 45
Blumle, A., Meerpohl, J. J., Rucker, G., Antes, G., Schumacher, M., & von Elm, E. (2011) · BMJ · doi:10.1136/bmj.d1828 · Cited by 44
Self-citationWhat’s more general than a whole population?
Alexander, N. D. E. (2015) · Emerging Themes in Epidemiology · doi:10.1186/s12982-015-0029-4 · Cited by 43
Pivetti, M., Battista, S. D., Paleari, F. G., & Hakoköngäs, E. J. (2021) · Journal of Pacific Rim Psychology · doi:10.1177/18344909211039893 · Cited by 42
Tranexamic acid for trauma: Filling the ‘ GAP ’ in evidence
Mitra, B., Mazur, S. M., Cameron, P. A., Bernard, S. A., Burns, B., Smith, A., Rashford, S., Fitzgerald, M., Smith, K. L., Gruen, R. L., et al. (2014) · Emergency Medicine Australasia · doi:10.1111/1742-6723.12172 · Cited by 41
Rationality and the generalization of randomized controlled trial evidence
Fuller, J. (2013) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12021 · Cited by 40
Hermida, J. R. F., Calafat, A., Becoña, E., Tsertsvadze, A., & Foxcroft, D. (2012) · Addiction · doi:10.1111/j.1360-0443.2012.03867.x · Cited by 40
Pijls, B. G. C. W., Dekkers, O. M., Middeldorp, S., Valstar, E. R., van der Heide, H. J. L., der Zwaag, H. M. J. V. D. L.-V., & Nelissen, R. G. H. H. (2011) · BMC Musculoskeletal Disorders · doi:10.1186/1471-2474-12-173 · Cited by 40
Self-citationGeneralizability of randomized controlled trials in heart failure with reduced ejection fraction
Lim, Y. M. F., Molnar, M., Vaartjes, I. C. H., Savarese, G., Eijkemans, M. J. C., Uijl, A., Vradi, E., Suzart‐Woischnik, K., Brugts, J. J., Rocca, H. B., et al. (2021) · European Heart Journal - Quality of Care and Clinical Outcomes · doi:10.1093/ehjqcco/qcab070 · Cited by 39
Interventions for mycosis fungoides
Valipour, A., Jäger, M., Wu, P. A., Schmitt, J. M., Bunch, C., & Weberschock, T. B. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008946.pub3 · Cited by 39
Improving the External Validity of Clinical Trials: The Case of Multiple Chronic Conditions
Fortin, M., & Susan, S. M. (2013) · Journal of Comorbidity · doi:10.15256/joc.2013.3.27 · Cited by 37
Checklists for external validity: a systematic review
Dyrvig, A., Kidholm, K., Gerke, O., & Vondeling, H. (2014) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12166 · Cited by 36
Battisti, N. M. L., Sehovic, M., & Extermann, M. (2017) · Clinical Lung Cancer · doi:10.1016/j.cllc.2017.03.005 · Cited by 35
Cristea, I. A., Karyotaki, E., Hollon, S. D., Cuijpers, P., & Gentili, C. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.03.022 · Cited by 34
“Moreover, while risk of bias indexes internal validity, establishing external validity is more complex and difficult to formalize (Dekkers et al., 2009).”
BackgroundInference with cross-lagged effects—Problems in time.
Driver, C. (2024) · Psychological Methods · doi:10.1037/met0000665 · Cited by 34
The Limitations of Artificial Intelligence in Head and Neck Oncology
Rao, K. N., Fernandez-Alvarez, V., Guntinas-Lichius, O., Sreeram, M. P., de Bree, R., Kowalski, L. P., Forastiere, A., Pace-Asciak, P., Rodrigo, J. P., Saba, N. F., et al. (2025) · Advances in Therapy · doi:10.1007/s12325-025-03198-4 · Cited by 33
Hohenschurz‐Schmidt, D., Cherkin, D. C., Rice, A. S. C., Dworkin, R. H., Turk, D. C., McDermott, M. P., Bair, M. J., DeBar, L., Edwards, R. R., Farrar, J. T., et al. (2023) · Pain · doi:10.1097/j.pain.0000000000002888 · Cited by 33
Searching the Clinical Fitness Landscape
Eppstein, M. J., Horbar, J. D., Buzas, J. S., & Kauffman, S. A. (2012) · PLoS ONE · doi:10.1371/journal.pone.0049901 · Cited by 33
Scheifes, A., Stolker, J. J., Egberts, T. C. G., Nijman, H. L. I., & Heerdink, E. R. (2010) · Journal of Intellectual Disability Research · doi:10.1111/j.1365-2788.2010.01353.x · Cited by 32
Kim, E., Caraballo, P. J., Castro, M. R., Pieczkiewicz, D. S., & Simon, G. (2019) · Journal of Medical Systems · doi:10.1007/s10916-019-1321-6 · Cited by 32
A bias‐adjusted evidence synthesis of RCT and observational data: the case of total hip replacement
Schnell‐Inderst, P., Iglesias, C. P., Arvandi, M., Ciani, O., Gothe, R. M., Peters, J., Blom, A. W., Taylor, R. S., & Siebert, U. (2017) · Health Economics · doi:10.1002/hec.3474 · Cited by 30
Chung, V. C., Ho, R. S. T., Wu, X., & Wu, J. C. Y. (2016) · European Journal of Integrative Medicine · doi:10.1016/j.eujim.2016.08.164 · Cited by 30
Jung, A., Balzer, J., Braun, T., & Luedtke, K. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01561-5 · Cited by 29
Cochrane systematic reviews in the field of addiction: What's there and what should be
Amato, L., Davoli, M., Vecchi, S., Ali, R. L., Farrell, M., Faggiano, F., Foxcroft, D., Ling, W. W., Minozzi, S., & Zhao, C. (2010) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2010.08.003 · Cited by 29
Nasser, M., van Weel, C., van Binsbergen, J. J., & de Laar, F. A. V. (2012) · Family Practice · doi:10.1093/fampra/cmr129 · Cited by 29
Wagenaar, L. P., Hamerlynck, T. W. O., Radder, C. M., Peters, L. W., Weyers, S., Schoot, B. C., & Vliet, H. A. V. (2023) · Fertility and Sterility · doi:10.1016/j.fertnstert.2023.08.956 · Cited by 28
Pibouleau, L., Boutron, I., Reeves, B. C., Nizard, R., & Ravaud, P. (2009) · BMJ · doi:10.1136/bmj.b4538 · Cited by 28
Exploring threats to generalisability in a large international rehabilitation trial (AVERT)
Bernhardt, J. A., Raffelt, A., Churilov, L. P., Lindley, R. I., Speare, S., Ancliffe, J., Ali, N. A. M., Hameed, S., Lennon, S., McRae, A., et al. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-008378 · Cited by 27
Felizardo, A. A., Marques, D. V. B., Caldas, I. S., Gonçalves, R. V., & Novaes, R. D. (2018) · Experimental Gerontology · doi:10.1016/j.exger.2018.01.022 · Cited by 27
Dekkers, O. M., Timmermans, J. M., Smit, J. W. A., Romijn, J. A., & Pereira, A. M. (2010) · European Journal of Endocrinology · doi:10.1530/eje-10-0621 · Cited by 26
Self-citationPool, J. J. M., Maissan, F., de Waele, N., Wittink, H. M., & Ostelo, R. (2019) · Musculoskeletal Science and Practice · doi:10.1016/j.msksp.2019.102098 · Cited by 26
Methods for pragmatic randomized clinical trials of pain therapies: IMMPACT statement
Hohenschurz‐Schmidt, D., Cherkin, D. C., Rice, A. S. C., Dworkin, R. H., Turk, D. C., McDermott, M. P., Bair, M. J., DeBar, L., Edwards, R. R., Evans, S. R., et al. (2024) · Pain · doi:10.1097/j.pain.0000000000003249 · Cited by 25
Randomized and nonrandomized studies: Complementary or competing?
Pandis, N., Tu, Y., Fleming, P. S., & Polychronopoulou, A. (2014) · American Journal of Orthodontics and Dentofacial Orthopedics · doi:10.1016/j.ajodo.2014.08.002 · Cited by 25
Koedel, U., Schuetze, C., Fischer, P. F., Bussmann, I., Sauer, P. K., Nixdorf, E., Kalbacher, T., Wichert, V., Rechid, D., Bouwer, L. M., et al. (2022) · Frontiers in Environmental Science · doi:10.3389/fenvs.2021.772666 · Cited by 24
Characteristics of acupuncture users among internal medicine patients in Germany
Cramer, H., Chung, V. C., Lauche, R., Zhang, Y., Zhang, A. L., Langhorst, J., & Dobos, G. J. (2015) · Complementary Therapies in Medicine · doi:10.1016/j.ctim.2015.04.009 · Cited by 24
Clinical drug trials in general practice: how well are external validity issues reported?
Brænd, A. M., Straand, J., & Klovning, A. (2017) · BMC Family Practice · doi:10.1186/s12875-017-0680-7 · Cited by 23
Evidence, External Validity, and Explanatory Relevance
Cartwright, N. D. (2011) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199738625.003.0002 · Cited by 23
Cuijpers, A. C. M., Linskens, F. G., Bongers, B. C., Stassen, L. P. S., Lubbers, T., & van Meeteren, N. L. U. (2022) · European Journal of Surgical Oncology · doi:10.1016/j.ejso.2022.04.012 · Cited by 22
Donovan, J., Young, G., Walsh, E. I., Metcalfe, C., Lane, J. A., Martin, R. M., Tazewell, M. K., Davis, M., Peters, T. J., Turner, E. L., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.12.019 · Cited by 22
A conceptual framework for external validity
Averitt, A., Ryan, P. B., Weng, C., & Perotte, A. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2021.103870 · Cited by 22
van Stein, R. M., Lok, C. A. R., Aalbers, A. G. J., de Hingh, I. H. J. T., Houwink, A. P. I., Stoevelaar, H. J., Sonke, G. S., & van Driel, W. J. (2022) · Gynecologic Oncology Reports · doi:10.1016/j.gore.2022.100945 · Cited by 22
Barrio, P., Ortega, L., Guardia, J., Roncero, C., Yuguero, L., & Gual, A. (2017) · Clinical Drug Investigation · doi:10.1007/s40261-017-0590-4 · Cited by 22
Reproducibility of Normal Flex tests in evaluating the flexibility of elderly women
Varejão, R. V., Figueira, H. A., Figueira, A. A., Conceição, M. C. D. S. C., Vale, R. G. S., & Dantas, E. H. M. (2014) · Research · doi:10.13070/rs.en.1.1266 · Cited by 21
Namjoo, S., Mirzaei, M., Foroughan, M., & Harouni, G. G. (2021) · Health Promotion Perspectives · doi:10.34172/hpp.2021.43 · Cited by 21
Randomization methods in emergency setting trials: a descriptive review
Corbett, M. S., Moe‐Byrne, T., Oddie, S., & McGuire, W. L. (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1163 · Cited by 21
Metadata
- Authors
- Dekkers, O. M., Elm, E. V., Algra, A., Romijn, J. A., & Vandenbroucke, J. P.
- Affiliations
- Leiden University Medical Center (NL)
- Corresponding author
- Dekkers, O M
- Venue
- International Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 2009-04-17
- Volume
- 39
- Issue
- 1
- Pages
- 89-94
- ISSN
- 0300-5771 (ISSN-L)
- 1464-3685
- OpenAlex
- W2145975717
- Cited by
- 219
- References
- 24
- Citations by year
- 2026: 3 · 2025: 8 · 2024: 18 · 2023: 17 · 2022: 20 · 2021: 14 · 2020: 11 · 2019: 13 · 2018: 7 · 2017: 12 · 2016: 11 · 2015: 15 · 2014: 14 · 2013: 14 · 2012: 15
- FWCI
- 11.62
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Journal of Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion)
- reroDoc Digital Library (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion)
- BORIS (University Library Bern) (submittedVersion, other-oa)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- generalizability, target population, study population representativeness, eligibility criteria, research design, biological plausibility, prior knowledge
- MeSH terms
- Age Factors, Clinical Trials as Topic, Humans, Research Design, Socioeconomic Factors, Time Factors, Reproducibility of Results, Epidemiologic Factors, Selection Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.