Meta-analyses including data from observational studies
Abstract
Observational studies represent a wide group of studies where the disease or condition of interest is naturally occurring and the investigator does not control allocation to interventions or exposures. Observational studies are used to test hypotheses about the efficacy of interventions or about exposure-disease relationships, to estimate incidence or prevalence of conditions, and to assess the sensitivity and specificity of diagnostic assays. Experimental-study designs and randomized controlled trials (RCTs) can also contribute to the body of evidence about such questions. Meta-analyses (either with or without systematic reviews) aim to combine information from primary research studies to better describe the entire body of work. The aim of meta-analyses may be to obtain a summary effect size, or to understand factors that affect effect sizes. In this paper, we discuss the role of observational studies in meta-analysis questions and some factors to consider when deciding whether a meta-analysis should include results from such studies. Our suggestion is that one should only include studies that are not at high risk of inherent bias when calculating a summary effect size. Study design however can be a meaningful variable in assessment of outcome heterogeneity.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Twenty articles (22%) gave recommendations on the key item "research question", with publication dates between 1997 and 2015 (median year 2009).
“Twenty articles (22%) gave recommendations on the key item “research question” [29, 30, 35, 36, 38–43, 45–54], with publication dates between 1997 and 2015 (median year of publication 2009).”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [42]
The articles recommended that the protocol should state the objectives, hypotheses to be tested and the rationale of the review, describe eligibility criteria, define the type of studies to be included, and give the reason when including observational studies in the review.
“The articles recommended that the protocol should state the objectives, hypotheses to be tested and the rationale of the review, [29] and that it should describe eligibility criteria [29, 33, 35, 36, 38, 39, 41], define the type of studies to be included [35, 37, 42, 44], and give the reason when including observational studies in the review [35, 37, 40].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [42]
The most common reason given for writing a protocol was to reduce bias in the selection of the studies by pre-specifying the study selection criteria.
“The most common reason given for writing a protocol was to reduce bias in the selection of the studies by pre-specifying the study selection criteria [37, 38, 40, 42].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [42]
It was recommended that the protocol should define the methods to be used for risk of bias assessment, meta-analysis and exploration of heterogeneity.
“Additionally, it was recommended that the protocol should define the methods to be used for risk of bias assessment, meta-analysis and exploration of heterogeneity [41, 42, 44].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [42]
Twenty-two articles (24%) provided recommendations on the key item "study eligibility", with publication dates between 1995 and 2016 (median year 2009).
“Twenty-two articles (24%) provided recommendations on the key item “study eligibility”, [12, 29, 30, 33, 34, 37–39, 41–43, 46–48, 54, 55, 61, 65, 70–73] with publication dates between 1995 and 2016 (median year of publication 2009).”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [42]
Twenty-five (27%) articles gave recommendations on item "considering different study designs", with publication dates between 1995 and 2016 (median year 2007).
“Twenty-five (27%) articles gave recommendations on item “considering different study designs” [8, 29, 34, 35, 37–39, 42, 44, 49, 51, 56, 61, 65, 70, 73, 75, 76, 77–83], with publication dates between 1995 and 2016 (median year of publication 2007).”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [42]
Some authors agreed that study design should be investigated as a potential source of between study heterogeneity, and others recommended presenting meta-analyses for each study type alongside results from an overall meta-analysis.
“Some authors agreed that study design should be investigated as a potential source of between study heterogeneity [8, 34, 42, 77], and others recommended presenting meta-analyses for each study type alongside results from an overall meta-analysis [42, 83].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [42]
A variation of this recommendation was to exclude studies that are at high risk of bias.
“A variation of this recommendation was to exclude studies that are at high risk of bias [42, 44, 54, 73, 82].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [42]
Cites
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164 · see citing works
Stroup, D. F. (2000) · JAMA · doi:10.1001/jama.283.15.2008 · Cited by 20,623 · see citing works
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.006 · Cited by 10,578 · see citing works
Chapter 9: Analysing Data and Undertaking Meta-Analyses
Deeks, J. J., Higgins, J. P. T., & Altman, D. G. (2008) · Cited by 2,782 · see citing works
Advanced methods in meta‐analysis: multivariate approach and meta‐regression
van Houwelingen, H. C., Arends, L. R., & Stijnen, T. (2002) · Statistics in Medicine · doi:10.1002/sim.1040 · Cited by 1,574 · see citing works
A comparison of methods to detect publication bias in meta‐analysis
Macaskill, P., Walter, S. D., & Irwig, L. (2001) · Statistics in Medicine · doi:10.1002/sim.698 · Cited by 1,286 · see citing works
The appropriateness of asymmetry tests for publication bias in meta-analyses: A large survey
Ioannidis, J. P. A., & Trikalinos, T. A. (2007) · Canadian Medical Association Journal · doi:10.1503/cmaj.060410 · Cited by 1,052 · see citing works
Authority, E. G. F. T. C. O. S. R. E. F. S. (2010) · EFSA Journal · doi:10.2903/j.efsa.2010.1637 · Cited by 693 · see citing works
Biases in the assessment of diagnostic tests
Begg, C. B. (1987) · Statistics in Medicine · doi:10.1002/sim.4780060402 · Cited by 495 · see citing works
Higgins, J. P., Ramsay, C., Reeves, B. C., Deeks, J. J., Shea, B., Valentine, J. C., Tugwell, P., & Wells, G. (2013) · Research Synthesis Methods · doi:10.1002/jrsm.1056 · Cited by 183 · see citing works
Vandenbroucke, J., & Psaty, B. (2008) · Journal of the American Medical Association (JAMA) · doi:10.1001/jama.2008.723 · Cited by 148 · see citing works
Reeves, B. C., Higgins, J. P. T., Ramsay, C. R., Shea, B., Tugwell, P., & Wells, G. A. (2013) · Research Synthesis Methods · doi:10.1002/jrsm.1068 · Cited by 111 · see citing works
Vandenbroucke, J. P. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.12.003 · Cited by 111 · see citing works
Sargeant, J. M., Elgie, R., Valcour, J. E., Saint-Onge, J., Thompson, A., Marcynuk, P., & Snedeker, K. G. (2009) · Preventive Veterinary Medicine · doi:10.1016/j.prevetmed.2009.06.002 · Cited by 89 · see citing works
Self-citationGroenwold, R. H. H., van Deursen, A. M. M., Hoes, A. W., & Hak, E. (2008) · Annals of Epidemiology · doi:10.1016/j.annepidem.2008.05.007 · Cited by 89 · see citing works
Dohoo, I. R., DesCôteaux, L., Leslie, K. D., Fredeen, A. H., Shewfelt, W., Preston, A., & Dowling, P. M. (2003) · Europe PMC (PubMed Central) · Cited by 83 · see citing works
Burns, M. J., & O'Connor, A. M. C. (2007) · Vaccine · doi:10.1016/j.vaccine.2007.10.014 · Cited by 69 · see citing works
Self-citationQuality of Reporting of Clinical Trials of Dogs and Cats and Associations with Treatment Effects
Sargeant, J. M., Thompson, A. M., Valcour, J. E., Elgie, R., Saint-Onge, J., Marcynuk, P., & Snedeker, K. G. (2009) · Journal of Veterinary Internal Medicine · doi:10.1111/j.1939-1676.2009.0386.x · Cited by 67 · see citing works
Self-citationDenagamage, T., O'Connor, A. M. C., Sargeant, J. M., Rajić, A., & McKean, J. D. (2007) · Foodborne Pathogens and Disease · doi:10.1089/fpd.2007.0013 · Cited by 66 · see citing works
Self-citationDohoo, I. R., Leslie, K. D., DesCôteaux, L., Fredeen, A. H., Dowling, P. M., Preston, A., & Shewfelt, W. (2003) · Europe PMC (PubMed Central) · Cited by 63 · see citing works
O'Connor, A. M. C., Auvermann, B. W., Bickett-Weddle, D. A., Kirkhorn, S. R., Sargeant, J. M., Ramirez, A., & Essen, S. G. V. (2010) · PLoS ONE · doi:10.1371/journal.pone.0009530 · Cited by 47 · see citing works
Self-citationCoppus, S. F. P. J., van der Veen, F., Bossuyt, P. M., & Mol, B. W. (2006) · Fertility and Sterility · doi:10.1016/j.fertnstert.2006.03.050 · Cited by 37 · see citing works
Selecting Observational Studies for Comparing Medical Interventions
Norris, S. L., Atkins, D. C., Bruening, W., Fox, S., Johnson, E. L., Kane, R. L., Morton, S. C., Oremus, M., Ospina, M. B., Randhawa, G., et al. (2008) · Europe PMC (PubMed Central) · Cited by 37 · see citing works
Sargeant, J. M., Torrence, M. E., Rajić, A., O'Connor, A. M. C., & Williams, J. (2006) · Foodborne Pathogens and Disease · doi:10.1089/fpd.2006.3.447 · Cited by 32 · see citing works
Self-citationReporting Heterogeneity in the Measurement of Health and Health-Related Quality of Life
Shmueli, A. (2002) · PharmacoEconomics · doi:10.2165/00019053-200220060-00005 · Cited by 32 · see citing works
BEYER, W. E. P. (2006) · Vaccine · doi:10.1016/j.vaccine.2006.05.038 · Cited by 28 · see citing works
O'Connor, A. M. C., Wellman, N. G., Evans, R. B., & Roth, D. R. (2006) · Animal Health Research Reviews · doi:10.1017/s146625230700120x · Cited by 27 · see citing works
Self-citationMethodologic standards for diagnostic test assessment studies
Begg, C. B. (1988) · Journal of General Internal Medicine · doi:10.1007/bf02595932 · Cited by 14 · see citing works
Moose Consort Strobe and Miame Stard Remark or how can we improve the quality of reporting studies
Verbeek, J. H. A. M. (2008) · Scandinavian Journal of Work Environment & Health · doi:10.5271/sjweh.1239 · Cited by 12 · see citing works
Meta-analysis methods for diagnostic accuracy
Begg, C. B. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.05.011 · Cited by 12 · see citing works
Improving the quality of reporting of studies of diagnostic accuracy: let's STARD now
Christopher, M. M. (2007) · Veterinary Clinical Pathology · doi:10.1111/j.1939-165x.2007.tb00173.x · Cited by 9 · see citing works
STARD and CONSORT: time for reflection
Ell, P. J. (2003) · European Journal of Nuclear Medicine and Molecular Imaging · doi:10.1007/s00259-003-1218-x · Cited by 2 · see citing works
Flagged first, then by citations
2 of the 34 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Zeng, X., Zhang, Y., Kwong, J. S. W., Zhang, C., Li, S., Sun, F., Niu, Y., & Du, L. (2015) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12141 · Cited by 2,082 · see citing works
Mueller, M., D’Addario, M., Egger, M., Cevallos, M., Dekkers, O., Mugglin, C., & Scott, P. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0495-9 · Cited by 400 · see citing works
Donath, J. L., Lüke, T., Graf, E., Tran, U. S., & Götz, T. (2023) · Educational Psychology Review · doi:10.1007/s10648-023-09752-2 · Cited by 199 · see citing works
Intimate partner violence and child attachment: A systematic review and meta-analysis
Noonan, C. B., & Pilkington, P. D. (2020) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2020.104765 · Cited by 83 · see citing works
Establishment of a typology for startups 4.0
Kaczam, F., Siluk, J. C. M., Guimarães, G. E., de Moura, G. L., da Silva, W. V., & da Veiga, C. P. (2021) · Review of Managerial Science · doi:10.1007/s11846-021-00463-y · Cited by 40 · see citing works
Salmonella in Animal Feeds: A Scoping Review
Sargeant, J. M., Totton, S. C., Plishka, M., & Vriezen, E. R. (2021) · Frontiers in Veterinary Science · doi:10.3389/fvets.2021.727495 · Cited by 34 · see citing works
Self-citationKaur, N., Dendukuri, N., Fellows, L. K., Brouillette, M., & Mayo, N. E. (2019) · AIDS Care · doi:10.1080/09540121.2019.1612017 · Cited by 33 · see citing works
Jambhekar, A., Robin, E., & Boedec, K. L. (2019) · Journal of Veterinary Internal Medicine · doi:10.1111/jvim.15568 · Cited by 31 · see citing works
Lo, K. B., Garvia, V., Stempel, J. M., Ram, P., & Rangaswami, J. (2019) · Heart & Lung · doi:10.1016/j.hrtlng.2019.10.007 · Cited by 31 · see citing works
Infectious and parasitic diseases of poultry in Ethiopia: a systematic review and meta-analysis
Asfaw, Y. T., Ameni, G., Medhin, G., Alemayehu, G., & Wieland, B. (2019) · Poultry Science · doi:10.3382/ps/pez521 · Cited by 29 · see citing works
An introduction to systematic reviews in animal health, animal welfare, and food safety
O'Connor, A. M. C., & Sargeant, J. M. (2014) · Animal Health Research Reviews · doi:10.1017/s146625231400005x · Cited by 26 · see citing works
Self-citationBoedec, K. L. (2017) · Journal of the American Veterinary Medical Association · doi:10.2460/javma.250.4.397 · Cited by 24 · see citing works
McMullen, C. K., Sargeant, J. M., Kelton, D. F., Churchill, K. J., Cousins, K. S., & Winder, C. B. (2021) · Journal of Dairy Science · doi:10.3168/jds.2020-19873 · Cited by 22 · see citing works
Self-citationClinical Examination and Making a Diagnosis
(2017) · Elsevier eBooks · doi:10.1016/b978-0-7020-5246-0.00001-2 · Cited by 20 · see citing works
Boelsen‐Robinson, T., Orellana, L., Backholer, K., Kurzeme, A., Jerebine, A., Gilham, B., Chung, A., & Peeters, A. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-029492 · Cited by 19 · see citing works
Cheurfa, C., Tsokani, S., Kontouli, K., Boutron, I., & Chaimani, A. (2024) · Systematic Reviews · doi:10.1186/s13643-024-02464-w · Cited by 18 · see citing works
Keay, S., Sargeant, J. M., O'Connor, A. M. C., Friendship, R. M., O’Sullivan, T. L., & Poljak, Z. (2020) · BMC Veterinary Research · doi:10.1186/s12917-020-02617-8 · Cited by 14 · see citing works
Self-citationMiotti, C., Cicotello, J., Archilla, G. A. S., Neder, V. E., Lucero, W. A., Calvinho, L. F., Signorini, M. L., Camussone, C., Zbrun, M. V., & Molineri, A. I. (2023) · Research in Veterinary Science · doi:10.1016/j.rvsc.2023.105032 · Cited by 10 · see citing works
Afifi, M., Kabera, F., Stryhn, H., Roy, J., Heider, L. C., Godden, S., Montelpare, W. J., Sánchez, J., & Dufour, S. (2018) · Animal Health Research Reviews · doi:10.1017/s1466252318000051 · Cited by 9 · see citing works
Vriezen, R., Sargeant, J. M., Vriezen, E. R., Winder, C. B., & O'Connor, A. M. C. (2019) · Animal Health Research Reviews · doi:10.1017/s146625231900029x · Cited by 9 · see citing works
Self-citationAzizpour, K., Schutte, P. J., Arts, M. P., Pondaag, W., Bouma, G. J., Coppes, M. H., Steyerberg, E. W., Peul, W. C., & Vleggeert-Lankamp, C. L. A. (2023) · Journal of Neurosurgery Spine · doi:10.3171/2022.12.spine22808 · Cited by 8 · see citing works
Thompson, A. C., & Smith, D. R. (2022) · The Bovine Practitioner · doi:10.21423/bovine-vol56no2p47-61 · Cited by 5 · see citing works
Wisener, L., Sargeant, J., O'Connor, A., O'Sullivan, T. L., Mcewen, S., Nwosu, A., & Rossi, T. (2019) · Animal Health Research Reviews · doi:10.1017/s1466252319000252 · Cited by 4 · see citing works
Self-citationDonath, J. L., Lüke, T., Graf, E., Tran, U. S., & Götz, T. (2022) · Research Square · doi:10.21203/rs.3.rs-2333341/v1 · Cited by 4 · see citing works
An investigative framework to facilitate epidemiological thinking during herd problem-solving
More, S. J., Doherty, M. L., & O’Grady, L. (2017) · Irish Veterinary Journal · doi:10.1186/s13620-017-0089-6 · Cited by 3 · see citing works
Nouira, M., Souayeh, N., Arfa, S., Maatouk, M., Hasnaoui, A., Bouchahda, H., & Ennigrou, S. (2023) · F1000Research · doi:10.12688/f1000research.141114.1 · Cited by 2 · see citing works
Καντζάνου, Μ., Kostares, E., Koumaki, V., Kostare, G., Kostares, M., & Tsakris, A. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14103558 · Cited by 2 · see citing works
Barros, R. D. J., Prazeres, M. P. C. D. S., Sousa, A. K. A., Vargens, M. L., & Coimbra, V. C. S. (2023) · Research Society and Development · doi:10.33448/rsd-v12i2.40147 · Cited by 2 · see citing works
Keay, S., Sargeant, J. M., O’Connor, A., Friendship, R. M., O’Sullivan, T., & Poljak, Z. (2020) · Figshare · doi:10.6084/m9.figshare.13181779
Self-citationKeay, S., Sargeant, J. M., O’Connor, A., Friendship, R. M., O’Sullivan, T., & Poljak, Z. (2020) · Figshare · doi:10.6084/m9.figshare.13181773
Self-citationCheurfa, C., Tsokani, S., Kontouli, K.-M., Boutron, I., & Chaimani, A. (2024) · Figshare · doi:10.6084/m9.figshare.26683834
Cheurfa, C., Tsokani, S., Kontouli, K.-M., Boutron, I., & Chaimani, A. (2024) · Figshare · doi:10.6084/m9.figshare.26683831
Keay, S., Sargeant, J. M., O’Connor, A., Friendship, R. M., O’Sullivan, T., & Poljak, Z. (2020) · Figshare · doi:10.6084/m9.figshare.13181776
Self-citationCheurfa, C., Tsokani, S., Kontouli, K.-M., Boutron, I., & Chaimani, A. (2024) · Figshare · doi:10.6084/m9.figshare.26683837
Cheurfa, C., Tsokani, S., Kontouli, K.-M., Boutron, I., & Chaimani, A. (2024) · Figshare · doi:10.6084/m9.figshare.26683828
Kanyama, J. C. B., Mavungu, G. N., Kassata, U. M., Lukamba, S. D., Embeya, V. O., Muyumba, G. M., & Ngoi, M. L. M. (2026) · Open Journal of Animal Sciences · doi:10.4236/ojas.2026.163019
- Authors
- O’Connor, A. M., & Sargeant, J. M.
- Affiliations
- University of Guelph (CA)
- ORCID
- Sargeant, Jan M.
- Venue
- Preventive Veterinary Medicine
- Publisher
- Elsevier BV
- Published
- 2014-02
- Volume
- 113
- Issue
- 3
- Pages
- 313-322
- ISSN
- 0167-5877 (ISSN-L)
- OpenAlex
- W2026697496
- Cited by
- 38
- References
- 34
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 5 · 2023: 6 · 2022: 2 · 2021: 3 · 2020: 6 · 2019: 6 · 2018: 3 · 2017: 3 · 2015: 1 · 2014: 1
- FWCI
- 0.73
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Preventive Veterinary Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- observational studies, meta-analysis, research design, outcome heterogeneity, exposure-disease relationship, sensitivity and specificity, inclusion criteria
- MeSH terms
- Veterinary Medicine, Meta-Analysis as Topic, Biomedical Research, Observational Studies as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
