Metabias: A Challenge for Comparative Effectiveness Research
The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
Editorials5 July 2011Metabias: A Challenge for Comparative Effectiveness ResearchSteven Goodman, MD, MHS, PhD and Kay Dickersin, MA, PhDSteven Goodman, MD, MHS, PhDFrom Johns Hopkins University, Baltimore, MD 21205.Search for more papers by this author and Kay Dickersin, MA, PhDFrom Johns Hopkins University, Baltimore, MD 21205.Search for more papers by this authorAuthor, Article, and Disclosure Informationhttps://doi.org/10.7326/0003-4819-155-1-201107050-00010 SectionsAboutFull TextPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissions ShareFacebookTwitterLinkedInRedditEmail Comparative effectiveness research encompasses both individual primary research studies and syntheses of the primary research, typically systematic reviews and meta-analyses. Before accepting the results of either form of study, decision makers must critically assess their methods to identify sources of potential bias.For primary research, critical appraisal involves close examination of research methods, including design, data, execution, analysis, and interpretation. For meta-analyses, individual studies are examined in the same way, but the collection of studies is also examined for heterogeneity. Studies are deemed heterogeneous if their methods or results differ from one another so much that the studies cannot be ...References1. Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess. 2003;7:1-76. [PMID: 12583822] CrossrefMedlineGoogle Scholar2. Dickersin K, Chalmers I. Recognising, investigating and dealing with incomplete and biased reporting of clinical research: from Francis Bacon to the World Health Organisation. Accessed at www.jameslindlibrary.org/essays/biased_reporting/biased_reporting.html on 23 May 2011. Google Scholar3. Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev. 2009;:MR000006. [PMID: 19160345] MedlineGoogle Scholar4. Dwan K, Altman DG, Cresswell L, Blundell M, Gamble CL, Williamson PR. Comparison of protocols and registry entries to published reports for randomised controlled trials. Cochrane Database Syst Rev. 2011;:MR000031. [PMID: 21249714] MedlineGoogle Scholar5. Song F, Parekh S, Hooper L, Loke YK, Ryder J, Sutton AJ, et al. Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess. 2010;14:iii, ix-xi, 1-193. [PMID: 20181324] CrossrefMedlineGoogle Scholar6. Vedula SS, Bero L, Scherer RW, Dickersin K. Outcome reporting in industry-sponsored trials of gabapentin for off-label use. N Engl J Med. 2009;361:1963-71. [PMID: 19907043] CrossrefMedlineGoogle Scholar7. Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291:2457-65. [PMID: 15161896] CrossrefMedlineGoogle Scholar8. Fontanarosa PB, Flanagin A, DeAngelis CD. Reporting conflicts of interest, financial aspects of research, and role of sponsors in funded studies [Editorial]. JAMA. 2005;294:110-1. [PMID: 15998899] CrossrefMedlineGoogle Scholar9. Eyding D, Lelgemann M, Grouven U, Härter M, Kromp M, Kaiser T, et al. Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials. BMJ. 2010;341:c4737. [PMID: 20940209] CrossrefMedlineGoogle Scholar10. McGauran N, Wieseler B, Kreis J, Schüler YB, Kölsch H, Kaiser T. Reporting bias in medical research—a narrative review. Trials. 2010;11:37. [PMID: 20388211] CrossrefMedlineGoogle Scholar11. Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ. 2003;326:1167-70. [PMID: 12775614] CrossrefMedlineGoogle Scholar12. Als-Nielsen B, Chen W, Gluud C, Kjaergard LL. Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events? JAMA. 2003;290:921-8. [PMID: 12928469] CrossrefMedlineGoogle Scholar13. Gøtzsche PC. Research integrity and pharmaceutical industry sponsorship [Editorial]. Med J Aust. 2005;182:549-50. [PMID: 15938676] CrossrefMedlineGoogle Scholar14. Dickersin K, Rennie D. Registering clinical trials. JAMA. 2003;290:516-23. [PMID: 12876095] CrossrefMedlineGoogle Scholar15. ClinicalTrials.gov. Protocol registration system. Accessed at prsinfo.clinicaltrials.gov/fdaaa.html on 23 May 2011. Google Scholar16. World Health Organization. International Clinical Trials Registry Platform (ICTRP). Accessed at www.who.int/ictrp/en/ on 23 May 2011. Google Scholar17. Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The ClinicalTrials.gov results database—update and key issues. N Engl J Med. 20
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Pharmaceutical industry sponsorship and research outcome and quality: systematic review
Lexchin, J., Bero, L. A., Djulbegovic, B., & Clark, O. (2003) · BMJ · doi:10.1136/bmj.326.7400.1167 · Cited by 2,147
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845
Clinical Trial Registration: A Statement from the International Committee of Medical Journal Editors
De Angelis, C., Drazen, J. M., Frizelle, F. A., Haug, C., Hoey, J., Horton, R., Kotzin, S., Laine, C., Marusic, A., Overbeke, A. J. P. M., et al. (2004) · New England Journal of Medicine · doi:10.1056/nejme048225 · Cited by 1,241
Egger, M., Jüni, P., Bartlett, C., Holenstein, F., & Sterne, J. (2003) · Health Technology Assessment · doi:10.3310/hta7010 · Cited by 1,181 · cited 1×
“Meta-epidemiologic studies, examining many meta-analyses on diverse topics, have been finding risk factors for bias in those metaanalyses—both in the methods that meta-analysts use and in the characteristics of studies included—that are not clearly associated with a bias-producing process in an individual study (1).”
BackgroundDissemination and publication of research findings: an updated review of related biases
Song, F., Parekh, Hooper, L., Loke, Y., Ryder, J., Sutton, A., Hing, C., Kwok, C., Pang, C., & Harvey, I. (2010) · Health Technology Assessment · doi:10.3310/hta14080 · Cited by 1,039
Publication bias in clinical trials due to statistical significance or direction of trial results
Hopewell, S., Loudon, K., Clarke, M. J., Oxman, A. D., & Dickersin, K. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000006.pub3 · Cited by 898 · cited 1×
“Probably the first to be recognized was publication bias, which is essentially the failure to publish a certain proportion of studies because of their findings (2, 3).”
BackgroundSelf-citationThe ClinicalTrials.gov Results Database — Update and Key Issues
Zarin, D. A., Tse, T., Williams, R. J., Califf, R. M., & Ide, N. C. (2011) · New England Journal of Medicine · doi:10.1056/nejmsa1012065 · Cited by 838
Als-Nielsen, B., Chen, W., Gluud, C., & Kjaergard, L. L. (2003) · JAMA · doi:10.1001/jama.290.7.921 · Cited by 746
Dickersin, K. (2003) · JAMA · doi:10.1001/jama.290.4.516 · Cited by 512
Self-citationReporting bias in medical research - a narrative review
McGauran, N., Wieseler, B., Kreis, J., Schüler, Y.-B., Kölsch, H., & Kaiser, T. (2010) · Trials · doi:10.1186/1745-6215-11-37 · Cited by 452
Research works printed but not linked: 7
- Hopewell S, 2009, Cochrane Database Syst Rev, MR000006
- Dwan K, 2011, Cochrane Database Syst Rev, MR000031
- 10.1056/NEJMsa0906126 · DOI · Paper page
- 10.1001/jama.294.1.110 · DOI
- 10.1136/bmj.c4737 · DOI · Paper page
- 10.5694/j.1326-5377.2005.tb06807.x · DOI
- 10.1001/jama.292.11.1363 · DOI · Paper page
Cited by
Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., & the PRISMA-P Group (2015) · BMJ · doi:10.1136/bmj.g7647 · Cited by 13,120
Industry sponsorship and research outcome
Lundh, A., Lexchin, J., Mintzes, B., Schroll, J. B., & Bero, L. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000033.pub3 · Cited by 1,697
Savović, J., Jones, H. E., Altman, D. G., Harris, R. J., Jüni, P., Pildal, J., Als-Nielsen, B., Balk, E. M., Gluud, C., Gluud, L. L., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-6-201209180-00537 · Cited by 1,049
Industry sponsorship and research outcome
Lundh, A., Sismondo, S., Lexchin, J., Busuioc, O. A., & Bero, L. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000033.pub2 · Cited by 605
Influence of trial sample size on treatment effect estimates: meta-epidemiological study
Dechartres, A., Trinquart, L., Boutron, I., & Ravaud, P. (2013) · BMJ · doi:10.1136/bmj.f2304 · Cited by 544
Metadata
- Authors
- Goodman, S., & Dickersin, K.
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2011-07-05
- Volume
- 155
- Issue
- 1
- Pages
- 61-62
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2067197221
- Cited by
- 67
- References
- 13
- FWCI
- 5.07
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Metabias: A Challenge for Comparative Effectiveness Research. https://app.cochat.ai/papers/10.7326/0003-4819-155-1-201107050-00010
- The paper
- Goodman, S., & Dickersin, K. (2011). Metabias: A Challenge for Comparative Effectiveness Research. Annals of Internal Medicine. https://doi.org/10.7326/0003-4819-155-1-201107050-00010
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.