The DOI matches its OpenAlex record; 35 of the 35 references it cites were found in a registry or an index.
The Importance of Quality of Primary Studies in Producing Unbiased Systematic Reviews
Abstract
BACKGROUND: Traditional and largely qualitative reviews of evidence are now giving way to much more structured systematic overviews that use a quantitative method to calculate the overall effect of treatment. The latter approach is dependent on the quality of primary studies, which may introduce bias if they are of poor methodologic quality. OBJECTIVE: To test the hypothesis that the inclusion of poor-quality trials in meta-analyses would bias the conclusions and produce incorrect estimates of treatment effect. METHODS: An overview of randomized trials of antiestrogen therapy in subfertile men with oligospermia was performed to test the hypothesis. Data sources included online searching of MEDLINE and Science Citation Index databases between 1966 and 1994, scanning the bibliography of known primary studies and review articles, and contacting experts in the field. After independent, blind assessment, nine of 149 originally identified studies met the inclusion criteria and were selected. We assessed study quality independently. Outcome data from each study were pooled and statistically summarized. RESULTS: There was a marginal improvement in pregnancy rate with antiestrogen treatment (odds ratio, 1.6; 95% confidence interval, 0.9 to 2.6). Sensitivity analyses on the basis of methodologic quality demonstrated that poor-quality studies produced a positive effect with treatment, whereas no benefit was observed with high-quality studies. CONCLUSION: The results of a meta-analysis are influenced by the quality of the primary studies included. Methodologically, poor studies tend to exaggerate the overall estimate of treatment effect and may lead to incorrect inferences.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical Aspects of the Analysis of Data From Retrospective Studies of Disease
(1959) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/22.4.719 · Cited by 14,689
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736
Statistical Methods in Cancer Research
Breslow, N. E. (1980) · Cited by 7,332
Schulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634
Statistical methods in cancer research. Volume I - The analysis of case-control studies.
Breslow, N. E., & Day, N. E. (1980) · PubMed · Cited by 4,988
Evidence-Based Medicine: A New Approach to Teaching the Practice of Medicine
Guyatt, G. (1992) · JAMA · doi:10.1001/jama.1992.03490170092032 · Cited by 4,202
A Fail-Safe N for Effect Size in Meta-Analysis
Orwin, R. G. (1983) · Journal of Educational Statistics · doi:10.2307/1164923 · Cited by 2,064
Bias in treatment assignment in controlled clinical trials
Chalmers, T. C., Celano, P., Sacks, H. S., & Smith, H. (1983) · New England Journal of Medicine · doi:10.1056/nejm198312013092204 · Cited by 864
Nurmohamed, M. T., Buller, H. R., Dekker, E., Hommes, D. W., Rosendaal, F. R., Briet, E., Rosendaal, F. R., & Vandenbroucke, J. P. (1992) · The Lancet · doi:10.1016/0140-6736(92)93223-a · Cited by 668
Cited by
Evaluating non-randomised intervention studies
Deeks, J., Dinnes, J., D'Amico, R., Sowden, A., Sakarovitch, C., Song, F., Petticrew, M., & Altman, D. (2003) · Health Technology Assessment · doi:10.3310/hta7270 · Cited by 2,907
The Hazards of Scoring the Quality of Clinical Trials for Meta-analysis
Jüni, P. (1999) · JAMA · doi:10.1001/jama.282.11.1054 · Cited by 1,981
“In 1 analysis evaluating the effect of antiestrogen treatment in male infertility, the results were reversed with adverse effects on pregnancy rates in studies of high quality.(56) Different scales had been used for assessing quality and, in light of our study, it is possible that the choice of the scale contributed t…”
ResultSex Differences in Insomnia: A Meta-Analysis
Zhang, B., & Wing, Y.-K. (2006) · Sleep · doi:10.1093/sleep/29.1.85 · Cited by 1,286
Linde, K., Clausius, N., Ramirez, G., Melchart, D., Eitel, F., Hedges, L. V., & Jonas, W. B. (1997) · The Lancet · doi:10.1016/s0140-6736(97)02293-9 · Cited by 979
Palliative Radiotherapy Trials for Bone Metastases: A Systematic Review
Chow, E., Harris, K., Fan, G., Tsao, M., & Sze, W. M. (2007) · Journal of Clinical Oncology · doi:10.1200/jco.2006.09.5281 · Cited by 908
Kunz, R., & Oxman, A. D. (1998) · BMJ · doi:10.1136/bmj.317.7167.1185 · Cited by 770
Gabapentin for chronic neuropathic pain and fibromyalgia in adults
Moore, R. A., Wiffen, P. J., Derry, S., & McQuay, H. J. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007938.pub2 · Cited by 690
Assessing the Quality of Randomized Trials
Clark, H. D., Wells, G. A., Huët, C., McAlister, F. A., Salmi, L. R., Fergusson, D., & Laupacis, A. (1999) · Controlled Clinical Trials · doi:10.1016/s0197-2456(99)00026-4 · Cited by 648
Veenstra, D. L., Saint, S., Saha, S., Lumley, T., & Sullivan, S. D. (1999) · JAMA · doi:10.1001/jama.281.3.261 · Cited by 571
Gabapentin for chronic neuropathic pain in adults
Wiffen, P. J., Derry, S., Bell, R. F., Rice, A. S., Tölle, T. R., Phillips, T., & Moore, R. A. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007938.pub4 · Cited by 561
A Systematic Review of the Frequency of Neurocyticercosis with a Focus on People with Epilepsy
Ndimubanzi, P. C., Carabin, H., Budke, C. M., Nguyen, H., Qian, Y.-J., Rainwater, E., Dickey, M., Reynolds, S., & Stoner, J. A. (2010) · PLoS Neglected Tropical Diseases · doi:10.1371/journal.pntd.0000870 · Cited by 478
Dolovich, L. R., Ginsberg, J. S., Douketis, J. D., Holbrook, A. M., & Cheah, G. (2000) · Archives of Internal Medicine · doi:10.1001/archinte.160.2.181 · Cited by 468
Quantitive systematic review of topically applied non-steroidal anti-inflammatory drugs
Moore, R. A., Tramer, M. R., Carroll, D., Wiffen, P. J., & McQuay, H. J. (1998) · BMJ · doi:10.1136/bmj.316.7128.333 · Cited by 446
Randomisation to protect against selection bias in healthcare trials
Odgaard-Jensen, J., Vist, G. E., Timmer, A., Kunz, R., Akl, E. A., Schünemann, H., Briel, M., Nordmann, A. J., Pregno, S., & Oxman, A. D. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000012.pub3 · Cited by 439
Oremus, M., Wolfson, C., Perrault, A., Demers, L., Momoli, F., & Moride, Y. (2001) · Dementia and Geriatric Cognitive Disorders · doi:10.1159/000051263 · Cited by 422
Bipat, S., Glas, A. S., Velden, J. V. D., Zwinderman, A. H., Bossuyt, P. M. M., & Stoker, J. (2003) · Gynecologic Oncology · doi:10.1016/s0090-8258(03)00409-8 · Cited by 393
Dahl, J. B., Jeppesen, I. S., Jørgensen, H., Wetterslev, J., & Møiniche, S. (1999) · Anesthesiology · doi:10.1097/00000542-199912000-00045 · Cited by 382
“Evidence” in chronic pain–establishing best practice in the reporting of systematic reviews
Moore, A. R., Eccleston, C., Derry, S., Wiffen, P., Bell, R. F., Straube, S., & McQuay, H. (2010) · Pain · doi:10.1016/j.pain.2010.05.011 · Cited by 370
Moher, Cook, Jadad, Tugwell, Moher, Jones, Pham, & Klassen (1999) · Health Technology Assessment · doi:10.3310/hta3120 · Cited by 368
Cigarette smoking during pregnancy and risk of preeclampsia: A systematic review
Conde-Agudelo, A., Althabe, F., Belizán, J. M., & Kafury-Goeta, A. C. (1999) · American Journal of Obstetrics and Gynecology · doi:10.1016/s0002-9378(99)70341-8 · Cited by 337
Large Trials vs Meta-analysis of Smaller Trials: How Do Their Results Compare?
Cappelleri, J. C. (1996) · JAMA · doi:10.1001/jama.1996.03540160054033 · Cited by 336
Saint, S., Elmore, J. G., Sullivan, S. D., Emerson, S. S., & Koepsell, T. D. (1998) · The American Journal of Medicine · doi:10.1016/s0002-9343(98)00240-x · Cited by 330
Mishara, A. L., & Goldberg, T. E. (2004) · Biological Psychiatry · doi:10.1016/j.biopsych.2004.01.027 · Cited by 310
Treatment of Acute Achilles Tendon Ruptures A Systematic Overview and Metaanalysis
Bhandari, M., Guyatt, G. H., Siddiqui, F., Morrow, F., Busse, J., Leighton, R. K., Sprague, S., & Schemitsch, E. H. (2002) · Clinical Orthopaedics and Related Research · doi:10.1097/00003086-200207000-00024 · Cited by 306
Gabapentin for chronic neuropathic pain and fibromyalgia in adults
Moore, R. A., Wiffen, P. J., Derry, S., & Rice, A. S. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007938.pub3 · Cited by 292
The effect of oral N-acetylcysteine in chronic bronchitis: a quantitative systematic review
Stey, C., Steurer, J., Bachmann, S., Medici, T. C., & Tramèr, M. R. (2000) · European Respiratory Journal · doi:10.1034/j.1399-3003.2000.16b12.x · Cited by 284
Bhandari, M., Guyatt, G. H., Tong, D., Adili, A., & Shaughnessy, S. G. (2000) · Journal of Orthopaedic Trauma · doi:10.1097/00005131-200001000-00002 · Cited by 281
Testosterone supplementation therapy for older men: potential benefits and risks
Gruenewald, D. A., & Matsumoto, A. M. (2003) · Journal of the American Geriatrics Society · doi:10.1034/j.1601-5215.2002.51018.x · Cited by 279
Pharmacologic interventions for treating phantom limb pain
Alviar, M. J. M., Hale, T., & Lim-Dungca, M. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006380.pub3 · Cited by 279
Knoll, G. A., & Bell, R. C. (1999) · BMJ · doi:10.1136/bmj.318.7191.1104 · Cited by 263
Hull, R. D. (1997) · Archives of Internal Medicine · doi:10.1001/archinte.157.22.2562 · Cited by 261
Systematic review of the use of honey as a wound dressing
Moore, O. A., Smith, L. A., Campbell, F., Seers, K., McQuay, H. J., & Moore, R. A. (2001) · BMC Complementary and Alternative Medicine · doi:10.1186/1472-6882-1-2 · Cited by 254
School programs targeting stress management in children and adolescents: A meta-analysis
Kraag, G., Zeegers, M. P., Kok, G., Hosman, C., & Abu-Saad, H. H. (2006) · Journal of School Psychology · doi:10.1016/j.jsp.2006.07.001 · Cited by 229
Fabrizi, F., Martin, P., Dixit, V., Bunnapradist, S., Kanwal, F., & Dulai, G. (2005) · American Journal of Transplantation · doi:10.1111/j.1600-6143.2005.01040.x · Cited by 208
Short- versus Long-Course Antibacterial Therapy for Community-Acquired Pneumonia: A Meta-Analysis
Dimopoulos, G., Matthaiou, D. K., Karageorgopoulos, D. E., Grammatikos, A. P., Athanassa, Z., & Falagas, M. E. (2008) · Drugs · doi:10.2165/00003495-200868130-00004 · Cited by 160
Single dose oral paracetamol (acetaminophen) for postoperative pain
Barden, J., Edwards, J., Moore, A., & McQuay, H. (2004) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004602 · Cited by 113
Rezende, M., Martins, A. C. R., da Silva, J. A., Reis, A., & de Geus, J. L. (2022) · Clinical Oral Investigations · doi:10.1007/s00784-021-04198-8 · Cited by 12
What works in mindfulness interventions for medically unexplained symptoms? A systematic review
Billones, R., Lukkahatai, N., & Saligan, L. (2020) · PubMed Central · doi:10.31372/20200501.1082 · Cited by 12
“The higher the score, the better the quality of the study (Khan et al., 1996).”
BackgroundCompliance of RCTs in endodontic instrumentation with the CONSORT Statement: A systematic review
Cabral, K. A., Martins, A. C. R., Wambier, L. M., Vivacqua-Gomes, N., Rezende, M., Reis, A., & Geus, J. L. D. (2021) · Research, Society and Development · doi:10.33448/rsd-v10i8.17350 · Cited by 2
“There is evidence that improper conduction of these steps in RCTs increases the likelihood of systematic errors (Pocock 1983); studies with poor methodological quality tend to overestimate the results, favoring the intervention in the test group (Schulz et al. 1995, Khan et al. 1996).”
Result
Metadata
- Authors
- Khan, K. S.
- Venue
- Archives of Internal Medicine
- Publisher
- American Medical Association
- Published
- 1996-03-25
- Volume
- 156
- Issue
- 6
- Pages
- 661-661
- ISSN
- 0003-9926 (ISSN-L)
- OpenAlex
- W2090642049
- Cited by
- 328
- References
- 35
- FWCI
- 28.28
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.