Retracted publications in pharmacy systematic reviews
Open accessThe DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
Objective: Systematic reviews and other evidence syntheses, the pinnacle of the evidence pyramid, embody comprehensiveness and rigor; however, retracted data are being incorporated into these publications. This study examines the use of retracted publications in the field of pharmacy, describes characteristics of retracted publications cited by systematic reviews, and discusses factors associated with citation likelihood. Methods: Using data from Retraction Watch, we identified retracted publications in the pharmacy field. We identified all articles citing these retracted publications in Web of Science and Scopus and limited results to systematic reviews. We classified the retraction reason, determined whether the citation occurred before or after retraction, and analyzed factors associated with the likelihood of systematic reviews citing a retracted publication. Results: Of 1,396 retracted publications, 283 were cited 1,096 times in systematic reviews. Most (65.0%) (712/1096) citations occurred before retraction. Citations were most often to items retracted due to data falsification or manipulation (39.2%), followed by items retracted due to ethical misconduct including plagiarism (30.4%), or concerns about or errors in data or methods (26.2%). Compared to those not cited in systematic reviews, cited items were significantly more likely to be retracted due to data falsification and manipulation, were published in high impact factor journals, and had longer delays between publication and retraction. Conclusions: Further analysis of systematic reviews citing retracted publications is needed to determine the impact of flawed data. Librarians understand the nuances involved and can advocate for greater transparency around the retraction process and increase awareness of challenges posed by retractions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rayyan-a web and mobile app for systematic reviews.
Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0384-4 · Cited by 26,092
JBI Manual for Evidence Synthesis
(2020) · JBI eBooks · doi:10.46658/jbimes-20-01 · Cited by 4,165
Biomedical research: increasing value, reducing waste.
Macleod, M. R., Michie, S., Roberts, I., Dirnagl, U., Chalmers, I., Ioannidis, J. P. A., Salman, R. A.-S., Chan, A.-W., & Glasziou, P. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62329-6 · Cited by 1,003
How Quickly Do Systematic Reviews Go Out of Date? A Survival Analysis
Shojania, K. G., Sampson, M., Ansari, M. T., Ji, J., Doucette, S., & Moher, D. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-4-200708210-00179 · Cited by 838 · cited 1×
“In one study of 100 systematic reviews published between 1995 and 2005, it was found that 23% of reviews were out of date within two years of publication, 15% within one year, and 7% were outdated by the time they were published [44].”
BackgroundAssessing scientists for hiring, promotion, and tenure
Moher, D., Naudet, F., Cristea, I. A., Miedema, F., Ioannidis, J. P. A., & Goodman, S. N. (2018) · PLOS Biology · doi:10.1371/journal.pbio.2004089 · Cited by 462 · cited 1×
“The use of the JIF as a proxy for scientific quality and rigor has previously been questioned [42, 43].”
BackgroundWhy has the number of scientific retractions increased?
Steen, R. G., Casadevall, A., & Fang, F. C. (2013) · PLoS ONE · doi:10.1371/journal.pone.0068397 · Cited by 415 · cited 2×
“Publications may be retracted for a variety of reasons that range from administrative publication errors and mistakes to scientific fraud and misconduct; a single paper may also be retracted for more than one reason [6].”
BackgroundCompliance with Results Reporting at ClinicalTrials.gov
Anderson, M. L., Chiswell, K., Peterson, E. D., Tasneem, A., Topping, J., & Califf, R. M. (2015) · New England Journal of Medicine · doi:10.1056/nejmsa1409364 · Cited by 380
Why and how do journals retract articles? An analysis of Medline retractions 1988–2008
Wager, E., & Williams, P. (2011) · Journal of Medical Ethics · doi:10.1136/jme.2010.040964 · Cited by 263 · cited 1×
“Ambiguity and a lack of context or detail may obscure the severity of the problem [10, 49, 50].”
BackgroundMoylan, E. C., & Kowalczuk, M. K. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-012047 · Cited by 212 · cited 2×
“Ambiguity and a lack of context or detail may obscure the severity of the problem [10, 49, 50].”
BackgroundGuidelines for retracting articles
Barbour, V., Kleinert, S., Wager, E., & Yentis, S. (2009) · doi:10.24318/cope.2019.1.4 · Cited by 185
Retractions in the medical literature: how many patients are put at risk by flawed research?
Steen, R. G. (2011) · Journal of Medical Ethics · doi:10.1136/jme.2011.043133 · Cited by 156
The visibility of scientific misconduct: A review of the literature on retracted journal articles
Hesselmann, F., Graf, V., Schmidt, M., & Reinhart, M. (2017) · Current Sociology · doi:10.1177/0011392116663807 · Cited by 153
The career effects of scandal: Evidence from scientific retractions
Azoulay, P., Bonatti, A., & Krieger, J. L. (2017) · Research Policy · doi:10.1016/j.respol.2017.07.003 · Cited by 118 · cited 1×
“Although the reputational and career implications of retraction are recognized by faculty [39], as are the ethical issues of student plagiarism and research misconduct and their implication for the student’s academic career [40], the clinical and educational implications of retractions as potential evidence have not b…”
BackgroundTemporal characteristics of retracted articles
Bar-Ilan, J., & Halevi, G. (2018) · Scientometrics · doi:10.1007/s11192-018-2802-y · Cited by 107
Trikalinos, N. A., Evangelou, E., & Ioannidis, J. P. A. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.11.019 · Cited by 100
Chambers, L., Michener, C., & Falcone, T. (2019) · BJOG: An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.15689 · Cited by 99 · cited 1×
“Retracted publications have been documented and analyzed in a variety of biomedical disciplines, including dentistry [10–12], radiation and oncology [13–15], mental health [16], emergency medicine [17], obstetrics and gynecology [18, 19], and surgery [20, 21].”
BackgroundReasons for article retraction in anesthesiology: a comprehensive analysis
Nair, S., Yean, C., Yoo, J., Leff, J., Delphin, E., & Adams, D. C. (2020) · Canadian Journal of Anesthesia/Journal canadien d'anesthésie · doi:10.1007/s12630-019-01508-3 · Cited by 90 · cited 1×
“Nearly half of all retracted publications in the field of anesthesia and analgesia are due to fraud or misconduct, although just four authors are responsible for nearly 60% of the retracted anesthesia literature [51].”
BackgroundRetractions in cancer research: a systematic survey
Bozzo, A., Bali, K., Evaniew, N., & Ghert, M. (2017) · Research Integrity and Peer Review · doi:10.1186/s41073-017-0031-1 · Cited by 83
Foo, J. Y. A. (2011) · Science and Engineering Ethics · doi:10.1007/s11948-010-9212-8 · Cited by 51
Evaluating Research and Researchers by the Journal Impact Factor: Is It Better than Coin Flipping?
Brito, R., & Rodríguez-Navarro, A. (2019) · Journal of Informetrics · doi:10.1016/j.joi.2019.01.009 · Cited by 47
Siebert, M., Gaba, J. F., Caquelin, L., Gouraud, H., Dupuy, A., Moher, D., & Naudet, F. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-038887 · Cited by 45
Timeliness and content of retraction notices for publications by a single research group
Grey, A., Avenell, A., & Bolland, M. (2022) · Accountability in Research · doi:10.1080/08989621.2021.1920409 · Cited by 41 · cited 1×
“However, retraction notices are often vague, lacking in sufficient detail to determine exactly which parts of the paper may be problematic and the nature of the concerns [48].”
Background
Cited by
Fake Publications in Biomedical Science: Red-flagging Method Indicates Mass Production
Sabel, B. A., Knaack, E., Gigerenzer, G., & Bilc, M. (2023) · doi:10.1101/2023.05.06.23289563 · Cited by 51
Bakker, C., Boughton, S., Faggion, C. M., Fanelli, D., Kaiser, K., & Schneider, J. (2024) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2022-111921 · Cited by 28
“27 Retractions may sometimes be preceded by an expression of concern, which aims to alert readers to a potential problem without definitively stating that the publication is flawed.”
BackgroundSelf-citationCitation Contamination by Paper Mill Articles in Systematic Reviews of the Life Sciences
Tang, G., & Cai, H. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.15160 · Cited by 25
An analysis of retracted papers in Computer Science
Shepperd, M., & Yousefi, L. (2023) · PLOS ONE · doi:10.1371/journal.pone.0285383 · Cited by 16
“Moreover, Brown et al. [17] explored these phenomena in pharmacology and found that of 1,396 retracted publications in the field approximately 20% (283) were cited by systematic reviews.”
Fake publications in biomedical science: red-flagging method indicates mass production
Sabel, B. A., Knaack, E., Gigerenzer, G., & Bilc, M.-I. (2026) · Naunyn-Schmiedeberg's Archives of Pharmacology · doi:10.1007/s00210-025-04275-9 · Cited by 51
Metadata
- Authors
- Brown, S. J., Bakker, C. J., & Theis-Mahon, N. R.
- Venue
- Journal of the Medical Library Association
- Published
- 2022-02-11
- Volume
- 110
- Issue
- 1
- Pages
- 47-55
- ISSN
- 1536-5050 (ISSN-L)
- 1558-9439
- OpenAlex
- W4210960884
- Cited by
- 34
- References
- 22
- FWCI
- 1.85
- Type
- journal article
- Language
- en
- Status
- Retracted
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.