Citations and metrics of journals discontinued from Scopus for publication concerns: the GhoS(t)copus Project
Open accessThe DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
Background: Scopus is a leading bibliometric database. It contains a large part of the articles cited in peer-reviewed publications . The journals included in Scopus are periodically re-evaluated to ensure they meet indexing criteria and some journals might be discontinued for 'publication concerns'. Previously published articles may remain indexed and can be cited. Their metrics have yet to be studied. This study aimed to evaluate the main features and metrics of journals discontinued from Scopus for publication concerns, before and after their discontinuation, and to determine the extent of predatory journals among the discontinued journals. Methods: We surveyed the list of discontinued journals from Scopus (July 2019). Data regarding metrics, citations and indexing were extracted from Scopus or other scientific databases, for the journals discontinued for publication concerns. Results: A total of 317 journals were evaluated. Ninety-three percent of the journals (294/317) declared they published using an Open Access model. The subject areas with the greatest number of discontinued journals were Medicine (52/317; 16%), Agriculture and Biological Science (34/317; 11%), and Pharmacology, Toxicology and Pharmaceutics (31/317; 10%). The mean number of citations per year after discontinuation was significantly higher than before (median of difference 16.89 citations, p<0.0001), and so was the number of citations per document (median of difference 0.42 citations, p<0.0001). Twenty-two percent (72/317) were included in the Cabell’s blacklist. The DOAJ currently included only 9 journals while 61 were previously included and discontinued, most for 'suspected editorial misconduct by the publisher'. Conclusions: Journals discontinued for 'publication concerns' continue to be cited despite discontinuation and predatory behaviour seemed common. These citations may influence scholars’ metrics prompting artificial career advancements, bonus systems and promotion. Countermeasures should be taken urgently to ensure the reliability of Scopus metrics for the purpose of scientific assessment of scholarly publishing at both journal- and author-level.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Predatory journals: no definition, no defence
Grudniewicz, A., Moher, D., Cobey, K. D., Bryson, G. L., Cukier, S., Allen, K., Ardern, C., Balcom, L., Barros, T., Berger, M., et al. (2019) · Nature · doi:10.1038/d41586-019-03759-y · Cited by 726
Self-citationHow Frequently Are Articles in Predatory Open Access Journals Cited
Björk, B.-C., Kanto-Karvonen, S., & Harviainen, J. T. (2020) · Publications · doi:10.3390/publications8020017 · Cited by 77
Research works printed but not linked: 14
- A Cortegiani, 2020, Inclusion of predatory journals in Scopus is inflating scholars’ metrics and advancing careers., Int J Public Health., 65, 3-4, 10.1007/s00038-019-01318-w · DOI
- M Bagues, 2019, A walk on the wild side: “Predatory” journals and information asymmetries in scientific evaluations., Research Policy., 48, 462-477, 10.1016/j.respol.2018.04.013 · DOI · Paper page
- D Hedding, 2019, Payouts push professors towards predatory journals., Nature., 565, 267, 10.1038/d41586-019-00120-1 · doi:10.1038/d41586-019-00120-1
- A Cortegiani, 2019, Predatory Open-Access Publishing in Anesthesiology., Anesth Analg., 128, 182-187, 10.1213/ANE.0000000000003803 · DOI
- A Severin, 2019, Readers beware! Predatory journals are infiltrating citation databases., Int J Public Health., 64, 1123-1124, 10.1007/s00038-019-01284-3 · DOI
- Scopus®, registered trademark of Elsevier B.V
- Journal Citation Reports
- M Strinzel, 2019, Blacklists and Whitelists To Tackle Predatory Publishing: a Cross-Sectional Comparison and Thematic Analysis., mBio., 10, 10.1128/mBio.00411-19 · DOI · Paper page
- A Cortegiani, 2020, Extended data Appendix 1., figshare.
- A Cortegiani, 2020, Extended data Table 1., figshare.
- J Hirsch, 2015, An index to quantify an individual’s scientific research output., Proc Natl Acad Sci U S A., 102, 16569-16572, 10.1073/pnas.0507655102 · DOI · Paper page
- J Butterworth, 2019, Predatory Journals Undermine Peer Review and Cheapen Scholarship., Anesth Analg., 128, 11-12, 10.1213/ANE.0000000000003862 · doi:10.1213/ane.0000000000003862
- K Cobey, 2019, Knowledge and motivations of researchers publishing in presumed predatory journals: a survey., BMJ Open., 9, e026516, 10.1136/bmjopen-2018-026516 · DOI
- B Pond, 2019, Faculty Applicants' Attempt to Inflate CVs Using Predatory Journals., Am J Pharm Educ., 83, 7210, 10.5688/ajpe7210 · DOI
Cited by
Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World
Pranckutė, R. (2021) · Publications · doi:10.3390/publications9010012 · Cited by 2,696
Factors affecting FinTech adoption: A systematic literature review
Firmansyah, E. A., Masri, M., Anshari, M., & Besar, M. H. A. (2022) · FinTech · doi:10.3390/fintech2010002 · Cited by 141
The entities enabling scientific fraud at scale are large, resilient, and growing rapidly
Richardson, R. A. K., Hong, S. S., Byrne, J. A., Stoeger, T., & Amaral, L. A. N. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2420092122 · Cited by 129
Who are the 100 largest scientific publishers by journal count? A webscraping approach
Nishikawa-Pacher, A. (2022) · Journal of Documentation · doi:10.1108/jd-04-2022-0083 · Cited by 65
Meho, L. I. (2025) · Scientometrics · doi:10.1007/s11192-025-05480-2 · Cited by 27
Metadata
- Authors
- Cortegiani, A., Ippolito, M., Ingoglia, G., Manca, A., Cugusi, L., Severin, A., Strinzel, M., Panzarella, V., Campisi, G., Manoj, L., Gregoretti, C., Einav, S., Moher, D., & Giarratano, A.
- Venue
- F1000Research
- Publisher
- F1000 Research Ltd
- Published
- 2020-08-26
- Volume
- 9
- Pages
- 415
- ISSN
- 2046-1402 (ISSN-L)
- OpenAlex
- W3026288581
- Cited by
- 40
- References
- 2
- FWCI
- 3.09
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- creativecommons.org/licenses/by/4.0/ (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Citations and metrics of journals discontinued from Scopus for publication concerns: the GhoS(t)copus Project. https://app.cochat.ai/papers/10.12688/f1000research.23847.2
- The paper
- Cortegiani, A., Ippolito, M., Ingoglia, G., Manca, A., Cugusi, L., Severin, A., Strinzel, M., Panzarella, V., Campisi, G., Manoj, L., Gregoretti, C., Einav, S., Moher, D., & Giarratano, A. (2020). Citations and metrics of journals discontinued from Scopus for publication concerns: the GhoS(t)copus Project. F1000Research. https://doi.org/10.12688/f1000research.23847.2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.