Detecting anomalous citation groups in journal networks
Open accessThe DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
The ever-increasing competitiveness in the academic publishing market incentivizes journal editors to pursue higher impact factors. This translates into journals becoming more selective, and, ultimately, into higher publication standards. However, the fixation on higher impact factors leads some journals to artificially boost impact factors through the coordinated effort of a “citation cartel” of journals. “Citation cartel” behavior has become increasingly common in recent years, with several instances being reported. Here, we propose an algorithm—named CIDRE—to detect anomalous groups of journals that exchange citations at excessively high rates when compared against a null model that accounts for scientific communities and journal size. CIDRE detects more than half of the journals suspended from Journal Citation Reports due to anomalous citation behavior in the year of suspension or in advance. Furthermore, CIDRE detects many new anomalous groups, where the impact factors of the member journals are lifted substantially higher by the citations from other member journals. We describe a number of such examples in detail and discuss the implications of our findings with regard to the current academic climate.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
![Figure 1 Statistics of anomalous citation groups detected by CIDRE. (a) Number of anomalous citation groups. (b) Number of journals in a group. A diamond indicates an outlier that does not fall in the range in [q0.25-1.5ΔIQR,q0.75+1.5ΔIQR], where q0.25 and q0.75 are the first and third quartiles, respectively, and ΔIQR=q0.75-q0.25.](https://figures.cochat.ai/paper-figures/22abf484f976458d9097218cf356b667/detecting-anomalous-citation-groups-in-journal-networks-figure-1.jpg)




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
From Louvain to Leiden: guaranteeing well-connected communities
Traag, V. A., Waltman, L., & van Eck, N. J. (2019) · Scientific Reports · doi:10.1038/s41598-019-41695-z · Cited by 5,635
Maps of random walks on complex networks reveal community structure
Rosvall, M., & Bergstrom, C. T. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0706851105 · Cited by 4,583
The History and Meaning of the Journal Impact Factor
Garfield, E. (2006) · JAMA · doi:10.1001/jama.295.1.90 · Cited by 2,770
Stochastic blockmodels and community structure in networks
Karrer, B., & Newman, M. E. J. (2011) · Physical Review E · doi:10.1103/physreve.83.016107 · Cited by 2,025
Bornmann, L., & Mutz, R. (2015) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23329 · Cited by 1,576 · cited 1×
“xd(i,U) := 1 sout i ∑ j∈U, j 6=i Wi jh(i, j), (1)”
The scientific impact of nations
King, D. A. (2004) · Nature · doi:10.1038/430311a · Cited by 1,245 · cited 1×
“λi j = sout i s in j Λgi,g j Sout gi S in g j , (5)”
BackgroundMartín-Martín, A., Thelwall, M., Orduna-Malea, E., & Delgado López-Cózar, E. (2021) · Scientometrics · doi:10.1007/s11192-020-03690-4 · Cited by 1,018
Martín‐Martín, A., Thelwall, M. A., Orduña‐Malea, E., & Delgado López-Cózar, E. (2020) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) · Cited by 916
Consensus clustering in complex networks
Lancichinetti, A., & Fortunato, S. (2012) · Scientific Reports · doi:10.1038/srep00336 · Cited by 795
Research works printed but not linked: 3
- Davis, P. When a Journal Sinks, Should the Editors Go Down with the Ship? 2014. https://scholarlykitchen.sspnet.org/2014/10/06/when-a-journal-sinks-should-the-editors-go-down-with-the-ship. Accessed 26 June 2020.
- Watch, R. Another Editor Resigns from Journal Hit by Citation Scandal, 2017. http://retractionwatch.com/2017/04/07/another-editor-resigns-journal-hit-citation-scandal/. Accessed 26 June 2020.
- Thomson Reuters. Web of Science Master Journal List, 2019. https://mjl.clarivate.com/home/. Accessed 26 June 2020.
Cited by
Deep Graph Learning for Anomalous Citation Detection
Liu, J., Xia, F., Xu, F., Ren, J. F., & Liu, H. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3145092 · Cited by 87
Citation manipulation through citation mills and pre-print servers
Ibrahim, H., Liu, F., Zaki, Y., & Rahwan, T. (2025) · Scientific Reports · doi:10.1038/s41598-025-88709-7 · Cited by 84
da Silva, J. A. T. (2022) · Journal for General Philosophy of Science · doi:10.1007/s10838-022-09607-4 · Cited by 60
“…have the responsibility of not manipulating peer review and editorial processing to favor themselves or their journal because this would introduce bias—and thus error—into the bibliometric profile of the editor, journal, and any papers that cite it (Kojaku et al. 2021; Lutmar and Reingewertz 2021).”
BackgroundChallenges posed by hijacked journals in Scopus
Abalkina, A. (2024) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24855 · Cited by 47
“Second, hijacked journals extensively cite each other, suggesting potential citation inflation and the formation of citation cartels (Fister Jr. et al., 2016; Kojaku et al., 2021).”
BackgroundSneaked references: Fabricated reference metadata distort citation counts
Besançon, L., Cabanac, G., Labbé, C., & Magazinov, A. (2024) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24896 · Cited by 31
“This kind of manipulation also arises at the journal level (Davis, 2016; Kojaku et al., 2021).”
Metadata
- Authors
- Kojaku, S., Livan, G., & Masuda, N.
- Affiliations
- Indiana University Bloomington (US)
- Systemic Risk Centre (GB)
- Waseda University (JP)
- Corresponding author
- Masuda, Naoki
- Venue
- Scientific Reports
- Published
- 2021-07-15
- Volume
- 11
- Issue
- 1
- Pages
- 14524-14524
- ISSN
- 2045-2322 (ISSN-L)
- PMC
- PMC8282695
- OpenAlex
- W3178476290
- Cited by
- 53
- References
- 35
- Citations by year
- 2026: 11 · 2025: 17 · 2024: 10 · 2023: 8 · 2022: 6 · 2021: 1
- FWCI
- 3.60
- Type
- journal article
- Language
- en
- Update notices
- correction (2022-08-29)
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Scientific Reports (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- London School of Economics and Political Science Research Online (London School of Economics and Political Science) (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UCL Discovery (University College London) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Engineering and Physical Sciences Research Council (EP/N006062/1)
- Air Force Office of Scientific Research (FA9550-, FA9550-19-1-7024)
- Topics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Education › Academic integrity and plagiarism
- Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
- Keywords
- citation cartels, citation manipulation, journal networks, Journal Citation Reports, scientific communities, scholarly publishing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Detecting anomalous citation groups in journal networks. https://app.cochat.ai/papers/10.1038/s41598-021-93572-3
- The paper
- Kojaku, S., Livan, G., & Masuda, N. (2021). Detecting anomalous citation groups in journal networks. Scientific Reports. https://doi.org/10.1038/s41598-021-93572-3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.