Detecting anomalous citation groups in journal networks
Open accessThe DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
The ever-increasing competitiveness in the academic publishing market incentivizes journal editors to pursue higher impact factors. This translates into journals becoming more selective, and, ultimately, into higher publication standards. However, the fixation on higher impact factors leads some journals to artificially boost impact factors through the coordinated effort of a “citation cartel” of journals. “Citation cartel” behavior has become increasingly common in recent years, with several instances being reported. Here, we propose an algorithm—named CIDRE—to detect anomalous groups of journals that exchange citations at excessively high rates when compared against a null model that accounts for scientific communities and journal size. CIDRE detects more than half of the journals suspended from Journal Citation Reports due to anomalous citation behavior in the year of suspension or in advance. Furthermore, CIDRE detects many new anomalous groups, where the impact factors of the member journals are lifted substantially higher by the citations from other member journals. We describe a number of such examples in detail and discuss the implications of our findings with regard to the current academic climate.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
![Figure 1 Statistics of anomalous citation groups detected by CIDRE. (a) Number of anomalous citation groups. (b) Number of journals in a group. A diamond indicates an outlier that does not fall in the range in [q0.25-1.5ΔIQR,q0.75+1.5ΔIQR], where q0.25 and q0.75 are the first and third quartiles, respectively, and ΔIQR=q0.75-q0.25.](https://figures.cochat.ai/paper-figures/22abf484f976458d9097218cf356b667/detecting-anomalous-citation-groups-in-journal-networks-figure-1.jpg)




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
From Louvain to Leiden: guaranteeing well-connected communities
Traag, V. A., Waltman, L., & van Eck, N. J. (2019) · Scientific Reports · doi:10.1038/s41598-019-41695-z · Cited by 5,635
Maps of random walks on complex networks reveal community structure
Rosvall, M., & Bergstrom, C. T. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0706851105 · Cited by 4,583
The History and Meaning of the Journal Impact Factor
Garfield, E. (2006) · JAMA · doi:10.1001/jama.295.1.90 · Cited by 2,770
Stochastic blockmodels and community structure in networks
Karrer, B., & Newman, M. E. J. (2011) · Physical Review E · doi:10.1103/physreve.83.016107 · Cited by 2,025
Bornmann, L., & Mutz, R. (2015) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23329 · Cited by 1,576 · cited 1×
“xd(i,U) := 1 sout i ∑ j∈U, j 6=i Wi jh(i, j), (1)”
The scientific impact of nations
King, D. A. (2004) · Nature · doi:10.1038/430311a · Cited by 1,245 · cited 1×
“λi j = sout i s in j Λgi,g j Sout gi S in g j , (5)”
BackgroundMartín-Martín, A., Thelwall, M., Orduna-Malea, E., & Delgado López-Cózar, E. (2021) · Scientometrics · doi:10.1007/s11192-020-03690-4 · Cited by 1,018
Martín‐Martín, A., Thelwall, M. A., Orduña‐Malea, E., & Delgado López-Cózar, E. (2020) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) · Cited by 916
Consensus clustering in complex networks
Lancichinetti, A., & Fortunato, S. (2012) · Scientific Reports · doi:10.1038/srep00336 · Cited by 795
An Overview of Microsoft Academic Service (MAS) and Applications
Sinha, A., Shen, Z., Song, Y., Ma, H., Eide, D., Hsu, B.-J. (., & Wang, K. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2740908.2742839 · Cited by 658
Impact factor: a valid measure of journal quality?
(2003) · Europe PMC (PubMed Central) · Cited by 584
Geographical proximity and scientific collaboration
Katz, J. S. (1994) · Scientometrics · doi:10.1007/bf02018100 · Cited by 476
Adam, D. (2002) · Nature · doi:10.1038/415726a · Cited by 447
Coercive Citation in Academic Publishing
Wilhite, A. W., & Fong, E. A. (2012) · Science · doi:10.1126/science.1212540 · Cited by 433
What I learned from predatory publishers
Beall, J. (2017) · Biochemia Medica · doi:10.11613/bm.2017.029 · Cited by 306
World citation and collaboration networks: uncovering the role of geography in science
Pan, R. K., Kaski, K., & Fortunato, S. (2012) · Scientific Reports · doi:10.1038/srep00902 · Cited by 304
Fast algorithms for determining (generalized) core groups in social networks
Batagelj, V., & Zaveršnik, M. (2011) · Advances in Data Analysis and Classification · doi:10.1007/s11634-010-0079-y · Cited by 288
Nonparametric Bayesian inference of the microcanonical stochastic block model
Peixoto, T. P. (2017) · Physical Review E · doi:10.1103/physreve.95.012317 · Cited by 196
(1992) · Science and Public Policy · doi:10.1093/spp/19.5.321 · Cited by 188 · cited 1×
“pi j = 1− Wi j−1 ∑ w=0 Pnull i j ( w; λ̂i j ) , (3)”
Scientific Communication--A Vanity Fair?
Franck, G. (1999) · Science · doi:10.1126/science.286.5437.53 · Cited by 174
Toward the discovery of citation cartels in citation networks
Fister, I., Fister, I., & Perc, M. (2016) · Frontiers in Physics · doi:10.3389/fphy.2016.00049 · Cited by 149
Brazilian citation scheme outed
Noorden, R. V. (2013) · Nature · doi:10.1038/500510a · Cited by 133
The h-index Debate: An Introduction for Librarians
Barnes, C. (2017) · The Journal of Academic Librarianship · doi:10.1016/j.acalib.2017.08.013 · Cited by 97 · cited 1×
“xr(i,U) := 1 sin i ∑ j∈U, j 6=i Wjih( j, i), (2)”
Categorizing Variants of Goodhart's Law
Manheim, D., & Garrabrant, S. (2018) · arXiv · doi:10.48550/arxiv.1803.04585 · Cited by 73
Recursive Self-Improvement Stability under Endogenous Yardstick DriftPreprint
Takahashi, K. (2026) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.19044633 · Cited by 63
Bornmann, L., & Bauer, J. (2015) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23396 · Cited by 46
The Network of Law Reviews: Citation Cartels, Scientific Communities, and Journal Rankings
Perez, O., Bar‐Ilan, J., Reuven Cohen, C., & Schreiber, N. (2019) · Modern Law Review · doi:10.1111/1468-2230.12405 · Cited by 34
Editorial Bias in Legal Academia
Yoon, A. H. (2013) · The Journal of Legal Analysis · doi:10.1093/jla/lat005 · Cited by 32
Reciprocity and impact in academic careers
Li, W., Aste, T., Caccioli, F., & Livan, G. (2019) · EPJ Data Science · doi:10.1140/epjds/s13688-019-0199-3 · Cited by 32
Self-citationStochastic block model reveals maps of citation patterns and their evolution in time
Hric, D., Kaski, K. K., & Kivelä, M. (2018) · Journal of Informetrics · doi:10.1016/j.joi.2018.05.004 · Cited by 31
Foo, J. Y. A. (2009) · Science and Engineering Ethics · doi:10.1007/s11948-009-9177-7 · Cited by 25
Excess reciprocity distorts reputation in online social networks
Livan, G., Caccioli, F., & Aste, T. (2017) · Scientific Reports · doi:10.1038/s41598-017-03481-7 · Cited by 20
Self-citationVisser, M. S., van Eck, N. J., & Waltman, L. (2019) · ISSI · Cited by 15
Self-renewal of neural stem cells: implications for future therapies
Raitano, S., Verfaillie, C. M., & Petryk, A. (2013) · Frontiers in Physiology · doi:10.3389/fphys.2013.00049 · Cited by 10
An Early Note on Flies as Transmitters of Disease
Gudger, E. W. (1910) · Science · doi:10.1126/science.31.784.31 · Cited by 9
Research works printed but not linked: 3
- Davis, P. When a Journal Sinks, Should the Editors Go Down with the Ship? 2014. https://scholarlykitchen.sspnet.org/2014/10/06/when-a-journal-sinks-should-the-editors-go-down-with-the-ship. Accessed 26 June 2020.
- Watch, R. Another Editor Resigns from Journal Hit by Citation Scandal, 2017. http://retractionwatch.com/2017/04/07/another-editor-resigns-journal-hit-citation-scandal/. Accessed 26 June 2020.
- Thomson Reuters. Web of Science Master Journal List, 2019. https://mjl.clarivate.com/home/. Accessed 26 June 2020.
Cited by
Deep Graph Learning for Anomalous Citation Detection
Liu, J., Xia, F., Xu, F., Ren, J. F., & Liu, H. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3145092 · Cited by 87
Citation manipulation through citation mills and pre-print servers
Ibrahim, H., Liu, F., Zaki, Y., & Rahwan, T. (2025) · Scientific Reports · doi:10.1038/s41598-025-88709-7 · Cited by 85
da Silva, J. A. T. (2022) · Journal for General Philosophy of Science · doi:10.1007/s10838-022-09607-4 · Cited by 62
“…have the responsibility of not manipulating peer review and editorial processing to favor themselves or their journal because this would introduce bias—and thus error—into the bibliometric profile of the editor, journal, and any papers that cite it (Kojaku et al. 2021; Lutmar and Reingewertz 2021).”
BackgroundChallenges posed by hijacked journals in Scopus
Abalkina, A. (2024) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24855 · Cited by 47
“Second, hijacked journals extensively cite each other, suggesting potential citation inflation and the formation of citation cartels (Fister Jr. et al., 2016; Kojaku et al., 2021).”
BackgroundSneaked references: Fabricated reference metadata distort citation counts
Besançon, L., Cabanac, G., Labbé, C., & Magazinov, A. (2024) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24896 · Cited by 31
“This kind of manipulation also arises at the journal level (Davis, 2016; Kojaku et al., 2021).”
Metadata
- Authors
- Kojaku, S., Livan, G., & Masuda, N.
- Affiliations
- Indiana University Bloomington (US)
- Systemic Risk Centre (GB)
- Waseda University (JP)
- Corresponding author
- Masuda, Naoki
- Venue
- Scientific Reports
- Published
- 2021-07-15
- Volume
- 11
- Issue
- 1
- Pages
- 14524-14524
- ISSN
- 2045-2322 (ISSN-L)
- PMC
- PMC8282695
- OpenAlex
- W3178476290
- Cited by
- 53
- References
- 35
- Citations by year
- 2026: 11 · 2025: 17 · 2024: 10 · 2023: 8 · 2022: 6 · 2021: 1
- FWCI
- 3.60
- Type
- journal article
- Language
- en
- Update notices
- correction (2022-08-29)
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Scientific Reports (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- London School of Economics and Political Science Research Online (London School of Economics and Political Science) (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UCL Discovery (University College London) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Engineering and Physical Sciences Research Council (EP/N006062/1)
- Air Force Office of Scientific Research (FA9550-, FA9550-19-1-7024)
- Topics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Education › Academic integrity and plagiarism
- Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
- Keywords
- citation cartels, citation manipulation, journal networks, Journal Citation Reports, scientific communities, scholarly publishing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.