CoChat

Detecting anomalous citation groups in journal networks

Open access
Kojaku, Sadamori, Livan, Giacomo, Masuda, Naoki · 2021 · Scientific Reports

The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.

Abstract

The ever-increasing competitiveness in the academic publishing market incentivizes journal editors to pursue higher impact factors. This translates into journals becoming more selective, and, ultimately, into higher publication standards. However, the fixation on higher impact factors leads some journals to artificially boost impact factors through the coordinated effort of a “citation cartel” of journals. “Citation cartel” behavior has become increasingly common in recent years, with several instances being reported. Here, we propose an algorithm—named CIDRE—to detect anomalous groups of journals that exchange citations at excessively high rates when compared against a null model that accounts for scientific communities and journal size. CIDRE detects more than half of the journals suspended from Journal Citation Reports due to anomalous citation behavior in the year of suspension or in advance. Furthermore, CIDRE detects many new anomalous groups, where the impact factors of the member journals are lifted substantially higher by the citations from other member journals. We describe a number of such examples in detail and discuss the implications of our findings with regard to the current academic climate.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41598-021-93572-3

Cites

36 of 36 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Statistics of anomalous citation groups detected by CIDRE. (a) Number of anomalous citation groups. (b) Number of journals in a group. A diamond indicates an outlier that does not fall in the range in [q0.25-1.5ΔIQR,q0.75+1.5ΔIQR], where q0.25 and q0.75 are the first and third quartiles, respectively, and ΔIQR=q0.75-q0.25.
Figure 1 Statistics of anomalous citation groups detected by CIDRE. (a) Number of anomalous citation groups. (b) Number of journals in a group. A diamond indicates an outlier that does not fall in the range in [q0.25-1.5ΔIQR,q0.75+1.5ΔIQR], where q0.25 and q0.75 are the first and third quartiles, respectively, and ΔIQR=q0.75-q0.25.CC BY · PubMed Central
Figure 2 Overlap between journal groups identified by JCR and CIDRE. (a) Years when the journal groups are identified. The circles with thick borders represent the groups suspended from JCR. The filled circles represent those detected by CIDRE. The hue of the circles indicates the value of the overlap, Oℓ,ℓ^(′), i.e., the fraction of journals in group ℓ^(′) detected by CIDRE that belong to group ℓ suspended from JCR. CIDRE detected 11 out of the 22 groups suspended from JCR in the year of the suspension or before. (b) Number of within-group citations per year from a donor journal excluding self-citations. If the group has multiple donor journals, we show the count for the donor journal that provides the largest number of within-group citations. The horizontal axis indicates the year relative to that in which the journal group is suspended from JCR. Therefore, a negative value indicates a year before JCR suspended the journal group. For groups J17, J19, and J22, the donor journals do not provide any within-group citations for the 3 years before the suspension. (c) Classification of journal groups identified by CIDRE and JCR.
Figure 2 Overlap between journal groups identified by JCR and CIDRE. (a) Years when the journal groups are identified. The circles with thick borders represent the groups suspended from JCR. The filled circles represent those detected by CIDRE. The hue of the circles indicates the value of the overlap, Oℓ,ℓ^(′), i.e., the fraction of journals in group ℓ^(′) detected by CIDRE that belong to group ℓ suspended from JCR. CIDRE detected 11 out of the 22 groups suspended from JCR in the year of the suspension or before. (b) Number of within-group citations per year from a donor journal excluding self-citations. If the group has multiple donor journals, we show the count for the donor journal that provides the largest number of within-group citations. The horizontal axis indicates the year relative to that in which the journal group is suspended from JCR. Therefore, a negative value indicates a year before JCR suspended the journal group. For groups J17, J19, and J22, the donor journals do not provide any within-group citations for the 3 years before the suspension. (c) Classification of journal groups identified by CIDRE and JCR.CC BY · PubMed Central
Figure 3 Sankey diagram for the classification of groups. We sequentially apply five classification criteria based on citation patterns (A–D) or the overlap of editorial board members (E). Groups are classified into categories A–D if more than 20% of within-group citations (A) come from a single paper, (B) go to a single paper, (C) come from a single author, or (D) go to a single author. For the groups that do not meet any of the criteria A–D, we classify a group into category E if its member journals have at least one overlapping editor. Otherwise, we classify it into category F.
Figure 3 Sankey diagram for the classification of groups. We sequentially apply five classification criteria based on citation patterns (A–D) or the overlap of editorial board members (E). Groups are classified into categories A–D if more than 20% of within-group citations (A) come from a single paper, (B) go to a single paper, (C) come from a single author, or (D) go to a single author. For the groups that do not meet any of the criteria A–D, we classify a group into category E if its member journals have at least one overlapping editor. Otherwise, we classify it into category F.CC BY · PubMed Central
Figure 4 Citation groups detected by CIDRE. The circles indicate journals. The color of edges indicates the source of citations. The width of edges is proportional to the number of citations made between two journals. Self-citation edges are omitted. The pie in color within each circle indicates the share of citations from the citing journal from the viewpoint of the cited journal. The white pie within each circle indicates the share of citations from outside the group. The journals on the left and right arcs indicate the donor and recipient, respectively. The journals that are simultaneously donor and recipient are located where the arcs overlap. The journals outside the group are agglomerated into a gray-colored circle at the bottom of each panel.
Figure 4 Citation groups detected by CIDRE. The circles indicate journals. The color of edges indicates the source of citations. The width of edges is proportional to the number of citations made between two journals. Self-citation edges are omitted. The pie in color within each circle indicates the share of citations from the citing journal from the viewpoint of the cited journal. The white pie within each circle indicates the share of citations from outside the group. The journals on the left and right arcs indicate the donor and recipient, respectively. The journals that are simultaneously donor and recipient are located where the arcs overlap. The journals outside the group are agglomerated into a gray-colored circle at the bottom of each panel.CC BY · PubMed Central
Figure 5 Citation groups that CIDRE detected for the network in 2019.
Figure 5 Citation groups that CIDRE detected for the network in 2019.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 36 references

Research works printed but not linked: 3

  1. Davis, P. When a Journal Sinks, Should the Editors Go Down with the Ship? 2014. https://scholarlykitchen.sspnet.org/2014/10/06/when-a-journal-sinks-should-the-editors-go-down-with-the-ship. Accessed 26 June 2020.
  2. Watch, R. Another Editor Resigns from Journal Hit by Citation Scandal, 2017. http://retractionwatch.com/2017/04/07/another-editor-resigns-journal-hit-citation-scandal/. Accessed 26 June 2020.
  3. Thomson Reuters. Web of Science Master Journal List, 2019. https://mjl.clarivate.com/home/. Accessed 26 June 2020.

Cited by

See all 60

Metadata

Authors
Kojaku, S., Livan, G., & Masuda, N.
Affiliations
  • Indiana University Bloomington (US)
  • Systemic Risk Centre (GB)
  • Waseda University (JP)
Corresponding author
Masuda, Naoki
Venue
Scientific Reports
Published
2021-07-15
Volume
11
Issue
1
Pages
14524-14524
ISSN
2045-2322 (ISSN-L)
OpenAlex
W3178476290
Cited by
53
References
35
Citations by year
2026: 11 · 2025: 17 · 2024: 10 · 2023: 8 · 2022: 6 · 2021: 1
FWCI
3.60
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • London School of Economics and Political Science Research Online (London School of Economics and Political Science) (acceptedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • UCL Discovery (University College London) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Engineering and Physical Sciences Research Council (EP/N006062/1)
  • Air Force Office of Scientific Research (FA9550-, FA9550-19-1-7024)
Topics
  • Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
  • Social Sciences › Education › Academic integrity and plagiarism
  • Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
Keywords
citation cartels, citation manipulation, journal networks, Journal Citation Reports, scientific communities, scholarly publishing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Detecting anomalous citation groups in journal networks. https://app.cochat.ai/papers/10.1038/s41598-021-93572-3
The paper
Kojaku, S., Livan, G., & Masuda, N. (2021). Detecting anomalous citation groups in journal networks. Scientific Reports. https://doi.org/10.1038/s41598-021-93572-3

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.