CoChat
Sign Up
Cited by 16scientometrics and bibliometrics researchPsychology Research and BibliometricsWeb visibility and informetrics

An Analysis of Peer-Reviewed Scores and Impact Factors with Different Citation Time Windows: A Case Study of 28 Ophthalmologic Journals

Open access
Liu, Xue-Li, Gai, Shuang-Shuang, Zhang, Shi-Le, Wang, Pu·2015·PLOS ONE
DOI 10.1371/journal.pone.0135583PMC PMC4546661
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: An important attribute of the traditional impact factor was the controversial 2-year citation window. So far, several scholars have proposed using different citation time windows for evaluating journals. However, there is no confirmation whether a longer citation time window would be better. How did the journal evaluation effects of 3IF, 4IF, and 6IF comparing with 2IF and 5IF? In order to understand these questions, we made a comparative study of impact factors with different citation time windows with the peer-reviewed scores of ophthalmologic journals indexed by Science Citation Index Expanded (SCIE) database. METHODS: The peer-reviewed scores of 28 ophthalmologic journals were obtained through a self-designed survey questionnaire. Impact factors with different citation time windows (including 2IF, 3IF, 4IF, 5IF, and 6IF) of 28 ophthalmologic journals were computed and compared in accordance with each impact factor's definition and formula, using the citation analysis function of the Web of Science (WoS) database. An analysis of the correlation between impact factors with different citation time windows and peer-reviewed scores was carried out. RESULTS: Although impact factor values with different citation time windows were different, there was a high level of correlation between them when it came to evaluating journals. In the current study, for ophthalmologic journals' impact factors with different time windows in 2013, 3IF and 4IF seemed the ideal ranges for comparison, when assessed in relation to peer-reviewed scores. In addition, the 3-year and 4-year windows were quite consistent with the cited peak age of documents published by ophthalmologic journals. RESEARCH LIMITATIONS: Our study is based on ophthalmology journals and we only analyze the impact factors with different citation time window in 2013, so it has yet to be ascertained whether other disciplines (especially those with a later cited peak) or other years would follow the same or similar patterns. ORIGINALITY/ VALUE: We designed the survey questionnaire ourselves, specifically to assess the real influence of journals. We used peer-reviewed scores to judge the journal evaluation effect of impact factors with different citation time windows. The main purpose of this study was to help researchers better understand the role of impact factors with different citation time windows in journal evaluation.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0135583

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A 12-month timeframe was deemed appropriate to provide an accurate indication of Altmetric attention score and citations because there is a quick uptake of attention-based indicators after publication and short-term citation counts are highly correlated with long-term citation counts in peer-reviewed research.

    “A 12-month timeframe was deemed appropriate to provide an accurate indication of Altmetric attention score and citations considering there appears to be a quick uptake of attention-based indicators after publication (Thelwall et al., 2013), and that short-term citation counts are highly correlated with long-term citation counts in peer-reviewed research (Liu et al., 2015).”

    From Graphical abstracts are associated with greater Altmetric attention scores, but not citations, in sport science (2023) · cites this paper as Liu et al., 2015

    Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 41

  1. [1]Ren SL. Eigenfactor:importance of analyzing journals and papers based on cited network. Chinese Journal of Scientific and Technology Periodicals. 2009; 20: 415– 418.
  2. [2]Jacsó P. Differences in the rank position of journals by Eigenfactor metrics and the five-year impact factor in the Journal Citation Reports and the Eigenfactor Project web site. Online Information Review. 2010; 34: 496– 508.
  3. [3]Vanclay JK. Impact factor: outdated artefact or stepping-stone to journal certification?. Scientometrics. 2012; 92 211– 238.
  4. [4]van Nierop E. The introduction of the 5-year impact factor: does it benefit statistics journals?. Statistica Neerlandica. 2010; 64: 71– 76.
  5. [6]Sombatsompop N, Markpin T, Premkamolnetr N. A modified method for calculating the Impact Factors of journals in ISI Journal Citation Reports: Polymer Science Category in 1997–2001. Scientometrics. 2004; 60: 217– 235.
  6. [7]van Leeuwen TN, Moed HF, Reedijk J. Critical comments on Institute for Scientific information impact factors: a sample of inorganic molecular chemistry journals. Journal of Information Science. 1999; 25: 489– 498.
  7. [8]Dorta-Gonzalez P, Dorta-Gonzalez MI. Impact maturity times and citation time windows: the 2-year maximum journal impact factor. Journal of Informetrics. 2013; 7: 593– 602.
  8. [9]Zhao X. An analysis of the 5-year impact factor in JCR. Journal of Library Science in China. 2010; 36: 120– 126.
  9. [10]Liu XL, Fang HL, Ding J, Wang MY. Correlation of publication frequency with impact factors and 5-year impact factor in 1058 medical journals in SCI-expanded database. Chinese Journal of Scientific and Technology Periodicals. 2011; 22: 211– 214.
  10. [12]Jacso P. Five-year impact factor data in the Journal Citation Reports. Online Information Review. 2009; 33: 603– 614.
  11. [13]Della-Sala S, Grafman J. Cortex 2009 5-year and 2-year Impact Factor: 4.1. Cortex. 2010; 46: 1069– 1069.
  12. [14]Della Sala S, Grafman J. Five-year impact factor. Cortex. 2009; 45: 911– 911.
  13. [15]Campanario JM. Empirical study of journal impact factors obtained using the classical 2-year citation window versus a five-year citation window. Scientometrics. 2011a; 87: 189– 204.
  14. [16]Garfield E. Citation indexes for science: a new dimension in documentation through association of ideas. Science. 1955; 122: 103– 111. 14385826
  15. [17]Finardi U. Correlation between Journal Impact Factor and Citation Performance: An experimental study. J. Informetr. 2013; 7: 357– 370.
  16. [18]Pérez-Hornero P, Arias-Nicolás JP, Pulgarín AA, Pulgarín A. An annual JCR impact factor calculation based on Bayesian credibility formulas,. Journal of Informetrics. 2013; 7: 1– 9.
  17. [19]Garfield E, Sher IH. New factors in the evaluation of scientific literature through citation indexing. American Documentation. 1963; 14: 195– 201.
  18. [20]Garfield E. Citation analysis as a tool in journal evaluation. Science & Justice. 1972; 178: 471– 479. 5079701
  19. [21]Betz CL. Impact factor and other new developments. Journal of Pediatric Nursing-Nursing Care of Children & Families. 2014; 29: 1– 2.
  20. [22]Tort ABL, Targino ZH, Amaral OB. Rising Publication Delays Inflate Journal Impact Factors. Plos One. 2012; 7: 1– 7. PMC3534064 23300920
  21. [23]Campanario JM, Coslado MA. Benford’s law and citations, articles and impact factors of scientific journals. Scientometrics. 2011; 88: 421– 432.
  22. [24]Rizkallah J, Sin DD. Integrative Approach to Quality Assessment of Medical Journals UsingImpact Factor, Eigenfactor, and Article Influence Scores. Plos One. 2010; 5: 1– 10. PMC2855371 20419115
  23. [25]Dorta-González P, Dorta-González MI. Comparing journals from different fields of science and social science through a JCR subject categories normalized impact factor. Scientometrics. 2013; 95: 645– 672.
  24. [26]Dorta-González P, Dorta-González MI, Suárez-Vega R. An approach to the author citation potential: Measures of scientific performance which are invariant across scientific fields. Scientometrics. 2015; 102: 1467– 1496.
  25. [27]Martyn J, Gilchrist A. An Evaluation of British Scientific Journals (1 ed.): Aslib.1968
  26. [28]Huang MH, Lin WYC. The influence of journal self-citations on journal impact factor and immediacy index. Online Information Review. 2012; 36: 639– 654.
  27. [29]Anonymous. Beware the impact factor. Nature Materials. 2013; 12: 89– 89. doi: 10.1038/nmat3566 23340463 · DOI
  28. [30]Leydesdorff L, Zhou P, Bornmann L. How Can Journal Impact Factors Be Normalized Across Fields of Science? An Assessment in Terms of Percentile Ranks and Fractional Counts. Journal of the American Society for Information Science and Technology. 2013; 64: 96– 107.
  29. [31]Vanclay JK. Ranking forestry journals using the h-index. Journal of Informetrics. 2008; 2: 326– 334.
  30. [32]Ha TC, Tan SB, Soo KC. The journal impact factor: Too much of an impact?. Annals Academy of Medicine Singapore. 2006; 35: 911– 916. 17219007
  31. [33]Leydesdorff L, Bornmann L. Integrated Impact Indicators Compared With Impact Factors: An Alternative Research Design With Policy Implications. Journal of the American Society for Information Science and Technology. 2011; 62: 2133– 2146.
  32. [34]Dorta-Gonzalez P, Dorta-Gonzalez MI, Santos-Penate DR, Suarez-Vega R. Journal topic citation potential and between-field comparisons: The topic normalized impact factor. Journal of Informetrics. 2014; 8: 406– 418.
  33. [36]Gai SS, Liu XL, Zhang SL. Prediction and structural analysis of impact factor for journals indexed in SCI: a case study of Nature. Chinese Journal of Scientific and Technology Periodicals. 2014; 25: 980– 984.
  34. [37]He Y, Qiu JP Method and demonstration research by scientometrics on the selection of peer-review experts. Library and Information Service. 2012; 56: 33– 37.
  35. [38]Weale AR, Bailey M, Lear P. The level of non-citation of articles within a journal as a measure of quality: a comparison to the impact factor. BMC Medical Research Methodology. 2004; 4: 1– 8. PMC434502 15169549
  36. [40]Shao PJ, Fang JM, Judy D. Research of the web-based survey on the development of E-Business in China, Canada, and Taiwan. Proceedings of the 5th Asian e-Business Workshop. KAIST Press, Jcju, Korea. 2005.
  37. [41]Wang P, Liu XL, Liu RY. SNIP, SJR its modified index SNIP2,SJR2 in the use of journal evaluation. Chinese Journal of Scientific and Technology Periodicals. 2014; 25: 833– 838.
  38. [42]Yoshikane F, Suzuki T. Diversity of fields in patent citations: synchronic and diachronic changes. Scientometrics, 2014; 98: 1879– 1897.
  39. [43]Shubert E. Use and misuse of the impact factor. Systematics and Biodiversity. 2012; 10: 391– 394.
  40. [44]Garfield E. Long-term vs. short-term journal impact: does it matter?. Scientist. 1988a; 12: 11– 12.
  41. [45]Garfield E. Long-term vs. short-term impact: part II: the second 100 highest-impact journals. Scientist. 1988b; 12: 12– 13.

Cited by

Sorted by citations

Authors
Liu, X.-L., Gai, S.-S., Zhang, S.-L., & Wang, P.
Affiliations
  • Henan Medical University (CN)
  • Xiamen University (CN)
Venue
PLOS ONE
Publisher
Public Library of Science (PLoS)
Published
2015-08-21
Volume
10
Issue
8
Pages
e0135583
ISSN
1932-6203 (ISSN-L)
DOI
10.1371/journal.pone.0135583
PMC
PMC4546661
OpenAlex
W1861673140
Cited by
16
References
2
Citations by year
2026: 1 · 2023: 1 · 2022: 2 · 2021: 2 · 2020: 2 · 2019: 2 · 2018: 2 · 2017: 1 · 2016: 2 · 2015: 1
FWCI
1.82
Type
journal article
Language
en
Open access
Licence
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (acceptedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
National Office for Philosophy and Social Sciences
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
  • Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
Keywords
journal impact factor, journal evaluation, ophthalmology journals
MeSH terms
Humans, Ophthalmology, Periodicals as Topic, Surveys and Questionnaires, Research Design, Time Factors, Bibliometrics, Databases, Bibliographic, Peer Review, Research, Journal Impact Factor

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review