A Comprehensive Examination of the Relation of Three Citation-Based Journal Metrics to Expert Judgment of Journal Quality
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Bibliometric indicators were generally shown to be positively correlated with peer-review evaluations.
“Although bibliometric indicators were generally shown to be positively correlated with peer-review evaluations”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [280]
Correlations between metrics and expert judgements vary greatly across disciplines, application contexts, and between metrics used.
“it was also observed that the correlations between metrics and expert judgements vary greatly across disciplines, application contexts, and between metrics used [44,276,280,282].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [280]
Cites
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879 · see citing works
Bibliometrics: The Leiden Manifesto for Research Metrics
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015) · Nature · doi:10.1038/520429a · Cited by 2,828 · see citing works
Why the impact factor of journals should not be used for evaluating research
Seglen, P. O. (1997) · BMJ · doi:10.1136/bmj.314.7079.497 · Cited by 2,447 · see citing works
Is citation analysis a legitimate evaluation tool?
Garfield, E. (1979) · Scientometrics · doi:10.1007/bf02019306 · Cited by 911 · see citing works
Measuring Contextual Citation Impact of Scientific Journals
Moed, H. F. (2010) · Journal of Informetrics · doi:10.1016/j.joi.2010.01.002 · Cited by 628 · see citing works
Martin, B. R., & Irvine, J. M. (1983) · Research Policy · doi:10.1016/0048-7333(83)90005-7 · Cited by 529 · see citing works
The Research Excellence Framework and the 'impact agenda': are we creating a Frankenstein monster?
Martin, B. R. (2011) · Research Evaluation · doi:10.3152/095820211x13118583635693 · Cited by 470 · see citing works
van Raan, A. F. J. (2003) · TATuP Zeitschrift für Technikfolgenabschätzung in Theorie und Praxis · doi:10.14512/tatup.12.1.20 · Cited by 341 · see citing works
Evaluating research: from informed peer review to bibliometrics
Abramo, G., & D’Angelo, C. A. (2011) · Scientometrics · doi:10.1007/s11192-011-0352-7 · Cited by 220 · see citing works
Langfeldt, L. (2001) · Social Studies of Science · doi:10.1177/030631201031006002 · Cited by 207 · see citing works
Some Modifications to the SNIP Journal Impact Indicator
Waltman, L., van Eck, N. J., van Leeuwen, T. N., & Visser, M. S. (2013) · Journal of Informetrics · doi:10.1016/j.joi.2012.11.011 · Cited by 191 · see citing works
Modifying the journal impact factor by fractional citation weighting: The audience factor
Zitt, M., & Small, H. G. (2008) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20880 · Cited by 184 · see citing works
Peer reviews and bibliometric indicators: a comparative study at a Norwegian university
Aksnes, D. W., & Taxt, R. E. (2004) · Research Evaluation · doi:10.3152/147154404781776563 · Cited by 179 · see citing works
Moed, H. F. (2007) · Science and Public Policy · doi:10.3152/030234207x255179 · Cited by 114 · see citing works
SJR and SNIP: two new journal metrics in Elsevier's Scopus
Colledge, L., Anegón, F. D. M., Guerrero‐Bote, V. P., López-Illescas, C., Aisati, M. E., & Moed, H. F. (2010) · Serials The Journal for the Serials Community · doi:10.1629/23215 · Cited by 98 · see citing works
Bibliometric evaluation vs. informed peer review: Evidence from Italy
Bertocchi, G., Gambardella, A., Jappelli, T., Nappi, C. A., & Peracchi, F. (2015) · Research Policy · doi:10.1016/j.respol.2014.08.004 · Cited by 86 · see citing works
Serenko, A., & Dohan, M. S. (2011) · Journal of Informetrics · doi:10.1016/j.joi.2011.06.002 · Cited by 80 · see citing works
Moed, H. F., van Leeuwen, T. N., & Reedijk, J. (1996) · Scientometrics · doi:10.1007/bf02093487 · Cited by 71 · see citing works
Citation analysis and peer ranking of Australian social science journals
Haddow, G., & Genoni, P. (2010) · Scientometrics · doi:10.1007/s11192-010-0198-4 · Cited by 67 · see citing works
Metrics or Peer Review? Evaluating the 2001 UK Research Assessment Exercise in Political Science
Butler, L., & McAllister, I. (2008) · Political Studies Review · doi:10.1111/j.1478-9299.2008.00167.x · Cited by 59 · see citing works
The new ERA of journal ranking: The consequences of Australia's fraught encounter with 'quality'
Cooper, S., & Poletti, A. (2011) · Australian universities' review · Cited by 46 · see citing works
An evaluation of the Australian Research Council's journal ranking
Vanclay, J. K. (2011) · Journal of Informetrics · doi:10.1016/j.joi.2010.12.001 · Cited by 44 · see citing works
An evaluation of the Australian Research Council's journal ranking
Vanclay, J. K. (2011) · Cross Connect (Southern Cross University) · Cited by 42 · see citing works
Research Policy: Problems with peer review and alternatives
Smith, J. (1988) · BMJ · doi:10.1136/bmj.296.6624.774 · Cited by 40 · see citing works
Ahlgren, P., & Waltman, L. (2014) · Journal of Informetrics · doi:10.1016/j.joi.2014.09.010 · Cited by 33 · see citing works
Jacsó, P. (2012) · Scientometrics · doi:10.1007/s11192-012-0769-7 · Cited by 29 · see citing works
ERA and the Ranking of Australian Humanities Journals
Genoni, P., & Haddow, G. (2009) · Australian Humanities Review · doi:10.22459/ahr.46.2009.01 · Cited by 27 · see citing works
Standards issues in a national bibliometric database: The Australian case
Bourke, P., & Butler, L. (1996) · Scientometrics · doi:10.1007/bf02018478 · Cited by 22 · see citing works
The devil is in the detail: Concerns about Vanclay's analysis of Australian journal rankings
Butler, L. (2011) · Journal of Informetrics · doi:10.1016/j.joi.2011.04.001 · Cited by 9 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World
Pranckutė, R. (2021) · Publications · doi:10.3390/publications9010012 · Cited by 2,696 · see citing works
- Authors
- Haddawy, P., Hassan, S.-U., Asghar, A., & Amin, S.
- Affiliations
- Mahidol University (TH)
- Information Technology University (PK)
- Corresponding author
- Haddawy, Peter
- Venue
- Journal of Informetrics
- Publisher
- Elsevier BV
- Published
- 2016-02
- Volume
- 10
- Issue
- 1
- Pages
- 162-173
- ISSN
- 1751-1577 (ISSN-L)
- OpenAlex
- W2270805614
- Cited by
- 45
- References
- 29
- Citations by year
- 2026: 4 · 2024: 4 · 2023: 4 · 2022: 2 · 2021: 5 · 2020: 5 · 2019: 8 · 2018: 5 · 2017: 6 · 2016: 2
- FWCI
- 6.59
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Informetrics (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
- Keywords
- Source Normalized Impact per Paper, journal impact factor, peer review, research evaluation, Excellence in Research for Australia
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
