The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective
Open accessAbstract
An extensive analysis of the presence of different altmetric indicators provided by Altmetric.com across scientific fields is presented, particularly focusing on their relationship with citations. Our results confirm that the presence and density of social media altmetric counts are still very low and not very frequent among scientific publications, with 15%–24% of the publications presenting some altmetric activity and concentrated on the most recent publications, although their presence is increasing over time. Publications from the social sciences, humanities, and the medical and life sciences show the highest presence of altmetrics, indicating their potential value and interest for these fields. The analysis of the relationships between altmetrics and citations confirms previous claims of positive correlations but is relatively weak, thus supporting the idea that altmetrics do not reflect the same kind of impact as citations. Also, altmetric counts do not always present a better filtering of highly‐cited publications than journal citation scores. Altmetric scores (particularly mentions in blogs) are able to identify highly‐cited publications with higher levels of precision than journal citation scores ( JCS ), but they have a lower level of recall. The value of altmetrics as a complementary tool of citation analysis is highlighted, although more research is suggested to disentangle the potential meaning and value of altmetric indicators for research evaluation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A recent study by Costas et al. also found that Altmetrics coverage of Mathematical and Computer Science papers tends to be much lower than disciplines like Biomedicine and Health or Natural Sciences and engineering [29].”
From Measuring scientific impact beyond academia: An assessment of existing impact metrics and proposed improvements (2017) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Why the impact factor of journals should not be used for evaluating research
Seglen, P. O. (1997) · BMJ · doi:10.1136/bmj.314.7079.497 · Cited by 2,447
Do Altmetrics Work? Twitter and Ten Other Social Web Services
Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. R. (2013) · PLoS ONE · doi:10.1371/journal.pone.0064841 · Cited by 907
The Leiden ranking 2011/2012: Data collection, indicators, and interpretation
Waltman, L., Calero‐Medina, C., Kosten, J., Noyons, E., Tijssen, R., van Eck, N. J., van Leeuwen, T. N., van Raan, A. F. J., Visser, M. S., & Wouters, P. F. (2012) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22708 · Cited by 452
Self-citationTowards a new crown indicator: Some theoretical considerations
Waltman, L., van Eck, N. J., van Leeuwen, T. N., Visser, M. S., & van Raan, A. F. J. (2010) · Journal of Informetrics · doi:10.1016/j.joi.2010.08.001 · Cited by 438
Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature
Haustein, S., Peters, I., Sugimoto, C. R., Thelwall, M. A., & Larivière, V. (2013) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23101 · Cited by 434
Zahedi, Z., Costas, R., & Wouters, P. F. (2014) · Scientometrics · doi:10.1007/s11192-014-1264-0 · Cited by 386
Self-citationAltmetrics in the wild: Using social media to explore scholarly impactPreprint
Priem, J. R., Piwowar, H., & Hemminger, B. M. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1203.4745 · Cited by 360
Sud, P., & Thelwall, M. A. (2013) · Scientometrics · doi:10.1007/s11192-013-1117-2 · Cited by 276
Altmetric: enriching scholarly content with article‐level discussion and metrics
Adie, E. A., & William, R. O. E. (2012) · Learned Publishing · doi:10.1087/20130103 · Cited by 224
Shema, H., Bar‐Ilan, J., & Thelwall, M. A. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23037 · Cited by 209
Users, narcissism and control – tracking the impact of scholarly publications in the 21st century
Costas, R., & Wouters, P. F. (2012) · Cited by 195
Self-citationResearch Blogs and the Discussion of Scholarly Information
Shema, H., Bar‐Ilan, J., & Thelwall, M. A. (2012) · PLoS ONE · doi:10.1371/journal.pone.0035869 · Cited by 187
Costas, R., van Leeuwen, T. N., & Bordons, M. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21348 · Cited by 185
Self-citationBeyond citations: Scholars' visibility on the social WebPreprint
Bar‐Ilan, J., Haustein, S., Peters, I., Priem, J. R., Shema, H., & Terliesner, J. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.5611 · Cited by 169
The effect of “open access” on citation impact: An analysis of ArXiv's condensed matter section
Moed, H. F. (2007) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20663 · Cited by 164
Altmetrics: Rethinking the Way We Measure
Galligan, F., & Dyas-Correia, S. (2013) · Serials Review · doi:10.1016/j.serrev.2013.01.003 · Cited by 155
Altmetrics: Rethinking the Way We Measure
Galligan, F., & Dyas-Correia, S. (2013) · Serials Review · doi:10.1080/00987913.2013.10765486 · Cited by 155
Self-citations at the meso and individual levels: Effects of different calculation methods
Costas, R., van Leeuwen, T. N., & Bordons, M. (2010) · Scientometrics · doi:10.1007/s11192-010-0187-7 · Cited by 148
Self-citationWaltman, L., & Costas, R. (2013) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23040 · Cited by 132
Self-citationRising Publication Delays Inflate Journal Impact Factors
Tort, A. B. L., Targino, Z. H., & Amaral, O. B. (2012) · PLoS ONE · doi:10.1371/journal.pone.0053374 · Cited by 109
The power of altmetrics on a CV
Piwowar, H., & Priem, J. R. (2013) · Bulletin of the American Society for Information Science and Technology · doi:10.1002/bult.2013.1720390405 · Cited by 86
The praxis of acknowledgement: from bibliometrics to influmetrics
Cronin, B., & Weaver, S. (1995) · Revista española de Documentación Científica · doi:10.3989/redc.1995.v18.i2.654 · Cited by 82
Studying Scientific Discourse on the Web using Bibliometrics: A Chemistry Blogging Case Study
Groth, P. T., & Gurney, T. (2010) · Data Archiving and Networked Services (DANS) · Cited by 60
A multi-metric approach for research evaluation
Ronald, R., & Fred, Y. Y. (2013) · Chinese Science Bulletin · doi:10.1007/s11434-013-5939-3 · Cited by 59
Effects of Print Publication Lag in Dual Format Journals on Scientometric Indicators
Heneberg, P. (2013) · PLoS ONE · doi:10.1371/journal.pone.0059877 · Cited by 39
On the relation between altmetrics and citations in medicine
Haustein, S. (2013) · Cited by 6
On the relation between altmetrics and citations in medicine
Larivière, V. (2013) · Cited by 1
Kellerer, A. M., & Lange, R. (2002) · Radiation and Environmental Biophysics · doi:10.1007/s00411-001-0126-3 · Cited by 1
Wu, X. (2003) · Knowledge and Information Systems · doi:10.1007/s10115-003-0113-x · Cited by 1
Flagged first, then by citations
Research works printed but not linked: 4
- Bar‐Ilan J.(2012).JASIST@ Mendeley. Presented at ACM Web Science Conference Workshop on Altmetrics Evanston IL 21 June 2012. Retrieved from:http://altmetrics.org/altmetrics12/bar‐ilan/
- Piwowar H., 2013, Value all research products, Nature, 493, 2013
- Schmidt M.(2012).Development and evaluation of a match key for linking references to cited articles. InProceedings of the 17th International Conference on Science and Technology Indicators Montréal; Science‐Metrix and OST Volume 2(pp.707–718).
- Zahedi Z., 2013, Proceedings of the 14th International Society of Scientometrics and Informetrics Conference, Vienna, Austria, 876
Cited by
Sorted by citations
Fortunato, S., Bergstrom, C. T., Börner, K., Evans, J. A., Helbing, D., Milojević, S., Petersen, A. M., Radicchi, F., Sinatra, R., Uzzi, B., et al. (2018) · Science · doi:10.1126/science.aao0185 · Cited by 1,497
The Academic, Economic and Societal Impacts of Open Access: An Evidence-Based Review
Tennant, J. P., Waldner, F., Jacques, D. C., Masuzzo, P., Collister, L. B., & Hartgerink, C. H. J. (2016) · F1000Research · doi:10.12688/f1000research.8460.3 · Cited by 560
Scholarly use of social media and altmetrics: A review of the literature
Sugimoto, C. R., Work, S., Larivière, V., & Haustein, S. (2017) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23833 · Cited by 680
Cheng, A., Nadkarni, V., Mancini, M. B., Hunt, E. A., Sinz, E. H., Merchant, R. M., Donoghue, A. J., Duff, J. P., Eppich, W., Auerbach, M. A., et al. (2018) · Circulation · doi:10.1161/cir.0000000000000583 · Cited by 457
OECD (2015) · OECD science, technology and industry policy papers · doi:10.1787/5jrs2f963zs1-en · Cited by 522
Metadata
- Authors
- Costas, R., Zahedi, Z., & Wouters, P.
- Affiliations
- Leiden University (NL)
- Corresponding author
- Costas, Rodrigo
- Zahedi, Zohreh
- Wouters, Paul
- Venue
- Journal of the Association for Information Science and Technology
- Publisher
- Wiley
- Published
- 2014-07-28
- Volume
- 66
- Issue
- 10
- Pages
- 2003-2019
- ISSN
- 2330-1635 (ISSN-L)
- 2330-1643
- OpenAlex
- W2164277894
- Cited by
- 711
- References
- 29
- Citations by year
- 2026: 27 · 2025: 37 · 2024: 48 · 2023: 55 · 2022: 56 · 2021: 72 · 2020: 96 · 2019: 70 · 2018: 88 · 2017: 63 · 2016: 37 · 2015: 46 · 2014: 16
- FWCI
- 52.66
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Journal of the Association for Information Science and Technology (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion, other-oa)
- RePEc: Research Papers in Economics (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Funders
- EU FP7 ACUMEN (266632)
- Topics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- altmetrics, citation analysis, Altmetric.com, social media engagement, research impact, highly cited papers, research evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
