The DOI matches its Crossref record; 42 of the 48 references it cites were found in a registry or an index.
Do Altmetrics Work? Twitter and Ten Other Social Web Services
Open accessAbstract
Altmetric measurements derived from the social web are increasingly advocated and used as early indicators of article impact and usefulness. Nevertheless, there is a lack of systematic scientific evidence that altmetrics are valid proxies of either impact or utility although a few case studies have reported medium correlations between specific altmetrics and citation rates for individual journals or fields. To fill this gap, this study compares 11 altmetrics with Web of Science citations for 76 to 208,739 PubMed articles with at least one altmetric mention in each case and up to 1,891 journals per metric. It also introduces a simple sign test to overcome biases caused by different citation and usage windows. Statistically significant associations were found between higher metric scores and higher citations for articles with positive altmetric scores in all cases with sufficient evidence (Twitter, Facebook wall posts, research highlights, blogs, mainstream media and forums) except perhaps for Google+ posts. Evidence was insufficient for LinkedIn, Pinterest, question and answer sites, and Reddit, and no conclusions should be drawn about articles with zero altmetric scores or the strength of any correlation between altmetrics and citations. Nevertheless, comparisons between citations and metric values for articles published at different times, even within the same year, can remove or reverse this association and so publishers and scientometricians should consider the effect of time when using altmetrics to rank articles. Finally, the coverage of all the altmetrics except for Twitter seems to be low and so it is not clear if they are prevalent enough to be useful in practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“A 12-month timeframe was deemed appropriate to provide an accurate indication of Altmetric attention score and citations considering there appears to be a quick uptake of attention-based indicators after publication (Thelwall et al., 2013), and that short-term citation counts are highly correlated with long-term citation counts in peer-reviewed research (Liu et al., 2015).”
From Graphical abstracts are associated with greater Altmetric attention scores, but not citations, in sport science (2023) · cites this paper as Thelwall et al., 2013
In the citing paper
“The WoS and Scopus platforms also provide researchers with usage counts for publications, enabling researchers to analyze the usage data and to explore usage patterns (Wang et al., 2016, Thelwall et al., 2013).”
From Core Elements in the Process of Citing Publications: Conceptual Overview of the Literature (2018) · cites this paper as Thelwall et al., 2013
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,417
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,421
Eysenbach, G. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.2012 · Cited by 1,088
Meho, L. I., & Yang, K. (2007) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20677 · Cited by 974
What is societal impact of research and how can it be assessed? a literature survey
Bornmann, L. (2012) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22803 · Cited by 732
Earlier Web usage statistics as predictors of later citation impact
Brody, T., Harnad, S. R., & Carr, L. A. (2006) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20373 · Cited by 403
Scientometrics 2.0: New metrics of scholarly impact on the social Web
Priem, J. R., & Hemminger, B. H. (2010) · First Monday · doi:10.5210/fm.v15i7.2874 · Cited by 382
Altmetrics in the wild: Using social media to explore scholarly impactPreprint
Priem, J. R., Piwowar, H., & Hemminger, B. M. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1203.4745 · Cited by 360
Shuai, X., Pepe, A., & Bollen, J. (2012) · PLoS ONE · doi:10.1371/journal.pone.0047523 · Cited by 327
The weakening relationship between the impact factor and papers' citations in the digital age
Lozano, G. A., Larivière, V., & Gingras, Y. (2012) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22731 · Cited by 293
Self-citationPriem, J. R., Groth, P., & Taraborelli, D. (2012) · PLoS ONE · doi:10.1371/journal.pone.0048753 · Cited by 280
Validating online reference managers for scholarly impact measurement
Li, X., Thelwall, M. A., & Giustini, D. (2011) · Scientometrics · doi:10.1007/s11192-011-0580-x · Cited by 258
Self-citationArticle-Level Metrics and the Evolution of Scientific Impact
Neylon, C., & Wu, S. (2009) · PLoS Biology · doi:10.1371/journal.pbio.1000242 · Cited by 244
Towards Electronic Journals: Realities for Scientists, Librarians, and Publishers,,,,
Tenopir, C., & King, D. W. (2000) · Cited by 230
Altmetric: enriching scholarly content with article‐level discussion and metrics
Adie, E. A., & William, R. O. E. (2012) · Learned Publishing · doi:10.1087/20130103 · Cited by 224
Google Scholar citations and Google Web/URL citations: A multi‐discipline exploratory analysis
Kousha, K., & Thelwall, M. A. (2007) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20584 · Cited by 211
Self-citationResearch Blogs and the Discussion of Scholarly Information
Shema, H., Bar‐Ilan, J., & Thelwall, M. A. (2012) · PLoS ONE · doi:10.1371/journal.pone.0035869 · Cited by 187
Self-citationBeyond citations: Scholars' visibility on the social WebPreprint
Bar‐Ilan, J., Haustein, S., Peters, I., Priem, J. R., Shema, H., & Terliesner, J. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.5611 · Cited by 169
Self-citationKurtz, M. J., & Bollen, J. (2010) · Annual Review of Information Science and Technology · doi:10.1002/aris.2010.1440440108 · Cited by 166
Moed, H. F. (2005) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20200 · Cited by 154
Applying social bookmarking data to evaluate journal usage
Haustein, S., & Siebenlist, T. (2011) · Journal of Informetrics · doi:10.1016/j.joi.2011.04.002 · Cited by 115
Self-citationTweeting the Meeting: An In-Depth Analysis of Twitter Activity at Kidney Week 2011
Desai, T., Shariff, A. I., Shariff, A., Кац, М. Д., Fang, X., Christiano, C., & Ferris, M. (2012) · PLoS ONE · doi:10.1371/journal.pone.0040253 · Cited by 111
Proceedings of the 17th International conference on science and technology indicators
Larivière, V. (2012) · Cited by 100
Self-citationGoogle book search: Citation analysis for social science and the humanities
Kousha, K., & Thelwall, M. A. (2009) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21085 · Cited by 99
Self-citationSchlögl, C., & Stock, W. G. (2004) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20070 · Cited by 81
Studies in Scientometrics I Transience and Continuance in Scientific Authorship
Price, D. D. S., & Gürsey, S. (1975) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) · doi:10.18225/ci.inf.v4i1.64 · Cited by 80
Soft peer review: social software and distributed scientific evaluation
Taraborelli, D. (2008) · UCL Discovery (University College London) · Cited by 72
Information science and scientists: JASIS, 1972-1990
Harter, S. P., & Hooten, P. A. (1992) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199210)43:9 · Cited by 69
Priem, J. R. (2014) · The MIT Press eBooks · doi:10.7551/mitpress/9445.003.0019 · Cited by 68
Kousha, K., & Thelwall, M. A. (2008) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20920 · Cited by 67
Self-citationYan, K., & Gerstein, M. (2011) · PLoS ONE · doi:10.1371/journal.pone.0019917 · Cited by 61
Scientific citations in Wikipedia
Nielsen, F. A. (2007) · First Monday · doi:10.5210/fm.v12i8.1997 · Cited by 61
Studying Scientific Discourse on the Web using Bibliometrics: A Chemistry Blogging Case Study
Groth, P. T., & Gurney, T. (2010) · Data Archiving and Networked Services (DANS) · Cited by 60
Buckingham Shum, S. (2000) · Journal of Interactive Media in Education · doi:10.5334/book-tenopir · Cited by 55
The journal download immediacy index (DII): experiences using a Chinese full-text database
Wan, J.-K., Hua, P.-H., Rousseau, R., & Sun, X. (2010) · Scientometrics · doi:10.1007/s11192-010-0171-2 · Cited by 54
The missing link: journal usage metrics
Rowlands, I., & Nicholas, D. (2007) · Aslib Journal of Information Management · doi:10.1108/00012530710752025 · Cited by 48
Google Scholar Metrics: an unreliable tool for assessing scientific journals
López‐Cózar, E. D., & Cabezas-Clavijo, Á. (2012) · El Profesional de la Informacion · doi:10.3145/epi.2012.jul.15 · Cited by 45
Research Dissemination and Impact: Evidence from Web Site Downloads
Pinkowitz, L. F. (2002) · The Journal of Finance · doi:10.1111/1540-6261.00429 · Cited by 40
Thelwall, M. A., & Kousha, K. (2008) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20803 · Cited by 40
Self-citationBar‐Ilan, J. (2012) · Bulletin of the American Society for Information Science and Technology · doi:10.1002/bult.2012.1720380607 · Cited by 35
Bibliographic References in Web 2.0
Bar‐Ilan, J., Shema, H., & Thelwall, M. A. (2014) · The MIT Press eBooks · doi:10.7551/mitpress/9445.003.0021 · Cited by 23
Self-citationF1000 recommendations as a new data source for research evaluation: A comparison with citationsPreprint
Waltman, L., & Costas, R. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1303.3875 · Cited by 1
Flagged first, then by citations
6 of the 48 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- Haustein S (2012) Readership metrics. In: Cronin B, Sugimoto C, editors. Beyond Bibliometrics: Harnessing Multi-dimensional Indicators of Performance. Cambridge, MA: MIT Press, in press.
- Bar-Ilan J (2012a) JASIST@mendeley. Presented at the ACM Web Science Conference Workshop on Altmetrics Evanston, IL. Available: http://altmetricsorg/altmetrics12/bar-ilan. Accessed 21 January 2013.
Cited by
Sorted by citations
Science Mapping: A Systematic Review of the Literature
Chen, C. (2017) · Journal of Data and Information Science · doi:10.1515/jdis-2017-0006 · Cited by 2,134
Bibliometrics: methods for studying academic publishing
Ninkov, A., Frank, J. R., & Maggio, L. A. (2021) · Perspectives on Medical Education · doi:10.1007/s40037-021-00695-4 · Cited by 1,048
Costas, R., Zahedi, Z., & Wouters, P. (2015) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23309 · Cited by 711
Bornmann, L. (2014) · Journal of Informetrics · doi:10.1016/j.joi.2014.09.005 · Cited by 578
Scholarly use of social media and altmetrics: A review of the literature
Sugimoto, C. R., Work, S., Larivière, V., & Haustein, S. (2017) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23833 · Cited by 680
Self-citation
Metadata
- Authors
- Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. R.
- Affiliations
- University of Wolverhampton (GB)
- Science-Metrix (Canada) (CA)
- Université du Québec à Montréal (CA)
- Indiana University Bloomington (US)
- Corresponding author
- Thelwall, Mike
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-05-28
- Volume
- 8
- Issue
- 5
- Pages
- e64841
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3665624
- OpenAlex
- W2073023209
- Cited by
- 907
- References
- 48
- Citations by year
- 2026: 29 · 2025: 36 · 2024: 48 · 2023: 42 · 2022: 61 · 2021: 82 · 2020: 89 · 2019: 75 · 2018: 103 · 2017: 101 · 2016: 73 · 2015: 85 · 2014: 68 · 2013: 15
- FWCI
- 88.64
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Papyrus : Institutional Repository (Université de Montréal) (submittedVersion, cc-by)
- Funders
- National Science Foundation (1208804)
- Joint Information Systems Committee
- Arts and Humanities Research Council
- Economic and Social Research Council
- Topics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- altmetrics, Twitter, social web services, article impact, citation analysis, PubMed articles, sign test, research highlights, mainstream media coverage
- MeSH terms
- Humans, Publishing, Bibliometrics, PubMed, Social Media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
