CoChat
Sign Up
Cited by 908scientometrics and bibliometrics researchWeb visibility and informetricsAcademic Publishing and Open Access

Do Altmetrics Work? Twitter and Ten Other Social Web Services

Open access
Thelwall, Mike, Haustein, Stefanie, Larivière, Vincent, Sugimoto, Cassidy R.·2013·PLoS ONE
DOI 10.1371/journal.pone.0064841PMC PMC3665624
Open this paper in CoChat — free accountOpen PDF

Abstract

Altmetric measurements derived from the social web are increasingly advocated and used as early indicators of article impact and usefulness. Nevertheless, there is a lack of systematic scientific evidence that altmetrics are valid proxies of either impact or utility although a few case studies have reported medium correlations between specific altmetrics and citation rates for individual journals or fields. To fill this gap, this study compares 11 altmetrics with Web of Science citations for 76 to 208,739 PubMed articles with at least one altmetric mention in each case and up to 1,891 journals per metric. It also introduces a simple sign test to overcome biases caused by different citation and usage windows. Statistically significant associations were found between higher metric scores and higher citations for articles with positive altmetric scores in all cases with sufficient evidence (Twitter, Facebook wall posts, research highlights, blogs, mainstream media and forums) except perhaps for Google+ posts. Evidence was insufficient for LinkedIn, Pinterest, question and answer sites, and Reddit, and no conclusions should be drawn about articles with zero altmetric scores or the strength of any correlation between altmetrics and citations. Nevertheless, comparisons between citations and metric values for articles published at different times, even within the same year, can remove or reverse this association and so publishers and scientometricians should consider the effect of time when using altmetrics to rank articles. Finally, the coverage of all the altmetrics except for Twitter seems to be low and so it is not clear if they are prevalent enough to be useful in practice.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0064841

Cites

42 of 42 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A 12-month timeframe was deemed appropriate to provide an accurate indication of Altmetric attention score and citations because there is a quick uptake of attention-based indicators after publication and short-term citation counts are highly correlated with long-term citation counts in peer-reviewed research.

    “A 12-month timeframe was deemed appropriate to provide an accurate indication of Altmetric attention score and citations considering there appears to be a quick uptake of attention-based indicators after publication (Thelwall et al., 2013), and that short-term citation counts are highly correlated with long-term citation counts in peer-reviewed research (Liu et al., 2015).”

    From Graphical abstracts are associated with greater Altmetric attention scores, but not citations, in sport science (2023) · cites this paper as Thelwall et al., 2013

  • The WoS and Scopus platforms provide researchers with usage counts for publications, enabling researchers to analyze the usage data and to explore usage patterns.

    “The WoS and Scopus platforms also provide researchers with usage counts for publications, enabling researchers to analyze the usage data and to explore usage patterns (Wang et al., 2016, Thelwall et al., 2013).”

    From Core Elements in the Process of Citing Publications: Conceptual Overview of the Literature (2018) · cites this paper as Thelwall et al., 2013

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

6 of the 48 references couldn’t be matched to a known paper.

Research works printed but not linked: 4

  1. [3]Seglen PO (1992) The skewness of science. Journal of the American Society for Information Science 43: 628– 638.
  2. [8]Haustein S (2012) Readership metrics. In: Cronin B, Sugimoto C, editors. Beyond Bibliometrics: Harnessing Multi-dimensional Indicators of Performance. Cambridge, MA: MIT Press, in press.
  3. [19]Priem J (2013) Altmetrics. In: Cronin B, Sugimoto C, editors. Bibliometrics and Beyond: Metrics-Based Evaluation of Scholarly Research, Cambridge: MIT Press, in press.
  4. [31]Bar-Ilan J (2012a) JASIST@mendeley. Presented at the ACM Web Science Conference Workshop on Altmetrics Evanston, IL. Available: http://altmetricsorg/altmetrics12/bar-ilan. Accessed 21 January 2013.

Cited by

Sorted by citations

Related

Authors
Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. R.
Affiliations
  • University of Wolverhampton (GB)
  • Science-Metrix (Canada) (CA)
  • Université du Québec à Montréal (CA)
  • Indiana University Bloomington (US)
Corresponding author
Thelwall, Mike
ORCID
Venue
PLoS ONE
Publisher
Public Library of Science (PLoS)
Published
2013-05-28
Volume
8
Issue
5
Pages
e64841
ISSN
1932-6203 (ISSN-L)
DOI
10.1371/journal.pone.0064841
PMC
PMC3665624
OpenAlex
W2073023209
Cited by
907
References
48
Citations by year
2026: 29 · 2025: 36 · 2024: 48 · 2023: 42 · 2022: 61 · 2021: 82 · 2020: 89 · 2019: 75 · 2018: 103 · 2017: 101 · 2016: 73 · 2015: 85 · 2014: 68 · 2013: 15
FWCI
88.64
Type
journal article
Language
en
Open access
Licence
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Papyrus : Institutional Repository (Université de Montréal) (submittedVersion, cc-by)
Funders
  • National Science Foundation (1208804)
  • Joint Information Systems Committee
  • Arts and Humanities Research Council
  • Economic and Social Research Council
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
  • Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
  • Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
Keywords
altmetrics, Twitter, social web services, article impact, citation analysis, PubMed articles, sign test, research highlights, mainstream media coverage
MeSH terms
Humans, Publishing, Bibliometrics, PubMed, Social Media

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review