Cited by 48Smoking Behavior and CessationSocial Media in Health EducationComputational and Text Analysis Methods

Methods for Coding Tobacco-Related Twitter Data: A Systematic Review

Open access
Lienemann, Brianna A, Unger, Jennifer B, Cruz, Tess Boley, Chu, Kar-Hai·2017·Journal of Medical Internet Research

Abstract

BACKGROUND: As Twitter has grown in popularity to 313 million monthly active users, researchers have increasingly been using it as a data source for tobacco-related research. OBJECTIVE: The objective of this systematic review was to assess the methodological approaches of categorically coded tobacco Twitter data and make recommendations for future studies. METHODS: Data sources included PsycINFO, Web of Science, PubMed, ABI/INFORM, Communication Source, and Tobacco Regulatory Science. Searches were limited to peer-reviewed journals and conference proceedings in English from January 2006 to July 2016. The initial search identified 274 articles using a Twitter keyword and a tobacco keyword. One coder reviewed all abstracts and identified 27 articles that met the following inclusion criteria: (1) original research, (2) focused on tobacco or a tobacco product, (3) analyzed Twitter data, and (4) coded Twitter data categorically. One coder extracted data collection and coding methods. RESULTS: E-cigarettes were the most common type of Twitter data analyzed, followed by specific tobacco campaigns. The most prevalent data sources were Gnip and Twitter's Streaming application programming interface (API). The primary methods of coding were hand-coding and machine learning. The studies predominantly coded for relevance, sentiment, theme, user or account, and location of user. CONCLUSIONS: Standards for data collection and coding should be developed to be able to more easily compare and replicate tobacco-related Twitter results. Additional recommendations include the following: sample Twitter's databases multiple times, make a distinction between message attitude and emotional tone for sentiment, code images and URLs, and analyze user profiles. Being relatively novel and widely used among adolescents and black and Hispanic individuals, Twitter could provide a rich source of tobacco surveillance data among vulnerable populations.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/jmir.7022

Cites

35 of 35 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The single coder approach allowed for consistency of coding and interpretation of codes.

    “As we used emergent coding and there was no a priori codebook, a single coder approach also allowed for consistency of coding and interpretation of codes, and this approach has been used in systematic reviews [21].”

    From Barriers to and facilitators of user engagement with digital mental health interventions: Systematic review (2021) · cites this paper as [21]

    The Review Limitations section (page unavailable) confirms that this systematic review used a single coder to select articles and extract data. It does not say that using one coder improved consistency of coding or interpretation; instead, it notes that decisions might have differed with another coder. (LLM, conf=0.85)

    Experimental — accuracy not yet measured
  • The risk of publication bias with significant results is small compared with a meta-analysis of outcomes.

    “Therefore, the risk of publication bias with significant results is small compared with a meta-analysis of outcomes [21].”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 36 references couldn’t be matched to a known paper.

Research works printed but not linked: 3

  1. [52]George D Mallery P SPSS for Windows step by step: A simple guide and reference. 11.0 update. 4th edition 2003 Boston, MA Allyn & Bacon
  2. [58]Burger JD Henderson J Kim G Zarrella G Discriminating gender on Twitter 2011 Proceedings of the Conference on Empirical Methods in Natural Language Processing July 2011 Edinburgh, Scotland, UK
  3. [60]Nguyen D Gravel R Trieschnigg D Meder T “How old do you think I am?”: a study of language and age in twitter 2013 Proceedings of the Seventh International AAAI Conference on WeblogsSocial Media July 2013 Cambridge, MA

Cited by

Sorted by citations

Authors
Lienemann, B. A., Unger, J. B., Cruz, T. B., & Chu, K.-H.
Affiliations
University of Southern California (US)
Corresponding author
Lienemann, Brianna A
Venue
Journal of Medical Internet Research
Published
2017-03-31
Volume
19
Issue
3
Pages
e91
ISSN
1438-8871 (ISSN-L)
OpenAlex
W2599314234
Cited by
48
References
36
Citations by year
2026: 2 · 2025: 1 · 2024: 4 · 2023: 6 · 2022: 8 · 2021: 8 · 2020: 6 · 2019: 6 · 2018: 7
FWCI
0.64
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
National Cancer Institute (T32CA009492, P50CA180905)
Topics
  • Health Sciences › Medicine › Physiology › Smoking Behavior and Cessation
  • Social Sciences › Health › Social Media in Health Education
  • Social Sciences › General Social Sciences › Computational and Text Analysis Methods
Keywords
Twitter, electronic cigarettes, sentiment analysis, machine learning, social media surveillance, coding standards, user profiles, vulnerable populations
MeSH terms
Data Collection, Humans, Smoking, Social Marketing, Social Media, Tobacco Products, Electronic Nicotine Delivery Systems

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review