CoChat
Sign Up
Cited by 32Computational and Text Analysis MethodsDigital Humanities and ScholarshipSentiment Analysis and Opinion Mining

Text as big data: Develop codes of practice for rigorous computational text analysis in energy social science

Open access
Müller-Hansen, Finn, Callaghan, Max W., Minx, Jan C.·2020·Energy Research & Social Science
DOI 10.1016/j.erss.2020.101691

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Augmenting traditional social science methods with computational analysis is crucial if we are to exploit the vast digital archives of text data that have become available over the past two decades. In this journal, Benites-Lazaro et al. (2018) showcase this in an application of topic modeling and other computational methods to an actor-specific examination of changes in policy discourse on ethanol in Brazil and point out methodological promises and challenges. However, their contribution also highlights the need for establishing codes of practice for computational text analysis. In this perspective, we discuss five areas for improvement when treating text as big data in light of guiding principles from computational research – transparency, reproducibility and validation – to facilitate rigorous research practice: (1) full transparency over data collection and corpus construction, (2) comprehensive method descriptions that enable reproducibility by other researchers, (3) application of rigorous model validation procedures, (4) results interpretation based on primary text and clear research design and (5) critical discussion and contextualization of main findings. We conclude that the energy social science community needs to develop codes of practice to build on the promising research within the field of computational text analysis and suggest first steps into this direction.

Integrity signals

Registry

Crossref · 10.1016/j.erss.2020.101691

Cites

82 of 82 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The authors develop a two-step attribution process which combines a transparent and reproducible machine learning approach to identifying studies on observed climate impacts with model-based assessments of detectable anthropogenic contributions to historical temperature and precipitation trends.

    “We develop a two-step attribution process which combines a transparent and reproducible 39,40 machine learning approach to identifying studies on observed climate impacts with model-based assessments of detectable anthropogenic contributions to historical temperature and precipitation trends.”

    From Machine-learning-based evidence and attribution mapping of 100,000 climate impact studies (2021) · cites this paper as [40]

Cites

Flagged first, then by citations

2 of the 84 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
Müller-Hansen, F., Callaghan, M. W., & Minx, J. C.
Affiliations
  • Leibniz Association (DE)
  • University of Leeds (GB)
Corresponding author
Müller-Hansen, Finn
ORCID
Venue
Energy Research & Social Science
Publisher
Elsevier BV
Published
2020-12
Volume
70
Pages
101691
ISSN
2214-6296 (ISSN-L)
DOI
10.1016/j.erss.2020.101691
OpenAlex
W3090090332
Cited by
32
References
84
Citations by year
2026: 2 · 2025: 4 · 2024: 10 · 2023: 4 · 2022: 4 · 2021: 8
FWCI
11.38
Type
journal article
Language
en
Open access
Licence
Locations
  • Energy Research & Social Science (publishedVersion)
  • Publication Database PIK (Potsdam Institute for Climate Impact Research (PIK)) (submittedVersion)
  • SocArXiv (OSF Preprints) (submittedVersion, cc-by-nc-nd)
  • SocArXiv (OSF Preprints) (submittedVersion)
Funders
  • Heinrich Böll Stiftung
  • Bundesministerium für Bildung und Forschung (03EK3046B)
Topics
  • Social Sciences › General Social Sciences › Computational and Text Analysis Methods
  • Social Sciences › Arts and Humanities › Literature and Literary Theory › Digital Humanities and Scholarship
  • Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
Keywords
computational text analysis, code of practice, research transparency, reproducibility, model validation, big data, topic modeling, policy discourse, ethanol policy, Brazil, research rigor

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review