Cited by 62Digital Humanities and ScholarshipTopic ModelingComputational and Text Analysis Methods

Simple, Interpretable and Stable Method for Detecting Words with Usage Change across Corpora

Open access
Gonen, Hila, Jawahar, Ganesh, Seddah, Djamé, Goldberg, Yoav·2020·Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

Abstract

The problem of comparing two bodies of text and searching for words that differ in their usage between them arises often in digital humanities and computational social science.This is commonly approached by training word embeddings on each corpus, aligning the vector spaces, and looking for words whose cosine distance in the aligned space is large.However, these methods often require extensive filtering of the vocabulary to perform well, and-as we show in this work-result in unstable, and hence less reliable, results.We propose an alternative approach that does not use vector space alignment, and instead considers the neighbors of each word.The method is simple, interpretable and stable.We demonstrate its effectiveness in 9 different setups, considering different corpus splitting criteria (age, gender and profession of tweet authors, time of tweet) and different languages (English, French and Hebrew).

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2020.acl-main.51

Cites

46 of 46 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Some common correlation metrics were criticized: Kendall's tau could be misleading at the tail end of distributions and Spearman correlation has limitations for global rankings.

    “We highlight that some common correlation metrics were criticized, such as Kendall's tau, which could be misleading at the tail end of distributions [165] and Spearman correlation has limitations for global rankings [82].”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 47 references couldn’t be matched to a known paper.

Research works printed but not linked: 3

  1. Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. 2018b. Unsupervised Neural Ma- chine Translation. In 6th International Conference on Learning Representations, ICLR.
  2. Mario Giullianelli. 2019. Lexical semantic change analysis with contextualised word representations. Master's thesis, Institute for Logic, Language and Computation,, University of Amsterdam, July.
  3. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Cor- rado, and Jeff Dean. 2013b. Distributed representa- tions of words and phrases and their compositional- ity. In Advances in neural information processing systems.

Cited by

Sorted by citations

Authors
Gonen, H., Jawahar, G., Seddah, D., & Goldberg, Y.
Affiliations
  • Bar-Ilan University (IL)
  • Centre Inria de Paris (FR)
Venue
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Published
2020
Pages
538-555
ISSN
0736-587X (ISSN-L)
OpenAlex
W3035345764
Cited by
59
References
47
Citations by year
2026: 2 · 2025: 2 · 2024: 12 · 2023: 14 · 2022: 13 · 2021: 14 · 2020: 2
FWCI
109.42
Type
conference paper
Language
en
Open access
Locations
  • Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • HAL (submittedVersion, other-oa)
  • arXiv (Cornell University)
Funders
  • European Commission (HORIZON2020)
  • Agence Nationale de la Recherche (ANR-16, ANR-16-CE33-0021, ANR-15-CE38-0011)
  • Ministry of Science, Technology and Space
  • H2020 European Research Council (802774)
Topics
  • Social Sciences › Arts and Humanities › Literature and Literary Theory › Digital Humanities and Scholarship
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Social Sciences › General Social Sciences › Computational and Text Analysis Methods
Keywords
word usage change, corpus comparison, word embeddings, computational social science, digital humanities, multilingual text analysis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review