CoChat

GLTR: Statistical Detection and Visualization of Generated Text

Open access
Gehrmann, Sebastian, Strobelt, Hendrik, Rush, Alexander · 2019 · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations

The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.

Abstract

The rapid improvement of language models has raised the specter of abuse of text generation systems.This progress motivates the development of simple methods for detecting generated text that can be used by and explained to non-experts.We develop GLTR, a tool to support humans in detecting whether a text was generated by a model.GLTR applies a suite of baseline statistical methods that can detect generation artifacts across common sampling schemes.In a human-subjects study, we show that the annotation scheme provided by GLTR improves the human detection-rate of fake text from 54% to 72% without any prior training.GLTR is open-source and publicly deployed, and has already been widely used to detect generated outputs.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/p19-3019

Cites

20 of 20 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 20 references

Research works printed but not linked: 4

  1. EA Grechnikov, GG Gusev, AA Kustarev, and A Raigorodsky. 2009. Detection of artificial texts. RCDL2009 Proceedings. Petrozavodsk, pages 306- 308.
  2. James V. Grimaldi. 2018. U.s. investigating fake com- ments on net neutrality. The Wall Street Journal.
  3. Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher. 2017. Learned in translation: Con- textualized word vectors. In Advances in Neural In- formation Processing Systems, pages 6294-6305.
  4. Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word rep- resentations. arXiv preprint arXiv:1802.05365. · arXiv

Cited by

See all 496

Metadata

Authors
Gehrmann, S., Strobelt, H., & Rush, A.
Affiliations
  • Harvard University (US)
  • IBM (United States) (US)
Venue
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations
Published
2019
Pages
111-116
OpenAlex
W2949028947
Cited by
470
References
19
Citations by year
2026: 108 · 2025: 123 · 2024: 75 · 2023: 76 · 2022: 19 · 2021: 34 · 2020: 31 · 2019: 4
FWCI
8.24
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
National Science Foundation (1845664)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
statistical detection methods, sampling artifacts, manual annotation, large language models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: GLTR: Statistical Detection and Visualization of Generated Text. https://app.cochat.ai/papers/10.18653/v1/p19-3019
The paper
Gehrmann, S., Strobelt, H., & Rush, A. (2019). GLTR: Statistical Detection and Visualization of Generated Text. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. https://doi.org/10.18653/v1/p19-3019

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.