CoChat

Can ChatGPT accurately identify the position of parties? A validation study with an expert survey in France

Bol, Damien, Bono, Pierre-Henri · 2025 · Research & Politics
Cited by 5

The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.

Abstract

We evaluate whether ChatGPT can be used to estimate the ideological positions of parties in real time. Compared to other methods, ChatGPT can be used at all times, with minimal cost, no technical skills, and for all parties, including new ones. In a validation exercise, we asked ChatGPT and a group of local experts to simultaneously estimate the positions of French political parties in the context of the 2024 European election on a series of issues from the Chapel Hill Expert Survey. We find that ChatGPT’s estimations are generally close to those of the expert survey, especially for major and salient political issues (left-right, European integration, immigration). However, it provides less accurate estimates on fringe issues (free market, asylum policies, urban–rural interests) and specific party characteristics (within-party division, issue blurring). Thus, ChatGPT shows strong potential as an alternative data source for estimating parties’ positions but should be used with caution.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1177/20531680251335653

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 22

  1. 10.1017/S0007123414000155 · DOI · Paper page
  2. 10.1017/S0007123409000635 · DOI · Paper page
  3. 10.4135/9781473957978 · DOI
  4. 10.1093/oso/9780199244003.001.0001 · DOI · Paper page
  5. 10.1111/j.1475-6765.1984.tb00080.x · DOI · Paper page
  6. Downs A (1957) An Economic Theory of Democracy. Manhattan, NY: Harper & Brothers.
  7. 10.1177/1354068810372100 · DOI · Paper page
  8. 10.1080/19331681.2024.2444587 · DOI
  9. 10.1111/1467-9248.12015 · DOI · Paper page
  10. 10.1145/3600211.3604672 · DOI · Paper page
  11. 10.1111/j.1475-6765.2009.01912.x · DOI · Paper page
  12. 10.1016/j.electstud.2021.102420 · DOI · Paper page
  13. 10.1017/S104909652300077X · DOI
  14. 10.1016/j.electstud.2006.03.007 · DOI · Paper page
  15. 10.1017/S0007123403000322 · DOI · Paper page
  16. 10.1177/0010414020938081 · DOI · Paper page
  17. 10.1017/gov.2018.32 · DOI
  18. 10.4159/harvard.9780674429147 · DOI · Paper page
  19. 10.1017/pan.2021.16 · DOI
  20. Reinl A-K, Braun D (2023) European Election Studies 2019: Manifesto Project. Cologne: GESIS. ZA7891 Data file Version 1.0.0.
  21. 10.1111/1475-6765.12032 · DOI · Paper page
  22. 10.1177/1465116507073284 · DOI · Paper page

Cited by

See all 5

Metadata

Authors
Bol, D., & Bono, P.-H.
Affiliations
CEVIPOF, Sciences Po Paris
Venue
Research & Politics
Published
2025-04
Volume
12
Issue
2
ISSN
2053-1680
Cited by
5
References
8
Language
en
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.