Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 190

The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.

The great Transformer: Examining the role of large language models in the political economy of AI

Open access
Luitse, Dieuwertje, Denkena, Wiebke · 2021 · Big Data & Society

Abstract

In recent years, AI research has become more and more computationally demanding. In natural language processing (NLP), this tendency is reflected in the emergence of large language models (LLMs) like GPT-3. These powerful neural network-based models can be used for a range of NLP tasks and their language generation capacities have become so sophisticated that it can be very difficult to distinguish their outputs from human language. LLMs have raised concerns over their demonstrable biases, heavy environmental footprints, and future social ramifications. In December 2020, critical research on LLMs led Google to fire Timnit Gebru, co-lead of the company’s AI Ethics team, which sparked a major public controversy around LLMs and the growing corporate influence over AI research. This article explores the role LLMs play in the political economy of AI as infrastructural components for AI research and development. Retracing the technical developments that have led to the emergence of LLMs, we point out how they are intertwined with the business model of big tech companies and further shift power relations in their favour. This becomes visible through the Transformer, which is the underlying architecture of most LLMs today and started the race for ever bigger models when it was introduced by Google in 2017. Using the example of GPT-3, we shed light on recent corporate efforts to commodify LLMs through paid API access and exclusive licensing, raising questions around monopolization and dependency in a field that is increasingly divided by access to large-scale computing power.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1177/20539517211047734

Cites

21 of 21 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 21 references

Research works printed but not linked: 33

  1. 10.1145/3461702.3462563 · DOI
  2. Ahmed N, Wahed M (2020) The de-democratization of ai: deep learning and the compute divide in artificial intelligence research. arXiv: 2010.15581 . Available at: http://arxiv.org/abs/2010.15581. · arXiv
  3. Amoore L, 2020, Cloud Ethics: Algorithms and the Attributes of Ourselves and Others
  4. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv:1409.0473 [cs, stat] . Available at: http://arxiv.org/abs/1409.0473. · arXiv
  5. Benjamin R, 2019, Race After Technology: Abolitionist Tools for the New Jim Code
  6. Cardon D, 2018, Réseaux, 211, 173, 10.3917/res.211.0173 · DOI · Paper page
  7. Dai AM, Le QV (2015) Semi-supervised sequence learning. arXiv:1511.01432 [cs] . Available at: http://arxiv.org/abs/1511.01432. · arXiv
  8. 10.2307/j.ctvj4sxc6 · DOI · Paper page
  9. 10.7551/mitpress/9780262062749.001.0001 · DOI · Paper page
  10. Graves A (2013) Generating sequences with recurrent neural networks. arXiv:1308. 0850. Available at: https://arxiv.org/abs/1308.0850. · arXiv
  11. Hernández A, Amigó JM (2020) Differentiable programming and its applications to dynamical systems. arXiv:1912.08168 [cs, math] . Available at: http://arxiv.org/abs/1912.08168. · arXiv
  12. 10.2139/ssrn.3147971 · DOI
  13. 10.1093/jeclap/lpy020 · DOI · Paper page
  14. Krizhevsky A, 2012, 25th International Conference on Neural Information Processing Systems—Volume 1 (NIPS'12), 1097
  15. Lee K-F, 2018, AI Superpowers: China, Silicon Valley, and the New World Order
  16. 10.7551/mitpress/12122.001.0001 · DOI
  17. Minsky M, 1969, Perceptrons: An Introduction to Computational Geometry
  18. 10.2139/ssrn.3532937 · DOI
  19. 10.1177/0163443718818384 · DOI · Paper page
  20. Pascanu R, 2013, Proceedings of the 30th International Conference on International Conference on Machine Learning (ICML’13)— Volume 28, Atlanta, GA, USA, 1310
  21. 10.1177/1461444816661553 · DOI · Paper page
  22. 10.1080/1369118X.2016.1181195 · DOI
  23. Rieder B, 2020, Engines of Order: A Mechanology of Algorithmic Techniques
  24. 10.1080/2157930X.2020.1855825 · DOI
  25. Srnicek N, 2017, Platform Capitalism
  26. Srnicek N, 2018, Economics for the Many, 153
  27. Staab P, 2019, Digitaler Kapitalismus: Markt und Herrschaft in der Ökonomie der Unknappheit
  28. Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. arXiv:1409.3215 [cs] . Available at: http://arxiv.org/abs/1409.3215. · arXiv
  29. Vaswani A, Shazeer N, Parmar N, et al. (2017) Attention is all you need. arXiv:1706.03762 [cs] . Available at: http://arxiv.org/abs/1706.03762. · arXiv
  30. 10.4324/9781315627731-14 · doi:10.4324/9781315627731-14
  31. Wooldridge M, 2020, The Road to Conscious Machines: The Story of AI
  32. 10.18653/v1/D19-3041 · DOI
  33. Zuboff S, 2019, The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power

Cited by

See all 190

Metadata

Authors
Luitse, D., & Denkena, W.
Venue
Big Data & Society
Published
2021-07
Volume
8
Issue
2
ISSN
2053-9517 (ISSN-L)
OpenAlex
W3202773593
Cited by
190
References
21
FWCI
31.20
Type
journal article
Language
en
Open access
gold

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The great Transformer: Examining the role of large language models in the political economy of AI. https://app.cochat.ai/papers/10.1177/20539517211047734
The paper
Luitse, D., & Denkena, W. (2021). The great Transformer: Examining the role of large language models in the political economy of AI. Big Data & Society. https://doi.org/10.1177/20539517211047734

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.