Who is GPT-3? An exploration of personality, values and demographics
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Language models such as GPT-3 have caused a furore in the research community.Some studies found that GPT-3 has some creative abilities and makes mistakes that are on par with human behaviour.This paper answers a related question: Who is GPT-3?We administered two validated measurement tools to GPT-3 to assess its personality, the values it holds and its selfreported demographics.Our results show that GPT-3 scores similarly to human samples in terms of personality and -when provided with a model response memory -in terms of the values it holds.We provide the first evidence of psychological assessment of the GPT-3 model and thereby add to our understanding of this language model.We close with suggestions for future research that moves social science closer to language models and vice versa.* Equal first-authorship contribution: authorship order for MM and NR was determined by a random number generator.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cognitive Reflection and Decision Making
Frederick, S. (2005) · Journal of Economic Perspectives · doi:10.1257/089533005775196732 · Cited by 5,256
Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment.
Tversky, A., & Kahneman, D. (1983) · Psychological Review · doi:10.1037/0033-295x.90.4.293 · Cited by 3,991
GPT-3: Its Nature, Scope, Limits, and Consequences
Floridi, L., & Chiriatti, M. (2020) · Minds and Machines · doi:10.1007/s11023-020-09548-1 · Cited by 2,519
Word embeddings quantify 100 years of gender and ethnic stereotypes
Garg, N., Schiebinger, L., Jurafsky, D., & Zou, J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1720347115 · Cited by 1,065
Creativity: Yesterday, today and tomorrow
GUILFORD, J. P. (1967) · The Journal of Creative Behavior · doi:10.1002/j.2162-6057.1967.tb00002.x · Cited by 1,008
Korngiebel, D. M., & Mooney, S. D. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00464-x · Cited by 259
GPT-NeoX-20B: An Open-Source Autoregressive Language Model
Black, S., Biderman, S., Hallahan, E., Anthony, Q., Gao, L., Golding, L., He, H., Leahy, C., McDonell, K., Phang, J., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.06745 · Cited by 67
Putting GPT-3's Creativity to the (Alternative Uses) Test
Stevenson, C., Smal, I., Baas, M., Grasman, R., & van der Maas, H. (2022) · arXiv · doi:10.48550/arxiv.2206.08932 · Cited by 51
Nye, M., Tessler, M. H., Tenenbaum, J. B., & Lake, B. M. (2021) · arXiv · doi:10.48550/arxiv.2107.02794 · Cited by 40
Cited by
Atari, M., Xue, M. J., Park, P. S., Blasi, D. E., & Henrich, J. (2023) · doi:10.31234/osf.io/5b26t · Cited by 151
Language Model Behavior: A Comprehensive Survey
Chang, T. A., & Bergen, B. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00492 · Cited by 105
Röttger, P., Hofmann, V., Pyatkin, V., Hinck, M., Kirk, H., Schuetze, H., & Hovy, D. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.816 · Cited by 89
Diminished diversity-of-thought in a standard large language model
Park, P. S., Schoenegger, P., & Zhu, C. (2024) · Behavior Research Methods · doi:10.3758/s13428-023-02307-x · Cited by 83
Hagendorff, T., Dasgupta, I., Binz, M., Chan, S. C. Y., Lampinen, A., Wang, J. X., Akata, Z., & Schulz, E. (2023) · arXiv · doi:10.48550/arxiv.2303.13988 · Cited by 68
Metadata
- Authors
- Miotto, M., Rossberg, N., & Kleinberg, B.
- Venue
- Proceedings of the Fifth Workshop on Natural Language Processing and Computational Social Science (NLP+CSS)
- Published
- 2022
- Pages
- 218-227
- OpenAlex
- W4385573216
- Cited by
- 87
- References
- 9
- FWCI
- 18.23
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Who is GPT-3? An exploration of personality, values and demographics. https://app.cochat.ai/papers/10.18653/v1/2022.nlpcss-1.24
- The paper
- Miotto, M., Rossberg, N., & Kleinberg, B. (2022). Who is GPT-3? An exploration of personality, values and demographics. Proceedings of the Fifth Workshop on Natural Language Processing and Computational Social Science (NLP+CSS). https://doi.org/10.18653/v1/2022.nlpcss-1.24
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.