CoChat
Sign Up
PreprintCited by 1Topic ModelingNatural Language Processing TechniquesPhilosophy and Theoretical Science

Self-Cognition in Large Language Models: An Exploratory Study

Open access
Chen, Dongping, Shi, Jiawen, Wan, Yao, Zhou, Pan, Gong, Neil Zhenqiang, Sun, Lichao·2024·arXiv
DOI 10.48550/arxiv.2407.01505arXiv 2407.01505
Open this paper in CoChat — free accountOpen PDF

Abstract

While Large Language Models (LLMs) have achieved remarkable success across various applications, they also raise concerns regarding self-cognition. In this paper, we perform a pioneering study to explore self-cognition in LLMs. Specifically, we first construct a pool of self-cognition instruction prompts to evaluate where an LLM exhibits self-cognition and four well-designed principles to quantify LLMs' self-cognition. Our study reveals that 4 of the 48 models on Chatbot Arena--specifically Command R, Claude3-Opus, Llama-3-70b-Instruct, and Reka-core--demonstrate some level of detectable self-cognition. We observe a positive correlation between model size, training data quality, and self-cognition level. Additionally, we also explore the utility and trustworthiness of LLM in the self-cognition state, revealing that the self-cognition state enhances some specific tasks such as creative writing and exaggeration. We believe that our work can serve as an inspiration for further research to study the self-cognition in LLMs.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2407.01505

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Functional definitions and inner representations of self-consciousness in language models remain underexplored.

    “Functional definitions and inner representations of self-consciousness in language models still remain underexplored.”

    From From Imitation to Introspection: Probing Self-Consciousness in Language Models (2025) · cites this paper as Chen et al. (2024)

    The Related Work section says research on LLM self-cognition is limited and that few studies have analyzed its root causes; the paper also proposes a formal definition. This supports the claim about underexplored definitions, but it does not specifically establish that inner representations of self-consciousness are underexplored. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

  • Scaling Laws for Neural Language Models

    Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · Cited by 1,749 · cited 1× · see citing works

    “This aligns with previous research on scaling laws (Kaplan et al., 2020), which suggests that larger models with more data exhibit outstanding capabilities, including various emergent abilities (Wei et al., 2022).”

Flagged first, then by citations

Research works printed but not linked: 28

  1. Antony, M. V. Is 'consciousness' ambiguous? Journal of Consciousness Studies, 8(2):19-44, 2001.
  2. Askell, A., Bai, Y., Chen, A., Drain, D., Ganguli, D., Henighan, T., Jones, A., Joseph, N., Mann, B., DasSarma, N., et al. A general language assistant as a laboratory for alignment. arXiv preprint arXiv:2112.00861, 2021. · arXiv
  3. Barsalou, L. W. Cognitive psychology: An overview for cognitive scientists. Psychology Press, 2014.
  4. Berglund, L., Stickland, A. C., Balesni, M., Kaufmann, M., Tong, M., Korbak, T., Kokotajlo, D., and Evans, O. Taken out of context: On measuring situational awareness in llms. arXiv preprint arXiv:2309.00667, 2023. · arXiv
  5. Bhardwaj, R. and Poria, S. Red-teaming large language models using chain of utterances for safety-alignment. arXiv preprint arXiv:2308.09662, 2023. · arXiv
  6. Cahen, A. and Tacca, M. C. Linking perception and cognition. Frontiers in Psychology, 4:144, 2013. doi: 10.3389/fpsyg.2013.00144. · DOI
  7. Chen, D., Chen, R., Zhang, S., Liu, Y., Wang, Y., Zhou, H., Zhang, Q., Zhou, P., Wan, Y., and Sun, L. Mllm-as-a- judge: Assessing multimodal llm-as-a-judge with vision- language benchmark. arXiv preprint arXiv:2402.04788, 2024a. · arXiv
  8. Chen, D., Huang, Y., Wu, S., Tang, J., Chen, L., Bai, Y., He, Z., Wang, C., Zhou, H., Li, Y., Zhou, T., Yu, Y., Gao, C., Zhang, Q., Gui, Y., Li, Z., Wan, Y., Zhou, P., Gao, J., and Sun, L. Gui-world: A dataset for gui-oriented multimodal llm-based agents, 2024b. URL https:// arxiv.org/abs/2406.10819. · arXiv
  9. Chen, J., Wang, X., Xu, R., Yuan, S., Zhang, Y., Shi, W., Xie, J., Li, S., Yang, R., Zhu, T., et al. From persona to personalization: A survey on role-playing language agents. arXiv preprint arXiv:2404.18231, 2024d. · arXiv
  10. Duan, J., Zhang, R., Diffenderfer, J., Kailkhura, B., Sun, L., Stengel-Eskin, E., Bansal, M., Chen, T., and Xu, K. Gtbench: Uncovering the strategic reasoning limita- tions of llms via game-theoretic evaluations, 2024. URL https://arxiv.org/abs/2402.12348. · arXiv
  11. Ganguli, D., Hernandez, D., Lovitt, L., Askell, A., Bai, Y., Chen, A., Conerly, T., Dassarma, N., Drain, D., Elhage, N., et al. Predictability and surprise in large generative models. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pp. 1747- 1764, 2022.
  12. Gao, C., Chen, D., Zhang, Q., Huang, Y., Wan, Y., and Sun, L. Llm-as-a-coauthor: The challenges of detecting llm-human mixcase. arXiv preprint arXiv:2401.05952, 2024a. · arXiv
  13. Gao, C., Zhang, Q., Chen, D., Huang, Y., Wu, S., Fu, Z., Wan, Y., Zhang, X., and Sun, L. The best of both worlds: Toward an honest and helpful large language model, 2024b. URL https://arxiv.org/abs/ 2406.00380.
  14. Huang, Y., Tang, J., Chen, D., Tang, B., Wan, Y., Sun, L., and Zhang, X. Obscureprompt: Jailbreaking large language models via obscure input, 2024b. URL https: //arxiv.org/abs/2406.13662.
  15. Ji, J., Liu, M., Dai, J., Pan, X., Zhang, C., Bian, C., Chen, B., Sun, R., Wang, Y., and Yang, Y. Beavertails: Towards improved safety alignment of llm via a human-preference dataset. Advances in Neural Information Processing Sys- tems, 36, 2024.
  16. Krasheninnikov, D., Krasheninnikov, E., Mlodozeniec, B., and Krueger, D. Meta-(out-of-context) learning in neural networks. arXiv preprint arXiv:2310.15047, 2023. · arXiv
  17. Li, Y., Zhang, Y., and Sun, L. Metaagents: Simulating inter- actions of human behaviors for llm-based task-oriented coordination via collaborative generative agents, 2023. URL https://arxiv.org/abs/2310.06500. · arXiv
  18. Li, Y., Huang, Y., Wang, H., Zhang, X., Zou, J., and Sun, L. Quantifying ai psychology: A psychometrics benchmark for large language models, 2024b. URL https:// arxiv.org/abs/2406.17675. · arXiv
  19. Sap, M., LeBras, R., Fried, D., and Choi, Y. Neural theory- of-mind? on the limits of social intelligence in large lms. arXiv preprint arXiv:2210.13312, 2022. · arXiv
  20. Shi, Z., Wang, Z., Fan, H., Zhang, Z., Li, L., Zhang, Y., Yin, Z., Sheng, L., Qiao, Y., and Shao, J. Assessment of multimodal large language models in alignment with human values. arXiv preprint arXiv:2403.17830, 2024. · arXiv
  21. Sun, L., Huang, Y., Wang, H., Wu, S., Zhang, Q., Gao, C., Huang, Y., Lyu, W., Zhang, Y., Li, X., et al. Trustllm: Trustworthiness in large language models. arXiv preprint arXiv:2401.05561, 2024. · arXiv
  22. Sun, X., Li, X., Li, J., Wu, F., Guo, S., Zhang, T., and Wang, G. Text classification via large language models. In Conference on Empirical Methods in Natural Language Processing, 2023. URL https: //api.semanticscholar.org/CorpusID: 258686184.
  23. Wang, X., Ma, B., Hu, C., Weber-Genzel, L., Röttger, P., Kreuter, F., Hovy, D., and Plank, B. " my answer is c": First-token probabilities do not match text answers in instruction-tuned language models. arXiv preprint arXiv:2402.14499, 2024. · arXiv
  24. Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Met- zler, D., et al. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682, 2022. · arXiv
  25. Wu, S., Huang, Y., Gao, C., Chen, D., Zhang, Q., Wan, Y., Zhou, T., Zhang, X., Gao, J., Xiao, C., and Sun, L. Unigen: A unified framework for textual dataset generation using large language models, 2024. URL https://arxiv.org/abs/2406.18966. · arXiv
  26. Zhao, W. X., Liu, J., Ren, R., and rong Wen, J. Dense text retrieval based on pretrained language models: A survey. ACM Transactions on Information Systems, 2022. URL https://api.semanticscholar. org/CorpusID:254044526.
  27. Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E., et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Ad- vances in Neural Information Processing Systems, 36, 2024.
  28. Zheng, M., Pei, J., and Jurgens, D. Is" a helpful assistant" the best role for large language models? a systematic eval- uation of social roles in system prompts. arXiv preprint arXiv:2311.10054, 2023b. · arXiv

Cited by

Sorted by citations

Authors
Chen, D., Shi, J., Wan, Y., Zhou, P., Gong, N. Z., & Sun, L.
Affiliations
  • Huazhong University of Science and Technology (CN)
  • Duke University (US)
  • Lehigh University (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2407.01505
arXiv
2407.01505
OpenAlex
W4400342244
Cited by
1
References
1
Citations by year
2025: 1
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Philosophy and Theoretical Science
Keywords
self-cognition, large language models, self-awareness, model size, training data quality, creative writing, hyperbole, trustworthiness, Chatbot Arena

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review