CoChat
Sign Up
PreprintCited by 15Explainable Artificial Intelligence (XAI)Topic ModelingAdvanced Neural Network Applications

Finding Neurons in a Haystack: Case Studies with Sparse Probing

Open access
Gurnee, Wes, Nanda, Neel, Pauly, Matthew, Harvey, Katherine, Troitskii, Dmitrii, Bertsimas, Dimitris·2023·arXiv
DOI 10.48550/arxiv.2305.01610arXiv 2305.01610
Open this paper in CoChat — free accountOpen PDF

Abstract

Despite rapid adoption and deployment of large language models (LLMs), the internal computations of these models remain opaque and poorly understood. In this work, we seek to understand how high-level human-interpretable features are represented within the internal neuron activations of LLMs. We train $k$-sparse linear classifiers (probes) on these internal activations to predict the presence of features in the input; by varying the value of $k$ we study the sparsity of learned representations and how this varies with model scale. With $k=1$, we localize individual neurons which are highly relevant for a particular feature, and perform a number of case studies to illustrate general properties of LLMs. In particular, we show that early layers make use of sparse combinations of neurons to represent many features in superposition, that middle layers have seemingly dedicated neurons to represent higher-level contextual features, and that increasing scale causes representational sparsity to increase on average, but there are multiple types of scaling dynamics. In all, we probe for over 100 unique features comprising 10 different categories in 7 different models spanning 70 million to 6.9 billion parameters.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2305.01610

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Higher-level reasoning and conceptual encodings in LLMs can be found in intermediate and deeper layers.

    “Notably, these units are mostly located in intermediate and deeper layers, matching results from previous research that show higher-level reasoning and conceptual encodings in LLMs can be found in these layers [26-29].”

    From From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Task (2025) · cites this paper as [29]

    The Introduction (p. 2) says higher-level contextual and linguistic features are seemingly encoded predominantly in middle layers, supporting the claim about intermediate-layer conceptual encodings. It does not establish higher-level reasoning encodings or place those features in deeper layers; Appendix A.14 (p. 21) instead discusses late layers as potentially converting semantic features into output tokens. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Gurnee et al. 2023 claim to observe superposition in the wild, but their study was limited to a few neurons in a specific layer of a single model, focusing on neuron superposition.

    “Gurnee et al. (2023) claim to observe superposition in the wild, their study was limited to a few neurons in a specific layer of a single model, focusing on neuron superposition.”

    From Knowledge in Superposition: Unveiling the Failures of Lifelong Knowledge Editing for Large Language Models (2024) · cites this paper as Gurnee et al. 2023

    Section 5.1 (p. 7) calls its compound-word example neuron superposition observed “in the wild,” supporting the central attribution. But it reports examples for 21 compound words across layers 1–3, not just a few neurons in one layer. Section 4 (p. 5) also describes experiments across seven Pythia models, so the claimed single-model limitation is inaccurate. (LLM, conf=0.90)

    Experimental — accuracy not yet measured

  • Middle layers often contain neurons that represent higher-level contextual features.

    “research has shown that neurons across different layers tend to specialize, with middle layers often containing neurons that represent higher-level contextual features [26-29].”

    From From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Task (2025) · cites this paper as [29]

  • These 286 units were predominantly located in middle layers, approximately layers 12-22.

    “These 286 units were predominantly located in the middle layers of the model (approximately layers 12-22), consistent with the distribution of units encoding the agent's position (Fig. 3) and with previous studies reporting that higher-level conceptual processing in LLMs tends to emerge in intermediate layers [26-29].”

    From From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Task (2025) · cites this paper as [29]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

Cited by 15

Research works printed but not linked: 83

  1. [1][1] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence , 35(8):1798-1828, 2013. -
  2. [2][2] Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. Decaf: A deep convolutional activation feature for generic visual recognition. In International conference on machine learning , pages 647-655. PMLR, 2014. -
  3. [3][3] Yonatan Belinkov. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics , 48(1):207-219, 2022. -
  4. [4][4] Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, et al. Toy models of superposition. arXiv preprint arXiv:2209.10652 , 2022. - · arXiv
  5. [5][5] Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An introduction to circuits. Distill , 5(3):e00024-001, 2020. -
  6. [6][6] Alec Radford, Rafal Jozefowicz, and Ilya Sutskever. Learning to generate reviews and discovering sentiment. arXiv preprint arXiv:1704.01444 , 2017. - · arXiv
  7. [7][7] David Bau, Jun-Yan Zhu, Hendrik Strobelt, Agata Lapedriza, Bolei Zhou, and Antonio Torralba. Understanding the role of individual units in a deep neural network. Proceedings of the National Academy of Sciences , 2020. -
  8. [8][8] Gabriel Goh, Nick Cammarata, Chelsea Voss, Shan Carter, Michael Petrov, Ludwig Schubert, Alec Radford, and Chris Olah. Multimodal neurons in artificial neural networks. Distill , 6(3):e30, 2021. -
  9. [9][9] Nick Cammarata, Gabriel Goh, Shan Carter, Chelsea Voss, Ludwig Schubert, and Chris Olah. Curve circuits. Distill , 2021. https://distill.pub/2020/circuits/curve-circuits. -
  10. [11][11] Adam Scherlis, Kshitij Sachan, Adam S Jermyn, Joe Benton, and Buck Shlegeris. Polysemanticity and capacity in neural networks. arXiv preprint arXiv:2210.01892 , 2022. - · arXiv
  11. [12][12] Jesse Mu and Jacob Andreas. Compositional explanations of neurons. Advances in Neural Information Processing Systems , 33:17153-17163, 2020. -
  12. [13][13] Omer Antverg and Yonatan Belinkov. On the pitfalls of analyzing individual neurons in language models. arXiv preprint arXiv:2110.07483 , 2021. - · arXiv
  13. [14][14] J Hewitt and P Liang. Designing and interpreting probes with control tasks. Proceedings of the 2019 Con , 2019. -
  14. [15][15] Abhilasha Ravichander, Yonatan Belinkov, and Eduard Hovy. Probing the probing paradigm: Does probing accuracy entail task relevance? arXiv preprint arXiv:2005.00719 , 2020. - · arXiv
  15. [18][18] Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644 , 2016. - · arXiv
  16. [20][20] Ian Tenney, Dipanjan Das, and Ellie Pavlick. Bert rediscovers the classical nlp pipeline. arXiv preprint arXiv:1905.05950 , 2019. - · arXiv
  17. [21][21] Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R Bowman, Dipanjan Das, et al. What do you learn from context? probing for sentence structure in contextualized word representations. arXiv preprint arXiv:1905.06316 , 2019. - · arXiv
  18. [23][23] Xavier Suau, Luca Zappella, and Nicholas Apostoloff. Finding experts in transformer models. arXiv preprint arXiv:2005.07647 , 2020. - · arXiv
  19. [24][24] Hassan Sajjad, Nadir Durrani, and Fahim Dalvi. Neuron-level interpretation of deep nlp models: A survey. Transactions of the Association for Computational Linguistics , 10:1285-1303, 2022. -
  20. [25][25] Xiaozhi Wang, Kaiyue Wen, Zhengyan Zhang, Lei Hou, Zhiyuan Liu, and Juanzi Li. Finding skill neurons in pre-trained transformer-based language models. arXiv preprint arXiv:2211.07349 , 2022. - · arXiv
  21. [26][26] Nadir Durrani, Hassan Sajjad, Fahim Dalvi, and Yonatan Belinkov. Analyzing individual neurons in pre-trained language models. arXiv preprint arXiv:2010.02695 , 2020. - · arXiv
  22. [27][27] Fahim Dalvi, Nadir Durrani, Hassan Sajjad, Yonatan Belinkov, Anthony Bau, and James Glass. What is one grain of sand in the desert? analyzing individual neurons in deep nlp models. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 33, pages 6309-6317, 2019. -
  23. [28][28] Jonathan Donnelly and Adam Roegiest. On interpretability and feature representations: an analysis of the sentiment neuron. In Advances in Information Retrieval: 41st European Conference on IR Research, ECIR 2019, Cologne, Germany, April 14-18, 2019, Proceedings, Part I 41 , pages 795-802. Springer, 2019. -
  24. [29][29] Rowan Hall Maudslay and Ryan Cotterell. Do syntactic probes probe syntax? experiments with jabberwocky probing. arXiv preprint arXiv:2106.02559 , 2021. - · arXiv
  25. [30][30] Yanai Elazar, Shauli Ravfogel, Alon Jacovi, and Yoav Goldberg. Amnesic probing: Behavioral explanation with amnesic counterfactuals. Transactions of the Association for Computational Linguistics , 9:160-175, 2021. -
  26. [31][31] Lucas Torroba Hennigen, Adina Williams, and Ryan Cotterell. Intrinsic probing through dimension selection. arXiv preprint arXiv:2010.02812 , 2020. - · arXiv
  27. [32][32] Elena Voita and Ivan Titov. Information-theoretic probing with minimum description length. arXiv preprint arXiv:2003.12298 , 2020. - · arXiv
  28. [33][33] Tiago Pimentel, Josef Valvoda, Rowan Hall Maudslay, Ran Zmigrod, Adina Williams, and Ryan Cotterell. Informationtheoretic probing for linguistic structure. arXiv preprint arXiv:2004.03061 , 2020. - · arXiv
  29. [34][34] Steven Cao, Victor Sanh, and Alexander M Rush. Low-complexity probing via finding subnetworks. arXiv preprint arXiv:2104.03514 , 2021. - · arXiv
  30. [35][35] Anthony Bau, Yonatan Belinkov, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and James Glass. Identifying and controlling important neurons in neural machine translation. arXiv preprint arXiv:1811.01157 , 2018. - · arXiv
  31. [36][36] Kedar Dhamdhere, Mukund Sundararajan, and Qiqi Yan. How important is a neuron? arXiv preprint arXiv:1805.12233 , 2018. - · arXiv
  32. [37][37] Rana Ali Amjad, Kairen Liu, and Bernhard C Geiger. Understanding neural networks and individual neuron importance via information-ordered cumulative ablation. IEEE Transactions on Neural Networks and Learning Systems , 2021. -
  33. [38][38] Nicola De Cao, Leon Schmid, Dieuwke Hupkes, and Ivan Titov. Sparse interventions in language models with differentiable masking. arXiv preprint arXiv:2112.06837 , 2021. - · arXiv
  34. [39][39] Nadir Durrani, Fahim Dalvi, and Hassan Sajjad. Linguistic correlation analysis: Discovering salient neurons in deepnlp models. arXiv preprint arXiv:2206.13288 , 2022. - · arXiv
  35. [40][40] Nicholas Goldowsky-Dill, Chris MacLeod, Lucas Sato, and Aryaman Arora. Localizing model behavior with path patching. arXiv preprint arXiv:2304.05969 , 2023. - · arXiv
  36. [42][42] Stephen Casper, Tilman Rauker, Anson Ho, and Dylan Hadfield-Menell. Sok: Toward transparent ai: A survey on interpreting the inner structures of deep neural networks. In First IEEE Conference on Secure and Trustworthy Machine Learning . -
  37. [44][44] Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: a circuit for indirect object identification in gpt-2 small. arXiv preprint arXiv:2211.00593 , 2022. - · arXiv
  38. [45][45] Neel Nanda, Lawrence Chan, Tom Liberum, Jess Smith, and Jacob Steinhardt. Progress measures for grokking via mechanistic interpretability. arXiv preprint arXiv:2301.05217 , 2023. - · arXiv
  39. [46][46] Bilal Chughtai, Lawrence Chan, and Neel Nanda. A toy model of universality: Reverse engineering how networks learn group operations. arXiv preprint arXiv:2302.03025 , 2023. - · arXiv
  40. [48][48] Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. arXiv preprint arXiv:2012.14913 , 2020. - · arXiv
  41. [49][49] Mor Geva, Avi Caciularu, Kevin Ro Wang, and Yoav Goldberg. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. arXiv preprint arXiv:2203.14680 , 2022. - · arXiv
  42. [50][50] Guy Dar, Mor Geva, Ankit Gupta, and Jonathan Berant. Analyzing transformers in embedding space. arXiv preprint arXiv:2209.02535 , 2022. - · arXiv
  43. [51][51] Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. Linear algebraic structure of word senses, with applications to polysemy. Transactions of the Association for Computational Linguistics , 6:483-495, 2018. -
  44. [52][52] Adam S Jermyn, Nicholas Schiefer, and Evan Hubinger. Engineering monosemanticity in toy models. arXiv preprint arXiv:2211.09169 , 2022. - · arXiv
  45. [54][54] David L Donoho. Compressed sensing. IEEE Transactions on information theory , 52(4):1289-1306, 2006. -
  46. [55][55] Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. Advances in neural information processing systems , 29, 2016. -
  47. [56][56] Hyunjik Kim and Andriy Mnih. Disentangling by factorising. In International Conference on Machine Learning , pages 2649-2658. PMLR, 2018. -
  48. [57][57] Fred Rieke, David Warland, Rob de Ruyter Van Steveninck, and William Bialek. Spikes: exploring the neural code . MIT press, 1999. -
  49. [58][58] Bruno A Olshausen and David J Field. Sparse coding with an overcomplete basis set: A strategy employed by v1? Vision research , 37(23):3311-3325, 1997. -
  50. [59][59] Horace Barlow. Redundancy reduction revisited. Network: computation in neural systems , 12(3):241, 2001. -
  51. [60][60] R Quian Quiroga, Leila Reddy, Gabriel Kreiman, Christof Koch, and Itzhak Fried. Invariant visual representation by single neurons in the human brain. Nature , 435(7045):1102-1107, 2005. -
  52. [61][61] William E Vinje and Jack L Gallant. Sparse coding and decorrelation in primary visual cortex during natural vision. Science , 287(5456):1273-1276, 2000. -
  53. [62][62] Alexandre Pouget, Peter Dayan, and Richard Zemel. Information processing with population codes. Nature Reviews Neuroscience , 1(2):125-132, 2000. -
  54. [63][63] Stefano Panzeri, Jakob H Macke, Joachim Gross, and Christoph Kayser. Neural population coding: combining insights from microscopic and mass signals. Trends in cognitive sciences , 19(3):162-172, 2015. -
  55. [65][65] Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712 , 2023. - · arXiv
  56. [67][67] Andreas M Tillmann, Daniel Bienstock, Andrea Lodi, and Alexandra Schwartz. Cardinality minimization, constraints, and regularization: a survey. arXiv preprint arXiv:2106.09606 , 2021. - · arXiv
  57. [68][68] Brian C Ross. Mutual information between discrete and continuous data sets. PloS one , 9(2):e87357, 2014. -
  58. [69][69] Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems , 35:17359-17372, 2022. -
  59. [71][71] Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027 , 2020. - · arXiv
  60. [72][72] Tolga Bolukbasi, Adam Pearce, Ann Yuan, Andy Coenen, Emily Reif, Fernanda Viégas, and Martin Wattenberg. An interpretability illusion for bert. arXiv preprint arXiv:2104.07143 , 2021. - · arXiv
  61. [74][74] Bruno A Olshausen and David J Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature , 381(6583):607-609, 1996. -
  62. [75][75] Tomáš Hromádka, Michael R DeWeese, and Anthony M Zador. Sparse representation of sounds in the unanesthetized auditory cortex. PLoS biology , 6(1):e16, 2008. -
  63. [76][76] Brett A Johnson and Michael Leon. Modular representations of odorants in the glomerular layer of the rat olfactory bulb and the effects of stimulus concentration. Journal of Comparative Neurology , 422(4):496-509, 2000. -
  64. [77][77] Stefan Leutgeb, Jill K Leutgeb, Carol A Barnes, Edvard I Moser, Bruce L McNaughton, and May-Britt Moser. Independent codes for spatial and episodic memory in hippocampal neuronal ensembles. Science , 309(5734):619-623, 2005. -
  65. [78][78] Philipp Koehn. Europarl: A parallel corpus for statistical machine translation. In Proceedings of machine translation summit x: papers , pages 79-86, 2005. -
  66. [79][79] Eric J Michaud, Ziming Liu, Uzay Girit, and Max Tegmark. The quantization model of neural scaling. arXiv preprint arXiv:2303.13506 , 2023. - · arXiv
  67. [81][81] Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682 , 2022. - · arXiv
  68. [82][82] Joseph Carlsmith. Is power-seeking ai an existential risk? arXiv preprint arXiv:2206.13353 , 2022. - · arXiv
  69. [83][83] Leo Z Liu, Yizhong Wang, Jungo Kasai, Hannaneh Hajishirzi, and Noah A Smith. Probing across time: What does roberta know and when? arXiv preprint arXiv:2104.07885 , 2021. - · arXiv
  70. [84][84] Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. Discovering latent knowledge in language models without supervision. arXiv preprint arXiv:2212.03827 , 2022. - · arXiv
  71. [86][86] Albert Reuther, Jeremy Kepner, Chansup Byun, Siddharth Samsi, William Arcand, David Bestor, Bill Bergeron, Vijay Gadepally, Michael Houle, Matthew Hubbell, Michael Jones, Anna Klein, Lauren Milechin, Julia Mullen, Andrew Prout, Antonio Rosa, Charles Yee, and Peter Michaleas. Interactive supercomputing on 40,000 cores for machine learning and data analysis. In 2018 IEEE High Performance extreme Computing Conference (HPEC) , pages 1-6. IEEE, 2018. -
  72. [87][87] Uri Alon. Design principles of biological circuits. Biophysical Journal , 96(3):15a-15a, 2009. -
  73. [88][88] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems , 33:1877-1901, 2020. -
  74. [89][89] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311 , 2022. - · arXiv
  75. [91][91] Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. Advances in neural information processing systems , 31, 2018. -
  76. [92][92] Blair Bilodeau, Natasha Jaques, Pang Wei Koh, and Been Kim. Impossibility theorems for feature attribution. arXiv preprint arXiv:2212.11870 , 2022. - · arXiv
  77. [93][93] Peter Hase, Mohit Bansal, Been Kim, and Asma Ghandeharioun. Does localization inform editing? surprising differences in causality-based localization vs. knowledge editing in language models. arXiv preprint arXiv:2301.04213 , 2023. - · arXiv
  78. [94][94] Amir-Hossein Karimi, Krikamol Muandet, Simon Kornblith, Bernhard Schölkopf, and Been Kim. On the relationship between explanation and prediction: A causal view. arXiv preprint arXiv:2212.06925 , 2022. - · arXiv
  79. [102][102] Kenneth Li, Aspen K Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Emergent world representations: Exploring a sequence model trained on a synthetic task. arXiv preprint arXiv:2210.13382 , 2022. - · arXiv
  80. [105][105] Rajiv Mehrotra, Kameswara Rao Namuduri, and Nagarajan Ranganathan. Gabor filter-based edge detection. Pattern recognition , 25(12):1479-1494, 1992. -
  81. [106][106] Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. An overview of early vision in inceptionv1. Distill , 2020. https://distill.pub/2020/circuits/early-vision. -
  82. [107][107] Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415 , 2016. - · arXiv
  83. [108][108] Fahim Dalvi, Hassan Sajjad, Nadir Durrani, and Yonatan Belinkov. Analyzing redundancy in pretrained transformer models. arXiv preprint arXiv:2004.04010 , 2020. · arXiv

Authors
Gurnee, W., Nanda, N., Pauly, M., Harvey, K., Troitskii, D., & Bertsimas, D.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2305.01610
arXiv
2305.01610
OpenAlex
W4368304611
Cited by
15
References
0
Citations by year
2025: 5 · 2024: 6 · 2023: 4
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
large language models, interpretability, internal activations

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review