The DOI matches its DataCite record; 2 of the 2 references it cites were found in a registry or an index.
Mixing Mechanisms: How Language Models Retrieve Bound Entities In-Context
Abstract
A key component of in-context reasoning is the ability of language models (LMs) to bind entities for later retrieval. For example, an LM might represent "Ann loves pie" by binding "Ann" to "pie", allowing it to later retrieve "Ann" when asked "Who loves pie?" Prior research on short lists of bound entities found strong evidence that LMs implement such retrieval via a positional mechanism, where "Ann" is retrieved based on its position in context. In this work, we find that this mechanism generalizes poorly to more complex settings; as the number of bound entities in context increases, the positional mechanism becomes noisy and unreliable in middle positions. To compensate for this, we find that LMs supplement the positional mechanism with a lexical mechanism (retrieving "Ann" using its bound counterpart "pie") and a reflexive mechanism (retrieving "Ann" through a direct pointer). Through extensive experiments on nine models and ten binding tasks, we uncover a consistent pattern in how LMs mix these mechanisms to drive model behavior. We leverage these insights to develop a causal model combining all three mechanisms that estimates next token distributions with 95% agreement. Finally, we show that our model generalizes to substantially longer inputs of open-ended text interleaved with entity groups, further demonstrating the robustness of our findings in more natural settings. Overall, our study establishes a more complete picture of how LMs bind and retrieve entities in-context.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Limitations in compositionality, entity binding, long-context evidence use, and numerical grounding are therefore especially relevant, but are rarely examined within one unified setting [11][12][13][14][15][16][17][18].”
From Diagnosing the limits of large language models for structured scientific evidence extraction (2026) · cites this paper as [14]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Levy, M., Jacoby, A., & Goldberg, Y. (2024) · arXiv · doi:10.48550/arxiv.2402.14848 · Cited by 119
Dissecting Recall of Factual Associations in Auto-Regressive Language Models
Geva, M., Bastings, J., Filippova, K., & Globerson, A. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.751 · Cited by 53
Self-citation
Cited by
Diagnosing the limits of large language models for structured scientific evidence extraction
Tan, Z., D’Souza, J., & Auer, S. (2026) · International Journal on Digital Libraries · doi:10.1007/s00799-026-00449-9
Metadata
- Authors
- Gur-Arieh, Y., Geva, M., & Geiger, A.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- arXiv
- 2510.06182
- References
- 2
- Type
- Preprint
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Mixing Mechanisms: How Language Models Retrieve Bound Entities In-Context. https://app.cochat.ai/papers/10.48550/arxiv.2510.06182
- The paper
- Gur-Arieh, Y., Geva, M., & Geiger, A. (2025). Mixing Mechanisms: How Language Models Retrieve Bound Entities In-Context. arXiv. https://doi.org/10.48550/arxiv.2510.06182
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.