The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index; 7 of the 8 sources its claims cite were checked for support.
HierMem: Context Curation Over Context Scaling — Hierarchical Memory with Invariant Constraint Placement for Long-Horizon LLM Conversations
Open accessAbstract
The industry assumes that larger context windows are the solution to long-horizon conversational AI. This assumption is fundamentally flawed. Flooding a Large Language Model (LLM) with raw historical turns actively degrades reasoning through the "lost in the middle" effect, dilutes critical user constraints, and incurs massive compute costs. LLMs do not need to process every past interaction; they require only highly relevant information, curated intelligently like human recall. I present HierMem, an OS-inspired systems architecture that explicitly challenges the full-history paradigm. HierMem replaces brute-force context dumping with a four-level hierarchical memory (L0-L3), an invariant non-evictable constraint zone, and a lightweight stateless curator model. By separating memory orchestration from response generation, constraint salience is guaranteed by placement policy, while topical relevance is handled by precision routing. Evaluated on 15 synthetic benchmark conversations (50 turns each) across software engineering, finance, legal, and healthcare domains, HierMem achieves a 4.7x architectural compression ratio. In the finalized Architecture C run (Qwen2.5-14B main model), HierMem obtains the best aggregate judge score (8.461) and a 93.3% constraint survival rate. Relative to Raw LLM, HierMem improves quality by +1.553 judge points while reducing average session compute cost by 36.9%. This work demonstrates that structured curation, rather than context scaling, is the optimal path for reliable long-horizon assistants.
Overview
CoChat Summary
TL;DR
HierMem is an OS-inspired context-management architecture that replaces full-history prompting with a four-level hierarchical memory (L0–L3), a non-evictable invariant constraint zone, and a small stateless curator model that handles routing while a larger model generates responses. Across 15 synthetic 50-turn conversations in four domains, it reports the best aggregate judge score (8.461) and constraint survival (93.3%) at the lowest session cost, though the legal domain is an acknowledged exception.
Key findings
- On the finalized Architecture C run (Qwen2.5-14B main model), HierMem scored 8.461 judge points vs. 6.908 for Raw LLM, 7.148 for RAG Summary, and 6.439 for RAG; constraint survival was 0.933 vs. 0.740, 0.760, and 0.667 respectively.
- Relative to Raw LLM, HierMem improved quality by +1.553 judge points while reducing average session compute cost by 36.9% (session cost 0.334 vs. 0.529; cost/turn 0.0069 vs. 0.0106).
- The paper reports a 4.7x architectural compression ratio: by turn 50 a typical conversation accumulates nearly 80,000 tokens of raw history, Raw LLM hits a hard 32k KV-cache ceiling by turn 25, while HierMem holds a stable prompt payload of roughly 16k–23k tokens.
- Domain aggregation shows HierMem leading quality, survival, and cost in software engineering, finance, and healthcare, but in legal datasets Raw LLM was slightly higher on quality and survival while HierMem remained lower-cost.
Methods
- Benchmark: 15 synthetic conversations of 50 turns each (750 turns total) — software engineering (6), finance (4), legal (3), healthcare (2) — each with 10 checkpoints, scored with a fixed LLM-as-judge template and aggregated into trajectory-level and end-of-run metrics.
- Compared systems under an identical turn/checkpoint schedule: HierMem, Raw LLM (full history), RAG (single-stage embedding retrieval, no chunking/reranking/query rewriting), and RAG Summary (retrieval + rolling summary).
- Judge: Gemini 3.1 Pro with a fixed weighted rubric over constraint adherence (CA), response quality/accuracy (RQ), and conversational coherence (CC), plus anti-vagueness and domain-drift penalties. Cost is estimated from runtime telemetry as dimensionless relative units (token components plus a retrieval-operation proxy), not billed invoices.
Limitations
- Stated by the paper: results come from a single finalized runtime configuration (Architecture C with Qwen2.5-14B), so sensitivity across model families and context-window scales is not characterized; parts of cost accounting rely on modeled components where provider billing metadata is not uniformly exposed; LLM-as-judge evaluation remains sensitive to prompt template and judge-model identity; domain coverage is broad but imbalanced (heavier software representation than healthcare); the corpus is synthetic and controlled, so conclusions are benchmark-scoped; and full ablation tables are not included, so the marginal contribution of each runtime component cannot be isolated.
- Evident from the text: the manuscript is a non-peer-reviewed preprint (the front matter states results "should not be considered conclusive"), and the RAG/RAG Summary baselines are deliberately lightweight single-stage systems that the authors say should not be read as fully equipped retrieval stacks, which bounds how much of the reported margin is attributable to the architecture versus baseline strength.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
5 Valid · 0 Not valid · 1 Not checked · 96 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“the 'Lost in the Middle' phenomenon demonstrates that relevant information placed in middle positions of long prompts is severely underutilized [4].”
What the sources say
- Supportedcites [4] [4] Nelson F. Liu et al. Lost in the Middle: How Language Models Use Long Contexts. TACL , 2023. - source
The Introduction reports that performance significantly degrades when models must use relevant information in the middle of a long input context. Section 2.3 likewise describes a U-shaped performance curve and substantial drops in multi-document question-answering performance, supporting the paraphrase that middle-position information is underutilized. (LLM, conf=0.94)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“sparse-attention (Longformer, BigBird) [10, 11] and positional bias studies (ALiBi) [12] prove that while models can physically process long sequences, they struggle to maintain strict instruction salience across multi-turn conversational updates.”
What the sources say
- Partly supportedcites [10] [10] Iz Beltagy, Matthew E. Peters, and Arman Cohan. Longformer: The Long-Document Transformer. arXiv preprint arXiv:2004.05150 , 2020. - sourceExperimental — accuracy not yet measured
The Introduction (p. 2) describes Longformer as using sparse attention to process long sequences. The Attention Pattern section (p. 4) discusses local and global attention for document tasks, but does not test multi-turn conversation or show that models fail to maintain instruction salience. The claimed proof of that failure is unsupported by this source. (LLM, conf=0.88)
- Partly supportedcites [11] [11] Manzil Zaheer et al. Big Bird: Transformers for Longer Sequences. In NeurIPS , 2020. - sourceExperimental — accuracy not yet measured
The Abstract (p. 1) says BIGBIRD can handle sequences up to eight times longer than previously possible on similar hardware. Neither that passage nor the reported sparse-attention results address instruction salience or multi-turn conversational updates, so the cited paper does not support that part of the claim. (LLM, conf=0.87)
- Could not verifycites [12] [12] Ofir Press, Noah A. Smith, and Mike Lewis. Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation. In ICLR , 2022. - sourceExperimental — accuracy not yet measured
The Introduction (p. 1) reports how positional methods affect perplexity as input sequences lengthen; it does not test whether instructions remain salient. Searches for instruction salience and multi-turn conversation found no passage addressing conversational updates, so this paper does not establish the attributed conclusion. (LLM, conf=0.75)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“HierMem achieves a 4.7x architectural compression ratio.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The curator included old segments because they were structurally helpful, not because they were exact semantic matches.”
plainstates a causeSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“By turn 50, a typical benchmark conversation accumulates nearly 80,000 tokens of raw history.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
24 more claims
Create a free account to see all 102 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 49
- It may state two separate claims as one. 42
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 1
- [1][1] Patrick Lewis et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. In NeurIPS , 2020. -
Metadata
- Authors
- Doke, Y.
- Affiliations
- Progressive Education Society's Modern College of Engineering
- Corresponding author
- Doke, Yash
- ORCID
- Doke, Yash
- Venue
- Research Square
- Published
- 2026-06-17
- OpenAlex
- W7164894427
- Cited by
- 0
- References
- 0
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Locations
- Research Square (acceptedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- hierarchical memory, constraint preservation, memory management
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: HierMem: Context Curation Over Context Scaling — Hierarchical Memory with Invariant Constraint Placement for Long-Horizon LLM Conversations. https://app.cochat.ai/papers/10.21203/rs.3.rs-10055780/v1
- The paper
- Doke, Y. (2026). HierMem: Context Curation Over Context Scaling — Hierarchical Memory with Invariant Constraint Placement for Long-Horizon LLM Conversations. Research Square. https://doi.org/10.21203/rs.3.rs-10055780/v1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.