xLSTM: Extended Long Short-Term Memory
Open accessThe DOI matches its Crossref record; 99 of the 101 references it cites were found in a registry or an index; 19 of the 35 sources its claims cite were checked for support.
Abstract
In the 1990s, the constant error carousel and gating were introduced as the central ideas of the Long Short-Term Memory (LSTM). Since then, LSTMs have stood the test of time and contributed to numerous deep learning success stories, in particular they constituted the first Large Language Models (LLMs). However, the advent of the Transformer technology with parallelizable self-attention at its core marked the dawn of a new era, outpacing LSTMs at scale. We now raise a simple question: How far do we get in language modeling when scaling LSTMs to billions of parameters, leveraging the latest techniques from modern LLMs, but mitigating known limitations of LSTMs? Firstly, we introduce exponential gating with appropriate normalization and stabilization techniques. Secondly, we modify the LSTM memory structure, obtaining: (i) sLSTM with a scalar memory, a scalar update, and new memory mixing, (ii) mLSTM that is fully parallelizable with a matrix memory and a covariance update rule. Integrating these LSTM extensions into residual block backbones yields xLSTM blocks that are then residually stacked into xLSTM architectures. Exponential gating and modified memory structures boost xLSTM capabilities to perform favorably when compared to state-of-the-art Transformers and State Space Models, both in performance and scaling.
Overview
CoChat Summary
TL;DR
xLSTM extends LSTM with exponential gating and new memory structures—sLSTM (scalar memory, memory mixing) and mLSTM (matrix memory, covariance update, parallelizable)—and shows favorable language-modeling performance versus Transformers and State Space Models. On 15B and 300B SlimPajama tokens, xLSTM variants achieve the best validation perplexity among compared methods and extrapolate better to longer contexts.
Key findings
- On 15B SlimPajama tokens, xLSTM[1:0] (409M params) reached 13.43 validation perplexity and xLSTM[7:1] (408M) 13.48, outperforming Mamba (423M, 13.70), Llama (407M, 14.25), GPT-3 (356M, 14.26), and other compared methods.
- On 300B SlimPajama tokens at 1.3B scale, xLSTM[1:0] reached 8.89 validation perplexity and xLSTM[7:1] 9.00, versus Mamba 9.14, Llama 9.44, and RWKV-4 9.83; at 16k context, xLSTM[7:1] reached 8.92 and xLSTM[1:0] 9.01, versus Mamba 14.00, RWKV-4 13.75, and Llama 337.83.
- On PALOMA, xLSTM[1:0] had lower perplexity than Mamba in 568/571 (99.5%) text domains, than Llama in 486/571 (85.1%), and than RWKV-4 in 570/571 (99.8%).
- Ablations attribute gains to both exponential gating and matrix memory: vanilla multi-layer LSTM had 2417.86 perplexity, adding ResNet backbone 35.46, up-projection backbone 26.01, exponential gating (xLSTM[0:1]) 17.70, and matrix memory (xLSTM[7:1]) 13.48.
Methods
- The paper introduces sLSTM with scalar memory, scalar update, exponential gating, normalizer/stabilizer states, and new memory mixing; and mLSTM with matrix memory, covariance (outer product) update rule, normalizer state, and full parallelizability.
- These are integrated into residual xLSTM blocks (post up-projection for sLSTM, pre up-projection for mLSTM) and residually stacked into xLSTM architectures; notation xLSTM[a:b] gives the ratio of mLSTM- to sLSTM-based blocks.
- Experiments include synthetic formal-language/state-tracking tasks, Multi-Query Associative Recall, Long Range Arena, and language modeling on SlimPajama with 15B and 300B tokens; main 300B comparisons use 125M, 350M, 760M, and 1.3B model sizes against RWKV-4, Llama, and Mamba.
Limitations
- The paper states: sLSTM memory mixing prevents parallelizable operations, though a fast CUDA kernel is less than two times slower than parallel mLSTM; mLSTM CUDA kernels are not optimized and are about 4 times slower than FlashAttention or Mamba’s scan; mLSTM’s d×d matrix memory has high computation complexity; forget-gate initialization must be chosen carefully; the matrix memory is independent of sequence length and may overload for longer contexts, though not up to 16k; and the architecture/hyperparameters were not fully optimized, especially for larger xLSTM models.
- Evident from the text: the main 300B-token LLM comparison excludes RWKV-5, RWKV-6, and HGRN2 because reasonable training precision was found only after the 300B experiments began; the largest main 300B models are 1.3B, with scaling-law plots extending to 2.7B in the appendix.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




![Figure 6 Method comparison on next token prediction when trained on 15B tokens from SlimPajama. Performance measure in validation perplexity for the best methods of each model class (see Table 1) are reported. The performance degradation of xLSTM[7:1] at 2.7B is due to initially slower training convergence that leads to an especially undertrained model. xLSTM is the best method at all sizes.](https://figures.cochat.ai/paper-figures/1c48bfb626c14c6795ef465de3ead032/xlstm-extended-long-short-term-memory-figure-6.png)


![Figure 9 Inference Generative Speed. Left: Generation times of different 1.3B models for a pre-fill context of 16 tokens (to mitigate cache initialization). The recurrent models (xLSTM[1:0], xLSTM[7:1], Mamba and RWKV-4) show linear behavior, whereas the Transformer (Llama) inference/decoding time is quadratic in sequence length. Right: Token throughput for different batch sizes on a A100-80GB GPU for 1.3B sized models. Note that the Transformer / Llama model goes out of memory (OOM) already for small batch sizes, whereas xLSTM and Mamba can sustain very large batch sizes. xLSTM[1:0] consistently outperforms Mamba in throughput.](https://figures.cochat.ai/paper-figures/f63789279b6e4ff4ba34314a9b59cd7d/xlstm-extended-long-short-term-memory-figure-9.png)






Verification
Claims the paper makes
10 Valid · 4 Not valid · 9 Not checked · 95 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This result is in agreement with findings that Transformers and State Space models are fundamentally less powerful than RNNs (Merrill et al., 2024;Merrill & Sabharwal, 2023;Delétang et al., 2023).”
§ xLSTM: Extended Long Short-Term Memory > 4.1 Synthetic Tasks and Long Range Arena
What the sources say
- Partly supportedcites Merrill et al., 2024 matched to “The Illusion of State in State-Space Models” (2024) sourceExperimental — accuracy not yet measured
The Introduction says that linear and Mamba-style SSMs, like transformers, cannot express certain sequential state-tracking problems that RNNs can. But the claim is broader than the paper’s result: the Discussion (§4.4) limits the separation to specified SSM architectures and assumptions, and the Conclusion (§7) describes an input-dependent SSM that can express the S5 word problem. (LLM, conf=0.87)
- Partly supportedcites Merrill & Sabharwal, 2023 matched to “The Parallelism Tradeoff: Limitations of Log-Precision Transformers” (2023) sourceExperimental — accuracy not yet measured
The Abstract and §2 support a computational limitation for log-precision Transformers, linking it to their parallelizability. They do not establish that State Space models are less powerful than RNNs, or make that comparison for Transformers; the paper’s result is bounded by its precision and model assumptions. The regular-grammar result is the citing authors’ own framing, not a finding attributed to this source. (LLM, conf=0.86)
- Not matchedcites Delétang et al., 2023
An author–year citation the paper’s reference list doesn’t settle. We don’t search these by name.
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Especially its quality of in-context learning seems to be strongly connected to this capability (Olsson et al., 2022).”
§ xLSTM: Extended Long Short-Term Memory > Why Multi-Query Associative Recall for Memory Tests of LLM Architectures.
What the sources say
- Supportedcites Olsson et al., 2022 matched to “In-context Learning and Induction Heads” (2022) source
The Induction Heads section (p. 5) describes retrieving a token associated with an earlier occurrence of the current token—the capability described here as associative recall. Assessing the Evidence (p. 15) reports that the formation of these heads correlates with a large increase in in-context-learning capacity, while cautioning that causation is not established. The claim says the capabilities seem connected, not that one causes the other. (LLM, conf=0.85)
boostedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In accordance to the results of Arora et al. (2023) H3, Hyena, RWKV-4 fail to solve the task with a smaller model dimension.”
§ xLSTM: Extended Long Short-Term Memory > Why Multi-Query Associative Recall for Memory Tests of LLM Architectures.
What the sources say
- Partly supportedcites Arora et al. (2023) matched to “Zoology: Measuring and Improving Recall in Efficient Language Models” (2023) sourceExperimental — accuracy not yet measured
The Setup section reports that the tested gated-convolution architectures—H3, Hyena, and RWKV—did not reach MQAR accuracy above 0.9 unless model dimension was at least the sequence length. Our copy calls the tested model RWKV, not specifically RWKV-4, so it does not establish the version named in the claim. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Associative Recall (AR), the ability to retrieve a specific value (information) associated with a given key (information), constitutes a key capability for LLM to perform well (Poli et al., 2024;Arora et al., 2023;Olsson et al., 2022).”
§ xLSTM: Extended Long Short-Term Memory > Why Multi-Query Associative Recall for Memory Tests of LLM Architectures.
What the sources say
- Supportedcites Poli et al., 2024 matched to “Mechanistic Design and Scaling of Hybrid Architectures” (2024) source
Section 3.1.1 'In-context recall' states that 'To answer a prompt well, language models must be able to understand and learn from new information presented in the prompt', and that 'the associative recall task ... is well-suited to test a specific subset of in-context learning ability: direct lookup', defined as 'Given an input sequence of key-value pairs, models are tasked with retrieving all values from the input sequence associated with keys that were already shown in the input sequence.' Appendix B.1.1 repeats the key–value retrieval definition. The claim's definition of AR and its framing as a capability required for good prompt-level performance therefore match what our copy attributes, with the minor nuance that the paper treats AR as a probe of a subset (direct lookup) of in-context learning ability. (LLM, conf=0.85)
- Supportedcites Arora et al., 2023 matched to “Zoology: Measuring and Improving Recall in Efficient Language Models” (2023) source
Section 3.1 defines associative recall as recalling an association previously seen in context, illustrated by retrieving “Rice” from “Tim.” The Introduction says this ability is predictive of in-context learning quality, and Section 3 reports that it accounts for 82% of the measured perplexity gap between the model architectures studied. Together, these support the claim that associative recall is an important capability for language-model performance. (LLM, conf=0.91)
- Partly supportedcites Olsson et al., 2022 matched to “In-context Learning and Induction Heads” (2022) sourceExperimental — accuracy not yet measured
The Induction Heads section (p. 5) describes retrieving a token B previously associated with token A, matching the claim’s description of associative recall. The paper presents the broader claim that this mechanism plays a major role in in-context learning as a preliminary hypothesis (p. 2); it does not establish associative recall as a general requirement for LLMs to perform well. (LLM, conf=0.82)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The results show that, in contrast to other methods (Katharopoulos et al., 2020;Sun et al., 2023;Qin et al., 2023;Katsch, 2023;Yang et al., 2023;Qin et al., 2024;Peng et al., 2024), having the gates both learnable and input dependent gives the best results.”
§ xLSTM: Extended Long Short-Term Memory > Additional information
What the sources say
- Partly supportedcites Katharopoulos et al., 2020 matched to “Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention” (2020) sourceExperimental — accuracy not yet measured
Section 5 (p. 8) identifies this paper’s method as a linear transformer, so it can serve as an example of an ‘other method.’ Its reported results compare linear attention with other transformer approaches, not learnable, input-dependent gates or mLSTM. The claimed best results for mLSTM are therefore not supported by this source. (LLM, conf=0.78)
- Supportedcites Sun et al., 2023 matched to “Retentive Network: A Successor to Transformer for Large Language Models” (2023) source
The citation identifies RetNet as a contrasting method, rather than attributing the mLSTM results to it. Section 2.2, “Gated Multi-Scale Retention” (page unavailable), says RetNet keeps its decay rates fixed, even though it also uses a learned, input-dependent swish gate. The paper does not report the mLSTM result; that result is presented as the citing authors’ own finding. (LLM, conf=0.85)
- Partly supportedcites Qin et al., 2023 matched to “Hierarchically Gated Recurrent Neural Network for Sequence Modeling” (2023) sourceExperimental — accuracy not yet measured
The Results (§4.2, page unavailable) report that HGRN performs worse with a data-independent forget gate and benefits from combining gating with a learnable lower bound. That supports the direction of the gate comparison, but these are HGRN results, not a finding that learnable, input-dependent gates give the best results for mLSTM. (LLM, conf=0.87)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
59 more claims
Create a free account to see all 118 claims and every source verdict.
- Probably not a substantial claim. 30
- It may state two separate claims as one. 33
- The quote only partly states this claim. 1
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ xLSTM: Extended Long Short-Term Memory > General Details on Ablation Studies.
“Mamba 793.2 182.74 22.58 10.47 36.25”
§ xLSTM: Extended Long Short-Term Memory > 4.3 xLSTM as Large Language Model
“Mamba 870.5 10.24 9.24 50.84 53.97 71.16 60.44 29.78 56.99 53.86”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ xLSTM: Extended Long Short-Term Memory > 4.3 xLSTM as Large Language Model
“Model sizes are 125M, 250M, 760M, and 1.3B.”
§ xLSTM: Extended Long Short-Term Memory > 4.3 xLSTM as Large Language Model
“We train different model sizes (125M, 350M, 760M, 1.3B)”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ xLSTM: Extended Long Short-Term Memory > A.2 sLSTM
“δ b = sign (n) ⊙ δ n ⊙ 1 if |b| > exp(-m) 0 otherwise (90”
§ xLSTM: Extended Long Short-Term Memory > A.2 sLSTM
“n = max (|b|, exp(-m))(84)”
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Part of this paper’s reference list could not be split into single entries.
Research works printed but not linked: 18
- J. Ba, G. E. Hinton, V. Mnih, J. Z. Leibo, and C. Ionescu. Using fast weights to attend to the recent past. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett (eds.), Advances in Neural Information Processing Systems 29, pp. 4331-4339.
- T. Brown, B. Mann, N. Ryder, et al. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 1877-1901.
- T. M. Cover. Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition. Electronic Computers, IEEE Transactions on, EC-14(3):326-334, 1965.
- T. Dao. Flashattention-2: Faster attention with better parallelism and work partitioning. In In- ternational Conference on Learning Representations (ICLR), volume 12, 2024. URL https: //openreview.net/forum?id=mZn2Xyh9Ec.
- S. Hochreiter. Untersuchungen zu dynamischen neuronalen Netzen. Master's thesis, Technische Universität München, 1991.
- S. Hochreiter and J. Schmidhuber. LSTM can solve hard long time lag problems. In M. C. Mozer, M. I. Jordan, and T. Petsche (eds.), Advances in Neural Information Processing Systems (NeurIPS), volume 9, pp. 473-479. MIT Press, Cambridge MA, 1997b.
- S. Hochreiter, Y. Bengio, P. Frasconi, and J. Schmidhuber. Gradient flow in recurrent nets: the difficulty of learning long-term dependencies. In J. Kolen and S. Kremer (eds.), A Field Guide to Dynamical Recurrent Networks. IEEE, 2000.
- A. Krizhevsky. Learning multiple layers of features from tiny images. Master's thesis, Deptartment of Computer Science, University of Toronto, 2009.
- D. Krotov and J. J. Hopfield. Dense associative memory for pattern recognition. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett (eds.), Advances in Neural Information Processing Systems, pp. 1172-1180.
- I. Loshchilov and F. Hutter. Decoupled weight decay regularization. In International Confer- ence on Learning Representations (ICLR), 2019. URL https://openreview.net/forum?id= Bkg6RiCqY7.
- A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts. Learning word vectors for sentiment analysis. In Annual Meeting of the Association for Computational Linguistics, volume 49, pp. 142-150, 2011.
- I. Schlag, K. Irie, and J. Schmidhuber. Linear transformers are secretly fast weight programmers. In M. Meila and T. Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning (ICML), volume 139 of Proceedings of Machine Learning Research, pp. 9355-9366.
- PMLR, 2021.
- J. Schmidhuber. Learning to control fast-weight memories: An alternative to recurrent nets. Neural Computation, 4(1):131-139, 1992.
- L. Soldaini, R. Kinney, A. Bhagia, et al. Dolma: an open corpus of three trillion tokens for language model pretraining research. ArXiv, 2306.01116, 2023. · arXiv
- R. K. Srivastava, K. Greff, and J. Schmidhuber. Training very deep networks. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (eds.), Advances in Neural Information Processing Systems (NeurIPS), volume 28.
- I. Sutskever, O. Vinyals, and Q. V. V. Le. Sequence to sequence learning with neural networks. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger (eds.), Advances in Neural Information Processing Systems 27 (NIPS'13), pp. 3104-3112.
- A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), volume 30, pp. 5998-6008.
Cited by
RULER: What's the Real Context Size of Your Long-Context Language Models?
Hsieh, C.-P., Sun, S., Kriman, S., Acharya, S., Rekesh, D., Jia, F., Zhang, Y., & Ginsburg, B. (2024) · arXiv · doi:10.48550/arxiv.2404.06654 · Cited by 1,094
A review of large language models and autonomous agents in chemistry
Ramos, M. C., Collison, C. J., & White, A. D. (2025) · Chemical Science · doi:10.1039/d4sc03921a · Cited by 274
Wang, F., Zhang, Z., Zhang, X., Wu, Z., Mo, T., Lu, Q., Wang, W., Li, R., Xu, J., Tang, X., et al. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3768165 · Cited by 106
Remaining useful life prediction of PEMFC based on matrix long short-term memory
Yi, F., Shu, X., Zhou, J., Zhang, J., Feng, C., Gong, H., Zhang, C., & Yu, W. (2025) · International Journal of Hydrogen Energy · doi:10.1016/j.ijhydene.2025.02.302 · Cited by 73
Nested Learning: The Illusion of Deep Learning Architectures
Behrouz, A., Razaviyayn, M., Zhong, P., & Mirrokni, V. (2025) · arXiv · doi:10.48550/arxiv.2512.24695 · Cited by 37
Transformer-based sensor fusion for autonomous vehicles: A comprehensive review
Abdulmaksoud, A., & Ahmed, R. (2025) · IEEE Access · doi:10.1109/access.2025.3545032 · Cited by 25
xLSTM 7B: A Recurrent LLM for Fast and Efficient Inference
Beck, M., Pöppel, K., Lippe, P., Kurle, R., Blies, P. M., Klambauer, G., Böck, S., & Hochreiter, S. (2025) · arXiv · doi:10.48550/arxiv.2503.13427 · Cited by 9
Self-citationxLSTMAD: A Powerful xLSTM-based Method for Anomaly Detection
Faber, K., Pietroń, M., Żurek, D., & Corizzo, R. (2025) · Proceedings · doi:10.1109/icdm65498.2025.00032 · Cited by 20
O’Shea, K., Yan, S., Yu, M., Chen, X., Mauceri, S., Dhariyal, B., Xu, L., O'Connor, N. E., & Liu, M. (2025) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2025.3634737 · Cited by 5
Shao, X., Xu, L., Zheng, T., et al. (2025) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2025.3641554 · Cited by 3
Li, T., Chen, J., Liu, T., et al. (2025) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2025.3643052 · Cited by 2
Odiaga, G. A., Masinde, N. W., & Yoga, C. A. (2025) · Journal of Soft Computing and Artificial Intelligence · doi:10.55195/jscai.1820478 · Cited by 5
Resilient Cable Rating for Overplanted Wind Farms Tackling Data Imbalance and Aging Risks
Yan, S., Karmokar, T., Niasar, M. G., et al. (2026) · IEEE Transactions on Power Delivery · doi:10.1109/tpwrd.2026.3670695 · Cited by 2
Nikandish, G., He, J., & Haghi, B. (2026) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2026.3669719 · Cited by 5
New Machine Learning Approaches for Intrusion Detection in ADS-B
Ngamboé, M., Marrocco, J.-S., Ouattara, J.-Y. S., Fernandez, J. M., & Nicolescu, G. (2025) · IEEE/AIAA Digital Avionics Systems Conference (DASC) · doi:10.1109/dasc66011.2025.11257260 · Cited by 3
Luo, F., Li, A., Huang, Z., Wu, K., Jiang, B., Sun, Z., & Wang, L. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3653448 · Cited by 1
Xie, Y. (2026) · International Journal on Semantic Web and Information Systems · doi:10.4018/ijswis.402041 · Cited by 2
Multi-Function Radar Work Mode Recognition Based on Extended Long Short-Term Memory Network
Ni, Z., Wang, M., Yu, X., Cui, G., & Kong, L. (2025) · International Conference on Vision, Image and Signal Processing (ICVISP) · doi:10.1109/icvisp68610.2025.11451701 · Cited by 1
Temporally-Preserving Quantum LSTM With Multiscale Attention Mechanisms for Soft Sensor Applications
Meng, X.-B., Guo, B.-C., Xu, K., et al. (2026) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2026.3680978 · Cited by 1
“Specifically, when comparing methods with similar network architectures, such as QLSTM [29], its recent variant xLSTM [28] and BLS-QLSTM [37], our method can achieve performance improvements ranging from 9.46% to 74.07% across three metrics.”
InfluentialDiptho, R. A., Ghosh, P., & Basak, S. (2025) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccit68739.2025.11491345 · Cited by 1
Thermal-GEMs: Generalized Models for Building Thermal Dynamics
Koch, F., Raisch, F., & Tischler, B. (2026) · ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation (BuildSys) · doi:10.1145/3744256.3812565 · Cited by 4
Dynamic Evolution: Continuous Change Monitoring in Time-Series Remote Sensing Images
Li, J., Sun, L., Wu, C., Du, B., & Zhang, L. (2026) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2026.3717221 · Cited by 1
Breast Cancer Postoperative Rehabilitation Nursing Action Recognition Based on Millimeter-Wave Radar
Hu, X.-Y., Hu, P., Shan, S., Liang, A., Zhang, H.-X., Wang, Y.-X., & Wang, F. (2026) · IEEE Sensors Journal · doi:10.1109/jsen.2025.3630323
Research on Indoor Visible Light Positioning Algorithm Based on Improved xLSTM Neural Network
Han, F., Qin, L., Shi, M., et al. (2025) · International Conference on Electronic Information Technology (EIT) · doi:10.1109/eit67313.2025.11232326
Demirbilek, B. H., Demіr, A., & Polat, F. (2025) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai66417.2025.00115 · Cited by 1
Combining Spatial and Temporal Network for Video Anomaly Detection
Liang, Z. (2025) · International Conference on Computer Vision and Data Mining (ICCVDM) · doi:10.1109/iccvdm66874.2025.11290667
Wu, X., Ren, J., Liu, Y., Fan, B., Wang, J., & Bo, Y. (2025) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2025.3649106
TriM-DTA: a Tri-Modal Fusion Framework for Drug-Target Binding Affinity Prediction
Zhu, H.-W., Yi, H.-C., & You, Z.-H. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11357191
Cai, J., Lou, X., Yan, J., et al. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11356227
WmLSTM: A Plug-and-Play Window-Level mLSTM-Based Temporal Encoder for Robust Visual Tracking
Chen, Y., Jia, G., Ma, D., Zha, Y., & Zhang, P. (2026) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2026.3662479
Pandagale, A. A., & Pandagale, L. V. P. A. A. (2026) · Statistical Applications in Genetics and Molecular Biology · doi:10.1515/sagmb-2025-0042
A Multi-Scale Spatio-Temporal Model for Eletric Load Forecasting
Tian, M., Ji, R., Xin, X., et al. (2025) · International Conference on Smart Power & Internet Energy Systems (SPIES) · doi:10.1109/spies67451.2025.11381496
Zhang, H., Tang, Z., Xie, Y., Zheng, Z., & Gui, W. (2026) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2026.3666051
da Silva, M. G. L., Vieira, J. R. D. S., Bezerra, B. L. D., & Oliveira, S. C. (2026) · IEEE Transactions on Power Delivery · doi:10.1109/tpwrd.2026.3673645
Exploring Step-Structure Adaptability of xLSTM Variants for Multi-Horizon Stock Price Prediction
Zheng, D. (2025) · IEEE International Conference on Electrical, Automation and Computer Engineering (ICEACE) · doi:10.1109/iceace67491.2025.11439315
Semantically Enhanced Image Captioning for Transmission Line Ice
Xiang, W., Yajie, W., Shuxian, F., et al. (2025) · International Conference on Computer and Communications (ICCC) · doi:10.1109/iccc68654.2025.11437783
Structural and dynamical strategies to prevent runaway excitation in reservoir computing
Metzner, C., Schilling, A., Maier, A., Kinfe, T. M., & Krauß, P. (2026) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2026.1845838
“…and hierarchical gated networks use task-trained, unit-or channel-specific gates to route and retain information (Hochreiter and Schmidhuber, 1997; Qin et al., 2023; Beck et al., 2024); our controller observes only one permutation-invariant population statistic and returns one global scalar.”
Li, Q., Li, G., Wang, M., et al. (2026) · Journal of Intelligent Manufacturing and Special Equipment · doi:10.1108/jimse-11-2025-0024
Improving Trust in Fact-Checking Systems with Synthetic Training Data and Explanations
Bussotti, J.-F. (2024) · theses.fr (ABES) · doi:10.70675/6f25f951z9b70z4941z86e9zaaa746e7effe
Online Continuous Image Classification with Memory-based Methods : Application to YouTube data
Michel, N. (2024) · theses.fr (ABES) · doi:10.70675/9afc967az785ez438dzb3fcz74648d8f9fbd
RKA-Mamba: A Lightweight Radar-Based Keystroke Authentication Using State-Space Model
Ma, J., Hong, Z., He, Y., et al. (2026) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2026.3684637
LGF-Net: A Local-Global Feature Learning Framework for Intrusion Detection
Liu, H., Zhao, M., & Liu, J. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463216
Jin, J., Yun, B.-S., Lin, Z., et al. (2026) · Expert systems with applications · doi:10.1016/j.eswa.2026.133986
Yardımcı, F., Ișık, M. S., Ertürk, A., & Erten, E. (2026) · Advances in Space Research · doi:10.1016/j.asr.2026.05.025
Research on supply chain management and ordering strategies for perishable goods in farmers' markets
Li, Y., Li, Z., Xia, L., Sun, J., Li, J., & Xu, Y. (2026) · RAIRO - Operations Research · doi:10.1051/ro/2026066 · Cited by 1
Hou, Z., Ma, J., & Hu, X. (2026) · Machines · doi:10.3390/machines14080948
Benchmarking Transformer and xLSTM for Time-Series Forecasting of Heat Consumption
Wahl, M., Bayer, D., Rausch, S., & Pruckner, M. (2026) · IEEE Conference on Technologies for Sustainability · doi:10.1109/sustech67720.2026.11536443 · Cited by 1
Zhu, L., Lin, Y., & Yang, S. (2026) · Interdisciplinary Sciences Computational Life Sciences · doi:10.1007/s12539-026-00849-0 · Cited by 1
Zhang, L., & Zhang, Y. (2026) · IEEE Sensors Letters · doi:10.1109/lsens.2026.3702086
Chou, J., Wang, F., & Hidayat, A. N. (2026) · IEEE Transactions on Transportation Electrification · doi:10.1109/tte.2026.3683650 · Cited by 1
Reliable Indoor Air Quality Prediction with Uncertainty-Aware Spatiotemporal GNN
Huang, C., Cheng, J. C. P., Hou, F., Wu, Z., & Kwok, H. L. (2026) · ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation (BuildSys) · doi:10.1145/3744256.3812590
InfiNAS: Architecture Search of Infinite-Dimensional Neural Networks
Chen, J. K., Li, M., Zheng, Z., Liu, C., Zhao, W. S., & Chen, X. (2026) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas66217.2026.11562843
Jin, H.-J., Sun, R.-C., Zhao, Y.-P., et al. (2026) · Thin-Walled Structures · doi:10.1016/j.tws.2026.115306
Physics-Enhanced Multistage Tornado Prediction Network Using Doppler Radar Data
Xie, J., Zhou, K., Han, L., Zhang, X., & Zheng, Y. (2026) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2026.3705532
Farbo, A., Parsons, D., Borgogno-Mondino, E., et al. (2026) · Remote Sensing of Environment · doi:10.1016/j.rse.2026.115545
An AI-driven Decision Support System for Air Quality Forecasting with Adaptive Model SelectionPreprint
Martínez‐España, R., Balderas-Díaz, S., Guerrero-Contreras, G., Terroso-Sáenz, F., & Morales-García, J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6981424
An AI-driven Decision Support System for Air Quality Forecasting with Adaptive Model SelectionPreprint
Martínez‐España, R., Balderas-Díaz, S., Guerrero-Contreras, G., Terroso-Sáenz, F., & Morales-García, J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6981431
Maplook, S., & Eguchi, K. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-1947-6_4
Fault Detection of Wind Turbine Based on Double Branch Fusion Model
He, Y., Xie, W., Zhang, L., & Mao, G. (2026) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc69976.2026.11560063
“Common one-dimensional networks include one-dimensional Convolutional Neural Network (1D-CNN), Long Short-term Memory Network (LSTM) and their variants such as xLSTM [5].”
Ma, J., Huang, H., Liu, D., Liu, Y., & Xu, Y. (2026) · Water · doi:10.3390/w18131574
Zhang, Y., Chen, J. Y., Li, T., Zhang, W., Li, Z., Li, B., & Zhang, W. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3706290
ELIRF: Enhanced Legal Information Representation and Fusion for Legal Judgment Prediction
Luo, Q., & Wang, S. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11581634
Xiao, Y., Wang, F., Liu, W., & Wan, F. (2026) · Journal of Energy Storage · doi:10.1016/j.est.2026.123570
Hu, J.-H., Zhou, W., Liao, J., et al. (2026) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3805712.3809730
A hybrid multi-model architecture for at-the-money options forecasting
Wu, M., Ko, K.-L., & Wu, J. M. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.115611
Yuan, Y., Chen, T., Xu, D., Zhou, H., Yang, C. R., Feng, G., Jie, T., Zhou, Z., Wei, Y., Ge, Q., et al. (2026) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2026.109544
xLSTM-Based Surgical Performance Classification for VR-Haptic Podiatry Simulation: A Pilot Study
Abounader, J., & Kim, K. (2026) · International Conference on Bio-engineering for Smart Technologies (BioSMART) · doi:10.1109/biosmart71257.2026.11598546
“The proposed classification framework uses the Extended Long Short-Term Memory (xLSTM) architecture [20] to model multimodal surgical time-series data.”
Mahmood, N., & Kim, K. (2026) · International Conference on Bio-engineering for Smart Technologies (BioSMART) · doi:10.1109/biosmart71257.2026.11598239
Lai, X., Deng, S., Yu, L., Lai, Y., Qiao, S., & Zhang, X. (2026) · Advanced Engineering Informatics · doi:10.1016/j.aei.2026.105047
Novel and Precise Lathe Machine Vibration Measurement and Predication with Heterodyne Interferometry
Zhang, H., Chet, T., & Zhou, W. (2026) · doi:10.1364/cleo_at.2026.am2b.7
An Extreme 0.15 kbps Speech Compression Model for Direct to Cell
Gong, C., Ouyang, M., Wang, L. G., Chen, C., & Zhu, X. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3714191
A New Regional ZTD Prediction Model Based on ProbSparse Attention Mechanism and Improved LSTM
Chang, H., Guo, Y.-M., Gao, S.-H., et al. (2026) · Advances in Space Research · doi:10.1016/j.asr.2026.07.047
DGHKT: Adaptive Dual-View Gated Hypergraph Networks for Knowledge Tracing
Zhao, B., Zhang, X., & Zheng, S. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3391-5_29
An integrated FFCM-iTransformer model for GNSS height time series prediction
Feng, T., Liu, H., Zhang, Q., & dong, M. (2026) · Advances in Space Research · doi:10.1016/j.asr.2026.07.066
Remote Sensing Techniques Shaping the Future of Wildfire Early Detection
Raza, M., Yin, X.-C., Abbas, T., Ahmad, I., Aslam, R. W., Tanveer, M. F., & Morimitsu, H. (2026) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70094
Unsupervised Representation Learning for Time Series Anomaly Detection: Overview and new Trends
Hamida, O. I., Boudane, F., & Berrichi, A. (2026) · International Conference on Pattern Analysis and Intelligent Systems (PAIS) · doi:10.1109/pais69842.2026.11605406
“The Extended Long Short-Term Memory (xLSTM) architecture, introduced by [8] in 2024, represents a significant paradigm shift in recurrent neural network (RNN) design.”
Truong, T. N. L., Vu, L. V., Phúc, Đ. H., Thọ, Đ. Đ., Mai, L. H., Tin, N. P., Lee, K., & Nguyen, T. C. N. (2026) · Buildings · doi:10.3390/buildings16142908
“The recently introduced xLSTM [28] modernizes this family by incorporating exponential gating and revised memory structures through two complementary blocks, scalar memory (sLSTM) and matrix memory (mLSTM), resulting in enhanced capacity and improved trainability.”
Yang, X., Li, C., Shao, C., Zhang, B., & Xie, C.-Z. (. (2026) · Journal of Statistical Mechanics Theory and Experiment · doi:10.1088/1742-5468/ae8434
SCST-Based Advantage Exploitation for Captioning: An Application in Traffic Scene Understanding
Zadeh, D. S., Moshiri, B., & Basir, O. A. (2026) · Conference proceedings - Canadian Conference on Electrical and Computer Engineering · doi:10.1109/ccece68150.2026.11610500
A multi-scale spatio-temporal learning framework for wind power forecasting
Bai, K., Zhong, W., Gu, Y., et al. (2026) · Applied Soft Computing · doi:10.1016/j.asoc.2026.116073
MS-BiViL: A Tiny Multi-Scale Vision-LSTM Variant for Matured Vision at the IoV Edge
Walid, A., & Mokhtar, B. (2026) · doi:10.1109/sm69703.2026.11614131
“The Vision-LSTM (ViL) architecture [7], [8] adapts the extended LSTM (xLSTM) for vision, offering linear computational complexity ( O ( N ) ).”
Yang, X., Qu, G., & Liu, Z. (2026) · Lecture notes in mechanical engineering · doi:10.1007/978-981-92-1319-1_5
Zhao, Y., & Qiao, G. (2026) · Journal of Forecasting · doi:10.1002/for.70196
Elansari, T., & Sulimani, H. H. (2026) · Evolving Systems · doi:10.1007/s12530-026-09865-3
“The extended LSTM (xLSTM) (Beck et al. 2024) generalizes recurrent architectures by combining different LSTM variants — notably the stabilized LSTM with neuron selection (sLSTM) and the matrix LSTM with neuron selection (mLSTM) — inside residual blocks .”
InfluentialRWKV-TS+: Adaptive Long-Range Prompt for Multiple Time Series Tasks
Pang, H., Xie, X., Li, Y., Jiang, Z., Zhang, X., Zhang, H., Wei, S., Zhao, Y., & Zhao, Y. (2026) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2026.3717305
Meng, X.-B., Xie, G., Xu, K., & Gao, X. (2026) · Measurement · doi:10.1016/j.measurement.2026.122647 · Cited by 1
Bidirectional mamba with adaptive feature modulators for turbulent flow forecasting
Wang, R., Luan, Z., Lei, H., Zhang, Z., & Wang, L. (2026) · International Journal of Heat and Fluid Flow · doi:10.1016/j.ijheatfluidflow.2026.110600
A Contrastive Learning Framework With Individualized Similarity for Industrial Soft Sensing
Shi, X.-D., Wang, J., Morales, H., Xiong, W.-L., & Amicarelli, A. (2026) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2026.3717823
Zhao, H., Zhu, H., Li, X., et al. (2026) · doi:10.1109/mrai70020.2026.11621429
“The proposed method utilizes a pretrained neural network based on the xLSTM architecture [10] as the internal prediction model.”
A Benchmark for Intelligent Healthcare Processes: Simulated Data Generation and Model Evaluation
Wu, Z., Lv, Q., Liang, Y., et al. (2025) · doi:10.1109/csaia65930.2025.11623753
Xu, A., Fang, S., Chen, J., & Chen, Z. (2026) · International Journal of Green Energy · doi:10.1080/15435075.2026.2710114
“(12) presents the process of xLSTM for temporal information extraction (Beck et al. 2024).”
PointxLSTM: Integrating xLSTM for 3D Point Cloud Completion and Analysis
Kumari, S., Patel, M., Ruthesha, A., et al. (2025) · doi:10.1145/3774521.3774584
Noor, T., Ramana, G. V., & Nayek, R. (2026) · E3S Web of Conferences · doi:10.1051/e3sconf/202673003006
“This stands in contrast to the matrix memory cell (mLSTM) of xLSTM [1], which employs an explicit outer-product update to the cell state, enabling the model to store and retrieve associative, covariance-like structure across the loading history.”
InfluentialHuang, C.-I., & Chang, W. (2026) · IET Intelligent Transport Systems · doi:10.1049/itr2.70304
Traffic Prediction-Assisted Resource Allocation for RAN Slicing Enabling URLLC and eMBB Coexistence
Bai, Y., & Wang, H. (2026) · IEEE Transactions on Communications · doi:10.1109/tcomm.2026.3720354
Enhancing Text-Based Sentiment and Emotion Classification with Quantum-Inspired Deep Learning
Kumar, S., Sharma, R., Rai, A. K., et al. (2026) · Innovations in Sustainable Technologies and Computing · doi:10.1007/978-3-032-22830-7_11
Short-Term Industrial Load Forecasting Method Based on HO-VMD-sLSTM-mLSTM
Wu, N., Huang, Q., Yang, J., et al. (2026) · doi:10.1109/sesg69629.2026.11634409
“The extended Long Short-Term Memory (xLSTM) network proposed in [5] introduces sLSTM and mLSTM units to enhance temporal representation.”
Richris, S. L., & Sebastian, A. (2026) · IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi69079.2026.00172
“Implements the un-gated outer-product matrix memory update of [1]: where k t and v t are randomly initialised (untrained) key and value projections.”
Remote Sensing Change Detection with Cross Mlstm
Ghazaei, E., & Aptoula, E. (2026) · Proceedings - International Conference on Image Processing · doi:10.1109/icip61757.2026.11630458
Poster: Towards Transparent NIDS: An xLSTM-Based Explainable Architecture
Joshi, A., & Fang, D. (2026) · Silicon Valley Cybersecurity Conference (SVCC) · doi:10.1109/svcc69905.2026.11642322
“To address these challenges, we introduce the application of the Extended Long Short-Term Memory (xLSTM) [6] architecture to NIDS [7]. xLSTM provides performance comparable to transformers but maintains linear time computational complexity and significantly lowers operational cost.”
Metadata
- Authors
- Beck, M., Pöppel, K., Spanring, M., Auer, A., Prudnikova, O., Kopp, M., Klambauer, G., Brandstetter, J., & Hochreiter, S.
- ORCID
- Venue
- Advances in Neural Information Processing Systems 37
- Published
- 2024
- Pages
- 107547-107603
- ISSN
- 1049-5258 (ISSN-L)
- OpenAlex
- W4415795940
- Cited by
- 286
- References
- 0
- Citations by year
- 2026: 137 · 2025: 126 · 2024: 23
- FWCI
- 83.96
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- cc-by
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- long short-term memory, recurrent neural networks, xLSTM, memory expansion, sequence modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.