Query-Key Normalization for Transformers
Open accessAbstract
Low-resource language translation is a challenging but socially valuable NLP task.Building on recent work adapting the Transformer's normalization to this setting, we propose QKNORM, a normalization technique that modifies the attention mechanism to make the softmax function less prone to arbitrary saturation without sacrificing expressivity.Specifically, we apply ℓ 2 normalization along the head dimension of each query and key matrix prior to multiplying them and then scale up by a learnable parameter instead of dividing by the square root of the embedding dimension.We show improvements averaging 0.928 BLEU over state-of-the-art bilingual benchmarks for 5 low-resource translation pairs from the TED Talks corpus and IWSLT'15. 1
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Employing bfloat16 and QK norm for training the dynamics model stabilizes training at large scale.
“we employ bfloat16 and QK norm for training our dynamics model, which has been shown to stabilize training at large scale (Dehghani et al., 2023; Henry et al., 2020).”
From Genie: Generative Interactive Environments (2024) · cites this paper as Henry et al., 2020
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · see citing works
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273 · see citing works
fairseq: A Fast, Extensible Toolkit for Sequence Modeling
Ott, M., Edunov, S., Baevski, A., Fan, A., Gross, S., Ng, N., Grangier, D., & Auli, M. (2019) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n19-4009 · Cited by 3,538 · see citing works
Loper, E., & Bird, S. (2002) · doi:10.3115/1118108.1118117 · Cited by 3,248 · see citing works
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · see citing works
A Call for Clarity in Reporting BLEU Scores
Post, M. (2018) · Proceedings of the Third Conference on Machine Translation: Research Papers · doi:10.18653/v1/w18-6319 · Cited by 2,112 · see citing works
NLTK: The Natural Language ToolkitPreprint
Loper, E., & Bird, S. (2002) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0205028 · Cited by 1,893 · see citing works
OpenNMT: Open-Source Toolkit for Neural Machine Translation
Klein, G., Kim, Y., Deng, Y., Sénellart, J., & Rush, A. M. (2017) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/p17-4012 · Cited by 1,771 · see citing works
What Does BERT Look at? An Analysis of BERT’s Attention
Clark, K., Khandelwal, U., Levy, O., & Manning, C. D. (2019) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/w19-4828 · Cited by 1,514 · see citing works
Statistical Significance Tests for Machine Translation Evaluation.
Koehn, P. (2004) · Cited by 1,456 · see citing works
Learning Deep Transformer Models for Machine Translation
Wang, Q., Li, B., Xiao, T., Zhu, J., Li, C., Wong, D. F., & Chao, L. S. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1176 · Cited by 673 · see citing works
Tensor2Tensor for Neural Machine TranslationPreprint
Vaswani, A., Bengio, S., Brevdo, E., Chollet, F., Gomez, A. N., Gouws, S., Jones, L., Kaiser, Ł., Kalchbrenner, N., Parmar, N., et al. (2018) · Conference of the Association for Machine Translation in the Americas · doi:10.48550/arxiv.1803.07416 · Cited by 556 · see citing works
The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation
Chen, M., Firat, O., Bapna, A., Johnson, M., Macherey, W., Foster, G. F., Jones, L., Schuster, M., Shazeer, N., Parmar, N., et al. (2018) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p18-1008 · Cited by 484 · see citing works
Massively Multilingual Neural Machine Translation
Aharoni, R., Johnson, M., & Firat, O. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1388 · Cited by 402 · see citing works
When and Why Are Pre-Trained Word Embeddings Useful for Neural Machine Translation?
Ye, Q., Sachan, D. S., Felix, M., Padmanabhan, S. J., & Neubig, G. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-2084 · Cited by 302 · see citing works
Root Mean Square Layer Normalization
Zhang, B., & Sennrich, R. (2019) · arXiv · doi:10.48550/arxiv.1910.07467 · Cited by 262 · see citing works
Understanding and Improving Layer NormalizationPreprint
Xu, J., Sun, X., Zhang, Z., Zhao, G., & Lin, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.07013 · Cited by 253 · see citing works
Guzmán, F., Chen, P., Ott, M., Pino, J., Lample, G., Koehn, P., Chaudhary, V., & Ranzato, M. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1632 · Cited by 231 · see citing works
BPE-Dropout: Simple and Effective Subword Regularization
Provilkov, I., Emelianenko, D., & Voita, E. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.170 · Cited by 220 · see citing works
Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks
Luo, C., Zhan, J., Xue, X., Wang, L., Ren, R., & Yang, Q. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01418-6_38 · Cited by 219 · see citing works
Weighted Transformer Network for Machine TranslationPreprint
Ahmed, K., Keskar, N., & Socher, R. (2017) · arXiv.org · doi:10.48550/arxiv.1711.02132 · Cited by 143 · see citing works
Explicit Sparse Transformer: Concentrated Attention Through Explicit SelectionPreprint
Zhao, G., Lin, J., Zhang, Z., Ren, X., Su, Q., & Sun, X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.11637 · Cited by 131 · see citing works
Accelerating Neural Transformer via an Average Attention Network
Zhang, B., Xiong, D., & Su, J. (2018) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p18-1166 · Cited by 129 · see citing works
Transformers without Tears: Improving the Normalization of Self-AttentionPreprint
Nguyen, T. Q., & Salazar, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.05895 · Cited by 124 · see citing works
The IWSLT 2015 Evaluation Campaign
Cettolo, M., Niehues, J., Stüker, S., Bentivogli, L., Cattoni, R., & Federico, M. (2015) · Repository KITopen (Karlsruhe Institute of Technology) · doi:10.5281/zenodo.1326521 · Cited by 95 · see citing works
Improving Lexical Choice in Neural Machine Translation
Nguyen, T. Q., & Chiang, D. (2017) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n18-1031 · Cited by 90 · see citing works
How Much Attention Do You Need? A Granular Analysis of Neural Machine Translation Architectures
Domhan, T. (2018) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p18-1167 · Cited by 81 · see citing works
PowerNorm: Rethinking Batch Normalization in TransformersPreprint
Shen, S., Yao, Z., Gholami, A., Mahoney, M. W., & Keutzer, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.07845 · Cited by 73 · see citing works
Stolen Probability: A Structural Weakness of Neural Language Models
Demeter, D., Kimmel, G. J., & Downey, D. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.198 · Cited by 24 · see citing works
On Optimal Transformer Depth for Low-Resource Language TranslationPreprint
van Biljon, E., Pretorius, A., & Kreutzer, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.04418 · Cited by 20 · see citing works
Language Model Prior for Low-Resource Neural Machine Translation
Baziotis, C., Haddow, B., & Birch, A. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.615 · Cited by 18 · see citing works
Normalized Attention Without Probability CagePreprint
Richter, O., & Wattenhofer, R. P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.09561 · Cited by 12 · see citing works
Flagged first, then by citations
4 of the 38 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
TransNeXt: Robust Foveal Visual Perception for Vision Transformers
Shi, D. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01683 · Cited by 452 · see citing works
A Survey on LLM-Based Code Generation for Low-Resource and Domain-Specific Programming Languages
Joel, S., Wu, J. J., & Fard, F. H. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3770084 · Cited by 93 · see citing works
Hutchins, D., Schlag, I., Wu, Y., Dyer, E., & Neyshabur, B. (2022) · arXiv · doi:10.48550/arxiv.2203.07852 · Cited by 43 · see citing works
BashExplainer: Retrieval-Augmented Bash Code Comment Generation based on Fine-tuned CodeBERT
Yu, C., Yang, G., Chen, X., Liu, K., & Zhou, Y. (2022) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme55016.2022.00016 · Cited by 38 · see citing works
DualSC: Automatic Generation and Summarization of Shellcode via Transformer and Dual Learning
Yang, G., Chen, X., Zhou, Y., & Yu, C. (2022) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner53432.2022.00052 · Cited by 36 · see citing works
Fine-grained Pseudo-code Generation Method via Code Feature Extraction and Transformer
Yang, G., Zhou, Y., Chen, X., & Yu, C. (2021) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec53868.2021.00029 · Cited by 35 · see citing works
Focused Transformer: Contrastive Training for Context Scaling
Tworkowski, S., Staniszewski, K., Pacek, M., Wu, Y., Michalewski, H., & Milos, P. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1849 · Cited by 18 · see citing works
Dual-Spatial Normalized Transformer for image captioning
Hu, J., Yang, Y., An, Y., & Lu, Y. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106384 · Cited by 16 · see citing works
A Critical Assessment of Modern Generative Models’ Ability to Replicate Artistic Styles
Asperti, A., George, F., Marras, T., Stricescu, R. C., & Zanotti, F. (2025) · Big Data and Cognitive Computing · doi:10.3390/bdcc9090231 · Cited by 15 · see citing works
Wang, C., Karimzadeh, M., Ravindra, N. G., Bounds, L. R., Alerasool, N., Huang, A., Ma, S., Gulbranson, D. R., Cui, H., Lee, Y., et al. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.03.18.712807 · Cited by 18 · see citing works
Genie: Generative Interactive Environments
Bruce, J., Dennis, M., Edwards, A., Parker-Holder, J., Shi, Y., Hughes, E., Lai, M., Mavalankar, A., Steigerwald, R., Apps, C., et al. (2024) · arXiv · doi:10.48550/arxiv.2402.15391 · Cited by 23 · see citing works
“Further, we employ bfloat16 and QK norm for training our dynamics model, which has been shown to stabilize training at large scale (Dehghani et al., 2023; Henry et al., 2020).”
MethodXiao, L., Lu, S., Pi, H., Fan, K., Pan, L., Zhou, Y., Feng, Z., Zhou, X., Peng, S., & Wang, J. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00940 · Cited by 19 · see citing works
Attention Approximates Sparse Distributed MemoryPreprint
Bricken, T., & Pehlevan, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.05498 · Cited by 14 · see citing works
Multi-Modal API Recommendation
Irsan, I. C., Zhang, T., Thung, F., Kim, K., & Lo, D. (2023) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner56733.2023.00034 · Cited by 10 · see citing works
EDM: Efficient Deep Feature Matching
Li, X., Rao, T., & Pan, C. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02431 · Cited by 7 · see citing works
Liu, X., Li, H., Qiao, Z., Huang, Y., Liu, X., Zhang, J., Qian, Z., Zhen, X., & Zhang, B. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02426-2 · Cited by 6 · see citing works
Dufter, P., Schmitt, M., & Schütze, H. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.324 · Cited by 6 · see citing works
RenderFormer: Transformer-based Neural Rendering of Triangle Meshes with Global Illumination
Zeng, C., Dong, Y., Peers, P., Wu, H., & Tong, X. (2025) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3721238.3730595 · Cited by 5 · see citing works
Applications of modular co-design for de novo 3D molecule generation
Reidenbach, D., Nikitin, F., Isayev, O., & Paliwal, S. (2026) · Digital Discovery · doi:10.1039/d5dd00380f · Cited by 5 · see citing works
Fast Text-to-Audio Generation with Adversarial Post-Training
Novack, Z., Evans, Z., Zukowski, Z., Taylor, J., Carr, C., Parker, J. D., Al-Sinan, A., Iodice, G. M., McAuley, J., Berg-Kirkpatrick, T., et al. (2025) · IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/waspaa66052.2025.11230941 · Cited by 3 · see citing works
SAGE-Amine: Generative Amine design with multi-property optimization for efficient CO2 capture
Lim, H., Cho, H., Kim, J.-H., & No, K. T. (2025) · Carbon Capture Science & Technology · doi:10.1016/j.ccst.2025.100447 · Cited by 3 · see citing works
DeepPseudo: Deep Pseudo-code Generation via Transformer and Code Feature Extraction.Preprint
Yang, G., Chen, X., Liu, K., & Yu, C. (2021) · arXiv (Cornell University) · Cited by 3 · see citing works
Mamba-3: Improved Sequence Modeling using State Space Principles
Lahoti, A., Li, K. Y., Chen, B., Wang, C., Bick, A., Kolter, J. Z., Dao, T., & Gu, A. (2026) · arXiv · doi:10.48550/arxiv.2603.15569 · Cited by 1 · see citing works
“Discretizing a complex SSM with the exponential-trapezoidal rule (Proposition 1) yields the recurrence Here, R is the block-diagonal rotation matrix defined in Proposition 2.”
NeuFrameQ: Neural Frame Fields for Scalable and Generalizable Anisotropic Quadrangulation
Liu, Y.-T., Li, J., Liu, Y.-T., Yu, X., Guo, Y.-C., Cao, Y.-P., Liang, D., Shamir, A., & Zhang, X. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02599 · Cited by 1 · see citing works
Application of One-click Conversion of Image Weather Styles Based on the CycleGAN
Yu, J., Yang, C. Z., Luo, T., Wu, X., Yuan, H., Guo, W., Yuan, X., Hu, J., & Wang, T. (2024) · International Conference on Vision, Image and Signal Processing (ICVISP) · doi:10.1109/icvisp64524.2024.10959673 · Cited by 1 · see citing works
Enigma: An Efficient Model for Deciphering Regulatory GenomicsPreprint
Jung, A., Zhu, H., Li, R., Gao, A. J., Lau, T., Chu, V. S., Lim, D., Cole, C. B., Lee, L. J., Celaj, A., et al. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2025.12.18.694875 · Cited by 1 · see citing works
IndexMem: Learned KV-Cache Eviction with Latent Memory for Long-Context LLM Inference
Yang, X., Gu, H., Xu, B., Li, L., Liu, B., Liu, J., Zhu, Q., Guo, Y., & Han, S. (2026) · arXiv · doi:10.48550/arxiv.2605.25475
“We first construct QK-Norm (Henry et al., 2020) features for stable training.”
Exact-Solution Volume and Length Generalization in Transformers
Zhu, Y. J., & Chiang, D. (2026) · arXiv · doi:10.48550/arxiv.2610.07676
“QKNorm ℓ 2 -normalizes complete query and key vectors and learns the attention scale (Henry et al., 2020), while scaled cosine attention likewise removes dependence on their magnitudes (Liu et al., 2022).”
Leveraging Cross-Utterance Context For ASR Decoding
Flynn, R., & Ragni, A. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-1941
A Large Financial Chat ModelPreprint
Ding, F. (2024) · doi:10.33774/coe-2024-lsc94
Xu, Y., Shang, Z., Zou, H., Yu, H., & Gao, S. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112206
All-atom protein design via SE(3) flow matching with ProteinZenPreprint
Li, A. J., & Kortemme, T. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.10.18.683228
CLAP: A Closed-Loop Diffusion Transformer Action Foundation Model for Robotic Manipulation
Li, M., Ye, D., Zhou, Y., & Yang, C. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246478
Towards foundation models for experimental readout systems combining discrete and continuous data
Giroux, J., & Fanelli, C. (2026) · Machine Learning Science and Technology · doi:10.1088/2632-2153/ae3d81
Trinity: Three-Dimensional Tensor Program Optimization via Tile-level Equality Saturation
Park, J., Kim, Y., An, H., Jeong, G., Kang, J., & Han, D. (2026) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3779212.3790240
GaussianGPT: Towards Autoregressive 3D Gaussian Scene Generation
von Lützow, N., Roessle, B., Schmid, K., & Nießner, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37044-0_34
Efficient Russian-English Neural Machine Translation with Architectural Optimizations
Berezin, G., & Lipatova, A. (2026) · Systems of Signals Generating and Processing in the Field of on Board Communications · doi:10.1109/ieeeconf68869.2026.11461110
Free-Range Gaussians: Non-Grid-Aligned Generative 3D Gaussian Reconstruction
Shabanov, A., Hedman, P., Weber, E., Li, Z., Rozumnyi, D., Lan, G. L., Dhingra, N., Luo, L., Vedaldi, A., Richardt, C., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37447-9_25
Volume Transformer: Revisiting Vanilla Transformers for 3D Scene Understanding
Yilmaz, K., Kruse, A., Höfer, T., de Geus, D., & Leibe, B. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37044-0_12
CapSpeech: Enabling Downstream Applications in Style-Captioned Text-to-Speech
Wang, H., Hai, J., Chong, D., Thakkar, K., Feng, T., Yang, D., Lee, J., Thebaud, T., Moro-Velázquez, L., Villalba, J., et al. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3703247
PointSplat: Compact Gaussian Splatting via Human-Centric Prediction
Guo, Y., Jin, Y., Qiu, L., Shen, Z., Xu, Z., Zhang, J., Shen, X., Bao, H., Peng, S., & Zhou, X. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37016-7_33
DSPose: Decoupled Real-Time Pose Estimation From Semantic-RGB Radiance Rays
Chen, T. L., Liang, S., Xiong, Y., Geng, Q., & Zhou, Z. H. (2026) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2026.3713498
Kowalik, M., Poruba, M., Grzymkowski, Ł., & Stefański, T. P. (2026) · IEEE Access · doi:10.1109/access.2026.3720484
OneLive: Dynamically Unified Generative Framework for Live-Streaming Recommendation
Wang, S., Huang, Y., Yang, R., Wen, S., Xu, P., Cao, J., Liu, Y., Cai, K., Guo, C., Wang, S., et al. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3818455
Developing GlassGPT: A Modern Day GPT-2 Alternative
Ravi, A. G., Devanand, A., Changethu, J. J., & Kavitha, K. V. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-30438-4_3
Fast Spatial Memory with Scalable Elastic Test-Time Training
Ma, Z., Yu, X., Zhen, H., Yang, Y., Chai, J., & Gan, C. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37393-9_14
Multi-Head Normalization for Wide Vision Transformers
Wei, G., Wang, F., Yuille, A., & Chellappa, R. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37550-6_28
Briden, M., Carrión, H., Mohammadmirzaei, M., Yang, H., Isseroff, R. R., Rolandi, M., Gomez, M., & Norouzi, N. (2026) · doi:10.2196/preprints.111274
Long-Lrm++: Preserving Fine Details in Feed-Forward Wide-Coverage Reconstruction
Ziwen, C., Tan, H., Wang, P., Xu, Z., & Fuxin, L. (2026) · doi:10.1109/cvprf71909.2026.00040
S 2 DiT: Sandwich Diffusion Transformer for Mobile Streaming Video Generation
Zhao, L., Wu, Y., Lebedev, A., Lahiri, D., Dong, M., Sahni, A., Vasilkovsky, M., Chen, H., Hu, J., Siarohin, A., et al. (2026) · doi:10.1109/cvprf71909.2026.00420
Long, M., Liu, J., YuFeng, L. I., Wang, K., Cao, Y., Ding, J., Xiong, H., & Yan, J. (2026) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2026.3728237
- Authors
- Henry, A., Dachapally, P. R., Pawar, S. S., & Chen, Y.
- ORCID
- Chen, Yuxuan
- Venue
- Findings of the Association for Computational Linguistics: EMNLP 2020
- Published
- 2020
- Pages
- 4246-4253
- OpenAlex
- W3099096615
- Cited by
- 45
- References
- 38
- Citations by year
- 2026: 20 · 2025: 12 · 2024: 4 · 2023: 3 · 2022: 2 · 2021: 3 · 2020: 1
- FWCI
- 0.86
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- low-resource translation, attention mechanism
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
