Quantification of Large Language Model Distillation
Open accessAbstract
Sunbowen Lee, Junting Zhou, Chang Ao, Kaige Li, Xeron Du, Sirui He, Haihong Wu, Tianci Liu, Jiaheng Liu, Hamid Alinejad-Rokny, Min Yang, Yitao Liang, Zhoufutu Wen, Shiwen Ni. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Detecting distillation between models is an active research area.
“Detecting distillation between models is an active research area.”
From Stealing Reasoning Traces from Proprietary LLM APIs (2026) · cites this paper as Lee et al. (2025)
Lee et al. (2025) quantify distillation through identity leakage and response similarity across models, and Rawat et al. (2026) attribute a student model's teacher through membership inference, which requires token likelihoods and an earlier checkpoint from the student's lineage.
“Lee et al. (2025) quantify it through identity leakage and response similarity across models, and Rawat et al. (2026) attribute a student model's teacher through membership inference, which requires token likelihoods and an earlier checkpoint from the student's lineage.”
From Stealing Reasoning Traces from Proprietary LLM APIs (2026) · cites this paper as Lee et al. (2025)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 5
- Peter F Brown, Vincent J Della Pietra, Peter V Desouza, Jennifer C Lai, and Robert L Mercer. 1992. Class- based n-gram models of natural language. Computa- tional linguistics, 18(4):467-480.
- Tom B Brown, Benjamin Mann, Nick Ryder, et al. 2020. Language models are few-shot learners. Advances in Neural Information Processing Systems, 33:1877- 1901.
- Wei Chen, Arjun Kumar, and Lin Yang. 2024. Distraction-based attack prompts: An effective jail- breaking method for llms. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL).
- John Doe and Jane Smith. 2024. Weak-to-strong jail- breaking attack on aligned large language models. OpenReview Preprint.
- Shiwen Ni, Min Yang, Ruifeng Xu, Chengming Li, and Xiping Xiping Hu. 2024b. Layer-wise regularized dropout for neural language models. In Proceedings of the 2024 Joint International Conference on Compu- tational Linguistics, Language Resources and Evalu- ation (LREC-COLING 2024), pages 10208-10218.
Cited by
Sorted by citations
Fang, L., Yu, X., Cai, J., Chen, Y., Wu, S., Liu, Z., Yang, Z., Lu, H., Gong, X., Liu, Y., et al. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11423-3 · Cited by 64 · see citing works
Self-citationDao, H. N., Hashimoto, Y., Paik, I., & Thang, T. C. (2025) · IEEE Access · doi:10.1109/access.2025.3592943 · Cited by 2 · see citing works
Stealing Reasoning Traces from Proprietary LLM APIs
Panfilov, A., Schmotz, D., Shumailov, I., Beurer-Kellner, L., Schaeffer, J., Prabhu, A., Geiping, J., & Andriushchenko, M. (2026) · arXiv · doi:10.48550/arxiv.2608.09867
Generalizing WiFi Gesture Recognition via Large-Model-Aware Semantic Distillation and Alignment
Cui, F.-Q., Guo, Y., Zheng, T., & Huang, J. (2025) · IEEE International Conference on Parallel and Distributed Systems (ICPADS) · doi:10.1109/icpads67057.2025.11323213
Agent-Based MLLM for Intelligent Buoy Analysis on Edge Devices
Wan, J., Guo, D., Wei, C., Tu, Y., Huang, Q., Wang, P., Wei, Z., & Wan, G. (2026) · Lecture notes in electrical engineering · doi:10.1007/978-981-92-3566-7_38
- Authors
- Lee, S., Zhou, J., Ao, C., Li, K., Du, X., He, S., Wu, H., Liu, T., Liu, J., Alinejad-Rokny, H., Yang, M., Liang, Y., Wen, Z., & Ni, S.
- Affiliations
- Chinese Academy of Sciences (CN)
- Peking University (CN)
- Nanjing University (CN)
- Venue
- Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2025
- Pages
- 4985-5004
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W4412886669
- Cited by
- 4
- References
- 0
- Citations by year
- 2026: 1 · 2025: 3
- FWCI
- 3.81
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- Funders
- National Natural Science Foundation of China (62376262)
- China Postdoctoral Science Foundation
- Natural Science Foundation of Guangdong Province
- National Science and Technology Major Project (2022ZD0114902)
- National Key Research and Development Program of China (2022YFF0902100)
- Basic and Applied Basic Research Foundation of Guangdong Province
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- knowledge distillation, quantification, model compression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
