MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices

Sun, Zhiqing, Yu, Hongkun, Song, Xiaodan, Liu, Renjie, Yang, Yiming, Zhou, Denny·2020·Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
DOI 10.18653/v1/2020.acl-main.195

Cited by (758)

Items 101–200 of 758

  1. Federated continual learning: Concepts, challenges, and solutions
    Parisa Hamedi, Roozbeh Razavi‐Far, Ehsan Hallaji · 2025 · Neurocomputing
  2. A Survey for Efficient Open Domain Question Answering
    Qin Zhang, Shangsi Chen, Dongkuan Xu, Qingqing Cao, Xiaojun Chen, Trevor Cohn, Meng Fang · 2023 · Proceedings of the conference - Association for Computational Linguistics. Meeting
  3. Greedy Layer Pruning: Decreasing Inference Time of Transformer Models
    David Peer, Sebastian Stabinger, Stefan Engl, Antonio J. Rodríguez-Sánchez · 2021 · Pattern Recognition Letters
  4. MixKD: Towards Efficient Distillation of Large-scale Language Models
    Kevin J Liang, Weituo Hao, Dinghan Shen, Yufan Zhou, Weizhu Chen, Changyou Chen, Lawrence Carin · 2020 · arXiv (Cornell University)
  5. Your fairness may vary: Pretrained language model fairness in toxic text classification
    Ioana Baldini, Dennis Wei, Karthikeyan Natesan Ramamurthy, Moninder Singh, Mikhail Yurochkin · 2022 · Findings of the Association for Computational Linguistics: ACL (Findings of ACL)
  6. When Attention Meets Fast Recurrence: Training Language Models with Reduced Compute
    Tao Lei · 2021 · Conference on Empirical Methods in Natural Language Processing (EMNLP)
  7. Revisiting Automatic Evaluation of Extractive Summarization Task: Can We Do Better than ROUGE?
    Mousumi Akter, Naman Bansal, S. Karmaker · 2022 · Findings
  8. MECLA: Memory-Compute-Efficient LLM Accelerator with Scaling Sub-matrix Partition
    Yubin Qin, Yang Wang, Zhiren Zhao, Xiaolong Yang, Yang Zhou, Shaojun Wei, Yang Hu, Shouyi Yin · 2024 · Proceedings - International Symposium on Computer Architecture/Proceedings
  9. Efficiency Optimization of Large-Scale Language Models Based on Deep Learning in Natural Language Processing Tasks
    Taiyuan Mei, Yun Zi, Xiaohan Cheng, Zijun Gao, Qi Wang, Haowei Yang · 2024 · IEEE International Conference on Sensors, Electronics and Computer Engineering (ICSECE)
  10. Pruning Redundant Mappings in Transformer Models via Spectral-Normalized Identity Prior
    Zi Jun Lin, Jeremiah Zhe Liu, Zi Yang, Nan Hua, Dan Roth · 2020 · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)
  11. A Lightweight Neural Model for Biomedical Entity Linking
    Lihu Chen, Gael P. Varoquaux, Fabian M. Suchanek · 2021 · Proceedings of the AAAI Conference on Artificial Intelligence
  12. NewsBERT: Distilling Pre-trained Language Model for Intelligent News Application
    Chuhan Wu, Fangzhao Wu, Yang Yu, Tao Qi, Yongfeng Huang, Qi Liu · 2021 · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)
  13. Efficient Fine-Tuning of BERT Models on the Edge
    Danilo Vucetic, Mohammadreza Tayaranian, Maryam Ziaeefard, James J. Clark, Brett H. Meyer, Warren J. Gross · 2022 · IEEE International Symposium on Circuits and Systems proceedings
  14. Cross-modal prototype based multimodal federated learning under severely missing modality
    Huy Q. Le, Chu Myaet Thwal, Yu Qiao, Ye Lin Tun, Minh N. H. Nguyen, Eui‐Nam Huh, Chaoning Zhang · 2025 · Information Fusion
  15. Optimizing Federated Learning With Aggregation Strategies: A Comprehensive Survey
    Naeem Khan, Shibli Nisar, Muhammad Asghar Khan, Yasar Abbas Ur Rehman, Fazal Noor, Gordana Barb · 2025 · IEEE Open Journal of the Computer Society
  16. Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains
    Tianqing Fang, Chengyu Wang, Minghui Qiu, Yichang Zhang, Yaliang Li, Jun Huang · 2021
  17. A social media event detection framework based on transformers and swarm optimization for public notification of crises and emergency management
    Abdelghani Dahou, Alhassan Mabrouk, Ahmed A. Ewees, Marwa A. Gaheen, Mohamed Abd Elaziz · 2023 · Technological Forecasting and Social Change
  18. Re-mine, Learn and Reason: Exploring the Cross-modal Semantic Correlations for Language-guided HOI detection
    Yichao Cao, Qingfei Tang, Feng Yu Yang, Xiu Qin Su, Shan You, Xiaobo Lu, Chang Xu · 2023 · IEEE/CVF International Conference on Computer Vision (ICCV)
  19. BK-SDM: A Lightweight, Fast, and Cheap Version of Stable Diffusion
    Bo-Kyeong Kim, Hyoung-Kyu Song, Thibault Castells, Shinkook Choi · 2024 · Lecture notes in computer science
  20. A Wireless AI-Generated Content (AIGC) Provisioning Framework Empowered by Semantic Communication
    Runze Cheng, Yao Sun, Dusit Tao Niyato, Lan Zhang, Lei Zhang, Muhammad Ali Imran · 2024 · IEEE Transactions on Mobile Computing
  21. Why Skip If You Can Combine: A Simple Knowledge Distillation Technique for Intermediate Layers
    Yimeng Wu, Peyman Passban, Mehdi Rezagholizadeh, Qun Liu · 2020 · Conference on Empirical Methods in Natural Language Processing (EMNLP)
  22. MATE-KD: Masked Adversarial TExt, a Companion to Knowledge Distillation
    Ahmad Rashid, Vasileios Lioutas, Mehdi Rezagholizadeh · 2021
  23. Automated essay scoring using efficient transformer-based language models
    Christopher M. Ormerod, Akanksha Malhotra, Jafari, Amir · 2021 · arXiv (Cornell University)
  24. Towards Accurate and Reliable Energy Measurement of NLP Models
    Qingqing Cao, Aruna Balasubramanian, Niranjan Balasubramanian · 2020 · Workshop on Simple and Efficient Natural Language Processing (SustaiNLP)
  25. Extremely Small BERT Models from Mixed-Vocabulary Training
    Sanqiang Zhao, Raghav Gupta, Yang Song, Denny Zhou · 2021 · Conference of the European Chapter of the Association for Computational Linguistics
  26. When the Edge Meets Transformers: Distributed Inference with Transformer Models
    Chenghao Hu, Baochun Li · 2024 · IEEE International Conference on Distributed Computing Systems (ICDCS)
  27. A 818–4094 TOPS/W Capacitor-Reconfigured Analog CIM for Unified Acceleration of CNNs and Transformers
    Kentaro Yoshioka · 2024 · IEEE Journal of Solid-State Circuits
  28. Comparative analysis of model compression techniques for achieving carbon efficient AI
    Eileen Paula, Jayesh Soni, Himanshu Upadhyay, Leonel E. Lagos · 2025 · Scientific Reports
  29. A Framework for Accelerating Transformer-Based Language Model on ReRAM-Based Architecture
    Myeonggu Kang, Hyein Shin, Lee‐Sup Kim · 2021 · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
  30. MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation
    Simiao Zuo, Qingru Zhang, Liang Chen, Pengcheng He, Tuo Zhao, Weizhu Chen · 2022 · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)
  31. Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression
    Canwen Xu, Wangchunshu Zhou, Tao Ge, Ke Xu, Julian McAuley, Furu Wei · 2021 · Conference on Empirical Methods in Natural Language Processing (EMNLP)
  32. Faster Depth-Adaptive Transformers
    Yijin Liu, Fandong Meng, Jie Zhou, Yufeng Chen, Jinan Xu · 2021 · Proceedings of the AAAI Conference on Artificial Intelligence
  33. Retrieval-Free Knowledge-Grounded Dialogue Response Generation with Adapters
    Yan Xu, Etsuko Ishii, Samuel Cahyawijaya, Zihan Liu, Genta Indra Winata, Andrea Madotto, Dan Su, Pascale Fung · 2022 · DialDoc Workshop on Document-grounded Dialogue and Conversational Question Answering (DialDoc)
  34. CE-BERT: Concise and Efficient BERT-Based Model for Detecting Rumors on Twitter
    Rini Anggrainingsih, Ghulam Mubashar Hassan, Amitava Datta · 2023 · IEEE Access
  35. 8-bit Transformer Inference and Fine-tuning for Edge Accelerators
    Jeffrey J. Yu, Kartik Prabhu, Yonatan Urman, Robert M. Radway, Eric Han, Priyanka Raina · 2024 · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS)
  36. STI: Turbocharge NLP Inference at the Edge via Elastic Pipelining
    Liwei Guo, Wonkyo Choe, Felix Xiaozhu Lin · 2023 · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS)
  37. Technological forecasting based on estimation of word embedding matrix using LSTM networks
    Necip Gözüaçık, C. Okan Sakar, Sercan Ozcan · 2023 · Technological Forecasting and Social Change
  38. Survey of Deep Learning Accelerators for Edge and Emerging Computing
    Alam, Shahanur, Yakopcic, Chris, Wu, Qing, Barnell, Mark, Khan, Simon, Taha, Tarek M. · 2024 · Electronics
  39. Disfluency Detection with Unlabeled Data and Small BERT Models
    Johann C. Rocholl, Vicky Zayats, Daniel D. Walker, Noah B. Murad, Aaron Schneider, Daniel J. Liebling · 2021 · Interspeech (USB)
  40. Distilling Linguistic Context for Language Model Compression
    Geondo Park, Gyeongman Kim, Eunho Yang · 2021 · Conference on Empirical Methods in Natural Language Processing (EMNLP)
  41. Simplified TinyBERT: Knowledge Distillation for Document Retrieval
    Xuanang Chen, Ben He, Kai Hui, Le Sun, Yingfei Sun · 2021 · Lecture notes in computer science
  42. Advancing Sentiment Analysis: Evaluating RoBERTa against Traditional and Deep Learning Models
    Pongsathon Pookduang, Rapeepat Klangbunrueang, Wirapong Chansanam, Tassanee Lunrasri · 2025 · Engineering Technology & Applied Science Research
  43. Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains
    Yunzhi Yao, Shaohan Huang, Wenhui Wang, Li Dong, Furu Wei · 2021 · Findings of the Association for Computational Linguistics: ACL (Findings of ACL)
  44. Designing a deep learning-based application for detecting fake online reviews
    Lê Quang Thảo, Do Trung Kien, Dang Thi Thanh Thuy, Luong Thi Minh Thuy, Ngo Chi Bach, Tran Xuan Duc, Hoang Gia Bach, Duong Duc Cuong · 2024 · Engineering Applications of Artificial Intelligence
  45. Intent detection for task‐oriented conversational agents: A comparative study of recurrent neural networks and transformer models
    Mourad Jbene, Abdellah Chehri, Rachid Saadane, Smail Tigani, Gwanggil Jeon · 2024 · Expert Systems
  46. AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language Models
    Yin, Yichun, Chen, Cheng, Shang, Lifeng, Jiang, Xin, Chen, Xiao, Liu, Qun · 2021 · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
  47. Improving Stance Detection with Multi-Dataset Learning and Knowledge Distillation
    Yingjie Li, Chenye Zhao, Cornelia Caragea · 2021 · Conference on Empirical Methods in Natural Language Processing (EMNLP)
  48. Multi-level semantic enhancement based on self-distillation BERT for Chinese named entity recognition
    Zepeng Li, Shuo Cao, Minyu Zhai, Nengneng Ding, Zhenwen Zhang, Bin Hu · 2024 · Neurocomputing
  49. Quantized Transformer Language Model Implementations on Edge Devices
    Mohammad Wali Ur Rahman, Murad Mehrab Abrar, Hunter Gibbons Copening, Salim Hariri, Sicong Shao, Pratik Satam, Soheil Salehi · 2023 · International Conference on Machine Learning and Applications
  50. Know what you don't need: Single-Shot Meta-Pruning for attention heads
    Zhengyan Zhang, Fanchao Qi, Zhiyuan Liu, Qun Liu, Maosong Sun · 2021 · AI Open
  51. GhostBERT: Generate More Features with Cheap Operations for BERT
    Zhiqi Huang, Lu Hou, Lifeng Shang, Xin Jiang, Xiao Dong Chen, Qun Liu · 2021
  52. How Good Is NLP? A Sober Look at NLP Tasks through the Lens of Social Impact
    Zhijing Jin, Geeticka Chauhan, Brian Tse, Mrinmaya Sachan, Rada F. Mihalcea · 2021 · Findings of the Association for Computational Linguistics: ACL (Findings of ACL)
  53. pNLP-Mixer: an Efficient all-MLP Architecture for Language
    Francesco Fusco, Damián Pascual, Peter Staar, Diego Antognini · 2023 · Proceedings of the conference - Association for Computational Linguistics. Meeting
  54. MoPE-CLIP: Structured Pruning for Efficient Vision-Language Models with Module-Wise Pruning Error Metric
    Haokun Lin, Haoli Bai, Zhili Liu, Lu Hou, Muyi Sun, Linqi Song, Ying Wei, Zhenan Surr · 2024 · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings
  55. Optimal Subarchitecture Extraction For BERT
    Adrian de Wynter, Perry, Daniel J. · 2020 · arXiv (Cornell University)
  56. FedTherapist: Mental Health Monitoring with User-Generated Linguistic Expressions on Smartphones via Federated Learning
    Jaemin Shin, H.S. Yoon, Seungjoo Lee, Sungjoon Park, Yunxin Liu, Jinho D. Choi, Sung-Ju Lee · 2023 · Conference on Empirical Methods in Natural Language Processing (EMNLP)
  57. AI-ssessment: Towards Assessment As a Sociotechnical System for Learning
    Alison Lentz, John Oliver Siy, Carol L. Carraccio · 2021 · Academic Medicine
  58. Kronecker Decomposition for GPT Compression
    Ali Edalati, Marzieh S. Tahaei, Ahmad Rashid, Vahid Partovi Nia, James J. Clark, Mehdi Rezagholizadeh · 2022 · Proceedings of the conference - Association for Computational Linguistics. Meeting
  59. Textual emotion detection utilizing a transfer learning approach
    Mahsa Hadikhah Mozhdehi, Amir Masoud Eftekhari Moghadam · 2023 · The Journal of Supercomputing
  60. Edge and Terminal Cooperation Enabled LLM Deployment Optimization in Wireless Network
    Wentao Zhao, Wenpeng Jing, Zhaoming Lu, Xiangming Wen · 2024 · IEEE/CIC International Conference on Communications in China (ICCC)
  61. NLP From Scratch Without Large-Scale Pretraining: A Simple and Efficient Framework
    Xingcheng Yao, Yanan Zheng, Xiaocong Yang, Zhilin Yang · 2021 · arXiv (Cornell University)
  62. Emotion Recognition on Edge Devices: Training and Deployment
    Vlad Pandelea, Edoardo Ragusa, Tommaso Apicella, Paolo Gastaldo, Erik Cambria · 2021 · Sensors
  63. EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation
    Chenhe Dong, Guangrun Wang, Hang Xu, Jiefeng Peng, Xiaozhe Ren, Xiaodan Liang · 2021 · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)
  64. Tiny-NewsRec: Effective and Efficient PLM-based News Recommendation
    Yang Yu, Fangzhao Wu, Chuhan Wu, Jingwei Yi, Qi Liu · 2022 · Conference on Empirical Methods in Natural Language Processing (EMNLP)
  65. Extracting laboratory test information from paper-based reports
    Mingwei Ma, Xianshu Gao, Zeyu Zhang, Shiyu Shang, Ling Jin, Peilin Liu, Feng Lv, Wei Ni, Yuchen Han, Hui Zong · 2023 · BMC Medical Informatics and Decision Making
  66. MicroBERT: Distilling MoE-Based Knowledge from BERT into a Lighter Model
    Dashun Zheng, Jiaxuan Li, Yunchu Yang, Yapeng Wang, Patrick Cheong‐Iao Pang · 2024 · Applied Sciences
  67. MINOTAUR: A Posit-Based 0.42–0.50-TOPS/W Edge Transformer Inference and Training Accelerator
    Kartik Prabhu, Robert M. Radway, Jeffrey J. Yu, Kai Bartolone, Massimo Giordano, Fabian Peddinghaus, Yonatan Urman, Win-San Khwa, Yu-Der Chih, Meng‐Fan Chang et al. · 2025 · IEEE Journal of Solid-State Circuits
  68. The Vision Behind MLPerf: Understanding AI Inference Performance
    Vijay Janapa Reddi, Christine Cheng, David E. Kanter, Peter R. Mattson, Guenther Schmuelling, Carole-Jean Wu · 2021 · IEEE Micro
  69. PictoBERT: Transformers for next pictogram prediction
    Jayr Pereira, David Macêdo, Cleber Zanchettin, Adriano L. I. Oliveira, Robson Fidalgo · 2022 · Expert Systems with Applications
  70. Dataset Distillation with Attention Labels for Fine-tuning BERT
    Aru Maekawa, Naoki Kobayashi, Kotaro Funakoshi, Manabu Okumura · 2023 · Proceedings of the conference - Association for Computational Linguistics. Meeting
  71. On the effectiveness of small, discriminatively pre-trained language representation models for biomedical text mining
    İbrahim Burak Özyurt · 2020 · Workshop on Scholarly Document Processing (SDP)
  72. Universal-KD: Attention-based Output-Grounded Intermediate Layer Knowledge Distillation
    Yimeng Wu, Mehdi Rezagholizadeh, Abbas Ghaddar, Md. Akmal Haidar, Ali Ghodsi · 2021 · Conference on Empirical Methods in Natural Language Processing (EMNLP)
  73. RAIL-KD: RAndom Intermediate Layer Mapping for Knowledge Distillation
    Md. Akmal Haidar, Nithin Anchuri, Mehdi Rezagholizadeh, Abbas Ghaddar, Philippe Langlais, Pascal Poupart · 2022 · Findings of the Association for Computational Linguistics: NAACL
  74. Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm
    Shaoyi Huang, Dongkuan Xu, Ian En-Hsu Yen, Yijue Wang, Sung-En Chang, Bingbing Li, Shiyang Chen, Mimi Xie, Sanguthevar Rajasekaran, Hang Liu et al. · 2022 · Proceedings of the conference - Association for Computational Linguistics. Meeting
  75. SkipBERT: Efficient Inference with Shallow Layer Skipping
    Jue Wang, Ke Chen, Gang Chen, Lidan Shou, Julian McAuley · 2022 · Proceedings of the conference - Association for Computational Linguistics. Meeting
  76. BEBERT: Efficient and robust binary ensemble BERT
    Tian, Jiayi, Fang, Chao, Wang, Haonan, Wang, Zhongfeng · 2023 · ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
  77. Reservoir Transformers
    Sheng Shen, Alexei Baevski, Ari S. Morcos, Kurt Keutzer, Michael Auli, Douwe Kiela · 2021 · Annual Meeting of the Association for Computational Linguistics
  78. AdapLeR: Speeding up Inference by Adaptive Length Reduction
    Modarressi, Ali, Mohebbi, Hosein, Pilehvar, Mohammad Taher · 2022 · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
  79. Intelligent data analysis in edge computing with large language models: applications, challenges, and future directions
    Xuanzheng Wang, Zhipeng Xu, Xingfei Sui · 2025 · Frontiers in Computer Science
  80. EssayGAN: Essay Data Augmentation Based on Generative Adversarial Networks for Automated Essay Scoring
    Yo-Han Park, Yong‐Seok Choi, Cheon-Young Park, Kong-Joo Lee · 2022 · Applied Sciences
  81. Neural Architecture Search for Parameter-Efficient Fine-tuning of Large Pre-trained Language Models
    Neal Lawton, Anoop Kumar, Govind Thattai, Aram Galstyan, Greg Ver Steeg · 2023 · Findings of the Association for Computational Linguistics: ACL (Findings of ACL)
  82. Efficient Processing of MLPerf Mobile Workloads Using Digital Compute-In-Memory Macros
    Xiaoyu Sun, Weidong Cao, Brian Crafton, Kerem Akarvardar, Haruki Mori, Hidehiro Fujiwara, Hiroki Noguchi, Yu-Der Chih, Meng‐Fan Chang, Yih Wang et al. · 2023 · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
  83. Selective Amnesia: On Efficient, High-Fidelity and Blind Suppression of Backdoor Effects in Trojaned Machine Learning Models
    Rui Zhu, Di Tang, Siyuan Tang, Xiaofeng Wang, Haixu Tang · 2023 · IEEE Symposium on Security and Privacy (S&P)
  84. Mini-3DCvT: a lightweight lip-reading method based on 3D convolution visual transformer
    Huijuan Wang, Boyan Cui, Quanbo Yuan, Gangqiang Pu, Xueli Liu, Jie Zhu · 2024 · The Visual Computer
  85. Stream: Design Space Exploration of Layer-Fused DNNs on Heterogeneous Dataflow Accelerators
    Arne Symons, Linyan Mei, Steven Colleman, Pouya Houshmand, Sebastian Karl, Marian Verhelst · 2024 · IEEE Transactions on Computers
  86. Will this Question be Answered? Question Filtering via Answer Model Distillation for Efficient Question Answering
    Siddhant Garg, Alessandro Moschitti · 2021 · Conference on Empirical Methods in Natural Language Processing (EMNLP)
  87. Knowledge Distillation Meets Few-Shot Learning: An Approach for Few-Shot Intent Classification Within and Across Domains
    Anna Sauer, Shima Asaadi, Fabian Kuech · 2022 · Workshop on NLP for Conversational AI (NLP4ConvAI)
  88. Distilling Large Language Models into Tiny and Effective Students using pQRNN
    Prabhu Kaliamoorthi, Aditya Siddhant, Edward Li, Melvin Johnson · 2021 · arXiv (Cornell University)
  89. FLiText: A Faster and Lighter Semi-Supervised Text Classification with Convolution Networks
    Chen Liu, Zhang Mengchao, FU Zhi-bing, Panpan Hou, Yu Li · 2021 · Conference on Empirical Methods in Natural Language Processing (EMNLP)
  90. Knowledge Distillation of Russian Language Models with Reduction of Vocabulary
    Alina Kolesnikova, Yuri Kuratov, Vasily Konovalov, Mikhail Mikhail · 2022 · Computational Linguistics and Intellectual Technologies
  91. BBVD: A BERT-based Method for Vulnerability Detection
    Wei‐Chang Huang, Shuyuan Lin, Chen Li · 2022 · International Journal of Advanced Computer Science and Applications
  92. Identifying Topics of Scientific Articles with BERT-Based Approaches and Topic Modeling
    Anna Valer'evna Glazkova · 2021 · Lecture notes in computer science
  93. Self-attention fusion and adaptive continual updating for multimodal federated learning with heterogeneous data
    Kangning Yin, Zhen Ding, Xinhui Ji, Zhiguo Wang · 2025 · Neural Networks
  94. Analysis of Machine Learning Techniques for Information Classification in Mobile Applications
    Sandra Pérez Arteaga, Ana Lucila Sandoval Orozco, Luis Javier García Villalba · 2023 · Applied Sciences
  95. Uncertainty-Driven Knowledge Distillation for Language Model Compression
    Tianyu Huang, Weisheng Dong, Fangfang Wu, Xin Li, Guangming Shi · 2023 · IEEE/ACM Transactions on Audio Speech and Language Processing
  96. A comparative study on the impact of model compression techniques on fairness in language models
    Ramesh, Krithika, Chavan, Arnav, Pandit, Shrey, Sitaram, Sunayana · 2023 · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
  97. Towards Data- and Compute-Efficient Fake-News Detection: An Approach Combining Active Learning and Pre-Trained Language Models
    Francesco Folino, Gianluigi Folino, Massimo Guarascio, Luigi Pontieri, Paolo Zicari · 2024 · SN Computer Science
  98. LPViT: Low-Power Semi-structured Pruning for Vision Transformers
    Kaixin Xu, Zhe Wang, Chunyun Chen, Xue Geng, Jie Lin, Xulei Yang, Min Wu, Xiaoli Li, Weisi Lin · 2024 · Lecture notes in computer science
  99. ROSITA: Refined BERT cOmpreSsion with InTegrAted techniques
    Yuanxin Liu, Zheng Lin, Fengcheng Yuan · 2021 · Proceedings of the AAAI Conference on Artificial Intelligence
  100. Improving question answering performance using knowledge distillation and active learning
    Yasaman Boreshban, Seyed Morteza Mirbostani, Gholamreza Ghassem-Sani, Seyed Abolghasem Mirroshandel, Shahin Amiriparian · 2023 · Engineering Applications of Artificial Intelligence