Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Models

Attention Is All You Need

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works

Language Models are Few-Shot Learners

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works

Proximal Policy Optimization Algorithms

Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · see citing works

Dissecting racial bias in an algorithm used to manage the health of populations

Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · see citing works

Knowledge Distillation: A Survey

Gou, J., Yu, B., Maybank, S. J., & Tao, D. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01453-z · Cited by 3,931 · see citing works

HuggingFace's Transformers: State-of-the-art Natural Language Processing

Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.03771 · Cited by 3,347 · see citing works

Transformer-XL: Attentive language models beyond a fixed-length context

Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · see citing works

The Power of Scale for Parameter-Efficient Prompt Tuning

Lester, B., Al-Rfou, R., & Constant, N. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.243 · Cited by 3,096 · see citing works

The Power of Scale for Parameter-Efficient Prompt Tuning

Lester, B., Al-Rfou, R., & Constant, N. (2021) · arXiv · doi:10.48550/arxiv.2104.08691 · Cited by 3,092

Big Data's Disparate Impact

Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · see citing works

Semantics derived automatically from language corpora contain human-like biases

Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887 · see citing works

DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation

Zhang, Y., Sun, S., Galley, M., Chen, Y.-C., Brockett, C., Gao, X., Gao, J., Liu, J., & Dolan, B. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.18653/v1/2020.acl-demos.30 · Cited by 1,076 · see citing works

Word embeddings quantify 100 years of gender and ethnic stereotypes

Garg, N., Schiebinger, L., Jurafsky, D., & Zou, J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1720347115 · Cited by 1,065 · see citing works

Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm

Reynolds, L., & McDonell, K. (2021) · Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411763.3451760 · Cited by 952 · see citing works

The Risk of Racial Bias in Hate Speech Detection

Sap, M., Card, D., Gabriel, S., Choi, Y., & Smith, N. A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1163 · Cited by 817 · see citing works

CTRL: A Conditional Transformer Language Model for Controllable Generation

Keskar, N. S., McCann, B., Varshney, L. R., Xiong, C., & Socher, R. (2019) · arXiv · doi:10.48550/arxiv.1909.05858 · Cited by 751 · see citing works

RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models

Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.301 · Cited by 693 · see citing works

Towards a Human-like Open-Domain Chatbot

Adiwardana, D., Luong, M.-T., So, D. R., Hall, J., Fiedel, N., Thoppilan, R., Yang, Z., Kulshreshtha, A., Nemade, G., Lu, Y., et al. (2020) · arXiv · doi:10.48550/arxiv.2001.09977 · Cited by 614 · see citing works

Learning to summarize from human feedback

Stiennon, N., Ouyang, L., Wu, J., Ziegler, D. M., Lowe, R., Voss, C., Radford, A., Amodei, D., & Christiano, P. (2020) · arXiv · doi:10.48550/arxiv.2009.01325 · Cited by 550 · see citing works

Artificial intelligence and algorithmic bias: implications for health systems

Panch, T., Mattie, H., & Atun, R. (2019) · Journal of Global Health · doi:10.7189/jogh.09.020318 · Cited by 533 · see citing works

Plug and Play Language Models: A Simple Approach to Controlled Text Generation

Dathathri, S., Madotto, A., Lan, J., Hung, J., Frank, E., Molino, P., Yosinski, J., & Liu, R. (2019) · arXiv · doi:10.48550/arxiv.1912.02164 · Cited by 508 · see citing works

The Pile: An 800GB Dataset of Diverse Text for Language Modeling

Gao, L., Biderman, S., Black, S., Golding, L., Hoppe, T., Foster, C., Phang, J., He, H., Thite, A., Nabeshima, N., et al. (2021) · arXiv · doi:10.48550/arxiv.2101.00027 · Cited by 492 · see citing works

Computational social science: Obstacles and opportunities

Lazer, D. M. J., Pentland, A., Watts, D. J., Aral, S., Athey, S., Contractor, N., Freelon, D., Gonzalez-Bailon, S., King, G., Margetts, H., et al. (2020) · Science · doi:10.1126/science.aaz8170 · Cited by 453 · see citing works

GeDi: Generative Discriminator Guided Sequence Generation

Krause, B., Gotmare, A. D., McCann, B., Keskar, N. S., Joty, S., Socher, R., & Rajani, N. F. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.424 · Cited by 220 · see citing works

DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts

Liu, A., Sap, M., Lu, X., Swayamdipta, S., Bhagavatula, C., Smith, N. A., & Choi, Y. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.522 · Cited by 199 · see citing works

QUARK: Controllable Text Generation with Reinforced Unlearning

Lu, X., Welleck, S., Hessel, J., Jiang, L., Qin, L., West, P., Ammanabrolu, P., & Choi, Y. (2022) · neural information processing systems · doi:10.52202/068431-2001 · Cited by 51 · see citing works

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805

LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models

Jiang, H., Wu, Q., Lin, C.-Y., Yang, Y., & Qiu, L. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.825 · Cited by 168 · see citing works

LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression

Pan, Z., Wu, Q., Jiang, H., Xia, M., Luo, X., Zhang, J., Lin, Q., Rühle, V., Yang, Y., Lin, C.-Y., et al. (2024) · Findings of the Association for Computational Linguistics: ACL 2024 · doi:10.18653/v1/2024.findings-acl.57 · Cited by 58 · see citing works

Adapting Language Models to Compress Contexts

Chevalier, A., Wettig, A., Ajith, A., & Chen, D. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.232 · Cited by 59 · see citing works

Resource-efficient Algorithms and Systems of Foundation Models: A Survey

Xu, M., Cai, D., Yin, W., Wang, S., Jin, X., & Liu, X. (2024) · ACM Computing Surveys · doi:10.1145/3706418 · Cited by 153 · see citing works

VoCo-LLaMA: Towards Vision Compression with Large Language Models

Ye, X., Gan, Y., Huang, X., Ge, Y., & Tang, Y. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02777 · Cited by 18 · see citing works

Discrete Prompt Compression With Reinforcement Learning

Jung, H., & Kim, K.-J. (2024) · IEEE Access · doi:10.1109/access.2024.3403426 · Cited by 17 · see citing works

A Unified and Resource-Aware Framework for Adaptive Inference Acceleration on Edge and Embedded Platforms

Wang, Y., & Zhao, J. (2025) · Electronics · doi:10.3390/electronics14112188 · Cited by 10 · see citing works

A Review of Wearables-Based Activities of Daily Living Recognition With LLMs: Overview, Progress, and Trends

Zheng, W., & Arakawa, Y. (2025) · IEEE Access · doi:10.1109/access.2025.3598088 · Cited by 10 · see citing works

Conversational Alignment With Artificial Intelligence in Context

Sterken, R., & Kirkpatrick, J. R. (2024) · Philosophical Perspectives · doi:10.1111/phpe.12205 · Cited by 10 · see citing works

Optimizing Retrieval-augmented Reader Models via Token Elimination

Berchansky, M., Izsak, P., Caciularu, A., Dagan, I., & Wasserblat, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.93 · Cited by 12 · see citing works

m2LLM: A Multi-Dimensional Optimization Framework for LLM Inference on Mobile Devices

Liu, K., Zhou, X., & Li, L. (2025) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2025.3587445 · Cited by 6 · see citing works

Make Large Language Models Efficient: A Review

Mussa, A., Туймебаев, Ж. К., & Мансурова, М. (2025) · IEEE Access · doi:10.1109/access.2025.3605110 · Cited by 5 · see citing works

FINCH: Prompt-guided Key-Value Cache Compression for Large Language Models

Corallo, G., & Papotti, P. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00716 · Cited by 5 · see citing works

CMT: A Memory Compression Method for Continual Knowledge Learning of Large Language Models

Li, D., Sun, Z., Hu, X., Hu, B., & Zhang, M. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i23.34619 · Cited by 4 · see citing works

A systematic review of toxicity in large language models: definitions, datasets, detectors, detoxification methods and challenges

Villate-Castillo, G., Ser, J. D., & Sanz, B. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11548-z · Cited by 4 · see citing works

AdaCoder: Adaptive Prompt Compression for Programmatic Visual Question Answering

Ukai, M., Kurita, S., Hashimoto, A., Ushiku, Y., & Inoue, N. (2024) · Proceedings of the 32nd ACM International Conference on Multimedia · doi:10.1145/3664647.3681010 · Cited by 1 · see citing works

Dynamic Prompt Compression for Efficient Inference of Large Language Models

Hu, J., Zhang, W., Wang, Y., Hu, Y., Xiao, B., Tan, M., & Du, Q. (2026) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2026.3669558 · Cited by 4 · see citing works

Coherent Entity Disambiguation via Modeling Topic and Categorical Dependency

Xiao, Z., Shou, L., Zhang, X., Wu, J., Gong, M., & Jiang, D. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.502 · Cited by 1 · see citing works

Context Compression and Extraction: Efficiency Inference of Large Language Models

Zhou, J., Du, R., Tan, Y., Yang, J., Yang, Z., Luo, W., Luo, Z., Zhou, X., & Hu, W. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-5663-6_19 · Cited by 1 · see citing works

Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality

Kong, Z., Li, Y., Zeng, F., Xin, L., Messica, S., Lin, X., Zhao, P., Kellis, M., Tang, H., & Zitnik, M. (2025) · arXiv · doi:10.48550/arxiv.2505.18227

Beyond Surface Linguistics

Šprogar, M. (2023) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-0230-9.ch005

GPT-C: Generative PrompT Compression

Liu, L., Wang, R., Jing, L., Lv, F., & Zhu, Z. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888801

Accelerating LLM Inference for Multi-Round Medical Q&A

Wang, Z., Xiong, A., & Shen, J. (2025) · International Symposium on Computer Technology and Information Science (ISCTIS) · doi:10.1109/isctis65944.2025.11065656

DynoInfer: Adaptive Resource Orchestration for LLM Inference on Resource-Constrained PCs

Chen, Y. Q., Lin, Q., Chen, Z., Ou, Y., & Chen, Z. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-99854-6_20

Al Compute Architecture and Evolution Trends [Feature]

Liang, B.-S. (2026) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2025.3650725 · Cited by 1 · see citing works

ModelFusion-RAG: A Prompt Compression Framework for Retrieval Enhanced Generation Based on Small Model Collaboration

Zhang, X., Pan, C., Xing, X., Wang, Y., Liang, R., & Li, J. (2025) · Asian Conference on Artificial Intelligence Technology (ACAIT) · doi:10.1109/acait67930.2025.11522017

SemCompress: Semantic-Aware Adaptive DocString Compression for Code Generation

Shen, Y., Xia, H., & Qiu, J. (2026) · International Conference on Information Science, Electrical and Automation Engineering (ISEAE) · doi:10.1109/iseae69422.2026.11544409

Graph-Aware Reinforcement Learning for Reusable Prompt Compression in Black-Box LLMsPreprint

Eskandarpour, M., Haddadnejad, P., & Jannati, M. J. (2026) · Research Square · doi:10.21203/rs.3.rs-10458568/v1

500xCompressor: Generalized Prompt Compression for Large Language Models

Li, Z., Su, Y., & Collier, N. (2025) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/2025.acl-long.1219 · Cited by 6 · see citing works

Style-Compress: An LLM-Based Prompt Compression Framework Considering Task-Specific Styles

Pu, X., He, T., & Wan, X. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.851

Say More with Less: Understanding Prompt Learning Behaviors through Gist CompressionPreprint

Li, X., Liu, Z., Xiong, C., Shi, Y., Yan, Y., Wang, S., & Yu, G. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.16058 · Cited by 1 · see citing works

LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models

Jiang, H., Wu, Q., Lin, C.-Y., Yang, Y., & Qiu, L. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.825 · Cited by 168 · see citing works

Plug and Play with Prompts: A Prompt Tuning Approach for Controlling Text GenerationPreprint

Ajwani, R. D., Zhu, Z., Rose, J., & Rudzicz, F. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.05143 · Cited by 2 · see citing works