Efficient Fine-Tuning Large Language Models for Knowledge-Aware Response Planning
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Sparsely-Gated Mixture-of-Experts Layer comprises numerous simple feed-forward expert networks and a trainable gating network responsible for expert selection, facilitating dynamic and sparse activation of experts for each input instance while maintaining high computational efficiency.
“The Sparsely-Gated Mixture-of-Experts Layer, a key component of these models, comprises numerous simple feed-forward expert networks and a trainable gating network responsible for expert selection, which can facilitate the dynamic and sparse activation of experts for each input instance, maintaining high computational efficiency [293], [294], [110].”
From From Google Gemini to OpenAI Q* (Q-Star): A Survey on Reshaping the Generative Artificial Intelligence (AI) Research Landscape (2025) · cites this paper as [293]
Expert capacity management strategies regulate the processing abilities of individual experts by setting thresholds on the number of tokens each can handle, effectively averting bottlenecks and ensuring more efficient model operation, leading to improved training processes and heightened performance during complex computational tasks.
“Concurrently, the integration of expert capacity management strategies, emerges as a crucial approach in MoE models to regulate the processing abilities of individual experts by setting thresholds on the number of tokens each can handle, effectively averting bottlenecks and ensuring a more efficient and streamlined model operation, leading to improved training processes and heightened performance”
From From Google Gemini to OpenAI Q* (Q-Star): A Survey on Reshaping the Generative Artificial Intelligence (AI) Research Landscape (2025) · cites this paper as [293]
Effective load balancing is essential in MoE models to guarantee a uniform distribution of computational load among experts.
“Effective load balancing is essential in MoE models to guarantee a uniform distribution of computational load among experts”
From From Google Gemini to OpenAI Q* (Q-Star): A Survey on Reshaping the Generative Artificial Intelligence (AI) Research Landscape (2025) · cites this paper as [293]
The router network in MoE layers plays a pivotal role in achieving load balance, which is fundamental to the stability and overall performance of MoE models.
“with the router network in MoE layers, responsible for selecting the appropriate experts for processing specific tokens, playing a pivotal role in achieving this balance, which is fundamental to the stability and overall performance of MoE models [293], [289], [288], [300], [110].”
From From Google Gemini to OpenAI Q* (Q-Star): A Survey on Reshaping the Generative Artificial Intelligence (AI) Research Landscape (2025) · cites this paper as [293]
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821 · see citing works
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · see citing works
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · cited 1× · see citing works
“As a result, the generated answer may seem plausible but could be factually incorrect [20, 76, 75, 64, 58, 54, 47, 46].”
BackgroundUniversal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1031 · Cited by 4,100 · see citing works
Dense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575 · see citing works
Teaching Machines to Read and ComprehendPreprint
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,779 · see citing works
Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks
Gururangan, S., Marasović, A., Swayamdipta, S., Lo, K., Beltagy, I., Downey, D., & Smith, N. A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.740 · Cited by 2,103 · see citing works
Proceedings for the 5th International Conference on Learning Representations
Srivastava, A., & Sutton, C. A. (2017) · Cited by 1,829 · see citing works
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · Cited by 1,749 · cited 1× · see citing works
“In contrast, CLM, exemplified by GPT [41], predicts the next token in a sequence given its preceding context, showing remarkable success in text generation and other downstream applications [42, 22, 43].”
BackgroundReading Wikipedia to answer open-domain questions
Chen, D., Fisch, A., Weston, J., & Bordes, A. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1171 · Cited by 1,546 · see citing works
Multi-Task Deep Neural Networks for Natural Language Understanding
Liu, X., He, P., Chen, W., & Gao, J. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1441 · Cited by 1,216 · see citing works
On Faithfulness and Factuality in Abstractive Summarization
Maynez, J., Narayan, S., Bohnet, B., & McDonald, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.173 · Cited by 1,185 · see citing works
Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering
Izacard, G., & Grave, E. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.74 · Cited by 1,109 · see citing works
Concerning nonnegative matrices and doubly stochastic matrices
Sinkhorn, R., & Knopp, P. (1967) · Pacific Journal of Mathematics · doi:10.2140/pjm.1967.21.343 · Cited by 882 · see citing works
WikiQA: A Challenge Dataset for Open-Domain Question Answering
Yang, Y., Yih, W.-T., & Meek, C. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1237 · Cited by 829 · see citing works
Recipes for Building an Open-Domain Chatbot
Roller, S., Dinan, E., Goyal, N., Ju, D., Williamson, M., Liu, Y., Xu, J., Ott, M., Smith, E. M., Boureau, Y.-L., et al. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.24 · Cited by 784 · see citing works
Learning to Rank Short Text Pairs with Convolutional Deep Neural Networks
Severyn, A., & Moschitti, A. (2015) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2766462.2767738 · Cited by 736 · see citing works
Gated Self-Matching Networks for Reading Comprehension and Question Answering
Wang, W., Yang, N., Wei, F., Chang, B., & Zhou, M. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1018 · Cited by 669 · see citing works
How Much Knowledge Can You Pack into the Parameters of a Language Model?
Roberts, A., Raffel, C., & Shazeer, N. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.437 · Cited by 638 · see citing works
Retrieval Augmentation Reduces Hallucination in Conversation
Shuster, K., Poff, S., Chen, M., Kiela, D., & Weston, J. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.320 · Cited by 575 · see citing works
UNIFIEDQA: Crossing Format Boundaries with a Single QA System
Khashabi, D., Min, S., Khot, T., Sabharwal, A., Tafjord, O., Clark, P., & Hajishirzi, H. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.171 · Cited by 514 · see citing works
Sequence-to-Sequence Learning as Beam-Search Optimization
Wiseman, S., & Rush, A. M. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1137 · Cited by 485 · see citing works
End-to-End Open-Domain Question Answering with BERTserini
Yang, W., Xie, Y., Lin, A., Li, X., Tan, L., Xiong, K., Li, M., & Lin, J. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-4013 · Cited by 472 · see citing works
Event Extraction as Machine Reading Comprehension
Liu, J., Chen, Y., Liu, K., Bi, W., & Liu, X. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.128 · Cited by 318 · see citing works
A Unified Generative Framework for Various NER Subtasks
Yan, H., Gui, T., Dai, J., Guo, Q., Zhang, Z., & Qiu, X. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.451 · Cited by 299 · cited 1× · see citing works
“These advantages have led to significant performance improvements for many NLP tasks such as event extraction [31], named entity recognition [66], and dependency parsing [29].”
BackgroundSuper-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks
Wang, Y., Mishra, S., Alipoormolabashi, P., Kordi, Y., Mirzaei, A., Naik, A., Ashok, A., Dhanasekaran, A. S., Arunkumar, A., Stap, D., et al. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.340 · Cited by 247 · see citing works
Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction
Lu, Y., Lin, H., Xu, J., Han, X., Tang, J., Li, A., Sun, L., Liao, M., & Chen, S. (2021) · doi:10.18653/v1/2021.acl-long.217 · Cited by 230 · see citing works
Question Answering Over Temporal Knowledge Graphs
Saxena, A., Chakrabarti, S., & Talukdar, P. P. (2021) · doi:10.18653/v1/2021.acl-long.520 · Cited by 153 · see citing works
The Curious Case of Hallucinations in Neural Machine Translation
Raunak, V., Menezes, A., & Junczys-Dowmunt, M. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.92 · Cited by 142 · see citing works
Combining Retrieval, Statistics, and Inference to Answer Elementary Science Questions
Clark, P. E., Etzioni, O., Khot, T., Sabharwal, A., Tafjord, O., Turney, P. D., & Khashabi, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10325 · Cited by 139 · see citing works
Knowledge-Based Question Answering as Machine Translation
Bao, J., Duan, N., Zhou, M., & Zhao, T. (2014) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/v1/p14-1091 · Cited by 132 · see citing works
Detecting Hallucinated Content in Conditional Neural Sequence Generation
Zhou, C., Neubig, G., Gu, J., Diab, M., Guzmán, F., Zettlemoyer, L., & Ghazvininejad, M. (2021) · Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 · doi:10.18653/v1/2021.findings-acl.120 · Cited by 122 · see citing works
On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation
Wang, C., & Sennrich, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.326 · Cited by 118 · see citing works
Reducing Quantity Hallucinations in Abstractive Summarization
Zhao, Z., Cohen, S. B., & Webber, B. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.203 · Cited by 99 · see citing works
On hallucination and predictive uncertainty in conditional language generation
Xiao, Y., & Wang, W. Y. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.236 · Cited by 79 · see citing works
FitzGerald, J., Ananthakrishnan, S., Arkoudas, K., Bernardi, D., Bhagia, A., Delli Bovi, C., Cao, J., Chada, R., Chauhan, A., Chen, L., et al. (2022) · Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3534678.3539173 · Cited by 68 · see citing works
A Compare-Aggregate Model with Latent Clustering for Answer Selection
Yoon, S., Dernoncourt, F., Kim, D. S., Bui, T., & Jung, K. (2019) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3357384.3358148 · Cited by 67 · see citing works
Open-Domain Question Answering
Chen, D., & Yih, W.-T. (2020) · doi:10.18653/v1/2020.acl-tutorials.8 · Cited by 67 · see citing works
Controlling hallucinations at word level in data-to-text generation
Rebuffel, C., Roberti, M., Soulier, L., Scoutheeten, G., Cancelliere, R., & Gallinari, P. (2022) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-021-00801-4 · Cited by 50 · see citing works
Asking Clarification Questions in Knowledge-Based Question Answering
Xu, J., Wang, Y., Tang, D., Duan, N., Yang, P., Zeng, Q., Zhou, M., & Sun, X. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1172 · Cited by 46 · see citing works
TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selection
Garg, S., Vu, T., & Moschitti, A. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6282 · Cited by 30 · see citing works
Self-citationAnswer Generation for Retrieval-based Question Answering Systems
Hsu, C.-C., Lind, E., Soldaini, L., & Moschitti, A. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.374 · Cited by 21 · see citing works
Answer Sentence Selection Using Local and Global Context in Transformer Models
Lauriola, I., & Moschitti, A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-72113-8_20 · Cited by 16 · see citing works
WDRASS: A Web-scale Dataset for Document Retrieval and Answer Sentence Selection
Zhang, Z., Vu, T., Gandhi, S., Chadha, A., & Moschitti, A. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557678 · Cited by 8 · see citing works
Self-citationDependency Parsing via Sequence Generation
Lin, B., Yao, Z., Shi, J., Cao, S., Tang, B., Li, S., Luo, Y., Li, J., & Hou, L. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.543 · Cited by 4 · see citing works
Knowledge Transfer from Answer Ranking to Answer Generation
Gabburo, M., Koncel-Kedziorski, R., Garg, S., Soldaini, L., & Moschitti, A. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.645 · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
McIntosh, T. R., Susnjak, T., Liu, T., Watters, P., Xu, D., Liu, D., & Halgamuge, M. N. (2025) · Technologies · doi:10.3390/technologies13020051 · Cited by 102 · see citing works
“Its two primary approaches are end-to-end fine-tuning, which adjusts all weights of the encoder and classifier [260], [261], and feature-extraction fine-tuning, where the encoder weights are frozen to extract features for a downstream classifier [262], [263], [264].”
BackgroundParameter-efficient fine-tuning of large language models using semantic knowledge tuning
Prottasha, N. J., Mahmud, A., Sobuj, M. S. I., Bhat, P., Kowsher, M., Yousefi, N., & Garibay, Ö. Ö. (2024) · Scientific Reports · doi:10.1038/s41598-024-75599-4 · Cited by 30 · see citing works
Abdullahi, S., Danyaro, K. U., & Chiroma, H. (2026) · Cluster Computing · doi:10.1007/s10586-025-05891-z · Cited by 7 · see citing works
Unraveling the thread: understanding and addressing sequential failures in human-robot interaction
Tisserand, L., Stephenson, B., Baldauf-Quilliatre, H., Lefort, M., & Armetta, F. (2024) · Frontiers in Robotics and AI · doi:10.3389/frobt.2024.1359782 · Cited by 7 · see citing works
Beyond Topicality: Including Multidimensional Relevance in Cross-encoder Re-ranking
Upadhyay, R., Askari, A., Pasi, G., & Viviani, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56027-9_16 · Cited by 4 · see citing works
In Situ Answer Sentence Selection at Web-scale
Zhang, Z., Vu, T., & Moschitti, A. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679946 · Cited by 4 · see citing works
Self-citationIjebu, F. F., Liu, Y., Sun, C., Jere, N. R., Mienye, I. D., & Usip, P. U. (2025) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2025.3615995 · Cited by 2 · see citing works
Tiwari, A., & Gupta, V. (2026) · Research Square · doi:10.21203/rs.3.rs-8589677/v1
- Authors
- Nguyen, M., Kishan, K. C., Nguyen, T., Chadha, A., & Vu, T.
- Affiliations
- Amazon (United States) (US)
- Corresponding author
- Vu, Thuy
- ORCID
- Venue
- Lecture Notes in Computer Science
- Publisher
- Springer Nature Switzerland
- Published
- 2023
- Pages
- 593-611
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- OpenAlex
- W4386804446
- Cited by
- 8
- References
- 49
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 4
- FWCI
- 1.42
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- Lecture notes in computer science (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- large language models, fine-tuning, knowledge retrieval, hallucinations, external knowledge, open-domain question answering, web pages, databases, response planning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
