Training Language Models to Follow Instructions with Human Feedback
The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
Making language models bigger does not inherently make them better at following a user's intent. For example, large language models can generate outputs that are untruthful, toxic, or simply not helpful to the user. In other words, these models are not aligned with their users. In this paper, we show an avenue for aligning language models with user intent on a wide range of tasks by fine-tuning with human feedback. Starting with a set of labeler-written prompts and prompts submitted through the OpenAI API, we collect a dataset of labeler demonstrations of the desired model behavior, which we use to fine-tune GPT-3 using supervised learning. We then collect a dataset of rankings of model outputs, which we use to further fine-tune this supervised model using reinforcement learning from human feedback. We call the resulting models InstructGPT. In human evaluations on our prompt distribution, outputs from the 1.3B parameter InstructGPT model are preferred to outputs from the 175B GPT-3, despite having 100x fewer parameters. Moreover, InstructGPT models show improvements in truthfulness and reductions in toxic output generation while having minimal performance regressions on public NLP datasets. Even though InstructGPT still makes simple mistakes, our results show that fine-tuning with human feedback is a promising direction for aligning language models with human intent.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Existing literature (Ouyang et al., 2022) does not fully describe the details of these models, such as the pretraining corpus, model sizes, and model biases.”
From Self-Refine: Iterative Refinement with Self-Feedback (2023) · cites this paper as Ouyang et al., 2022
Experimental — accuracy not yet measuredIn the citing paper
“Specifically, providers mitigate race and gender bias through reinforcement learning from human feedback (RLHF), adversarial stress-testing known as red teaming, and hard-coded refusals governed by the provider's own content policies [6, 43, 71].”
From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [71]
In the citing paper
“Fine-tuning a model from human feedback at scale, using preference data from paid raters or drawn from a large and willing user base, is an especially potent technique.It has the potential not only to shape a model's responses to better reflect user norms (for better or worse), but also to filter out toxic language, improve factual accuracy, and mitigate the tendency to fabricate information.”
From Talking about Large Language Models (2024) · cites this paper as [26]
In the citing paper
“These include general competencies such as in-context learning (ICL) [13], chain-of-thought (CoT) [17] reasoning, and instruction following [25], as well as task-specialized skills like mathematical reasoning [30].”
From A Survey of Large Language Models (2026) · cites this paper as [25]
In the citing paper
“When external reward models or ground-truth answers are available, rejection sampling can also be applied to select higher-quality responses from multiple candidates [25].”
From A Survey of Large Language Models (2026) · cites this paper as [25]
41 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Items 501–600 of 4,340
Wen, B., Norel, R., Liu, J. C., Stappenbeck, T. S., Zulkernine, F. H., & Chen, H. (2024) · IEEE International Conference on Digital Health (ICDH) · doi:10.1109/icdh62654.2024.00027 · Cited by 59
MP5: A Multi-modal Open-ended Embodied System in Minecraft via Active Perception
Qin, Y., Zhou, E., Liu, Q., Yin, Z., Lu, S., Zhang, R., Qiao, Y., & Shao, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01543 · Cited by 113
GPT4Motion: Scripting Physical Motions in Text-to-Video Generation via Blender-Oriented GPT Planning
Lv, J., Huang, Y., Yan, M., Huang, J., Liu, J., Liu, Y., Wen, Y., Chen, X., & Chen, S. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00150 · Cited by 95
Abstractive Meeting Summarization: A Survey
Rennard, V., Shang, G., Hunter, J., & Vazirgiannis, M. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00578 · Cited by 73
Large Language Models Can Learn Temporal Reasoning
Xiong, S., Payani, A., Kompella, R., & Fekri, F. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.563 · Cited by 47
Systematic Biases in LLM Simulations of Debates
Taubenfeld, A., Dover, Y., Reichart, R., & Goldstein, A. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.16 · Cited by 47
“Self Alignment In recent years, the task of aligning LLMs with human intentions has become a significant area of research (Ouyang et al., 2022; Wang et al., 2023).”
Zandi, R., Fahey, J. D., Drakopoulos, M., Bryan, J. M., Dong, S., Bryar, P. J., Bidwell, A. E., Bowen, R. C., Lavine, J. A., & Mirza, R. G. (2024) · Bioengineering · doi:10.3390/bioengineering11020120 · Cited by 52
MaViLa: Unlocking new potentials in smart manufacturing through vision language models
Fan, H., Liu, C., Janvisloo, N. E., Bian, S., Fuh, J. Y. H., Lu, W. F., & Li, B. (2025) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2025.02.017 · Cited by 52
The Foundation Model Transparency Index
Bommasani, R., Klyman, K., Longpre, S., Kapoor, S., Maslej, N., Xiong, B., Zhang, D., & Liang, P. (2023) · arXiv · doi:10.48550/arxiv.2310.12941 · Cited by 46
Decentralized Machine Learning Governance: Overview, Opportunities, and Challenges
Alsagheer, D., Xu, L., & Shi, W. (2023) · IEEE Access · doi:10.1109/access.2023.3311713 · Cited by 61
Azadi, A., Gorjinejad, F., Mohammad‐Rahimi, H., Tabrizi, R., Alam, M. M. R., & Golkar, M. (2024) · Oral Surgery Oral Medicine Oral Pathology and Oral Radiology · doi:10.1016/j.oooo.2024.02.018 · Cited by 48
Yang, L., Wang, Z., Li, Z., Na, J., & Yu, J. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103724 · Cited by 59
AgentLens: Visual Analysis for Agent Behaviors in LLM-Based Autonomous Systems
Lu, J., Pan, B., Chen, J., Feng, Y., Hu, J., Peng, Y., & Chen, W. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3394053 · Cited by 64
Song, B., Yang, H., Wu, Y., Zhang, P., Wang, B., & Han, G. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3411398 · Cited by 56
ChatGPT, the perfect virtual teaching assistant? Ideological bias in learner-chatbot interactions
Poucke, M. V. (2024) · Computers & composition/Computers and composition · doi:10.1016/j.compcom.2024.102871 · Cited by 48
X-VARS: Introducing Explainability in Football Refereeing with Multi-Modal Large Language Models
Held, J., Itani, H., Cioppa, A., Giancola, S., Ghanem, B., & Droogenbroeck, M. V. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00332 · Cited by 65
Incremental learning of humanoid robot behavior from natural interaction and large language models
Bärmann, L., Kartmann, R., Peller-Konrad, F., Niehues, J., Waibel, A., & Asfour, T. (2024) · Frontiers in Robotics and AI · doi:10.3389/frobt.2024.1455375 · Cited by 60
Ding, W., Abdel‐Basset, M., Ali, A. M., & Moustafa, N. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.112663 · Cited by 54
Virtual Cities: From Digital Twins to Autonomous AI Societies
Nechesov, A. V., Dorokhov, I., & Ruponen, J. (2025) · IEEE Access · doi:10.1109/access.2025.3531222 · Cited by 51
A large language model-based building operation and maintenance information query
Li, Y., Ji, M., Chen, J., Wei, X., Xiao-hong, G., & Tang, J. (2025) · Energy and Buildings · doi:10.1016/j.enbuild.2025.115515 · Cited by 53
Using fine-tuned large language models to parse clinical notes in musculoskeletal pain disorders
Vaid, A., Landi, I., Nadkarni, G., & Nabeel, I. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00202-9 · Cited by 45
Exploring ChatGPT for next-generation information retrieval: Opportunities and challenges
Huang, Y., & Huang, J. X. (2024) · Web Intelligence · doi:10.3233/web-230363 · Cited by 56
Are large language models qualified reviewers in originality evaluation?
Huang, S., Huang, Y., Liu, Y., Luo, Z., & Lu, W. P. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103973 · Cited by 58
Federated Learning in Practice: Reflections and Projections
Daly, K., Eichner, H., Kairouz, P., McMahan, H. B., Ramage, D., & Xu, Z. (2024) · IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA) · doi:10.1109/tps-isa62245.2024.00026 · Cited by 70
CHEAT: A Large-scale Dataset for Detecting ChatGPT-writtEn AbsTracts
Yu, P., Chen, J., Feng, X., & Xia, Z. (2023) · arXiv · doi:10.48550/arxiv.2304.12008 · Cited by 45
SF-IQA: Quality and Similarity Integration for AI Generated Image Quality Assessment
Yu, Z., Guan, F., Lu, Y., Li, X., & Chen, Z. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00663 · Cited by 67
Applications and perspectives of Generative Artificial Intelligence in agriculture
Pallottino, F., Violino, S., Figorilli, S., Pane, C., Aguzzi, J., Colle, G., Nemmi, E. N., Montaghi, A., Chatzievangelou, D., Antonucci, F., et al. (2025) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2025.109919 · Cited by 50
Chen, D., & Huang, Y. (2025) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2025.110028 · Cited by 54
Optical multilayer thin film structure inverse design: From optimization to deep learning
Ma, T., Ma, M., & Guo, L. J. (2025) · iScience · doi:10.1016/j.isci.2025.112222 · Cited by 60
LLMs: A game-changer for software engineers?
Haque, M. A. (2025) · BenchCouncil Transactions on Benchmarks Standards and Evaluations · doi:10.1016/j.tbench.2025.100204 · Cited by 55
Enhanced automated code vulnerability repair using large language models
de-Fitero-Dominguez, D., Garcia-Lopez, E., Garcia-Cabot, A., & Martinez-Herraiz, J.-J. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109291 · Cited by 43
Learning Hierarchical Modular Networks for Video Captioning
Li, G., Ye, H., Qi, Y., Wang, S., Qing, L., Huang, Q., & Yang, M. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3327677 · Cited by 47
Wang, H., & Li, Y. (2023) · International Conference on System Reliability and Safety Engineering (SRSE) · doi:10.1109/srse59585.2023.10336112 · Cited by 58
A review of dose prediction methods for tumor radiation therapy
Kui, X., Liu, F., Yang, M., Wang, H., Liu, C., Huang, D., Li, Q., Chen, L., & Zou, B. (2024) · Meta-Radiology · doi:10.1016/j.metrad.2024.100057 · Cited by 45
A Review of Advancements and Applications of Pre-Trained Language Models in Cybersecurity
Liu, Z. (2024) · International Symposium on Digital Forensics and Security (ISDFS) · doi:10.1109/isdfs60797.2024.10527236 · Cited by 56
RadVLM: a multitask conversational vision-language model for radiology
Deperrois, N., Matsuo, H., Ruipérez-Campillo, S., Vandenhirtz, M., Laguna, S., Ryser, A., Fujimoto, K., Nishio, M., Sutter, T. M., Vogt, J. E., et al. (2026) · Scientific Reports · doi:10.1038/s41598-026-66181-1 · Cited by 50
Yi, Z., Xiao, T., & Albert, M. V. (2025) · Information · doi:10.3390/info16020136 · Cited by 53
The Responsible Development of Automated Student Feedback with Generative AI
Lindsay, E. D., Zhang, M., Johri, A., & Bjerva, J. (2025) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon62633.2025.11016572 · Cited by 51
Large models in medical imaging: Advances and prospects
Fang, M., Wang, Z., Pan, S., Feng, X., Zhao, Y., Hou, D., Wu, L., Xie, X., Zhang, X.-Y., Tian, J., et al. (2025) · Chinese Medical Journal · doi:10.1097/cm9.0000000000003699 · Cited by 59
CAST: Counterfactual Labels Improve Instruction Following in Vision-Language-Action Models
Glossop, C., Chen, W., Bhorkar, A., Shah, D., & Levine, S. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3726383 · Cited by 55
Chain-of-Verification Reduces Hallucination in Large Language Models
Dhuliawala, S., Komeili, M., Xu, J., Raileanu, R., Li, X., Celikyilmaz, A., & Weston, J. (2023) · arXiv · doi:10.48550/arxiv.2309.11495 · Cited by 42
“CoVe-based Llama outperforms InstructGPT, ChatGPT and PerplexityAI Applying CoVe to Llama 65B lifts its performance above ChatGPT and InstructGPT, as well as PerplexityAI.”
BackgroundResultEvent-based Simultaneous Localization and Mapping: A Comprehensive Survey
Huang, K., Zhang, S., Zhang, J., & Tao, D. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02845-9 · Cited by 45
Brewer, J., Patel, D., Kim, D., & Murray, A. M. (2024) · Business Horizons · doi:10.1016/j.bushor.2024.04.011 · Cited by 49
Guo, J., Wang, M., Yin, H., Song, B., Chi, Y., Yu, F. R., & Yuen, C. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3496491 · Cited by 51
Prompt-to-SQL Injections in LLM-Integrated Web Applications: Risks and Defenses
Pedro, R., Coimbra, M. E., de Castro, D. A., Carreira, P., & Santos, N. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00007 · Cited by 72
DeepMesh: Auto-Regressive Artist-Mesh Creation with Reinforcement Learning
Zhao, R., Ye, J., Wang, Z., Liu, G., Chen, Y., Wang, Y., & Zhu, J. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00988 · Cited by 87
Cheung, T.-H., & Lam, K. (2023) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipaasc58517.2023.10317251 · Cited by 98
A Framework for LLM-Assisted Smart Policing System
Sarzaeim, P., Mahmoud, Q. H., & Azim, A. (2024) · IEEE Access · doi:10.1109/access.2024.3404862 · Cited by 48
The Evolution of Large Language Model: Models, Applications and Challenges
Sindhu, B., Prathamesh, R. P., Sameera, M. B., & Kumaraswamy, S. (2024) · International Conference on Current Trends in Advanced Computing (ICCTAC) · doi:10.1109/icctac61556.2024.10581180 · Cited by 69
Chhun, C., Suchanek, F. M., & Clavel, C. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00689 · Cited by 77
Language Models as Black-Box Optimizers for Vision-Language Models
Liu, S., Yu, S., Lin, Z., Pathak, D., & Ramanan, D. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01206 · Cited by 72
Zhang, Y., Li, J., Wang, Z., He, Z., Guan, Q., Lin, J., & Yu, W. (2024) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2024.104312 · Cited by 65
RAGVA: Engineering retrieval augmented generation-based virtual assistants in practice
Yang, R., Fu, M. C., Tantithamthavorn, C. K., Arora, C., Vandenhurk, L., & Chua, J. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112436 · Cited by 47
A Large-Scale Empirical Study on Fine-Tuning Large Language Models for Unit Testing
Shang, Y., Zhang, Q., Fang, C., Gu, S., Zhou, J., & Chen, Z. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728951 · Cited by 54
R1-T1: Fully Incentivizing Translation Capability in LLMs via Reasoning Learning
He, M., Liu, Y., Tao, S., Zeng, H., Zhang, J., Luo, Y., Zhang, L., Wei, D., Meng, W., & Yoshie, O. (2026) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.793 · Cited by 41
VisRL: Intention-Driven Visual Perception via Reinforced Reasoning
Chen, Z., Luo, X., & Li, D. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00245 · Cited by 52
PoseFix: Correcting 3D Human Poses with Natural Language
Delmas, G., Weinzaepfel, P., Moreno-Noguer, F., & Rogez, G. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01379 · Cited by 68
Luo, S. X., Chen, W., Tian, W., Liu, R., Hou, L., Zhang, X., Shen, H., Wu, R., Geng, S., Zhou, Y., et al. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3406372 · Cited by 63
How to Prompt Your Robot: A PromptBook for Manipulation Skills with Code as Policies
Arenas, M. G., Xiao, T. G., Singh, S., Jain, V., Ren, A. Z., Vuong, Q., Varley, J., Herzog, A., Leal, I. M. P., Kirmani, S., et al. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610784 · Cited by 64
A Critical Review of Methods and Challenges in Large Language Models
Asgari, R., Moradi, M., Yan, K., Colwell, D., & Samwald, M. (2025) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2025.061263 · Cited by 48
MoE 2 : Optimizing Collaborative Inference for Edge Large Language Models
Jin, L., Zhang, Y., Li, Y., Wang, S., Yang, H. H., Wu, J., & Zhang, M. (2026) · IEEE Transactions on Networking · doi:10.1109/ton.2026.3677371 · Cited by 47
Self-citationLeng, Z., Kwon, H., & Ploetz, T. (2023) · Proceedings of the 2023 International Symposium on Wearable Computers · doi:10.1145/3594738.3611361 · Cited by 39
Cultural Adaptation of Recipes
Cao, Y., Kementchedjhieva, Y., Cui, R., Karamolegkou, A., Zhou, L., Dare, M., Donatelli, L. E., & Hershcovich, D. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00634 · Cited by 82
Potential to transform words to watts with large language models in battery research
Zhao, S., Chen, S., Zhou, J., Li, C., Tang, T., Harris, S. J., Liu, Y., Wan, J., & Li, X. (2024) · Cell Reports Physical Science · doi:10.1016/j.xcrp.2024.101844 · Cited by 53
Open-World Recognition in Remote Sensing: Concepts, challenges, and opportunities
Fang, L., Yang, Z., Ma, T., Yue, J., Xie, W., Ghamisi, P., & Li, J. (2024) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2024.3382510 · Cited by 41
How Good (Or Bad) Are LLMs at Detecting Misleading Visualizations?
Lo, L. Y., & Qu, H. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3456333 · Cited by 59
Ask-EDA: A Design Assistant Empowered by LLM, Hybrid RAG and Abbreviation De-hallucination
Shi, L., Kazda, M., Sears, B., Shropshire, N., & Puri, R. (2024) · IEEE LLM Aided Design Workshop (LAD) · doi:10.1109/lad62341.2024.10691824 · Cited by 40
A safety realignment framework via subspace-oriented model fusion for large language models
Yi, X., Zheng, S., Wang, L., Wang, X., & He, L. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112701 · Cited by 63
Exploring Embodied Multimodal Large Models: Development, datasets, and future directions
Chen, S., Wu, Z., Zhang, K., Li, C., Zhang, B., Ma, F., Yu, F. R., & Li, Q. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103198 · Cited by 57
Xu, C., & Cho, S.-E. (2025) · Sustainability · doi:10.3390/su17104335 · Cited by 42
Xie, H., Wen, B., Chen, Z., Hong, F., & Liu, Z. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02999-6 · Cited by 51
Artificial general intelligence for radiation oncology
Liu, C., Liu, Z., Holmes, J., Zhang, L., Zhang, L., Ding, Y., Shu, P., Wu, Z., Dai, H., Li, Y., et al. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100045 · Cited by 50
Hallucination of Multimodal Large Language Models: A Survey
Bai, Z., Wang, P., Xiao, T., He, T., Han, Z., Zhang, Z., & Shou, M. Z. (2024) · arXiv · doi:10.48550/arxiv.2404.18930 · Cited by 38
“(RLHF) [19, 84, 94] emerges as a notable approach for achieving alignment through reinforcement learning.”
Organizational preparedness for the use of large language models in pathology informatics
Hart, S. N., Hoffman, N. G., Gershkovich, P., Christenson, C., McClintock, D. S., Miller, L. J., Jackups, R., Azimi, V., Spies, N., & Brodsky, V. (2023) · Journal of Pathology Informatics · doi:10.1016/j.jpi.2023.100338 · Cited by 36
Self-citationUnderstanding natural language: Potential application of large language models to ophthalmology
Yang, Z., Wang, D., Zhou, F., Song, D., Zhang, Y., Jiang, J., Kong, K., Liu, X., Qiao, Y., Chang, R. T., et al. (2024) · Asia-Pacific Journal of Ophthalmology · doi:10.1016/j.apjo.2024.100085 · Cited by 50
Wang, M., Su, L., Yan, C., Xu, S. H., Yuan, P., Jiang, X., & Zhang, B. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3439395 · Cited by 54
Self-citationChatAssert: LLM-Based Test Oracle Generation With External Tools Assistance
Hayet, I., Scott, A., & d’Amorim, M. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3519159 · Cited by 47
Xu, K., Zhang, K., Li, J., Huang, W. D., & Wang, Y. (2024) · Electronics · doi:10.3390/electronics14010047 · Cited by 51
Emo-DPO: Controllable Emotional Speech Synthesis through Direct Preference Optimization
Gao, X., Zhang, C., Chen, Y., Zhang, H., & Chen, N. F. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888737 · Cited by 90
Self-citationA Technical Survey of Reinforcement Learning Techniques for Large Language Models
Srivastava, S. S., & Aggarwal, V. (2026) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3834858 · Cited by 46
Reinforcement learning control of quantum error correction
Sivak, V., Morvan, A., Broughton, M., Cortiñas, R. G., Bausch, J., Senior, A., Neeley, M., Eickbusch, A., Shutty, N., Beni, L. A., et al. (2026) · Nature · doi:10.1038/s41586-026-10759-2 · Cited by 61
Alignment of Diffusion Models: Fundamentals, Challenges, and Future
Liu, B., Shao, S., Li, B., Bai, L., Xu, Z., Xiong, H., Kwok, J. T.-Y., Helal, S., & Xie, Z. (2026) · ACM Computing Surveys · doi:10.1145/3796982 · Cited by 48
Yuan, X., Ning, L., Ye, Q., Fan, W., & Li, Q. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809680 · Cited by 45
MELTing Point: Mobile Evaluation of Language Transformers
Laskaridis, S., Katevas, K., Minto, L., & Haddadi, H. (2024) · Proceedings of the 30th Annual International Conference on Mobile Computing and Networking · doi:10.1145/3636534.3690668 · Cited by 37
“Last, alignment is usually the final step of model tuning, typically through reinforcement learning from human [75] or automated [9] feedback, to promote a certain style or content or response that “aligns” with values of the creator (e.g., safety).”
Hartmann, J., Schwenzow, J., & Witte, M. (2023) · arXiv · doi:10.48550/arxiv.2301.01768 · Cited by 37
“For details on the training procedure, see (48) and (53).”
BackgroundFine-grained Affective Processing Capabilities Emerging from Large Language Models
Broekens, J., Hilpert, B., Verberne, S., Baraka, K., Gebhard, P., & Plaat, A. (2023) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii59096.2023.10388177 · Cited by 73
Alomari, E. A. (2024) · Computer Modeling in Engineering & Sciences · doi:10.32604/cmes.2024.052256 · Cited by 51
Bang, J., Choi, Y., Kim, M., Kim, Y., & Rhu, M. (2024) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro61859.2024.00021 · Cited by 57
Next Generation XR Systems—Large Language Models Meet Augmented and Virtual Reality
Afzal, M. Z., Ali, S. A., Stricker, D., Eisert, P., Hilsmann, A., Pérez-Marcos, D., Bianchi, M., Crottaz‐Herbette, S., Ioris, R. D., Mangina, E., et al. (2025) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2025.3548554 · Cited by 41
Men, C., Han, Y., Wang, P., Tao, J., & Huang, C. (2025) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2025.3550999 · Cited by 37
Large Language Models for Human-Like Autonomous Driving: A Survey
Li, Y., Katsumata, K., Javanmardi, E., & Tsukada, M. (2024) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc58415.2024.10919629 · Cited by 53
AvatarShield: Visual Reinforcement Learning for Human-Centric Synthetic Video Detection
Xu, Z., Zhang, X., Huang, Q., Zhou, X., & Zhang, J. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02954-5 · Cited by 45
Xue, X., Yu, X., & Wang, F. (2023) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2023.123585 · Cited by 37
Exploring the effectiveness of instruction tuning in biomedical language processing
Rohanian, O., Nouriborji, M., Kouchaki, S., Nooralahzadeh, F., Clifton, L., & Clifton, D. A. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.103007 · Cited by 46
Large Language Models Cannot Self-Correct Reasoning Yet
Huang, J., Chen, X., Mishra, S., Zheng, H. S., Yu, A. W., Song, X., & Zhou, D. (2023) · arXiv · doi:10.48550/arxiv.2310.01798 · Cited by 712
“Expanding upon this, a notable advancement for LLMs was the incorporation of Reinforcement Learning from Human Feedback (RLHF) (Ouyang et al., 2022).”
MethodJARVIS -1: Open-World Multi-Task Agents With Memory-Augmented Multimodal Language Models
Wang, Z., Cai, S., Liu, A., Jin, Y., Hou, J., Zhang, B., Lin, H., He, Z., Zheng, Z., Yang, Y., et al. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3511593 · Cited by 36
INSTRUCTEVAL: Towards Holistic Evaluation of Instruction-Tuned Large Language Models
Chia, Y. K., Hong, P., Bing, L., & Poria, S. (2023) · arXiv · doi:10.48550/arxiv.2306.04757 · Cited by 36
“While most instruction-tuned LLMs are trained with supervised fine-tuning, this may not capture the nuances of human preferences compared to reinforcement learning from human feedback (Ouyang et al., 2022).”
BackgroundPlanGenLLMs: A Modern Survey of LLM Planning Capabilities
Wei, H., Zhang, Z., He, S., Xia, T., Pan, S., & Liu, F. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.958 · Cited by 36
Generating dermatopathology reports from gigapixel whole slide images with HistoGPT
Tran, M., Schmidle, P., Guo, R. R., Wagner, S. J., Koch, V., Lupperger, V., Novotny, B., Murphree, D. H., Hardway, H. D., D’Amato, M., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-60014-x · Cited by 36
Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Models
Ning, M., Zhu, B., Xie, Y., Lin, B., Cui, J., Yuan, L., Chen, D., & Yuan, L. (2023) · arXiv · doi:10.48550/arxiv.2311.16103 · Cited by 36
“5-based [ 35] metric.”
BackgroundMethodInfluential
Metadata
- Authors
- Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., & Lowe, R.
- Venue
- Advances in Neural Information Processing Systems 35
- Published
- 2022
- Pages
- 27730-27744
- ISSN
- 1049-5258 (ISSN-L)
- OpenAlex
- W7133211372
- Cited by
- 3,117
- References
- 0
- Citations by year
- 2026: 1421 · 2025: 544 · 2024: 852 · 2023: 297 · 2022: 3
- FWCI
- 234.27
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- neural information processing systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- instruction following, language models, human feedback, reinforcement learning from human feedback, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.