Training Language Models to Follow Instructions with Human Feedback
The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
Making language models bigger does not inherently make them better at following a user's intent. For example, large language models can generate outputs that are untruthful, toxic, or simply not helpful to the user. In other words, these models are not aligned with their users. In this paper, we show an avenue for aligning language models with user intent on a wide range of tasks by fine-tuning with human feedback. Starting with a set of labeler-written prompts and prompts submitted through the OpenAI API, we collect a dataset of labeler demonstrations of the desired model behavior, which we use to fine-tune GPT-3 using supervised learning. We then collect a dataset of rankings of model outputs, which we use to further fine-tune this supervised model using reinforcement learning from human feedback. We call the resulting models InstructGPT. In human evaluations on our prompt distribution, outputs from the 1.3B parameter InstructGPT model are preferred to outputs from the 175B GPT-3, despite having 100x fewer parameters. Moreover, InstructGPT models show improvements in truthfulness and reductions in toxic output generation while having minimal performance regressions on public NLP datasets. Even though InstructGPT still makes simple mistakes, our results show that fine-tuning with human feedback is a promising direction for aligning language models with human intent.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Existing literature (Ouyang et al., 2022) does not fully describe the details of these models, such as the pretraining corpus, model sizes, and model biases.”
From Self-Refine: Iterative Refinement with Self-Feedback (2023) · cites this paper as Ouyang et al., 2022
Experimental — accuracy not yet measuredIn the citing paper
“Specifically, providers mitigate race and gender bias through reinforcement learning from human feedback (RLHF), adversarial stress-testing known as red teaming, and hard-coded refusals governed by the provider's own content policies [6, 43, 71].”
From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [71]
In the citing paper
“Fine-tuning a model from human feedback at scale, using preference data from paid raters or drawn from a large and willing user base, is an especially potent technique.It has the potential not only to shape a model's responses to better reflect user norms (for better or worse), but also to filter out toxic language, improve factual accuracy, and mitigate the tendency to fabricate information.”
From Talking about Large Language Models (2024) · cites this paper as [26]
In the citing paper
“These include general competencies such as in-context learning (ICL) [13], chain-of-thought (CoT) [17] reasoning, and instruction following [25], as well as task-specialized skills like mathematical reasoning [30].”
From A Survey of Large Language Models (2026) · cites this paper as [25]
In the citing paper
“When external reward models or ground-truth answers are available, rejection sampling can also be applied to select higher-quality responses from multiple candidates [25].”
From A Survey of Large Language Models (2026) · cites this paper as [25]
41 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Items 1,101–1,200 of 4,340
HexT5: Unified Pre-Training for Stripped Binary Code Information Inference
Xiong, J., Chen, G., Chen, K., Gao, H., Cheng, S., & Zhang, W. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00099 · Cited by 30
A Survey of Knowledge Graph Construction Using Machine Learning
Zhao, Z., Luo, X., Chen, M., & Ma, L. (2024) · Computer Modeling in Engineering & Sciences · doi:10.32604/cmes.2023.031513 · Cited by 24
What’s Next in Affective Modeling? Large Language Models
Yongsatianchot, N., Thejll-Madsen, T., & Marsella, S. (2023) · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) · doi:10.1109/aciiw59127.2023.10388124 · Cited by 20
Xu, Y., Liang, J., Zhuo, Y., Liu, L., Xiao, Y., & Zhou, L. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108103 · Cited by 15
Speak While You Think: Streaming Speech Synthesis During Text Generation
Dekel, A., Shechtman, S., Fernandez, R., Haws, D., Kons, Z., & Hoory, R. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446214 · Cited by 13
Controllable Speaking Styles Using A Large Language Model
Sigurgeirsson, A., & King, S. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448400 · Cited by 19
Multi-view fusion for instruction mining of large language model
Huang, H., Xu, B., Liang, X., Chen, K., Yang, M., Zhao, T., & Zhu, C. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102480 · Cited by 21
Li, M., Liu, Y., Giunchiglia, F., Li, X., Feng, X., & Guan, R. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792237 · Cited by 13
A Comparison of Large Language Models and Genetic Programming for Program Synthesis
Sobania, D., Petke, J., Briesch, M., & Rothlauf, F. (2024) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2024.3410873 · Cited by 13
Jones, C. R., Trott, S., & Bergen, B. K. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00674 · Cited by 35
CrowdTransfer: Enabling Crowd Knowledge Transfer in AIoT Community
Liu, Y., Guo, B., Li, N., Ding, Y., Zhang, Z., & Yu, Z. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3423319 · Cited by 17
A Large Language Model Approach to Code and Privacy Policy Alignment
Morales, G. A., Pragyan, K. C., Jahan, S., Hosseini, M. B., & Slavin, R. (2024) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner60148.2024.00016 · Cited by 19
ChatChisel: Enabling Agile Hardware Design with Large Language Models
Liu, T., Tian, Q., Ye, J., Fu, L., Su, S., Li, J., Wan, G.-W., Zhang, L., Wong, S.-Z., Wang, X., et al. (2024) · International Symposium of Electronics Design Automation (ISEDA) · doi:10.1109/iseda62518.2024.10618053 · Cited by 13
MindLLM: Lightweight large language model pre-training, evaluation and domain application
Yang, Y., Sun, H., Li, J., Liu, R., Li, Y., Liu, Y., Gao, Y., & Huang, H. (2024) · AI Open · doi:10.1016/j.aiopen.2024.08.001 · Cited by 18
SOK-Bench: A Situated Video Reasoning Benchmark with Aligned Open-World Knowledge
Wang, A., Wu, B., Chen, S., Chen, Z., Guan, H., Lee, W.-N., Li, L. E., & Gan, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01271 · Cited by 44
Ni, F., Hao, J., Wu, S., Kou, L., Liu, J., Zheng, Y., Wang, B., & Zhuang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01327 · Cited by 48
In Situ AI Prototyping: Infusing Multimodal Prompts into Mobile Settings with MobileMaker
Petridis, S., Liu, M. X., Fiannaca, A. J., Tsai, V., Terry, M. A., & Cai, C. J. (2024) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc60511.2024.00023 · Cited by 14
Cho, S., Kim, J., & Kim, J. H. (2024) · IEEE Access · doi:10.1109/access.2024.3502219 · Cited by 26
Gopali, S., Siami‐Namini, S., Abri, F., & Namin, A. S. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2024.100120 · Cited by 15
Empowering IoT-Based Autonomous Driving via Federated Instruction Tuning With Feature Diversity
Chen, J., He, J., Chen, F., Lv, Z., Tang, J., & Jia, Y. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3518615 · Cited by 19
Wang, P., Holmes, J., Liu, Z., Chen, D., Liu, T., Shen, J., & Liu, W. (2025) · Frontiers in Oncology · doi:10.3389/fonc.2025.1557064 · Cited by 17
EX-CODE: A Robust and Explainable Model to Detect AI-Generated Code
Bulla, L., Midolo, A., Mongiovì, M., & Tramontana, E. (2024) · Information · doi:10.3390/info15120819 · Cited by 17
Zhang, Z., Zhou, Y., & Ma, J. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.126251 · Cited by 15
Kuan, C.-Y., Yang, C.-K., Huang, W., Lu, K.-H., & Lee, H.-Y. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832184 · Cited by 32
Zhou, X., Yuan, Y., Yang, S., & Hao, J. (2025) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2025.3529902 · Cited by 15
Co-LLaVA: Efficient Remote Sensing Visual Question Answering via Model Collaboration
Liu, F., Dai, W., Zhang, C., Zhu, J., Yao, L. X., & Li, X. (2025) · Remote Sensing · doi:10.3390/rs17030466 · Cited by 16
Self-citationYang, L., Zhou, Y., Qi, J., Zhen, X., Sun, L., Shi, S. R., Su, Q., & Yang, X. (2025) · European Journal of Radiology · doi:10.1016/j.ejrad.2025.111984 · Cited by 14
Selective Reviews of Bandit Problems in AI via a Statistical View
Zhou, P., Wei, H., & Zhang, H. (2025) · Mathematics · doi:10.3390/math13040665 · Cited by 14
Mu, B., Wei, K., Shao, Q., Xu, Y., & Xie, L. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888133 · Cited by 25
Multi-View User Preference Modeling for Personalized Text-to-Image Generation
Zhang, H., Wu, T., & Wei, Y. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3557683 · Cited by 20
Videogamebunny: Towards Vision Assistants for Video Games
Taesiri, M. R., & Bezemer, C. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00144 · Cited by 19
Zhu, Z., Yu, T., Wang, Y., & Xu, J. (2025) · IEEE Access · doi:10.1109/access.2025.3567750 · Cited by 20
An Exploratory Assessment of LLMs’ Potential for Flight Trajectory Reconstruction Analysis
Zhang, Q., & Mott, J. H. (2025) · Mathematics · doi:10.3390/math13111775 · Cited by 13
Wang, B., Liu, Y., Tian, H., Hua, R., Chang, K., Xia, J., Dai, X., Gao, Z., Liu, S., Wang, R., et al. (2025) · Tsinghua Science & Technology · doi:10.26599/tst.2024.9010125 · Cited by 14
The Muse Benchmark: Probing Music Perception and Auditory Relational Reasoning in Audio LLMs
Carone, B. J., Roman, I. R., & Ripollés, P. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11462479 · Cited by 16
Reinforcement Fine-Tuning of Flow-Matching Policies for Vision-Language-Action Models
Lyu, M., Sun, Y., Lin, E., Li, H., Chen, R., Zhao, F., & Zeng, Y. (2026) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57385.2026.11696958 · Cited by 14
Value Drifts: Tracing Value Alignment During LLM Post-Training
Bhatia, M., Nayak, S., Kamath, G., Mosbach, M., Stańczak, K., Shwartz, V., & Reddy, S. (2026) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.790 · Cited by 20
State media control influences large language models
Waight, H. C., Yang, E., Yuan, Y., Messing, S., Roberts, M. E., Stewart, B. M., & Tucker, J. A. (2026) · Nature · doi:10.1038/s41586-026-10506-7 · Cited by 32
Reasoning BO: Enhancing Bayesian Optimization With the Reasoning Power of LLMs
Yang, Z., Wang, D., Ge, L., Wang, B., Fu, T., & Li, Y. (2026) · Materials Genome Engineering Advances · doi:10.1002/mgea.70086 · Cited by 15
Understanding Reward Hacking in Text-to-Image Reinforcement Learning
Hong, Y., Kao, K.-C., Zhou, H., & Hsieh, C. (2026) · doi:10.1109/cvprf71909.2026.00410 · Cited by 15
PoE: A Panel of Experts for Generalized Automatic Dialogue Assessment
Zhang, C., D’Haro, L. F., Zhang, Q., Friedrichs, T., & Li, H. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3250825 · Cited by 15
Self-citationEnhancing Conversational Model With Deep Reinforcement Learning and Adversarial Learning
Tran, Q.-D. L., Le, A.-C., & Huynh, V. (2023) · IEEE Access · doi:10.1109/access.2023.3297652 · Cited by 15
A Taxonomic Survey of Model Extraction Attacks
Genç, D., Özuysal, M., & Tomur, E. (2023) · IEEE International Conference on Cyber Security and Resilience (CSR) · doi:10.1109/csr57506.2023.10224959 · Cited by 20
Robust visual question answering via semantic cross modal augmentation
Mashrur, A., Luo, W., Zaidi, N. A., & Robles‐Kelly, A. (2023) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2023.103862 · Cited by 22
MDM: Meta diffusion model for hard-constrained text generation
Ke, W., Guo, Y., Liu, Q., Chen, W., Wang, P., Luo, H., & Luo, Z. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.111147 · Cited by 13
Agile Modeling: From Concept to Classifier in Minutes
Stretcu, O., Vendrow, E. B., Hata, K., Viswanathan, K., Ferrari, V., Tavakkol, S., Zhou, W., Avinash, A., Luo, E., Alldrin, N., et al. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02040 · Cited by 21
A Comparison of Classical and Deep Reinforcement Learning Methods for HVAC Control
Wang, M., Willes, J., Jiralerspong, T., & Moezzi, M. (2023) · IEEE Smart World Congress (SWC) · doi:10.1109/swc57546.2023.10448598 · Cited by 13
Entropy-based guidance of deep neural networks for accelerated convergence and improved performance
Meni, M. J., White, R. T., Mayo, M. L., & Pilkiewicz, K. R. (2024) · Information Sciences · doi:10.1016/j.ins.2024.121239 · Cited by 13
Object-Centric Instruction Augmentation for Robotic Manipulation
Wen, J., Zhu, Y., Zhu, M., Li, J., Xu, Z., Che, Z., Shen, C., Peng, Y., Liu, D., Feng, F., et al. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10609992 · Cited by 26
Roadmap on Incentive Compatibility for AI Alignment and Governance in Sociotechnical Systems
Zhang, Z., Bai, F., Wang, M., Ye, H., Ma, C., & Yang, Y. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-00800-8_33 · Cited by 17
The Causal Impact of Tool Affordance on Safety Alignment in LLM Agents
Yu, S., Carroll, F., & Bentley, B. L. (2026) · International Conference on Electrical, Computer and Energy Technologies (ICECET) · doi:10.1109/icecet65726.2026.11632769 · Cited by 12
Yin, J., Wang, H., Bao, P., Xu, J., & Wang, Y. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770854.3783953 · Cited by 15
Wang, S., Yu, L., Gao, C., Zheng, C., Liu, S., Lu, R., Dang, K., Chen, X., Yang, J., Zhang, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.01939 · Cited by 14
“…Before the advent of reasoning-capable models like OpenAI’s o1 (Ope-nAI, 2024), reinforcement learning (RL) was widely used in reinforcement learning from human feedback (RLHF) to improve large language models’ (LLMs) instruction-following and alignment with human preferences (Ouyang et al., 2022).”
TextGrad: Automatic "Differentiation" via Text
Yuksekgonul, M., Bianchi, F., Boen, J., Liu, S., Huang, Z., Guestrin, C., & Zou, J. (2024) · arXiv · doi:10.48550/arxiv.2406.07496 · Cited by 12
Miceli Barone, A. V., Barez, F., Cohen, S. B., & Konstas, I. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.19 · Cited by 12
Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems
Dalrymple, D. "., Skalse, J., Bengio, Y., Russell, S., Tegmark, M., Seshia, S., Omohundro, S., Szegedy, C., Goldhaber, B., Ammann, N., et al. (2024) · arXiv · doi:10.48550/arxiv.2405.06624 · Cited by 12
Decision Support Systems in Architecture—A Future Perspective
Schubert, G., Bratoev, I., & Petzold, F. (2023) · Buildings · doi:10.3390/buildings13081952 · Cited by 12
Fu, D., Abawi, F., & Wermter, S. (2023) · IEEE RO-MAN · doi:10.1109/ro-man57019.2023.10309334 · Cited by 14
Enriching Music Descriptions with A Finetuned-LLM and Metadata for Text-to-Music Retrieval
Doh, S., Lee, M., Jeong, D., & Nam, J. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446380 · Cited by 32
Enhancing Chinese abbreviation prediction with LLM generation and contrastive evaluation
Liu, J., Tian, X., Tong, H., Xie, C., Ruan, T., Lin, C., Wu, B., & Wang, H. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103768 · Cited by 12
Wang, S., Ma, S., Lai, G.-H., & Chao, C.-H. (2024) · Future Generation Computer Systems · doi:10.1016/j.future.2024.07.029 · Cited by 14
Gitzel, R., Hoffmann, M. W., Heiden, P. Z., Skolik, A., Kaltenpoth, S., Müller, O., Kanak, C., Kandiah, K., Stroh, M.-F., Boos, W., et al. (2024) · IEEE Access · doi:10.1109/access.2024.3437195 · Cited by 15
Survey of Large Language Models for Answering Questions Across Various Fields
Shailendra, P., Ghosh, R. C., Kumar, R., & Sharma, N. (2024) · International Conference on Advanced Computing and Communication Systems (ICACCS) · doi:10.1109/icaccs60874.2024.10717078 · Cited by 24
Self-supervised contrastive learning for implicit collaborative filtering
Song, S., 刘, 斌., Teng, F., & Li, T. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109563 · Cited by 14
Ulug, E., Gunesli, I., Acıkgoz Pinar, A., & Yildiz, B. O. (2025) · Nutrition Research · doi:10.1016/j.nutres.2024.11.005 · Cited by 12
Towards bias-aware visual question answering: Rectifying and mitigating comprehension biases
Chen, C., Han, D., Guo, Z., & Chang, C. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125817 · Cited by 12
Cross-modal event extraction via Visual Event Grounding and Semantic Relation Filling
Liu, M., Zhou, B., Hu, H., Qiu, C., & Zhang, X. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.104027 · Cited by 12
MIRA-ChatGLM: A Fine-Tuned Large Language Model for Intelligent Risk Assessment in Coal Mining
Sun, Y., Zhang, C., Wang, C., & Han, Y. (2024) · Applied Sciences · doi:10.3390/app142412072 · Cited by 13
Self-citationSC4ANM: Identifying optimal section combinations for automated novelty prediction in academic papers
Wu, W., Zhang, C., Bao, T., & Zhao, Y. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126778 · Cited by 19
Self-citationAssessing and improving syntactic adversarial robustness of pre-trained models for code translation
Yang, G., Zhou, Y., Zhang, X., Chen, X., Han, T., & Chen, T. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107699 · Cited by 19
How to leverage large language models for automatic ICD coding
Yoo, Y., & Kim, S. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.109971 · Cited by 15
Gen AI and research integrity: Where to now?
Vasconcelos, S. M. R., & Marušić, A. (2025) · EMBO Reports · doi:10.1038/s44319-025-00424-6 · Cited by 15
Miao, H., Xu, R., Zhao, Y. L., Wang, S., Wang, J., Yu, P. S., & Jensen, C. S. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3558964 · Cited by 12
ChatGPT vs. Human Journalists: Analyzing News Summaries Through BERTScore and Moderation Standards
Kim, H.-S., Kang, J.-W., & Choi, S. (2025) · Electronics · doi:10.3390/electronics14112115 · Cited by 13
Training LLMs for Generating IEC 61131-3 Structured Text with Online Feedback
HAAG, A. L., Fuchs, B., Kacan, A., & Lohse, O. (2025) · International Workshop on Large Language Models for Code (LLM4Code) · doi:10.1109/llm4code66737.2025.00013 · Cited by 17
An explainable framework for assisting the detection of AI-generated textual content
Yan, S., Wang, Z., & Dobolyi, D. G. (2025) · Decision Support Systems · doi:10.1016/j.dss.2025.114498 · Cited by 17
SIGMAN: Scaling 3D Human Gaussian Generation with Millions of Assets
Yang, Y., Liu, F., Lu, Y., Zhao, Q., Wu, P., Zhai, W., Yi, R., Cao, Y., Ma, L., Zha, Z.-J., et al. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00487 · Cited by 18
Function Calling in Large Language Models: Industrial Practices, Challenges, and Future Directions
Wang, M., Zhang, Y., Yu, B., Hao, B., Peng, C., Chen, Y.-C., Zhou, W., Gu, J., Zhuang, C., Guo, R., et al. (2026) · ACM Computing Surveys · doi:10.1145/3788284 · Cited by 39
The Many Faces of Indeterminacy in Interactive Deadbots
Kozlovski, A., Harbinja, E., & Dobbe, R. (2026) · Philosophy & Technology · doi:10.1007/s13347-026-01089-2 · Cited by 13
Kibria, M. R., Lafond, S., & Arslan, J. (2026) · Information Fusion · doi:10.1016/j.inffus.2026.104405 · Cited by 16
BEAR: Towards Beam-Search-Aware Optimization for Recommendation with Large Language Models
Yang, W., Wang, B., Xu, Z., Chen, J., Zhang, S., Chen, J., Jin, C., & Wang, C. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809533 · Cited by 14
Finding structure during incremental speech comprehension
Lyu, B., Marslen‐Wilson, W. D., Fang, Y., & TYLER, L. K. (2024) · eLife · doi:10.7554/elife.89311 · Cited by 13
Temporally-Extended Prompts Optimization for SAM in Interactive Medical Image Segmentation
Shen, C., Li, W., Zhang, Y., Wang, Y., & Wang, X. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385291 · Cited by 17
Code Detection for Hardware Acceleration Using Large Language Models
Martínez, P. A., Bernabé, G., & García, J. M. (2024) · IEEE Access · doi:10.1109/access.2024.3372853 · Cited by 12
Everson, K., Gu, Y., Yang, H., Shivakumar, P. G., Lin, G.-T., Kolehmainen, J., Bulyko, I., Gandhe, A., Ghosh, S., Hamza, W., et al. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447938 · Cited by 12
Enhance Multi-Domain Sentiment Analysis of Review Texts Through Prompting Strategies
Wang, Y., & Luo, Z. (2023) · International Conference on High Performance Big Data and Intelligent Systems (HPBD&IS) · doi:10.1109/hdis60872.2023.10499502 · Cited by 25
Domain-Specific Improvement on Psychotherapy Chatbot Using Assistant
Kang, C., Novák, D., Urbanova, K., Cheng, Y., & Hu, Y. S. (2024) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw62465.2024.10626529 · Cited by 14
Diverse AI Feedback For Large Language Model Alignment
Yu, T., Lin, T.-E., Wu, Y., Yang, M., Huang, F., & Li, Y. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00746 · Cited by 21
Enhancing Collaborative Semantics of Language Model-Driven Recommendations via Graph-Aware Learning
Guan, Z., Wu, L., Zhao, H., He, M. X., & Fan, J. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3581606 · Cited by 15
Learning Fine-Grained Grounded Citations for Attributed Large Language Models
Huang, L., Feng, X., Ma, W., Gu, Y., Zhong, W., Feng, X., Yu, W., Peng, W., Tang, D., Tu, D., et al. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.838 · Cited by 12
A Preference-Driven Methodology for Efficient Code Generation
Li, Y., Zhou, Z., Peng, Z., Dong, J., You, H., Yan, R., Wen, S., Tian, Y., & Huang, T. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3639527 · Cited by 36
Autonomous navigation of intelligent microrobotic swarms in unknown environments
An, X., Luo, S., Zhang, H., Yang, Q., Ma, Y., Wang, B., Du, J., & Wang, Q. (2026) · Nature Machine Intelligence · doi:10.1038/s42256-026-01252-6 · Cited by 11
$π_0$: A Vision-Language-Action Flow Model for General Robot Control
Black, K., Brown, N., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., Groom, L., Hausman, K., Ichter, B., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.24164 · Cited by 13
“This is perhaps the most important ingredient, as much of the recent progress with large models in NLP and computer vision has relied heavily on delicate strategies for curating pre-training and post-training data [35].”
LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks
Kambhampati, S., Valmeekam, K., Guan, L., Verma, M., Stechly, K., Bhambri, S., Saldyt, L., & Murthy, A. (2024) · arXiv · doi:10.48550/arxiv.2402.01817 · Cited by 237
“5 OpenAI (2022), InstructGPT-3 Ouyang et al. (2022) and GPT-3 Brown et al. (2020)), as well as fine-tuning does not seem to have a major effect on this dismal performance.”
BackgroundDebating with More Persuasive LLMs Leads to More Truthful Answers
Khan, A., Hughes, J., Valentine, D., Ruis, L., Sachan, K., Radhakrishnan, A., Grefenstette, E., Bowman, S. R., Rocktäschel, T., & Perez, E. (2024) · arXiv · doi:10.48550/arxiv.2402.06782 · Cited by 13
Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge
Ye, J., Wang, Y., Huang, Y., Chen, D., Zhang, Q., Moniz, N., Gao, T., Geyer, W., Huang, C., Chen, P.-Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.02736 · Cited by 11
Deanthropomorphising NLP: Can a language model be conscious?
Shardlow, M., & Przybyła, P. (2024) · PLoS ONE · doi:10.1371/journal.pone.0307521 · Cited by 16
A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents
Su, H., Luo, J., Liu, C., Yang, X., Zhang, Y., Dong, Y., & Zhu, J. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3688650 · Cited by 11
Beamaggr: Beam aggregation reasoning over multi-source knowledge for multi-hop question answering
Chu, Z., Chen, J., Chen, Q., Wang, H., Zhu, K., Du, X., Yu, W., Liu, M., & Qin, B. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.67 · Cited by 11
CrossCodeBench: Benchmarking Cross-Task Generalization of Source Code Models
Niu, C., Li, C., Ng, V., & Luo, B. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00055 · Cited by 11
Metadata
- Authors
- Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., & Lowe, R.
- Venue
- Advances in Neural Information Processing Systems 35
- Published
- 2022
- Pages
- 27730-27744
- ISSN
- 1049-5258 (ISSN-L)
- OpenAlex
- W7133211372
- Cited by
- 3,117
- References
- 0
- Citations by year
- 2026: 1421 · 2025: 544 · 2024: 852 · 2023: 297 · 2022: 3
- FWCI
- 234.27
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- neural information processing systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- instruction following, language models, human feedback, reinforcement learning from human feedback, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.