Training Language Models to Follow Instructions with Human Feedback
The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
Making language models bigger does not inherently make them better at following a user's intent. For example, large language models can generate outputs that are untruthful, toxic, or simply not helpful to the user. In other words, these models are not aligned with their users. In this paper, we show an avenue for aligning language models with user intent on a wide range of tasks by fine-tuning with human feedback. Starting with a set of labeler-written prompts and prompts submitted through the OpenAI API, we collect a dataset of labeler demonstrations of the desired model behavior, which we use to fine-tune GPT-3 using supervised learning. We then collect a dataset of rankings of model outputs, which we use to further fine-tune this supervised model using reinforcement learning from human feedback. We call the resulting models InstructGPT. In human evaluations on our prompt distribution, outputs from the 1.3B parameter InstructGPT model are preferred to outputs from the 175B GPT-3, despite having 100x fewer parameters. Moreover, InstructGPT models show improvements in truthfulness and reductions in toxic output generation while having minimal performance regressions on public NLP datasets. Even though InstructGPT still makes simple mistakes, our results show that fine-tuning with human feedback is a promising direction for aligning language models with human intent.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Existing literature (Ouyang et al., 2022) does not fully describe the details of these models, such as the pretraining corpus, model sizes, and model biases.”
From Self-Refine: Iterative Refinement with Self-Feedback (2023) · cites this paper as Ouyang et al., 2022
Experimental — accuracy not yet measuredIn the citing paper
“Specifically, providers mitigate race and gender bias through reinforcement learning from human feedback (RLHF), adversarial stress-testing known as red teaming, and hard-coded refusals governed by the provider's own content policies [6, 43, 71].”
From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [71]
In the citing paper
“Fine-tuning a model from human feedback at scale, using preference data from paid raters or drawn from a large and willing user base, is an especially potent technique.It has the potential not only to shape a model's responses to better reflect user norms (for better or worse), but also to filter out toxic language, improve factual accuracy, and mitigate the tendency to fabricate information.”
From Talking about Large Language Models (2024) · cites this paper as [26]
In the citing paper
“These include general competencies such as in-context learning (ICL) [13], chain-of-thought (CoT) [17] reasoning, and instruction following [25], as well as task-specialized skills like mathematical reasoning [30].”
From A Survey of Large Language Models (2026) · cites this paper as [25]
In the citing paper
“When external reward models or ground-truth answers are available, rejection sampling can also be applied to select higher-quality responses from multiple candidates [25].”
From A Survey of Large Language Models (2026) · cites this paper as [25]
41 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Items 3,501–3,600 of 4,340
Zeng, C., & Kang, S.-W. (2026) · 2026 8th Asia Conference on Machine Learning and Computing (ACMLC) · doi:10.1109/acmlc70381.2026.11700470
Reservoir Modeling Assisted with Knowledge-Enhanced Large Language Models
Sun, W., Jin, Z., & Gao, M. (2026) · Processes · doi:10.3390/pr14193076
Evaluating Large‐Language Models in Bioinformatics Applications
Yin, H., Cui, Z., Li, D., Yang, Y., Chang, Y., Zhu, X., Zhang, J., Hu, P., Zhou, X., & Hu, L. (2026) · Advanced Computing · doi:10.1002/adco.70007
Simulated Morality, Misplaced Trust: The Risks of Treating AI as a Moral Partner
Josifović, S. (2026) · Philosophy & Technology · doi:10.1007/s13347-026-01180-8
Sblendorio, E., Dentamaro, V., Maria, M. D., Tempesta, S., Barile, E., Napolitano, D., Tallini, M., Nigrelli, D., Cicolini, G., & Piredda, M. (2026) · JMIR Medical Informatics · doi:10.2196/90854
Liu, Z., Luo, S., Li, D., Ma, Y., & Li, X. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.117113
Intelligent Recommendation Engine for Vacancy-Resume Matching Using Hybrid ML Methods
Матковский, М., Spivak, I., Krepych, S., Spivak, M., Chudyk, N., & Andrii, K. (2026) · International Conference on Advanced Computer Information Technologies (ACIT) · doi:10.1109/acit69831.2026.11700074
Fodor, J., Murawski, C., & Suzuki, S. (2026) · eLife · doi:10.7554/elife.108442.3 · Cited by 1
Yodihartomo, F. (2026) · International Conference on Information Management and Technology (ICIMTech) · doi:10.1109/icimtech202671078.2026.11698964
AI Agent with Model Selection, Tool Synthesis, and Proactive Knowledge Base Management
Jelínek, J. (2026) · International Conference on Advanced Computer Information Technologies (ACIT) · doi:10.1109/acit69831.2026.11700205
Measuring Position Bias in LLM-as-a-Judge for Hiring
Aminou, L., Daaif, A., Soulami, M., & Youssfi, M. (2026) · International Conference on Advanced Computer Information Technologies (ACIT) · doi:10.1109/acit69831.2026.11699959
Öztürk, T., Yüksel, M. E., & Sevgen, S. (2026) · Applied Sciences · doi:10.3390/app16199625
A Learning-Based Framework for Quantifying Structured Prompt Refinement in Large Language Models
Kumar, B. S., Giri, A., Pattamsetty, A. B., & Datta, T. R. S. (2026) · IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT) · doi:10.1109/conecct68978.2026.11700679
A Validation Topos: A Sheaf-Theoretic Framework for AI Reliability and Creative Inference
Hernández, M., & Sánchez-Soto, E. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-37444-8_3
When GenAI Negotiates Like Humans: Evidence of the Anchoring Bias in Large Language Models’ Outputs
Maaravi, Y., & Gur, T. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2732484
Missing context: a review of natural language processing for archives
Dunagan, L., Johnston, J., & Card, D. (2026) · Culture & Society · doi:10.1017/cfc.2026.10031
Salah, K. M., Mahmoud, S. A., & Ibrahim, M. (2026) · Intelligent Methods, Systems, and Applications (IMSA) · doi:10.1109/imsa70415.2026.11700363
Large Language Models: A Review with its Role, Applications and Challenges & Future Directions
Chatterjee, C., Patel, U., Vala, J., Baria, R. N., Tiwari, S., & Trivedi, P. (2026) · International Conference on Secure Information Systems and Technologies (ICSIST) · doi:10.1109/icsist70215.2026.11705743
Mitigating Sycophancy and Alignment Failures in Large Language Models
Gollapalli, V. P., Dasari, S. M., Kadam, S., Changalva, R., Gupta, S., & Hegde, A. (2026) · International Conference on Secure Information Systems and Technologies (ICSIST) · doi:10.1109/icsist70215.2026.11706246
A cognitive hierarchy-guided abstract forest knowledge base based on cognitive load theory
Zhao, F., Xu, K., & Feng, J. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.116241
How Model Choice and Memory Shape Preference Consistency in Large Language Models
Chung, Y. (2026) · Sociological Methods & Research · doi:10.1177/00491241261492203
Alves, L. W. A., Fazzioni, D., de Andrade, M. L., MATOS, A., Jordão, S. A. A., Pereira, L. A., de Oliveira, S. S. T., & Camilo-Junior, C. G. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39898-7_36
LLM-Lobotomy: A Representation Engineering Framework for LLM Behavioral Steering
Neto, E. B., Ferreira, L. N., Interian, R., & Dias, Z. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39882-6_39
周, 启. (2026) · Vocational Education · doi:10.12677/ve.2026.1510419
LMTracer: Fine-Grained and Real-Time Performance Profiling for Production LLM Systems
Liu, W., He, Y., Zhao, B., Wang, H., Li, Z., & Zhao, J. (2026) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/3830418.3843871
Empirical Evaluation of Fine-Tuned LLMs for Hierarchical HS Code Prediction
Gökçül, I., Ozaydin, B., Karaosmanoğlu, E., Günel, C., Koç, H. Ş., & Arslan, S. (2026) · International Conference on Computer Technology Applications (ICCTA) · doi:10.1109/iccta70480.2026.11706479
Kumar, B. V. C., Ramadevi, N., Arbaz, S., Kedhar, B. H., Shaklain, S. B., & Raju, T. (2026) · International Conference on Secure Information Systems and Technologies (ICSIST) · doi:10.1109/icsist70215.2026.11706385
Efficient GPU Multitasking with Morphable Kernels
Ren, T., Fan, R., Guo, H., Xie, M., Gao, S. Y., Shu, J., & Lu, Y. (2026) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/3830418.3843904
Kang, J.-H., Lee, S.-H., & Bae, Y.-S. (2026) · World Electric Vehicle Journal · doi:10.3390/wevj17100507
Adaptive Low-Rank Perturbation Allocation via Efficient Evolution Strategy for LLM Post-Training
Lin, T., Huang, T., Wang, C., Hu, X., Lin, Z., He, Y., & He, C. (2026) · International Conference on Data-driven Optimization of Complex Systems (DOCS) · doi:10.1109/docs70981.2026.11698564
DIVE-Bench: Codec-Style High-FPS Video Understanding
Zhang, H., Chai, W., He, S., Li, A., & Fu, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-36987-1_23
From bits to atoms: A survey of cross-layer safety in Embodied AI
Feng, T., Zhang, Y., Zhou, S., Fan, Z., Wang, W., Chi, X., Shen, T., Li, X., Bai, X., Zhu, Z., et al. (2026) · AI Plus · doi:10.59717/ipj.aiplus.2026.100013 · Cited by 1
Ponduru, P. S., Nandanavanam, P. P. V., & Ponduru, S. K. K. (2024) · International Journal of Scientific Research in Science Engineering and Technology · doi:10.32628/ijsrset2613385
Sejdiu, N. P., & Sejdiu, S. (2026) · Applied Linguistics · doi:10.1093/applin/amag089
Barad, K., Gunjan, R., & Butalia, A. (2026) · International Conference on Innovation & Growth in Next-Gen Intelligent Technology & Engineering (IGNITE) · doi:10.1109/ignite69592.2026.11705780
Omri, M. N., & Louhichi, B. (2026) · Neural Computing and Applications · doi:10.1007/s00521-026-12365-9
The shift in ethical governance from traditional philosophy to algorithmic decision making
de Marcellis-Warin, N., Melchior, C., & Warin, T. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-01974-x
How philosophical reasoning shapes the alignment problem: evolutionary debunking as a case study
Dale, M. T., & Saad, B. (2026) · Synthese · doi:10.1007/s11229-026-05796-3
Kim, S., Kim, H. M., Kim, K., & Zhu, D. (2026) · Social Science & Medicine · doi:10.1016/j.socscimed.2026.119855
Li, C., Wang, F., Pan, M., Li, Z., Zeng, M., Zhang, B., Yu, X., Wang, B., Hua, K., & Li, X. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832097
Learning Consistency in Reward Modeling for Multi-modal Reasoning
Li, C., Ni, B., Zhang, B., Zhang, J., Ye, K., Peng, H., Hu, H., & Zheng, N. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37258-1_16
Gómez-Gutiérrez, O. A., la Torre, O. A. D., Cassagne, G. M., Flores-Orduña, J., Leon, L., & Fabiani, M. A. (2026) · Phlebology The Journal of Venous Disease · doi:10.1177/02683555261494674
Datura: Progressive Red Teaming Testing for Tool Invocation Chain in LLM Agents
Shao, Y., Bao, Z., Huang, Y., Shi, Y., Weng, M., Sun, Y., Yang, L., Ma, L., Su, T., & Wan, C. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832101
Towards More Realistic Assertion Generation under Mixed-Assertion Scenario
Li, H., Kunpeng, E., Sun, W., Zhang, Q., & Yan, M. L. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832214
Wu, X., Li, S., Liu, S., & Guo, Z. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.105205
CAST: A Compiler-Based Framework for Systematically Testing LLM Compositional Safety
Yan, L., Zhang, Z., Xu, X., An, S., Shen, G., Xuan, Z., Chen, X., & Zhang, X. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832237
Juarto, B., Tiara, K. A., Rahmania, R., & Chen, V. F. (2026) · International Conference on ICT for Smart Society (ICISS) · doi:10.1109/iciss71466.2026.11715381
Boosting Vision-Language Model for Cross-Device GUI Agent Grounding and Navigation
Chen, B.-Y., Shih, L.-H., & Chiang, C.-K. (2026) · IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr70517.2026.00011
Advertising with HAART: Introducing Human–AI Asymmetric Relationship Theory
Thomas, V. L., Shoenberger, H., Gvirtz, A., Sands, S., & Willemsen, L. M. (2026) · Journal of Advertising · doi:10.1080/00913367.2026.2735425
Qin, C., Chen, Y., & Yan, Y. (2026) · IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr70517.2026.00087
Odilovna, M. Z., Mamatkulov, D. M., Иброгимовна, Т. Р., Rakhmonova, N. T., Shavqiyev, O., & Umbarovna, Z. S. (2026) · Eurasia Journal of Mathematics Science and Technology Education · doi:10.29333/ejmste/19481
Zyra: An AI Smart Mirror for Personalized Styling
Joji, C., Manmadhan, G. P., Jose, J., Anantharaman, B., & K, A. R. (2026) · International Conference on Innovations in Engineering and Technology (ICIET) · doi:10.1109/iciet70550.2026.11712734
Reliable Text-to-SQL in Resource-Constrained Settings with Small LLMs via Iterative Retry Workflows
Benziane, B. A., & Schepler, X. (2026) · International Conference on Automation and Computing (ICAC) · doi:10.1109/icac70325.2026.11712531
Beyond Constrained Decoding: Grammar-Aware Fine-Tuning for Small Language Models
Silva, P. H. C., Oliveira, J. M. G., & Silva, F. A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39904-5_18
Gunawardana, R., Randika, A., Perera, D., & Karunanayaka, K. T. (2026) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3776574.3831155
Zhang, W., Li, Q., Zhang, G., Zhang, W., & Xie, Z. (2026) · The Computer Journal · doi:10.1093/comjnl/bxag102
Post-Training for Reasoning LLMs with Reinforcement Learning: A Stability–Efficiency Perspective
Yang, L., Zhu, H., Zhong, Z., Bai, N., Tan, X., Yin, Q., Yan, Z., & Tang, Z. (2026) · International Conference on Automation and Computing (ICAC) · doi:10.1109/icac70325.2026.11712300
Zhang, W., Guo, H., Wang, W., Xie, H., & Wang, F. L. (2026) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3776574.3831125
CLARiF: clue alignment and retrieval-informed fusion for fine-grained pest recognition
Li, Q., Pan, Y., Xiang, N., Zhang, H., Li, Z., Huang, X., & Xu, Z. (2026) · Applied Intelligence · doi:10.1007/s10489-026-07522-5
Building global sample graph for effective code clone detection
Li, C., Xu, T., Dou, S., Li, J., Jia, J., Wu, Y., & Zou, D. (2026) · Journal of Systems and Software · doi:10.1016/j.jss.2026.113117
EREAL: Enhancing Mathematical Reasoning through Error-Aware Learning in Large Language Models
Yu, Y., Li, Z., Zou, L., Li, C., & Wang, Q. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.105197
Ramli, M. S. B. (2026) · Research Square · doi:10.21203/rs.3.rs-11225296/v1
Securing Agentic AI via Hybrid CNN–Transformer Detection and Cryptographic Intent Binding
Sindhu, N., Jagruthi, H., Pai, S., Chiranth, S., Aishvarya, V., & Pavan, J. (2026) · Algorithms for intelligent systems · doi:10.1007/978-3-032-25627-0_31
Too Much Mind Extension? Towards a Virtue Ethics of the Extended Mind
Chlebowski, S., Ratajczyk, D., & Wyrwa, M. (2026) · Roczniki Filozoficzne · doi:10.18290/rf26743.6
From Static to Adaptive Inference: A Survey on Test‐Time Learning
Wang, M., Chen, J., Li, Q., Shi, J., & Nappi, M. (2026) · Expert Systems · doi:10.1111/exsy.70446
Yao, N., Yang, X., & Xiao, F. (2026) · Information · doi:10.3390/info17100981
Ma, Y., Zheng, S., Tian, H., 秦茂惠, Zheng, P., Leng, J., & Hong, J. (2026) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2026.09.020
Zhou, W., Jia, S., Xu, B., Niu, Y., & Zha, D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38702-8_21
Sun, J., Zhang, W., Zhao, Q., Gu, Q., & Liu, C. (2026) · Cell Reports Physical Science · doi:10.1016/j.xcrp.2026.103578
Fatima, S. S., Reddy, B. B., Ramadevi, N., Venkat, N., Imran, S., & Sameer, S. M. (2026) · International Conference on Soft Computing for Security Applications (ICSCSA) · doi:10.1109/icscsa69367.2026.11707969
Autenrieth, D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37979-5_5
Jumaila, M., Sundar, G. N., & Narmadha, D. (2026) · International Conference on Emerging Techniques in Computational Intelligence (ICETCI) · doi:10.1109/icetci71772.2026.11710407
Enhancing Formal Text Generation Through Direct Preference Optimization (DPO)
Senthilkumar, P., Sheokand, P., Vaishnavi, R., Chintalapati, S. S., & Joshi, A. M. (2026) · Lecture notes in electrical engineering · doi:10.1007/978-981-92-1243-9_33
Zhang, B. (2026) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2026.105149
Effective Context Management for LLM Agents via Hybrid Chat History Optimization
Songa, S., Bayyavarapu, S. R. K. V., Gulkotwar, N., Pasupuleti, V., Allala, S. R., & Tyagi, S. (2026) · IEEE World Conference on Applied Intelligence and Computing (AIC) · doi:10.1109/aic70896.2026.11717333
SecRefine: Enhancing Cybersecurity LLM through Two-Stage Instruction Fine-Tuning
Liu, Y. (2026) · International Conference on Computer Science and Blockchain (CCSB) · doi:10.1109/ccsb71176.2026.11716369
Mechanism design meets large language models: Foundations and frontiers
Chen, B., Zhu, T., Li, X., Zhang, Y., & Dai, X. (2026) · Statistical Learning and Data Science · doi:10.1016/j.slads.2026.100033
A Comprehensive Dataset and Benchmark for Empathetic Responses in Online Communities
Qi, S., Lyu, J., Liao, M., Xiang, R., & Li, J. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-5696-9_16
Comparing large language models and prompting techniques to control an embodied persuasive coach
Corrò, C., & Chittaro, L. (2026) · Behaviour and Information Technology · doi:10.1080/0144929x.2026.2742725
Secure Knowledge Extraction From Large Language Model
Koneru, N. M. K., & Vadlamudi, S. K. (2026) · International Conference on Natural Language Processing and Computer Vision (ICNPCV) · doi:10.1109/icnpcv69434.2026.11708929
Less is more: Interpretable prefix-based jailbreaking of MoE language models
Xie, Q., Li, Y., Ou, S., Chen, Q., Wang, T., Chen, Y., Shen, Y., Pan, Z., Yu, L., Ma, H., et al. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.117130
Bridging Offline and Online Multi-Objective Reinforcement Learning
Duan, Y., & Jin, Q. (2026) · Communications in computer and information science · doi:10.1007/978-981-92-5712-6_26
Hosni, N. A. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1824854
Designing Prompt Literacy: Teaching and Learning With AI in Higher Education
Ojha, M. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8332-3.ch007
Large Language Models in Cybersecurity: Explainability, Trust, and Responsible Deployment
Sharma, D., Girdhar, N., & Modi, A. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-1970-2.ch009
From model trust to software topology: a Perspective on structurally governed agentic AI systems
Morin, M., & Sufi, F. K. (2026) · Frontiers in Computer Science · doi:10.3389/fcomp.2026.1919950
BanditFuzz: A Synergistic Differential Fuzzing Framework for Compilers
Li, D., Li, Y., Xu, J., Zhao, B., Wang, N., Yao, J., & Liu, L. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834548
Trust but Verify: Evaluating Backdoor-Defended LLMs After Quantization
Ge, Y., & Zhang, Y. (2026) · ACM Turing Celebration Conference - China (ACM TURC) · doi:10.1145/3833142.3833191
Optimization-Free Repair of Code LLMs: A Closed-Form Fisher-Guided Bayesian Approach
Duan, Z., Bian, C., Zhang, Y., & Yang, S. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834442
From Prompts to Publications: Rethinking Academic Writing and Research With Generative AI
Tyagi, V. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8332-3.ch013
Sycophancy Negatively Affects LLM-as-a-Judge in Conflict Evaluation
Farzi, N., Dietz, L., & Carton, S. (2026) · Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics (GEM) · doi:10.18653/v1/2026.gem-main.45
RubricEval: A Rubric-Level Meta-Evaluation Benchmark for LLM Judges in Instruction Following
Pan, T., Lin, X., Yang, W., He, Q., Chen, S., Qi, L., Xu, W., Feng, H., Xu, B., & Xiao, Y. (2026) · arXiv · doi:10.48550/arxiv.2603.25133
“Instruction following (IF) is a fundamental capability of large language models (LLMs), as it directly affects task completion quality and user experience(Ouyang et al., 2022; Achiam et al., 2023).”
InfluentialCaring Without Feeling: Affective Dynamics as the Control Layer of Human-AI Agent Collaboration
Xu, J., Wu, X., Zhang, Z., Xu, Y., Yang, Y., Zhu, J., Xiao, L., Wu, W., & He, L. (2026) · arXiv · doi:10.48550/arxiv.2606.18259
“Second, instruction tuning and reinforcement learning from human feedback shape responses toward helpful, polite and socially acceptable forms [15].”
Memory for Autonomous LLM Agents:Mechanisms, Evaluation, and Emerging Frontiers
Du, P. (2026) · arXiv · Cited by 1
Experience Compression Spectrum: Unifying Memory, Skills, and Rules in LLM Agents
Zhang, X., Wang, G., Cui, Y., Qiu, W., Li, Z., Zhu, B., & He, P. (2026) · arXiv
“…“missing diagonal” (Section 2); (3) quantify the community disconnect ( < 1% cross-citation) and expose four structural insights invisible from within either community (Section 3); and (4) articulate open problems and design principles for scalable, full-spectrum agent learning (Section 4).”
Compression Method Matters: Benchmark-Dependent Output Dynamics in LLM Prompt Compression
Johnson, W. (2026) · arXiv · Cited by 1
“Current RLHF pipelines (Ouyang et al., 2022) optimize for helpfulness and harmlessness on well-formed prompts but do not explicitly penalize output explosion under degraded inputs.”
Towards Embodiment Scaling Laws in Robot Locomotion
Ai, B., Dai, L., Bohlinger, N., Li, D., Mu, T., Wu, Z., Fay, K., Christensen, H. I., Peters, J., & Su, H. (2025) · arXiv · doi:10.48550/arxiv.2505.05753
“…260, 261, 266, 272, 278, 280, 282, 286, 290, 298, 308, 312, 313, 316, 320, 327] Hexapod [0, 7, 8, 20, 31, 32, 37, 41, 46, 47, 48, 50, 51, 55, 71, 72, 75, 97, 104, 111, 113, 122, 124, 128, 132, 133, 144, 149, 154, 155, 158, 161, 163, 166, 169, 170, 181, 183, 197, 204, 207, 215, 222, 226, 229,…”
Limits to scalable evaluation at the frontier: LLM as Judge won't beat twice the data
Dorner, F. E., Nastl, V. Y., & Hardt, M. (2024) · arXiv · doi:10.48550/arxiv.2410.13341 · Cited by 31
“Initially, specialized fine-tuned models were used to provide training signals for a second model (Ouyang et al., 2022; Bai et al., 2022; Dorner et al., 2023).”
Mohamed, A. S., Fritz, C., Saber, A. M., Ma, Y., Kaur, R., Al-Darwbi, M., Friedrich, D., & Kundur, D. (2026) · arXiv · doi:10.48550/arxiv.2609.00171
GIER: Gap-Driven Self-Refinement for Large Language Models
Dewri, R. (2025) · arXiv · doi:10.48550/arxiv.2509.00325
“A final strand of work focuses on aligning models with human preferences via reinforcement learning with human feedback (RLHF) (Ouyang et al., 2022; Bai et al., 2022a), preference modeling (Rafailov et al., 2023), critiques (Bai et al., 2022b), or by using high-quality examples for supervised fine-tuning (Zhou et al.,…”
Metadata
- Authors
- Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., & Lowe, R.
- Venue
- Advances in Neural Information Processing Systems 35
- Published
- 2022
- Pages
- 27730-27744
- ISSN
- 1049-5258 (ISSN-L)
- OpenAlex
- W7133211372
- Cited by
- 3,117
- References
- 0
- Citations by year
- 2026: 1421 · 2025: 544 · 2024: 852 · 2023: 297 · 2022: 3
- FWCI
- 234.27
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- neural information processing systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- instruction following, language models, human feedback, reinforcement learning from human feedback, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.