Training Language Models to Follow Instructions with Human Feedback
The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
Making language models bigger does not inherently make them better at following a user's intent. For example, large language models can generate outputs that are untruthful, toxic, or simply not helpful to the user. In other words, these models are not aligned with their users. In this paper, we show an avenue for aligning language models with user intent on a wide range of tasks by fine-tuning with human feedback. Starting with a set of labeler-written prompts and prompts submitted through the OpenAI API, we collect a dataset of labeler demonstrations of the desired model behavior, which we use to fine-tune GPT-3 using supervised learning. We then collect a dataset of rankings of model outputs, which we use to further fine-tune this supervised model using reinforcement learning from human feedback. We call the resulting models InstructGPT. In human evaluations on our prompt distribution, outputs from the 1.3B parameter InstructGPT model are preferred to outputs from the 175B GPT-3, despite having 100x fewer parameters. Moreover, InstructGPT models show improvements in truthfulness and reductions in toxic output generation while having minimal performance regressions on public NLP datasets. Even though InstructGPT still makes simple mistakes, our results show that fine-tuning with human feedback is a promising direction for aligning language models with human intent.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Existing literature (Ouyang et al., 2022) does not fully describe the details of these models, such as the pretraining corpus, model sizes, and model biases.”
From Self-Refine: Iterative Refinement with Self-Feedback (2023) · cites this paper as Ouyang et al., 2022
Experimental — accuracy not yet measuredIn the citing paper
“Specifically, providers mitigate race and gender bias through reinforcement learning from human feedback (RLHF), adversarial stress-testing known as red teaming, and hard-coded refusals governed by the provider's own content policies [6, 43, 71].”
From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [71]
In the citing paper
“Fine-tuning a model from human feedback at scale, using preference data from paid raters or drawn from a large and willing user base, is an especially potent technique.It has the potential not only to shape a model's responses to better reflect user norms (for better or worse), but also to filter out toxic language, improve factual accuracy, and mitigate the tendency to fabricate information.”
From Talking about Large Language Models (2024) · cites this paper as [26]
In the citing paper
“These include general competencies such as in-context learning (ICL) [13], chain-of-thought (CoT) [17] reasoning, and instruction following [25], as well as task-specialized skills like mathematical reasoning [30].”
From A Survey of Large Language Models (2026) · cites this paper as [25]
In the citing paper
“When external reward models or ground-truth answers are available, rejection sampling can also be applied to select higher-quality responses from multiple candidates [25].”
From A Survey of Large Language Models (2026) · cites this paper as [25]
41 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Items 3,001–3,100 of 4,340
AI-Driven Social Media Marketing Assistant Using Agentic Workflows and Long-Term Memory
Vedant, B., Shirole, H., Kadam, P., Kadu, A., Shirke, A., Singh, P., & Kunkulol, A. (2026) · International Conference on Computational Intelligence and Communication Networks (CICN) · doi:10.1109/cicn70047.2026.11594391
NeurPIU: Neurobiologically Inspired Personalized Intent Understanding in Large Language Models
Liao, Z., Liao, J., & Zhao, X. H. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809593
Li, Q., Chen, C. Q., Wu, X., Xie, X., Gu, N., & Niu, J. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-2856-0_23
Evolva: A Multi-turn Contextual Attack for Long-Reasoning LLMs
Du, W., Yang, X., Liu, Z., Zhang, B., & Xue, J. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-2852-2_46 · Cited by 1
Hard Problems, Soft Guards: Softening LLM Guards with Olympiad-Grade Problems
Zhong, T., Xu, M., & Yang, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-2859-1_13
Zhang, X., Pan, H., Huang, J., Jiang, H. Q., Ye, H., Wu, X., Zhang, D. H., Wang, P., & Wang, B. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-2862-1_28
Kallel, Y., Zouari, A., Ketata, M., & Loukil, Z. (2026) · IFIP advances in information and communication technology · doi:10.1007/978-3-032-30809-2_29
SecPrompt v2: Protect LLM-Integrated Applications from Adversarial Attack
Zhang, J., Wu, J., Lu, Z., Ma, L., Xie, J., Yang, B., Zhang, W., & Huang, Q. (2026) · Communications in computer and information science · doi:10.1007/978-981-92-3541-4_4
Self-citationFestiyed, Makrooni, G., Ismail, I. A., Arif, K., Karimi, A., & Mingxuan, Z. (2026) · Multidisciplinary Science Journal · doi:10.31893/multiscience.2026791
Examples of Using LLMs as Metaheuristics
Iba, H., Batista, J. E., & Xu, J. (2026) · Natural computing series · doi:10.1007/978-981-95-8597-7_2
Dual-Loop Multi-agent Framework and Policy Optimization for Numbered Musical Notation Digitization
Lei, Z., Bai, P., & Shi, X. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3447-9_18
Liu, T., Wang, J., Hao, X., Liao, P., & Chen, G. (2026) · Journal of Management in Engineering · doi:10.1061/jmenea.meeng-7392
Çulha, D. (2026) · IFIP advances in information and communication technology · doi:10.1007/978-3-032-30801-6_9
Backdooring Societal Bias into Instruction-Tuned Large Language Models Via Virtual Prompt Injection
Haque, F., Ghosh, A., Xu, D., & Yuan, S. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-2014-4_6
Li, H., Uygun, Ö., Yu, X., Zhou, Y., Yang, Z., & Chen, C. (2026) · Computers & Industrial Engineering · doi:10.1016/j.cie.2026.112240 · Cited by 1
Intelligence beyond knowledge: control, architecture, and the structural law of artificial agency
Han, H. (2026) · AI and Ethics · doi:10.1007/s43681-026-01256-3
Assisting or resisting patriarchy? a critical discourse analysis of chatgpt’s responses on feminism
Biswas, A., Sulzer, M., & Hasan, M. U. (2026) · AI and Ethics · doi:10.1007/s43681-026-01220-1
EnMask: Jailbreaking Multimodal Large Language Models Through Distraction
Dou, Z., & Yan, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3492-9_1
Breaking Memory Wall for Fast Edge LLM Inference Using Contextual Sparsity
Wei, Z., Zhou, Y., Zhao, J., Li, K., & Fu, X. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3712257
Cui, J., Li, Q., Liu, W., Hu, J., Yin, X., Liu, N., Sun, B.-X., Liu, J., & Yang, B. X. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3712056
REVA-PO: Stabilizing Reinforcement Learning for Chest X-Ray Report Generation
Guo, L., Tahir, A. M., & Wang, Z. J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37321-2_31
MedQueryIntent: A Large-Scale Benchmark Dataset for Medical Search Intent Classification
Schnell, S., Kottam, S., & Sun, Y. (2026) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3805713.3820435
Are Generative Recommenders Truly Fair? Auditing Bias with Conformal Calibration
Tao, C., Pan, Y., & Fan, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3444-8_22
Vo, T. K. T., Nguyen, N. H., & Tan, H. M. (2026) · International Conference on Multimedia Analysis and Pattern Recognition (MAPR) · doi:10.1109/mapr72750.2026.11685719
Instruction Fine-Tuning of Large Language Models for Structured Educational Response Generation
Karatay, A., & Zhartybayeva, M. (2026) · IEEE International Conference on Smart Information Systems and Technologies (SIST) · doi:10.1109/sist61674.2026.11596315
Towards Adaptive and Retriever-friendly Retrieval-augmented Generation via Reinforcement Learning
FANG, Y., Yu, H.-T., Joho, H., & Fujita, S. (2026) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3805713.3820432
Gómez-Gutiérrez, O. A., Imbrognio, F., la Torre, O. A. D., Vegas, D. H., & Fabiani, M. A. (2026) · Annals of Vascular Surgery · doi:10.1016/j.avsg.2026.07.004
Outlier-Aware and Task-Oriented Quantization-Based Low-Rank Adaptation of Large Language Models
Zhang, C., Zhou, G., & Fu, T. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3444-8_30
Self-citationA Cascaded Generative Approach for e-Commerce Recommendations
Hasani, M., Shahidi, H., Levinson, T., Zhong, Y., Shu, G., Gudla, V., & Tenneti, T. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808458
Towards End-to-End Alignment of User Satisfaction via Questionnaire in Video Recommendation
Li, N., Yu, J., Xie, M., He, T., Xu, X., Wang, Z., Hu, L., Liu, Y., Li, H., Zhan, K., et al. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809715
Preliminary Study of an Evaluation Benchmark for Vision–Language Models in Fashion E-Commerce
Shimizu, R., HtaungKham, S., Sakurai, S., & Shimizu, Y. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808442
If attention scaled intelligence, can it scale morality?
Bombaerts, G. J. T., Delisse, B., & Kaymak, U. (2026) · AI and Ethics · doi:10.1007/s43681-026-01239-4
Shitara, K., Chaiprabha, K., Piankitrungreang, P., Takano, K., Chancharoen, R., & Phanomchoeng, G. (2026) · Journal of Advances in Information Technology · doi:10.12720/jait.17.6.1162-1176
Captioning daily activity images in early childhood education: benchmark and algorithm
Li, S., Gu, Z., Zhang, Z., Pan, W., Li, B., Wang, Y., & Liu, H. (2026) · Multimedia Systems · doi:10.21203/rs.3.rs-10234796/v1
Ethical and Legal Limits of Aligning Large Language Models: The Illusion of Control
Schmitt, V., Ostermann, S., & Solopova, V. (2026) · Artificial intelligence · doi:10.5772/intechopen.1016018
Yu, Y., Park, S. H., Kim, H.-G., & Cho, B. H. (2026) · doi:10.2196/preprints.107087
Singh, A. K., Chawla, M., & Tiwari, N. (2026) · International Conference on Secure Cyber Computing and Communications (ICSCCC) · doi:10.1109/icsccc69031.2026.11600352
Lorusso, R., Maci, A., & Coscia, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-25029-2_11
Xu, G., Huang, Y., Zhang, K., Li, X., & Zan, H. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3417-2_30
Fu, C., Zhang, W., Zhang, J., & Si, N. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3417-2_13
An Educational Prompt Framework Based on Complex Adaptive Systems Theory
汤, 雅. (2026) · Creative Education Studies · doi:10.12677/ces.2026.147497
A Research Agenda for Usability and Generalisation in Reinforcement Learning
Soemers, D. J. N. J., Samothrakis, S., Driessens, K., & Winands, M. H. M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-25032-2_33
Peng, R., & Bao, J. (2026) · Electronics · doi:10.3390/electronics15143143
Hua, W., Fang, Q., Sun, M., Xu, S., & Zhang, M. (2026) · Multimedia Systems · doi:10.1007/s00530-026-02499-0
GenAI vs Traditional AI: A Comparative Framework Across Five Domains
Khanna, S., Yadav, A., & Kapoor, K. (2026) · IEEE International Conference on Automatic Control and Intelligent Systems (I2CACIS) · doi:10.1109/i2cacis69435.2026.11600276
StoryCheck: A Scalable Framework for Automated Evaluation of Creative Writing using AI Agents
Shittu, L., Akintayo, M., Arabambi, A., Okeleye, F., Owah, O. L., Akinfenwa, O., Olajide-Shokunbi, A., Jaiyesimi, B. S., Fagbohun, O., & Adetunji, R. A. (2026) · Cognitive Models and Artificial Intelligence Conference (AICCONF) · doi:10.1109/aicconf69182.2026.11600738
A Governed Closed-Domain Question Answering System for Decision Assistance in Education
Elnozahy, W. A., Khayat, G. A. E., & Aboulsafa, E. I. (2026) · IEEE International Conference on Smart Sustainable Systems for Computer and Engineering Applications (3SCEA) · doi:10.1109/3scea68071.2026.11603208
Freezing the Crime Scene: A State Snapshot Paradigm for Reproducible Agentic SRE Evaluation
Yu, G., Wang, Y., & Lyu, M. R.-T. (2026) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3803437.3806092
Prakash, N., & Desai, A. (2026) · IEEE International Conference on Smart Sustainable Systems for Computer and Engineering Applications (3SCEA) · doi:10.1109/3scea68071.2026.11602961
Liu, L., Yao, S., Liu, Y. P., Wang, Y., Xia, H., & Tang, X. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.105027
LLMs as judges: Toward the LLM-assisted review of GSN-compliant assurance cases
Yu, G., Sivakumar, M., Belle, A. B., Ghari, S., Wang, S., & Lethbridge, T. C. (2026) · Journal of Systems and Software · doi:10.1016/j.jss.2026.113036
BiasAgent: Exploiting Agent Bias for Preference Manipulation Attacks on Model Context Protocol
Guo, J. H., Wang, Z., Jiang, W., Zhang, R. Y., Xiong, J., Song, Q., Wu, H. B., & Liu, Y.-J. (2026) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2026.3714063
Kassab, M., & Ibrahim, Y. (2026) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3803437.3806716
Kumar, R., Khullar, S., & Arora, A. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-0368-8.ch003
Lu, Y., Li, W., & Cai, Y. (2026) · Systems · doi:10.3390/systems14070846
PR-TDMPC: Preference-Based Reinforcement Learning for Humanoid Control
Yao, T., Wu, M., Lu, G., Zhu, S., & Zheng, B. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3381-6_35
Ma, J., Wang, C., Zhang, Y., Wang, Q., Yang, W., & Zhao, Z. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3403-5_9
SARE: Soft Alignment Reward for Reinforcement Learning in Generative Recommendation
Yang, Y., Wang, Z., Li, L., & Zeng, D. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3391-5_45
Jiang, X., & Li, T. F. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3520-9_5
Self-citationMixture of Experts Approach for Domain Adaptation in Finance
Suzuki, M., Sakaji, H., & Izumi, K. (2026) · Journal of Advanced Computational Intelligence and Intelligent Informatics · doi:10.20965/jaciii.2026.p1093
Siddiqui, S. M. T., Ononto, A. J., Singhal, A., & Khan, L. R. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-33260-8_14
Exploring Open-Source Language Model in Solving Physics Word Problems
Liang, Y., Zou, D., Xie, H., & Wang, F. L. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3761-6_11
LALUN: LLM-Enhanced Adaptive Label Update Network for Aspect Sentiment Triplet Extraction
Ye, Y., & Zeng, B. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3426-4_24
Agentic AI Unleashed: Power, Promise, and Ethical Frontiers
Wang, M., Wang, Y., & Ma, Q. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3761-6_3
Liu, T., Chen, J., Lai, J., & Huang, Z. (2026) · International Conference on Artificial Intelligence and Digital Ethics (ICAIDE) · doi:10.1109/icaide69370.2026.11604549
Li, Q. (2026) · International Conference on Artificial Intelligence and Digital Ethics (ICAIDE) · doi:10.1109/icaide69370.2026.11604521
JAIL: Adaptive multi-turn jailbreak attacks reveal limitations of LLM safety alignment
Feng, Y., Lao, M., Wang, J., Li, Y., Li, X., Xie, Y., & Guo, Y. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.133684
Ma, N., Fan, B., Tang, Q., & Yan, W. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3761-6_4
RefZVC: Refinable Zero-Shot Video Captioning by Test-Time Reinforcement Polishing
Bao, Q., Liu, F., Jiao, L., Liu, Y., Li, L., & Liu, X. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3713012
Störmer, M., & Rehfeld, K.-M. (2026) · Springer eBooks · doi:10.1007/978-3-658-47898-8_22-1
Perera, K. J. P. G., Nanayakkara, S., & Huff, E. W. (2026) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3816046.3816225
Entangled Visions: Cocreativity and Biomimicry in Contemporary AI Art
Wright, R., & Sade, G. (2026) · Leonardo · doi:10.1162/leon.a.2734
A Comprehensive Review of Vision-Based Fatigue Detection AlgorithmsPreprint
Tang, C., Liu, J., Xiong, J., Han, J., Cao, M. L., Chi, H., Yu, Z., Zhou, B., & Yang, L. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7152575
Evaluating reinforcement learning from human feedback for task‐oriented dialogue systems
Gwon, H.-M., Lee, Y., Huang, J., & Lee, J. (2026) · ETRI Journal · doi:10.4218/etrij.2025-0313
Miličević, M., Rovis, M., Karlović, R., Šegota, S. B., Mrzljak, V., Lorencin, I., & Etinger, D. (2026) · Information · doi:10.3390/info17070706
Cloud and Edge Systems for Large AI Models
Yuan, H., Bi, J., Zhou, M., & Buyya, R. (2026) · Internet of things · doi:10.1007/978-3-032-18400-9_15
Benchmarking Large Language Models for Java Code Smell Detection
Ghadimi, E. B., Şahin, D. Ö., Demirci, S., & Yıldız, D. (2026) · International Symposium on AI-Driven Engineering Systems (ISADES) · doi:10.1109/isades69945.2026.11608239
Delaflor, M., & Toxtli, C. (2026) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3816046.3816233
Liao, G.-C., & Liao, B.-T. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7152488
Instruction-Tuned vs Chat-Tuned LLMs: A Comparative Study
K.Kalidhas, & P.Pradeepa (2025) · IEEE International Conference on Innovate for Humanitarian: Tech Solutions for Global Challenge (ICIH) · doi:10.1109/icih67754.2025.11608679
Saju, G., Akhi, M., & 仇, 长. (2026) · IEEE International Conference on AI and Data Analytics (ICAD) · doi:10.1109/icad69378.2026.11608529
Privacy-Preserving Entity Transformation: A Semantic Compatibility Framework for Language Models
Desai, A., & Prakash, N. (2026) · International Conference on Machine Learning and Intelligent Systems Engineering (MLISE) · doi:10.1109/mlise70044.2026.11607655
Clustering-guided graph contrastive learning for short-text classification
Zhou, J., Zhang, C., Dong, H., & Liu, X. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.133695
Self-citationCross-Modal Misalignment as a Systemic Vulnerability in MLLMs
Angelis, V. D., Annunziata, D., Izzo, S., Giampaolo, F., & Piccialli, F. (2026) · International Conference on Machine Learning and Intelligent Systems Engineering (MLISE) · doi:10.1109/mlise70044.2026.11607572
Chen, K., Wang, Q., Shen, Z., Zhao, Y., Zeng, Y., Qin, X., Li, T., & Luo, H. (2026) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-026-03238-7
Zhang, X., WILSON, C. L., Eyre, H., Reed, D. E., Wai, T. H., Kloehn, A. T., Rosser, E. W., Luo, G., & Zeliadt, S. B. (2026) · JMIR Research Protocols · doi:10.2196/96346
LLMs or Naive Bayes? Old Gems or New Ways
Sada, M. F., Graham, J. A., Tatineni, M., Mishin, D., Kim, S., & Würthwein, F. (2026) · Practice and Experience in Advanced Research Computing · doi:10.1145/3785462.3815880
Jatavallabha, A., Matam, V., & Vijay, V. (2026) · IEEE International Conference on AI and Data Analytics (ICAD) · doi:10.1109/icad69378.2026.11608900
Encounter Ontology: Relational Intelligence and Generative Emergence in Human–AI Interaction
Sfakianakis, M., & Kampi, M. E. (2026) · Electronics · doi:10.3390/electronics15143218
Evaluating Summaries of Social Media Posts: Balancing Fairness for Readers and Authors
Chhikara, G., Ghosh, K., Ghosh, S., & Chakraborty, A. (2026) · ACM Transactions on the Web · doi:10.1145/3833377 · Cited by 1
Yamakawa, H., & Taniguchi, T. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-33195-3_31
Li, H., Kong, Y., & Wang, M. (2026) · International Conference on Machine Learning and Intelligent Systems Engineering (MLISE) · doi:10.1109/mlise70044.2026.11607601
Large Language Models: Technical Pathways and Application Challenges
Zhang, Y., & Bian, X. (2025) · International Conference on Intelligent Decision-making and Machine Learning (ICIDML) · doi:10.1109/icidml69845.2025.00010
High-precision time-domain parallelism photonic computing
Che, J., Wang, G., Zhou, H., Mu, Y., Shen, J., Cheng, Z., & Zhou, P. (2026) · Science Advances · doi:10.1126/sciadv.aef7533
Elsayed, S. (2026) · ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies (COMPASS) · doi:10.1145/3811242.3819088
Internal Reasoning and External Control
Chang, E. Y. (2026) · ACM eBooks · doi:10.1145/3822380.3822385
Chang, E. Y. (2026) · ACM eBooks · doi:10.1145/3822380.3822392
Retrieval Augmented Generation Enhanced RLHF Framework for Factuality Verification Improvement
Zhou, X. (2025) · International Conference on Intelligent Decision-making and Machine Learning (ICIDML) · doi:10.1109/icidml69845.2025.00022
Chang, E. Y. (2026) · ACM eBooks · doi:10.1145/3822380.3822386
From Spine to Mind: The Causal Abstraction Bridge
Chang, E. Y. (2026) · ACM eBooks · doi:10.1145/3822380.3822387
Metadata
- Authors
- Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., & Lowe, R.
- Venue
- Advances in Neural Information Processing Systems 35
- Published
- 2022
- Pages
- 27730-27744
- ISSN
- 1049-5258 (ISSN-L)
- OpenAlex
- W7133211372
- Cited by
- 3,117
- References
- 0
- Citations by year
- 2026: 1421 · 2025: 544 · 2024: 852 · 2023: 297 · 2022: 3
- FWCI
- 234.27
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- neural information processing systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- instruction following, language models, human feedback, reinforcement learning from human feedback, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.