The DOI matches its DataCite record; 45 of the 45 references it cites were found in a registry or an index.
Fine-Tuning Language Models from Human Preferences
Open accessAbstract
Reward learning enables the application of reinforcement learning (RL) to tasks where reward is defined by human judgment, building a model of reward by asking humans questions. Most work on reward learning has used simulated environments, but complex information about values is often expressed in natural language, and we believe reward learning for language is a key to making RL practical and safe for real-world tasks. In this paper, we build on advances in generative pretraining of language models to apply reward learning to four natural language tasks: continuing text with positive sentiment or physically descriptive language, and summarization tasks on the TL;DR and CNN/Daily Mail datasets. For stylistic continuation we achieve good results with only 5,000 comparisons evaluated by humans. For summarization, models trained with 60,000 comparisons copy whole sentences from the input but skip irrelevant preamble; this leads to reasonable ROUGE scores and very good performance according to our human labelers, but may be exploiting the fact that labelers rely on simple heuristics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, while the combination of RL and human feedback has been extensively studied [30,82], it is only recently that RLHF with LLMs has achieved significant success in sequence-to-sequence tasks such as Summarization [177,194,222], providing reliable answers with citations to queries [53,123], creating Helpful, Harmless and Honest dialogue agents aligned with broad human values [12,136].”
From RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs (2026) · cites this paper as [222]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“It appears that with recent increases in scale, LLMs have encoded in ever more granular detail nuanced language patterns, or styles (65), that bring about these mimicking abilities, further enhanced via fine-tuning for human-like conversation through reinforcement learning from human feedback (48).”
From The benefits and dangers of anthropomorphic conversational agents (2025) · cites this paper as [48]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This is the motivation for fine-tuning efforts via RLHF or other methods: to align the outputs of a pretrained model with an acceptable imitation of an ideal for an obedient assistant (Ziegler et al. 2019;Bai et al. 2022).”
From Epistemic Injustice in Generative AI (2024) · cites this paper as Ziegler et al. 2019
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Unlike traditional finetuning, RL does not require large datasets; instead, models learn from comparisons rather than absolute labels, reducing redundancy and cost (Ziegler et al., 2019).”
From YESciEval: Robust LLM-as-a-Judge for Scientific Question Answering (2025) · cites this paper as Ziegler et al., 2019
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Akey limitation of our study is the exclusive use of models that have undergone Reinforcement Learning with Human Feedback (RLHF; Christiano et al., 2017; Ziegler et al., 2019).”
From Simulating Opinion Dynamics with Networks of LLM-based Agents (2024) · cites this paper as Ziegler et al., 2019
In the citing paper
“Yet another approach is to fine-tune using constitutional rules provided by humans 190 .”
From Machine culture (2023) · cites this paper as [190]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Self-citationGoogle's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319
Universal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1031 · Cited by 4,100
Teaching Machines to Read and ComprehendPreprint
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,855
Image-Based Recommendations on Styles and Substitutes
McAuley, J., Targett, C., Shi, Q., & van den Hengel, A. (2015) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2766462.2767755 · Cited by 2,348
Zhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., & Fidler, S. (2015) · arXiv · doi:10.48550/arxiv.1506.06724 · Cited by 2,264
Zhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., & Fidler, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.11 · Cited by 2,264
Flagged first, then by citations
Research works printed but not linked: 3
- Paul Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. In Advances in Neural Information Processing Systems , pages 4302-4310, 2017.
- Borja Ibarz, Jan Leike, Tobias Pohlen, Geoffrey Irving, Shane Legg, and Dario Amodei. Reward learning from human preferences and demonstrations in Atari. In Advances in Neural Information Processing Systems , 2018. URL https://arxiv.org/abs/1811.06521 . · arXiv
- Natasha Jaques, Shixiang Gu, Dzmitry Bahdanau, José Miguel Hernández-Lobato, Richard E Turner, and Douglas Eck. Sequence tutor: Conservative fine-tuning of sequence generation models with kl-control. In Proceedings of the 34th International Conference on Machine Learning-Volume 70 , pages 1645-1654. JMLR. org, 2017.
Cited by
Sorted by citations
Items 101–200 of 468
Liu, W., Li, J., Tang, Y., Zhao, Y., Liu, C., Song, M., Ju, Z., Kumar, S. V., Lu, Y., Akbani, R., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-57430-4 · Cited by 51
A Review of Current Trends, Techniques, and Challenges in Large Language Models (LLMs)
Patil, R., & Gudivada, A. V. (2024) · doi:10.20944/preprints202402.0357.v1 · Cited by 296
Plug-and-Play Conversational Models
Madotto, A., Ishii, E., Lin, Z., Dathathri, S., & Fung, P. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.219 · Cited by 51
Compilable Neural Code Generation with Compiler Feedback
Wang, X., Wang, Y., Wan, Y., Mi, F., Li, Y., Zhou, P., Liu, J., Wu, H., Jiang, X., & Liu, Q. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.2 · Cited by 56
Large Language Models for Text Classification: From Zero-Shot Learning to Instruction-TuningPreprint
Chae, Y., & Davidson, T. (2023) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/sthwk · Cited by 54
The Foundation Model Transparency Index
Bommasani, R., Klyman, K., Longpre, S., Kapoor, S., Maslej, N., Xiong, B., Zhang, D., & Liang, P. (2023) · arXiv · doi:10.48550/arxiv.2310.12941 · Cited by 46
Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation
Fernandes, P., Madaan, A., Liu, E., Farinhas, A., Martins, P. H., Bertsch, A., de Souza, J. G. C., Zhou, S., Wu, T., Neubig, G., et al. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00626 · Cited by 94
Generative AI vs. humans in online hotel review management: A Task-Technology Fit perspective
Zhang, H., Xiang, Z., & Zach, F. (2025) · Tourism Management · doi:10.1016/j.tourman.2025.105187 · Cited by 61
Scaling Unlocks Broader Generation and Deeper Functional Understanding of Proteins
Bhatnagar, A., Jain, S., Beazer, J., Curran, S. C., Hoffnagle, A. M., Ching, K., Martyn, M., Nayfach, S., Ruffolo, J. A., & Madani, A. (2025) · neural information processing systems · doi:10.52202/085713-1536 · Cited by 55
Mix and Match: Learning-free Controllable Text Generationusing Energy Language Models
Mireshghallah, F., Goyal, K., & Berg-Kirkpatrick, T. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.31 · Cited by 61
An information fusion based approach to context-based fine-tuning of GPT models
Nguyen-Mau, T., Le, A.-C., Pham, D.-H., & Huynh, V. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102202 · Cited by 45
Generative Steganography Based on Long Readable Text Generation
Cao, Y., Zhou, Z., Chakraborty, C., Wang, M., Wu, Q. M. J., Sun, X., & Yu, K. (2022) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2022.3174013 · Cited by 42
Are Large Language Models All You Need for Task-Oriented Dialogue?
Hudeček, V., & Dušek, O. (2023) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2023.sigdial-1.21 · Cited by 105
New Paradigm for Economic and Financial Research With Generative AI: Impact and Perspective
Zheng, X., Li, J., Lu, M., & Wang, F. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2023.3334306 · Cited by 51
Human-centric dialog training via offline reinforcement learning
Jaques, N., Shen, J. H., Ghandeharioun, A., Ferguson, C., Lapedriza, À., Jones, N. C., Gu, S., & Picard, R. W. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.327 · Cited by 81
GPT-4 vs. GPT-3Preprint
Koubâa, A. (2023) · doi:10.36227/techrxiv.22312330.v1 · Cited by 46
The Plastic Surgery Hypothesis in the Era of Large Language Models
Xia, C. S., Ding, Y., & Zhang, L. (2023) · 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00047 · Cited by 67
ChatGPT for digital pathology research
Omar, M., Ullanat, V., Loda, M. F., Marchionni, L., & Umeton, R. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00114-6 · Cited by 40
Applications and perspectives of Generative Artificial Intelligence in agriculture
Pallottino, F., Violino, S., Figorilli, S., Pane, C., Aguzzi, J., Colle, G., Nemmi, E. N., Montaghi, A., Chatzievangelou, D., Antonucci, F., et al. (2025) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2025.109919 · Cited by 44
LLM-PBE: Assessing Data Privacy in Large Language Models
Li, Q., Hong, J., Xie, C., Tan, J., Xin, R., Hou, J., Yin, X., Wang, Z., Hendrycks, D., Wang, Z., et al. (2024) · Proceedings of the VLDB Endowment · doi:10.14778/3681954.3681994 · Cited by 42
SayCanPay: Heuristic Planning with Large Language Models Using Learnable Domain Knowledge
Hazra, R., Martires, P. Z. D., & Raedt, L. D. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i18.29991 · Cited by 51
A Review of Reinforcement Learning for Fixed-Wing Aircraft Control Tasks
Richter, D. J., Calix, R. A., & Kim, K. (2024) · IEEE Access · doi:10.1109/access.2024.3433540 · Cited by 41
REFINER: Reasoning Feedback on Intermediate RepresentationsPreprint
Paul, D., Ismayilzada, M., Peyrard, M., Borges, B., Bosselut, A., West, R., & Faltings, B. V. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.01904 · Cited by 35
Using ChatGPT for medical education: the technical perspective
Chan, K. Y., Yuen, T. H., & Co, M. (2025) · BMC Medical Education · doi:10.1186/s12909-025-06785-9 · Cited by 34
Liang, H., Li, X., Xiao, D., Liu, J., Zhou, Y., Wang, A., & Li, J. (2023) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2023.3252523 · Cited by 37
Language Models Meet World Models: Embodied Experiences Enhance Language Models
Xiang, J., Tao, T., Gu, Y., Shu, T., Wang, Z., Yang, Z., & Hu, Z. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-3295 · Cited by 34
Large language models for automated scholarly paper review: A survey
Zhuang, Z., Chen, J., Xu, H., Jiang, Y., & Lin, J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103332 · Cited by 34
Personalized Impression Generation for PET Reports Using Large Language Models
Tie, X., Shin, M., Pirasteh, A., Ibrahim, N., Huemann, Z., Castellino, S. M., Kelly, K. M., Garrett, J., Hu, J., Cho, S. Y., et al. (2024) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-024-00985-3 · Cited by 35
ChatGPT: Keuntungan, Risiko, Dan Penggunaan Bijak Dalam Era Kecerdasan Buatan
Misnawati, M. (2023) · PROSIDING SEMINAR NASIONAL PENDIDIKAN BAHASA SASTRA SENI DAN BUDAYA · doi:10.55606/mateandrau.v2i1.221 · Cited by 32
Buchlak, Q. D., Esmaili, N., Bennett, C. C., & Farrokhi, F. R. (2021) · Acta neurochirurgica. Supplementum · doi:10.1007/978-3-030-85292-4_32 · Cited by 34
CQG: A Simple and Effective Controlled Generation Framework for Multi-hop Question Generation
Fei, Z., Zhang, Q., Gui, T., Liang, D., Wang, S., Wu, W., & Huang, X. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.475 · Cited by 35
Risks from Language Models for Automated Mental Healthcare: Ethics and Structure for Implementation
Grabb, D., Lamparth, M., & Vasan, N. (2024) · doi:10.1101/2024.04.07.24305462 · Cited by 40
Generative AI Meets Wireless Networking: An Interactive Paradigm for Intent-Driven Communications
Qin, X., Sun, M., Dai, J., Ma, P., Cao, Y., Zhang, J., Wang, J., Xu, X., Zhang, P., & Niyato, D. T. (2025) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2025.3567613 · Cited by 36
Davidson, T., & Karell, D. (2025) · Sociological Methods & Research · doi:10.1177/00491241251339184 · Cited by 40
Summarizing Text on Any Aspects: A Knowledge-Informed Weakly-Supervised Approach
Tan, B., Qin, L., Xing, E. P., & Hu, Z. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.510 · Cited by 35
Tailor: A Soft-Prompt-Based Approach to Attribute-Based Controlled Text Generation
Yang, K., Liu, D., Lei, W., Yang, B., Xue, M., Chen, B., & Xie, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.25 · Cited by 61
Toward Grounded Commonsense Reasoning
Kwon, M., Hu, H., Myers, V., Karamcheti, S., Dragan, A. D., & Sadigh, D. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611218 · Cited by 41
A psychometric framework for evaluating and shaping personality traits in large language models
Serapio-García, G., Safdari, M., Crepy, C., Sun, L., Fitz, S., Romero, P., Abdulhai, M., Faust, A., & Matarić, M. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01115-6 · Cited by 30
Zhou, J., & Ma, Z. (2025) · Automation in Construction · doi:10.1016/j.autcon.2025.106151 · Cited by 35
Domain-specific chatbots for science using embeddings
Yager, K. G. (2023) · Digital Discovery · doi:10.1039/d3dd00112a · Cited by 35
Conducting Qualitative Interviews with AI
Chopra, F., & Haaland, I. (2026) · CESifo · doi:10.65864/ch6grwpore · Cited by 38
“Personhood and AI: Why large language models don’t understand us”
Browning, J. (2024) · AI & SOCIETY · doi:10.1007/s00146-023-01724-y · Cited by 27
Automatic Title Generation for Text with Pre-trained Transformer Language Model
Mishra, P., Diwan, C., Srinivasa, S., & Srinivasaraghavan, G. (2021) · IEEE International Conference on Semantic Computing (ICSC) · doi:10.1109/icsc50631.2021.00009 · Cited by 29
AI’s Epistemic Harm: Reinforcement Learning, Collective Bias, and the New AI Culture War
Humphreys, D. (2025) · Philosophy & Technology · doi:10.1007/s13347-025-00928-y · Cited by 26
Attribute Alignment: Controlling Text Generation from Pre-trained Language Models
Yu, D., Yu, Z., & Sagae, K. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.194 · Cited by 33
Robust Deep Reinforcement Learning for Extractive Legal Summarization
Nguyen, D.-H., Nguyen, B.-S., Nghiem, N. V. D., Le, D. T., Khatun, M. A., Nguyen, M.-T., & Lê, H. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-92310-5_69 · Cited by 28
Gradient-based Constrained Sampling from Language Models
Kumar, S., Paria, B., & Tsvetkov, Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.144 · Cited by 57
Reinforcement Learning from Human Feedback in LLMs: Whose Culture, Whose Values, Whose Perspectives?
Barman, K. G., Lohse, S., & de Regt, H. W. (2025) · Philosophy & Technology · doi:10.1007/s13347-025-00861-0 · Cited by 43
Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedback
Roit, P., Ferret, J., Shani, L., Aharoni, R., Cideron, G., Dadashi, R., Geist, M., Girgin, S., Hussenot, L., Keller, O., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.344 · Cited by 63
An agent-based approach for the automatic generation of valid SysMLv2 Models in industrial contexts
Cibrián, E., Olivert-Iserte, J., Lloréns, J., & Álvarez-Rodríguez, J. M. (2025) · Computers in Industry · doi:10.1016/j.compind.2025.104350 · Cited by 27
Neural Language Generation: Formulation, Methods, and EvaluationPreprint
Gârbacea, C., & Mei, Q. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.15780 · Cited by 45
Language Models Meet World Models: Embodied Experiences Enhance Language Models
Xiang, J., Tao, T., Gu, Y., Shu, T., Wang, Z., Yang, Z., & Hu, Z. (2023) · arXiv · Cited by 34
When to Show a Suggestion? Integrating Human Feedback in AI-Assisted Programming
Mozannar, H., Bansal, G., Fourney, A., & Horvitz, E. (2023) · arXiv · doi:10.48550/arxiv.2306.04930 · Cited by 22
GPT-4 vs. GPT-3.5: A Concise ShowdownPreprint
Koubâa, A. (2023) · doi:10.36227/techrxiv.22312330 · Cited by 32
RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning
Gehring, J., Zheng, K., Copet, J., Mella, V., Carbonneaux, Q., Cohen, T., & Synnaeve, G. (2024) · arXiv · doi:10.48550/arxiv.2410.02089 · Cited by 23
Reinforcement Learning from Automatic Feedback for High-Quality Unit Test Generation
Steenhoek, B. J., Tufano, M., Sundaresan, N., & Svyatkovskiy, A. (2025) · IEEE/ACM International Workshop on Deep Learning for Testing and Testing for Deep Learning (DeepTest) · doi:10.1109/deeptest66595.2025.00011 · Cited by 72
A reliable knowledge processing framework for combustion science using foundation models
Sharma, V., & Raman, V. (2024) · Energy and AI · doi:10.1016/j.egyai.2024.100365 · Cited by 24
Hou, Y., Bert, C., Gomaa, A., Lahmer, G., Höfler, D., Weißmann, T., Voigt, R., Schubert, P., Schmitter, C., Depardon, A., et al. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1493716 · Cited by 30
When to Show a Suggestion? Integrating Human Feedback in AI-Assisted Programming
Mozannar, H., Bansal, G., Fourney, A., & Horvitz, E. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i9.28878 · Cited by 23
Liu, J., Shen, H., Chen, K., & Li, X. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae430 · Cited by 20
Large language model-based methodology for data-driven health prediction of Lithium-ion batteries
Tunçel, S., Çınar, H., Güçyetmez, M., & Erdoğan, N. (2025) · Energy Reports · doi:10.1016/j.egyr.2025.05.027 · Cited by 21
ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?
Heck, M. C., Lubis, N., Ruppik, B., Vukovic, R., Feng, S., Geishauser, C., Lin, H.-C., van Niekerk, C., & Gašić, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.81 · Cited by 41
Opportunities and Risks of LLMs for Scalable Deliberation with Polis
Small, C. T., Vendrov, I., Durmus, E., Homaei, H., Barry, E., Cornebise, J., Suzman, T., Ganguli, D., & Megill, C. (2023) · arXiv · doi:10.48550/arxiv.2306.11932 · Cited by 20
“These human judgments can be used to as a feedback to the models and better align the automated summarization with what participants feel captures the essence of the discourse, a sort of Deliberative Reinforcement Learning from Human Feedback (RLHF) (Ziegler et al., 2020; Lambert et al., 2022).”
BackgroundReducing Non-Normative Text Generation from Language Models
Peng, X., Li, S., Frazier, S., & Riedl, M. O. (2020) · International Conference on Natural Language Generation · doi:10.18653/v1/2020.inlg-1.43 · Cited by 28
Benefits and Harms of Large Language Models in Digital Mental Health
De Choudhury, M., Pendse, S. R., & Kumar, N. (2023) · arXiv · doi:10.48550/arxiv.2311.14693 · Cited by 78
Controllable Summarization with Constrained Markov Decision Process
Chan, H. P., Wang, L., & King, I. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00423 · Cited by 23
Bidimensional Leaderboards: Generate and Evaluate Language Hand in Hand
Kasai, J., Sakaguchi, K., Bras, R. J. L., Dunagan, L., Morrison, J., Fabbri, A. R., Choi, Y., & Smith, N. A. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.259 · Cited by 21
BeautifulPrompt: Towards Automatic Prompt Engineering for Text-to-Image Synthesis
Cao, T., Wang, C., Liu, B., Wu, Z., Zhu, J., & Huang, J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-industry.1 · Cited by 42
Belief-based Generation of Argumentative Claims
Alshomary, M., Chen, W.-F., Gurcke, T., & Wachsmuth, H. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.17 · Cited by 20
Transforming agency: On the mode of existence of large language models
Barandiaran, X. E., & Almendros, L. S. (2025) · Phenomenology and the Cognitive Sciences · doi:10.1007/s11097-025-10094-3 · Cited by 35
Pre-trained Language Model-based Retrieval and Ranking for Web Search
Zou, L., Lu, W., Liu, Y., Cai, H., Chu, X., Ma, D., Shi, D., Sun, Y., Cheng, Z., Gu, S., et al. (2022) · ACM Transactions on the Web · doi:10.1145/3568681 · Cited by 19
Aligning Large Language Models through Synthetic Feedback
Kim, S., Bae, S., Shin, J., Kang, S., Kwak, D., Yoo, K. M., & Seo, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.844 · Cited by 80
Understanding the Effects of RLHF on LLM Generalisation and Diversity
Kirk, R., Mediratta, I., Nalmpantis, C., Luketina, J., Hambro, E., Grefenstette, E., & Raileanu, R. (2023) · arXiv · doi:10.48550/arxiv.2310.06452 · Cited by 17
GPT-4 vs. GPT-3.5: A Concise Showdown
Koubaa, A. (2023) · doi:10.20944/preprints202303.0422.v1 · Cited by 37
Demonstrations of the Potential of AI-based Political Issue Polling
Sanders, N. E., Ulinich, A., & Schneier, B. (2023) · Harvard Data Science Review · doi:10.1162/99608f92.1d3cf75d · Cited by 43
Play the Shannon Game with Language Models: A Human-Free Approach to Summary Evaluation
Egan, N., Vasilyev, O., & Bohannon, J. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21304 · Cited by 22
DeepClone: Modeling Clones to Generate Code Predictions
Hammad, M., Babur, Ö., Basit, H. A., & van den Brand, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-64694-3_9 · Cited by 17
Adaptive and Personalized Exercise Generation for Online Language Learning
Cui, P., & Sachan, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.567 · Cited by 37
DreamReward: Text-to-3D Generation with Human Preference
Ye, J., Liu, F., Li, Q., Wang, Z., Wang, Y., Wang, X., Duan, Y., & Zhu, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72897-6_15 · Cited by 44
Diversity driven Query Rewriting in Search Advertising
Mohankumar, A. K., Begwani, N., & Singh, A. P. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467202 · Cited by 17
Spurious Correlations in Reference-Free Evaluation of Text Generation
Durmus, E., Ladhak, F., & Hashimoto, T. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.102 · Cited by 16
A National Security Research Agenda for Cybersecurity and Artificial Intelligence
Buchanan, B. (2020) · doi:10.51593/2020ca001 · Cited by 16
Pellert, M., Lechner, C. M., Wagner, C., Rammstedt, B., & Strohmaier, M. (2022) · doi:10.31234/osf.io/jv5dt · Cited by 16
Identification and Description of Emotions by Current Large Language ModelsPreprint
Patel, S., & Fan, J. (2023) · Research Square · doi:10.1101/2023.07.17.549421 · Cited by 18
Hassan, M. A., Rehmat, A., Khan, M. U. G., & Yousaf, M. H. (2022) · Mathematical Problems in Engineering · doi:10.1155/2022/6825555 · Cited by 15
Mining Twitter for Insights into ChatGPT Sentiment: A Machine Learning Approach
Sharma, S. R. R., Aggarwal, R., & Kumar, M. (2023) · International Conference on Distributed Computing and Electrical Circuits and Electronics (ICDCECE) · doi:10.1109/icdcece57866.2023.10150620 · Cited by 15
trlX: A Framework for Large Scale Reinforcement Learning from Human Feedback
Havrilla, A., Zhuravinskyi, M., Phung, D., Tiwari, A. K., Tow, J., Biderman, S., Anthony, Q. G., & Castricato, L. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.530 · Cited by 30
Reinforcement Learning with Human Feedback for Realistic Traffic Simulation
Cao, Y., Ivanovic, B., Xiao, C., & Pavone, M. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610878 · Cited by 26
ChatGPT in the public eye: Ethical principles and generative concerns in social media discussions
Cohen, M., Khavkin, M., Davidow, D. M., & Toch, E. (2024) · New Media & Society · doi:10.1177/14614448241279034 · Cited by 23
Parks, A. C., Travers, E., Perera-Delcourt, R. P., Major, M., Economides, M., & Mullan, P. (2025) · Journal of Participatory Medicine · doi:10.2196/69534 · Cited by 15
Deep Learning for Text Style Transfer: A SurveyPreprint
Jin, D., Jin, Z., Hu, Z., Olga Vechtomova, A., & Mihalcea, R. F. (2020) · arXiv (Cornell University) · doi:10.3929/ethz-b-000543432 · Cited by 15
Skill Preferences: Learning to Extract and Execute Robotic Skills from Human FeedbackPreprint
Wang, X., Lee, K., Hakhamaneshi, K., Abbeel, P., & Laskin, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.05382 · Cited by 17
Using In-Context Learning to Improve Dialogue Safety
Meade, N., Gella, S., Hazarika, D., Gupta, P., Jin, D., Reddy, S., Liu, Y., & Hakkani-Tur, D. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.796 · Cited by 13
Query Rewriting for Retrieval-Augmented Large Language Models
Ma, X., Gong, Y., He, P., Zhao, H., & Duan, N. (2023) · arXiv · doi:10.48550/arxiv.2305.14283 · Cited by 14
“Besides, a KL-divergence regularization is added to prevent the model from deviating too far from the initialization (Ramamurthy et al., 2022; Ziegler et al., 2019).”
MethodAI generations: from AI 1.0 to AI 4.0
Wu, J., You, H., & Du, J. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1585629 · Cited by 27
Self-citationDual-process theory and decision-making in large language models
Brady, O., Nulty, P., Zhang, L., Ward, T. E., & McGovern, D. P. (2025) · Nature Reviews Psychology · doi:10.1038/s44159-025-00506-1 · Cited by 19
JASMINE: Arabic GPT Models for Few-Shot Learning
Nagoudi, E. M. B., Abdul-Mageed, M., Elmadany, A., Inciarte, A. A., & Khondaker, M. T. I. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.1040 · Cited by 25
Leveraging Large Language Models for Literature Review Tasks - A Case Study Using ChatGPT
Zimmermann, R., Staab, M., Nasseri, M., & Brandtner, P. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-48858-0_25 · Cited by 16
PRDP: Proximal Reward Difference Prediction for Large-Scale Reward Finetuning of Diffusion Models
Deng, F., Wang, Q., Wei, W., Hou, T., & Grundmann, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00709 · Cited by 25
Metadata
- Authors
- Ziegler, D. M., Stiennon, N., Wu, J., Brown, T. B., Radford, A., Amodei, D., Christiano, P., & Irving, G.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1909.08593
- OpenAlex
- W2973379954
- Cited by
- 413
- References
- 45
- Citations by year
- 2026: 19 · 2025: 89 · 2024: 95 · 2023: 109 · 2022: 33 · 2021: 41 · 2020: 25 · 2019: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- reward learning, human preference data, reinforcement learning from human feedback, language model fine-tuning, text summarization, sentiment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
