The DOI matches its DataCite record; 45 of the 45 references it cites were found in a registry or an index.
Fine-Tuning Language Models from Human Preferences
Open accessAbstract
Reward learning enables the application of reinforcement learning (RL) to tasks where reward is defined by human judgment, building a model of reward by asking humans questions. Most work on reward learning has used simulated environments, but complex information about values is often expressed in natural language, and we believe reward learning for language is a key to making RL practical and safe for real-world tasks. In this paper, we build on advances in generative pretraining of language models to apply reward learning to four natural language tasks: continuing text with positive sentiment or physically descriptive language, and summarization tasks on the TL;DR and CNN/Daily Mail datasets. For stylistic continuation we achieve good results with only 5,000 comparisons evaluated by humans. For summarization, models trained with 60,000 comparisons copy whole sentences from the input but skip irrelevant preamble; this leads to reasonable ROUGE scores and very good performance according to our human labelers, but may be exploiting the fact that labelers rely on simple heuristics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, while the combination of RL and human feedback has been extensively studied [30,82], it is only recently that RLHF with LLMs has achieved significant success in sequence-to-sequence tasks such as Summarization [177,194,222], providing reliable answers with citations to queries [53,123], creating Helpful, Harmless and Honest dialogue agents aligned with broad human values [12,136].”
From RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs (2026) · cites this paper as [222]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“It appears that with recent increases in scale, LLMs have encoded in ever more granular detail nuanced language patterns, or styles (65), that bring about these mimicking abilities, further enhanced via fine-tuning for human-like conversation through reinforcement learning from human feedback (48).”
From The benefits and dangers of anthropomorphic conversational agents (2025) · cites this paper as [48]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This is the motivation for fine-tuning efforts via RLHF or other methods: to align the outputs of a pretrained model with an acceptable imitation of an ideal for an obedient assistant (Ziegler et al. 2019;Bai et al. 2022).”
From Epistemic Injustice in Generative AI (2024) · cites this paper as Ziegler et al. 2019
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Unlike traditional finetuning, RL does not require large datasets; instead, models learn from comparisons rather than absolute labels, reducing redundancy and cost (Ziegler et al., 2019).”
From YESciEval: Robust LLM-as-a-Judge for Scientific Question Answering (2025) · cites this paper as Ziegler et al., 2019
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Akey limitation of our study is the exclusive use of models that have undergone Reinforcement Learning with Human Feedback (RLHF; Christiano et al., 2017; Ziegler et al., 2019).”
From Simulating Opinion Dynamics with Networks of LLM-based Agents (2024) · cites this paper as Ziegler et al., 2019
In the citing paper
“Yet another approach is to fine-tune using constitutional rules provided by humans 190 .”
From Machine culture (2023) · cites this paper as [190]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Self-citationGoogle's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319
Universal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1031 · Cited by 4,100
Teaching Machines to Read and ComprehendPreprint
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,855
Image-Based Recommendations on Styles and Substitutes
McAuley, J., Targett, C., Shi, Q., & van den Hengel, A. (2015) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2766462.2767755 · Cited by 2,348
Zhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., & Fidler, S. (2015) · arXiv · doi:10.48550/arxiv.1506.06724 · Cited by 2,264
Zhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., & Fidler, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.11 · Cited by 2,264
Research works printed but not linked: 3
- Paul Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. In Advances in Neural Information Processing Systems , pages 4302-4310, 2017.
- Borja Ibarz, Jan Leike, Tobias Pohlen, Geoffrey Irving, Shane Legg, and Dario Amodei. Reward learning from human preferences and demonstrations in Atari. In Advances in Neural Information Processing Systems , 2018. URL https://arxiv.org/abs/1811.06521 . · arXiv
- Natasha Jaques, Shixiang Gu, Dzmitry Bahdanau, José Miguel Hernández-Lobato, Richard E Turner, and Douglas Eck. Sequence tutor: Conservative fine-tuning of sequence generation models with kl-control. In Proceedings of the 34th International Conference on Machine Learning-Volume 70 , pages 1645-1654. JMLR. org, 2017.
Cited by
Sorted by citations
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Self-citationTraining language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
Self-citationA Survey on Evaluation of Large Language Models
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., et al. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3641289 · Cited by 4,254
Feuerriegel, S., Hartmann, J., Janiesch, C., & Zschech, P. (2024) · Business & Information Systems Engineering · doi:10.1007/s12599-023-00834-7 · Cited by 2,267
Autonomous chemical research with large language models
Boiko, D. A., MacKnight, R., Kline, B., & Gomes, G. (2023) · Nature · doi:10.1038/s41586-023-06792-0 · Cited by 1,050
A survey on multimodal large language models
Yin, S., Fu, C., Zhao, S., Li, K., Sun, X., Xu, T., & Chen, E. (2024) · National Science Review · doi:10.1093/nsr/nwae403 · Cited by 893
A survey of human-in-the-loop for machine learning
Wu, X., Xiao, L., Sun, Y., Zhang, J., Ma, T., & He, L. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.05.014 · Cited by 800
Simulating 500 million years of evolution with a language model
Hayes, T., Rao, R., Akin, H., Sofroniew, N. J., Oktay, D., Lin, Z., Verkuil, R., Tran, V. Q., Deaton, J., Wiggert, M., et al. (2025) · Science · doi:10.1126/science.ads0018 · Cited by 937
Taxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764
“Though the standard LM training objective encourages LMs to mirror language found in the training data, generated language can be further constrained and steered by additional objectives during training [90, 207] or specific sampling techniques [42].”
BackgroundSelf-citationRealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.301 · Cited by 693
Decision Transformer: Reinforcement Learning via Sequence ModelingPreprint
Chen, L.-L., Lu, K., Rajeswaran, A., Lee, K., Grover, A., Laskin, M., Abbeel, P., Srinivas, A., & Mordatch, I. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.01345 · Cited by 849
SummEval: Re-evaluating Summarization Evaluation
Fabbri, A. R., Kryściński, W., McCann, B., Xiong, C., Socher, R., & Radev, D. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00373 · Cited by 587
Regulating ChatGPT and other Large Generative AI Models
Hacker, P., Engel, A., & Mauer, M. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594067 · Cited by 466
“Typically, LGAIM developers will pre-train a large model, which may then be fine-tuned by deployers, potentially in collaboration with developers [81, 82], while users ultimately make the decision what the AI system is used for specifically (e.”
BackgroundBenchmarking Large Language Models for News Summarization
Zhang, T., Ladhak, F., Durmus, E., Liang, P., McKeown, K., & Hashimoto, T. B. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00632 · Cited by 415
Red Teaming Language Models with Language Models
Perez, E., Huang, S., Song, F., Cai, T., Ring, R., Aslanides, J., Glaese, A., McAleese, N., & Irving, G. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.225 · Cited by 363
Self-citationUsing large language models in psychology
Demszky, D., Yang, D., Yeager, D. S., Bryan, C. J., Clapper, M., Chandhok, S., Eichstaedt, J. C., Hecht, C., Jamieson, J., Johnson, M., et al. (2023) · Nature Reviews Psychology · doi:10.1038/s44159-023-00241-5 · Cited by 369
Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models
Ferrara, E. (2023) · arXiv · doi:10.48550/arxiv.2304.03738 · Cited by 282
A review of large language models and autonomous agents in chemistry
Ramos, M. C., Collison, C. J., & White, A. D. (2025) · Chemical Science · doi:10.1039/d4sc03921a · Cited by 274
Query Rewriting in Retrieval-Augmented Large Language Models
Ma, X., Gong, Y., He, P., Zhao, H., & Duan, N. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.322 · Cited by 602
Do Users Write More Insecure Code with AI Assistants?
Perry, N., Srivastava, M., Kumar, D., & Boneh, D. (2023) · Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3576915.3623157 · Cited by 259
Should ChatGPT be biased? Challenges and risks of bias in large language models
Ferrara, E. (2023) · First Monday · doi:10.5210/fm.v28i11.13346 · Cited by 282
Zhou, H., Hu, C., Yuan, Y., Cui, Y., Jin, Y., Chen, C., Wu, H., Yuan, D., Jiang, L., Wu, D., et al. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3465447 · Cited by 344
AI deception: A survey of examples, risks, and potential solutions
Park, P. S., Goldstein, S., O’Gara, A., Chen, M., & Hendrycks, D. (2024) · Patterns · doi:10.1016/j.patter.2024.100988 · Cited by 243
AI Deception: A Survey of Examples, Risks, and Potential Solutions
Park, P. S., Goldstein, S., O'Gara, A., Chen, M., & Hendrycks, D. (2023) · arXiv · doi:10.48550/arxiv.2308.14752 · Cited by 243
Kang, Y., & Kim, J. (2024) · Nature Communications · doi:10.1038/s41467-024-48998-4 · Cited by 330
Vision-language models for medical report generation and visual question answering: a review
Hartsock, I., & Rasool, G. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1430984 · Cited by 298
A Survey of Controllable Text Generation Using Transformer-based Pre-trained Language Models
Zhang, H., Song, H., Li, S., Zhou, M., & Song, D. (2024) · ACM Computing Surveys · doi:10.1145/3617680 · Cited by 317
Ten deep learning techniques to address small data problems with remote sensing
Safonova, A. N., Ghazaryan, G., Stiller, S., Main-Knorn, M., Nendel, C., & Ryo, M. (2023) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2023.103569 · Cited by 242
Reformulating Unsupervised Style Transfer as Paraphrase Generation
Krishna, K., Wieting, J., & Iyyer, M. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.55 · Cited by 229
Applications, Challenges, and Future Directions of Human-in-the-Loop Learning
Kumar, S., Datta, S., Singh, V., Datta, D., Singh, S. K., & Sharma, R. (2024) · IEEE Access · doi:10.1109/access.2024.3401547 · Cited by 204
RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning
Deng, M., Wang, J., Hsieh, C.-P., Wang, Y., Guo, H. W., Shu, T., Song, M., Xing, E. P., & Hu, Z. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.222 · Cited by 406
Evaluating large language models on a highly-specialized topic, radiation oncology physics
Holmes, J., Liu, Z., Zhang, L., Ding, Y., Sio, T. T. W., McGee, L. A., Ashman, J. B., Li, X., Liu, T., Shen, J., et al. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1219326 · Cited by 198
Freyer, O., Wiest, I. C., Kather, J. N., & Gilbert, S. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00124-9 · Cited by 172
Exploring the capabilities and limitations of large language models in the electric energy sector
Majumder, S., Dong, L., Doudi, F., Cai, Y., Tian, C., Kalathil, D. M., Ding, K. Q., Thatte, A. A., Li, N., & Xie, L. (2024) · Joule · doi:10.1016/j.joule.2024.05.009 · Cited by 167
DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature
Mitchell, E., Lee, Y., Khazatsky, A., Manning, C. D., & Finn, C. (2023) · arXiv · doi:10.48550/arxiv.2301.11305 · Cited by 151
Detecting and Preventing Hallucinations in Large Vision Language Models
Gunjal, A., Yin, J., & Bas, E. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29771 · Cited by 161
ChatGPT and Bard exhibit spontaneous citation fabrication during psychiatry literature search
McGowan, A., Gui, Y., Dobbs, M., Shuster, S., Cotter, M., Selloni, A., Goodman, M., Srivastava, A., Cecchi, G. A., & Corcoran, C. M. (2023) · Psychiatry Research · doi:10.1016/j.psychres.2023.115334 · Cited by 140
Personality Traits in Large Language ModelsPreprint
Serapio‐García, G., Safdari, M., Crepy, C., Sun, L., Fitz, S., Abdulhai, M., Faust, A., & Matarić, M. J. (2023) · Research Square · doi:10.21203/rs.3.rs-3296728/v1 · Cited by 137
Eight Things to Know about Large Language Models
Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2304.00612 · Cited by 137
Eight Things to Know about Large Language Models
Bowman, S. R. (2024) · Critical AI · doi:10.1215/2834703x-11556011 · Cited by 305
Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review
Wong, M.-F., Guo, S., Hang, C. N., Ho, S., & Tan, C. W. (2023) · Entropy · doi:10.3390/e25060888 · Cited by 169
Large Language Models for Therapy Recommendations Across 3 Clinical Specialties: Comparative Study
Wilhelm, T. I., Roos, J., & Kaczmarczyk, R. (2023) · Journal of Medical Internet Research · doi:10.2196/49324 · Cited by 133
Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data
Xu, C., Guo, D., Duan, N., & McAuley, J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.385 · Cited by 348
Brinkmann, L., Baumann, F., Bonnefon, J.-F., Derex, M., Müller, T. F., Nussberger, A.-M., Czaplicka, A., Acerbi, A., Griffiths, T. L., Henrich, J., et al. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01742-2 · Cited by 132
Datasets for large language models: a comprehensive survey
Liu, Y., Cao, J., Liu, C., Ding, K., & Jin, L. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11403-7 · Cited by 268
Kirk, H. R., Vidgen, B., Röttger, P., & Hale, S. A. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00820-y · Cited by 122
Rawte, V., Chakraborty, S., Pathak, A., Sarkar, A., Tonmoy, S. M. T. I., Chadha, A., Sheth, A., & Das, A. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.155 · Cited by 119
Liesenfeld, A., Lopez, A., & Dingemanse, M. (2023) · Proceedings of the 5th International Conference on Conversational User Interfaces · doi:10.1145/3571884.3604316 · Cited by 111
Tian, K., Mitchell, E., Zhou, A., Sharma, A., Rafailov, R., Yao, H., Finn, C., & Manning, C. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.330 · Cited by 123
The benefits and dangers of anthropomorphic conversational agents
Peter, S., Riemer, K., & West, J. D. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2415898122 · Cited by 110
On the creativity of large language models
Franceschelli, G., & Musolesi, M. (2025) · AI & SOCIETY · doi:10.1007/s00146-024-02127-3 · Cited by 113
Perils and opportunities in using large language models in psychological research
Abdurahman, S., Atari, M., Karimi-Malekabadi, F., Xue, M. J., Trager, J., Park, P. S., Golazizian, P., Omrani, A., & Dehghani, M. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae245 · Cited by 111
“A commitment to rigor and replicability should guide the integration of AI into psychological research, not convenience.”
Asai, A., Wu, Z., Wang, Y., Sil, A., & Hajishirzi, H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.11511 · Cited by 101
“Reflection tokens, inspired by reward models used in reinforcement learning (Ziegler et al., 2019; Ouyang et al., 2022), are inserted offline into the original corpus by a trained critic model.”
MethodMultimodal Large Language Models in Medical Imaging: Current State and Future Directions
Nam, Y., Kim, D., Kyung, S., Seo, J., Song, J. M., Kwon, J., Kim, J., Jo, W., Park, H. J., Sung, J., et al. (2025) · Korean Journal of Radiology · doi:10.3348/kjr.2025.0599 · Cited by 106
Improving Factuality and Reasoning in Language Models through Multiagent Debate
Du, Y., Li, S., Torralba, A., Tenenbaum, J. B., & Mordatch, I. (2023) · arXiv · doi:10.48550/arxiv.2305.14325 · Cited by 1,202
“Our results indicate that RLHF significantly improves consensus in multiagent debate.”
InfluentialCode Generation Using Machine Learning: A Systematic Review
Dehaerne, E., Dey, B., Halder, S., Gendt, S. D., & Meert, W. (2022) · IEEE Access · doi:10.1109/access.2022.3196347 · Cited by 119
Pellert, M., Lechner, C. M., Wagner, C., Rammstedt, B., & Strohmaier, M. (2024) · Perspectives on Psychological Science · doi:10.1177/17456916231214460 · Cited by 97
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Casper, S., Davies, X., Shi, C., Gilbert, T. K., Scheurer, J., Rando, J., Freedman, R., Korbak, T., Lindner, D., Freire, P., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.15217 · Cited by 101
“Reinforcement learning from human feedback (RLHF) has emerged as a prominent technique to adapt machine learning models to difficult-to-specify goals [Christiano et al., 2017, Ziegler et al., 2019, Bai et al., 2022a].”
BackgroundMethodReframing Human-AI Collaboration for Generating Free-Text Explanations
Wiegreffe, S., Hessel, J., Swayamdipta, S., Riedl, M. O., & Choi, Y. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.47 · Cited by 121
Simulating Opinion Dynamics with Networks of LLM-based Agents
Chuang, Y.-S., Goyal, A., Harlalka, N., Suresh, S., Hawkins, R., Yang, S., Shah, D., Hu, J., & Rogers, T. (2024) · Findings of the Association for Computational Linguistics: NAACL 2024 · doi:10.18653/v1/2024.findings-naacl.211 · Cited by 85
GeDi: Generative Discriminator Guided Sequence Generation
Krause, B., Gotmare, A. D., McCann, B., Keskar, N. S., Joty, S., Socher, R., & Rajani, N. F. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.424 · Cited by 220
Retracted: Chat With ChatGPT on Interactive Engines for Intelligent Driving
Gao, Y., Tong, W., Wu, E. Q., Chen, W., Zhu, G., & Wang, F. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3252571 · Cited by 84
RetractedGenerating Role-Playing Game Quests With GPT Language Models
Värtinen, S., Hämäläinen, P., & Guckelsberger, C. (2022) · IEEE Transactions on Games · doi:10.1109/tg.2022.3228480 · Cited by 98
GPT-4 vs. GPT-3Preprint
Koubâa, A. (2023) · doi:10.36227/techrxiv.22312330.v2 · Cited by 90
Fill in the BLANC: Human-free quality estimation of document summaries
Vasilyev, O., Dharnidharka, V., & Bohannon, J. (2020) · Workshop on Evaluation and Comparison of NLP Systems (Eval4NLP) · doi:10.18653/v1/2020.eval4nlp-1.2 · Cited by 107
Preference Ranking Optimization for Human Alignment
Song, F., Yu, B., Li, M., Yu, H., Huang, F., Li, Y., & Wang, H. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29865 · Cited by 77
Preference Ranking Optimization for Human Alignment
Song, F., Yu, B., Li, M., Yu, H., Huang, F., Li, Y., & Wang, H. (2023) · arXiv · doi:10.48550/arxiv.2306.17492 · Cited by 77
Humans or LLMs as the Judge? A Study on Judgement Bias
Chen, G. H., Chen, S., Liu, Z., Jiang, F., & Wang, B. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.474 · Cited by 76
“The collected feedback can be used to improve model performance (Kreutzer et al., 2018; Zhou and Xu, 2020; Leike et al., 2018; Ziegler et al., 2019; Stiennon et al., 2020; Böhm et al., 2019; Ouyang et al., 2022; Christiano et al., 2023) or to serve as an indicator of output quality as in Chatbot…”
MethodRLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs
Chaudhari, S., Aggarwal, P., Murahari, V., Rajpurohit, T., Kalyan, A., Narasimhan, K., Deshpande, A., & Castro da Silva, B. (2026) · ACM Computing Surveys · doi:10.1145/3743127 · Cited by 92
Multitask-Based Evaluation of Open-Source LLM on Software Vulnerability
Yin, X., Ni, C., & Wang, S. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3470333 · Cited by 101
Irfan, B., Kuoppamäki, S., Hosseini, A., & Skantze, G. (2025) · Autonomous Robots · doi:10.1007/s10514-025-10190-y · Cited by 92
Ha, J., Jeon, H., Han, D. S., Seo, J., & Oh, C. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642472 · Cited by 96
del Río, A., Jiménez, D., & Serrano, J. (2024) · IEEE Access · doi:10.1109/access.2024.3472473 · Cited by 82
Hindi, M., Mohammed, L., Maaz, O., & Alwarafy, A. (2025) · IEEE Access · doi:10.1109/access.2025.3550145 · Cited by 108
Data-Driven Traffic Simulation: A Comprehensive Review
Chen, D., Zhu, M., Yang, H., Wang, X., & Wang, Y. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3367919 · Cited by 86
Enhancing large language model capabilities for rumor detection with Knowledge-Powered Prompting
Yan, Y., Zheng, P., & Wang, Y. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108259 · Cited by 69
Language in Brains, Minds, and Machines
Tuckute, G., Kanwisher, N., & Fedorenko, E. G. (2024) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-120623-101142 · Cited by 78
Generative AIPreprint
Feuerriegel, S., Hartmann, J., Janiesch, C., & Zschech, P. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4443189 · Cited by 65
Introduction to large language models (LLMs) for dementia care and research
Treder, M. S., Lee, S., & Tsvetanov, K. A. (2024) · Frontiers in Dementia · doi:10.3389/frdem.2024.1385303 · Cited by 63
A guide to prompt design: foundations and applications for healthcare simulationists
Maaz, S., Palaganas, J. C., Palaganas, G., & Bajwa, M. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2024.1504532 · Cited by 69
Epistemic Injustice in Generative AI
Kay, J., Kasirzadeh, A., & Mohamed, S. (2024) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1609/aies.v7i1.31671 · Cited by 76
The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems
Ziems, C., Yu, J., Wang, Y., Halevy, A. Y., & Yang, D. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.261 · Cited by 107
Giuffrè, M., Kresevic, S., Pugliese, N., You, K., & Shung, D. L. (2024) · Liver International · doi:10.1111/liv.15974 · Cited by 62
CoCon: A Self-Supervised Approach for Controlled Text GenerationPreprint
Chan, A., Ong, Y.-S., Pung, B. T. W., Zhang, A., & Fu, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.03535 · Cited by 73
Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
Li, K., Patel, O., Viégas, F., Pfister, H., & Wattenberg, M. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1797 · Cited by 117
“However, we are unsure of the result from a task-specific RLHF (Ziegler et al., 2019) with 5% samples.”
BackgroundInfluentialKim, S. S. Y., Vaughan, J. W., Liao, Q. V., Lombrozo, T., & Russakovsky, O. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714020 · Cited by 58
“Since LLMs are often fine-tuned on human preference data via approaches such as Reinforcement Learning from Human Feedback (RLHF) [23, 91, 137], such preferences would shape the form of their outputs.”
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
Gou, Z., Shao, Z., Gong, Y., Shen, Y., Yang, Y., Duan, N., & Chen, W. (2023) · arXiv · doi:10.48550/arxiv.2305.11738 · Cited by 561
“, 2019) are commonly used for reinforcement learning (Ziegler et al., 2019; Lu et al., 2022) and generate-then-rank framework (Chen et al.”
Benefits and Harms of Large Language Models in Digital Mental Health
De Choudhury, M., Pendse, S. R., & Kumar, N. (2023) · doi:10.31234/osf.io/y8ax9 · Cited by 78
Lazaridou, A., Potapenko, A., & Tieleman, O. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.685 · Cited by 84
Steering Llama 2 via Contrastive Activation Addition
Rimsky, N., Gabrieli, N., Schulz, J., Tong, M., Hubinger, E., & Turner, A. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.828 · Cited by 57
Advancing the search frontier with AI agents
White, R. W. (2024) · Communications of the ACM · doi:10.1145/3655615 · Cited by 55
Thus Spoke GPT-3: Interviewing a Large-Language Model on Climate Finance
Leippold, M. (2022) · Finance research letters · doi:10.1016/j.frl.2022.103617 · Cited by 62
Ding, P., Kuang, J., Ma, D., Cao, X., Xian, Y., Chen, J., & Huang, S. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.118 · Cited by 58
Systematic review: The use of large language models as medical chatbots in digestive diseases
Giuffrè, M., Kresevic, S., You, K., Dupont, J., Huebner, J., Grimshaw, A. A., & Shung, D. L. (2024) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.18058 · Cited by 60
Herrera-Poyatos, D., Peláez-González, C., Zuheros, C., Herrera-Poyatos, A., Tejedor, V., Herrera, F., & Montes, R. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1609097 · Cited by 93
Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model
Yang, K., Tao, J., Lyu, J., Ge, C., Chen, J., Shen, W., Zhu, X., & Li, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00854 · Cited by 109
Large Language Models are Not Yet Human-Level Evaluators for Abstractive Summarization
Shen, C., Cheng, L., Nguyen, X.-P., You, Y., & Bing, L. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.278 · Cited by 112
Marasović, A., Bhagavatula, C., Park, J. S., Bras, R. J. L., Smith, N. A., & Choi, Y. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.253 · Cited by 58
Large Language Models for Text Classification: From Zero-Shot Learning to Instruction-Tuning
Chae, Y., & Davidson, T. (2025) · Sociological Methods & Research · doi:10.1177/00491241251325243 · Cited by 72
Franceschelli, G., & Musolesi, M. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15278 · Cited by 54
Metadata
- Authors
- Ziegler, D. M., Stiennon, N., Wu, J., Brown, T. B., Radford, A., Amodei, D., Christiano, P., & Irving, G.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1909.08593
- OpenAlex
- W2973379954
- Cited by
- 413
- References
- 45
- Citations by year
- 2026: 19 · 2025: 89 · 2024: 95 · 2023: 109 · 2022: 33 · 2021: 41 · 2020: 25 · 2019: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- reward learning, human preference data, reinforcement learning from human feedback, language model fine-tuning, text summarization, sentiment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
