OpenAssistant Conversations - Democratizing Large Language Model Alignment
Open accessAbstract
Aligning large language models (LLMs) with human preferences has proven to drastically improve usability and has driven rapid adoption as demonstrated by ChatGPT. Alignment techniques such as supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF) greatly reduce the required skill and domain knowledge to effectively harness the capabilities of LLMs, increasing their accessibility and utility across various domains. However, state-of-the-art alignment techniques like RLHF rely on high-quality human feedback data, which is expensive to create and often remains proprietary. In an effort to democratize research on large-scale alignment, we release OpenAssistant Conversations, a human-generated, human-annotated assistant-style conversation corpus consisting of 161,443 messages in 35 different languages, annotated with 461,292 quality ratings, resulting in over 10,000 complete and fully annotated conversation trees. The corpus is a product of a worldwide crowd-sourcing effort involving over 13,500 volunteers. Models trained on OpenAssistant Conversations show consistent improvements on standard benchmarks over respective base models. We release our code and data under a fully permissive licence.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 19
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
- Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. · arXiv
- Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. Pythia: A suite for analyzing large language models across training and scaling. arXiv preprint arXiv:2304.01373, 2023. · arXiv
- Iason Gabriel. Artificial intelligence, values, and alignment. Minds and Machines, 30(3):411- 437, September 2020.
- Kawin Ethayarajh, Yejin Choi, and Swabha Swayamdipta. Understanding dataset difficulty with V-usable information. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 5988- 6008. PMLR, 17-23 Jul 2022.
- Jacob Menick, Maja Trebacz, Vladimir Mikulik, John Aslanides, Francis Song, Martin Chad- wick, Mia Glaese, Susannah Young, Lucy Campbell-Gillingham, Geoffrey Irving, et al. Teaching language models to support answers with verified quotes. arXiv preprint arXiv:2203.11147, 2022. · arXiv
- Ratanond Koonchanok, Ya-Chen Pan, and Hyeju Jang. Tracking public attitudes toward chatgpt on twitter using sentiment analysis and topic modeling. ArXiv, abs/2306.12951, 2023. · arXiv
- John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. · arXiv
- Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language model with self generated instruc- tions. arXiv preprint arXiv:2212.10560, 2022. · arXiv
- Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning with gpt-4. arXiv preprint arXiv:2304.03277, 2023. · arXiv
- Herbert H. Clark and Susan E. Brennan. Grounding in communication. In Lauren Resnick, Levine B., M. John, Stephanie Teasley, and D., editors, Perspectives on Socially Shared Cognition, pages 13-1991. American Psychological Association, 1991.
- Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. Evaluating large language models trained on code, 2021.
- Wojciech Kryściński, Bryan McCann, Caiming Xiong, and Richard Socher. Evaluating the factual consistency of abstractive text summarization. arXiv preprint arXiv:1910.12840, 2019. · arXiv
- Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. Retrieval augmenta- tion reduces hallucination in conversation. arXiv preprint arXiv:2104.07567, 2021. · arXiv
- Peter Henderson, Koustuv Sinha, Nicolas Angelard-Gontier, Nan Rosemary Ke, Genevieve Fried, Ryan Lowe, and Joelle Pineau. Ethical challenges in data-driven dialogue systems. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 123-129, 2018.
- Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Measuring and mitigating unintended bias in text classification. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 67-73, 2018.
- Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al. Ethical and social risks of harm from language models. arXiv preprint arXiv:2112.04359, 2021. · arXiv
- T. N. Tideman. Independence of clones as a criterion for voting rules. Social Choice and Welfare, 4(3):185-206, September 1987.
- David Rozado. The political biases of chatgpt. Social Sciences, 12(3):148, Mar 2023.
Cited by
Sorted by citations
Evaluation and mitigation of the limitations of large language models in clinical decision-making
Hager, P., Jungmann, F., Holland, R., Bhagat, K., Hubrecht, I., Knauer, M., Vielhauer, J., Makowski, M., Braren, R., Kaissis, G., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-03097-1 · Cited by 798 · see citing works
“We test both generalist versions such as Llama 2 Chat (70B) 32 , Open Assistant (OASST) (70B) 33 and WizardLM (70B) 34 , as well as medical-domain aligned models such as Clinical Camel (70B) 19 and Meditron (70B) 35 .”
QLoRA: Efficient Finetuning of Quantized LLMs
Dettmers, T., Pagnoni, A., Holtzman, A., & Zettlemoyer, L. (2023) · arXiv · doi:10.48550/arxiv.2305.14314 · Cited by 507 · see citing works
“OASST1 The OpenAssistant dataset [31] was collected via crowd-sourcing.”
MethodResultInfluentialJudging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05685 · Cited by 1,888 · see citing works
“• Conversational benchmarks , like CoQA [35], MMDialog [15] and OpenAssistant [23], are closest to our intended use cases.”
BackgroundEnabling Large Language Models to Generate Text with Citations
Gao, T., Yen, H., Yu, J., & Chen, D. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.398 · Cited by 216 · see citing works
Takita, H., Kabata, D., Walston, S. L., Tatekawa, H., Saito, K., Tsujimoto, Y., Miki, Y., & Ueda, D. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01543-z · Cited by 203 · see citing works
Emotional intelligence of Large Language Models
Wang, X., Li, X., Yin, Z., Wu, Y., & Liu, J. (2023) · Journal of Pacific Rim Psychology · doi:10.1177/18344909231213958 · Cited by 128 · see citing works
Enhancing Chat Language Models by Scaling High-quality Instructional Conversations
Ding, N., Chen, Y., Xu, B., Qin, Y., Hu, S., Liu, Z., Sun, M., & Zhou, B. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.183 · Cited by 130 · see citing works
Benchmarking Cognitive Biases in Large Language Models as Evaluators
Koo, R., Lee, M., Raheja, V., Park, J. I., Kim, Z. M., & Kang, D. (2023) · arXiv · doi:10.48550/arxiv.2309.17012 · Cited by 72 · see citing works
Are Large Language Models a Threat to Digital Public Goods? Evidence from Activity on Stack Overflow
del Rio-Chanona, M., Laurentsyeva, N., & Wachs, J. (2023) · arXiv · doi:10.48550/arxiv.2307.07367 · Cited by 62 · see citing works
MELTing Point: Mobile Evaluation of Language Transformers
Laskaridis, S., Katevas, K., Minto, L., & Haddadi, H. (2024) · Proceedings of the 30th Annual International Conference on Mobile Computing and Networking · doi:10.1145/3636534.3690668 · Cited by 37 · see citing works
INSTRUCTEVAL: Towards Holistic Evaluation of Instruction-Tuned Large Language Models
Chia, Y. K., Hong, P., Bing, L., & Poria, S. (2023) · arXiv · doi:10.48550/arxiv.2306.04757 · Cited by 36 · see citing works
“…et al., 2022) 12M 62 General Human-Annotation Databricks Dolly (Databricks Labs, 2023) 15K 12K General Human-Annotation OpenAssistant Conversations (Köpf et al., 2023) 161K 161K Dialogue Human-Annotated Anthropic HH (Bai et al., 2022) 161K 161K Safety Human-Annotated such efforts, we are deeply…”
Effective Long-Context Scaling of Foundation Models
Xiong, W., Liu, J., Molybog, I., Zhang, H., Bhargava, P., Hou, R., Martin, L., Rungta, R., Sankararaman, K. A., Oguz, B., et al. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.260 · Cited by 33 · see citing works
RA-DIT: Retrieval-Augmented Dual Instruction Tuning
Lin, X. V., Chen, X., Chen, M., Shi, W., Lomeli, M., James, R., Rodriguez, P., Kahn, J., Szilvasy, G., Lewis, M., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.01352 · Cited by 26 · see citing works
OctoPack: Instruction Tuning Code Large Language Models
Muennighoff, N., Liu, Q., Zebaze, A., Zheng, Q., Hui, B., Zhuo, T. Y., Singh, S., Tang, X., von Werra, L., & Longpre, S. (2023) · arXiv · doi:10.48550/arxiv.2308.07124 · Cited by 17 · see citing works
“C OMMIT P ACK FT+OASST yields best performance All instruction datasets provide similar boosts for code synthesis (H UMAN E VAL S YNTHESIZE ), which has been the focus of all prior work on code instruction models (Wang et al., 2023c; Luo et al., 2023; Muennighoff et al., 2022b).”
BackgroundMethodInfluentialUnderstanding the Effects of RLHF on LLM Generalisation and Diversity
Kirk, R., Mediratta, I., Nalmpantis, C., Luketina, J., Hambro, E., Grefenstette, E., & Raileanu, R. (2023) · arXiv · doi:10.48550/arxiv.2310.06452 · Cited by 17 · see citing works
“This is a set of inputs taken from a variety of open source instruction following and dialogue training and evaluation datasets (Bai et al., 2022; Ganguli et al., 2022; Gudibande, 2023; Köpf et al., 2023; Wang et al., 2023; Zheng et al., 2023), curated by Dubois et al. (2023).”
BackgroundModel Tells You What to Discard: Adaptive KV Cache Compression for LLMs
Ge, S., Zhang, Y., Liu, L., Zhang, M., Han, J., & Gao, J. (2023) · arXiv · doi:10.48550/arxiv.2310.01801 · Cited by 13 · see citing works
Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models
Zhang, H., Edelman, B. L., Francati, D., Venturi, D., Ateniese, G., & Barak, B. (2023) · arXiv · doi:10.48550/arxiv.2311.04378 · Cited by 12 · see citing works
SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHF
Dong, Y., Wang, Z., Sreedhar, M., Wu, X., & Kuchaiev, O. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.754 · Cited by 15 · see citing works
Effective Long-Context Scaling of Foundation Models
Xiong, W., Liu, J., Molybog, I., Zhang, H., Bhargava, P., Hou, R., Martin, L., Rungta, R., Sankararaman, K. A., Oguz, B., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.16039 · Cited by 9 · see citing works
KIWI: A Dataset of Knowledge-Intensive Writing Instructions for Answering Research Questions
Xu, F., Lo, K., Soldaini, L., Kuehl, B., Choi, E., & Wadden, D. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.770 · Cited by 3 · see citing works
From Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMs
Wu, Y., Liang, S., Zhang, C., Wang, Y., Zhang, Y., Guo, H., Tang, R., & Liu, Y. (2025) · arXiv · doi:10.48550/arxiv.2504.15965 · Cited by 3 · see citing works
“ChatGPT [26], DeepSeek-Chat [27], Claude [28], QWEN-CHAT [29], Llama 2-Chat [30], Gemini [31], PANGU-BOT [32], ChatGLM [33], OpenAssistant [34] II Personal Non-Parametric Long-Term ChatGPT Memory [18], Apple Intelligence [19], Microsoft Recall [35], Me.bot [36] Open-SourceFramework MemoryScope…”
Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence
Feng, S., Wang, Z., Wang, Y., Ebrahimi, S., Palangi, H., Miculicich, L., Kulshrestha, A., Rauschmayr, N., Choi, Y., Tsvetkov, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.11163 · Cited by 2 · see citing works
Preference Leakage: A Contamination Problem in LLM-as-a-judge
Li, D., Sun, R., Huang, Y., Zhong, M., Jiang, B., Han, J., Zhang, X., Wang, W., & Liu, H. (2025) · arXiv · doi:10.48550/arxiv.2502.01534 · Cited by 2 · see citing works
Shen, Y., Li, K., Zhou, W., & Hu, S. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.370 · Cited by 2 · see citing works
On the Way to LLM Personalization: Learning to Remember User Conversations
Magister, L. C., Metcalf, K., Zhang, Y., & ter Hoeve, M. (2024) · arXiv · doi:10.48550/arxiv.2411.13405 · Cited by 6 · see citing works
Play Favorites: A Statistical Method to Measure Self-Bias in LLM-as-a-Judge
Spiliopoulou, E., Fogliato, R., Burnsky, H., Soliman, T., Ma, J., Horwood, G., & Ballesteros, M. (2025) · arXiv · doi:10.48550/arxiv.2508.06709
“HELM-Instruct itself aggregates prompts from diverse sources [Bai et al., 2022a, Perez et al., 2022, Geng et al., 2023, Team, 2023, Köpf et al., 2023, Wang et al., 2023, Gridfiti, 2023].”
YESciEval: Robust LLM-as-a-Judge for Scientific Question Answering
D'Souza, J., Giglou, H. B., & Münch, Q. (2025) · arXiv · doi:10.48550/arxiv.2505.14279
“Notably, despite requiring GPU compute, model access incurred zero monetary cost—whereas using proprietary models like Ope-nAI’s GPT would have cost at least 1,000 euros or dollars for these evaluations.”
- Authors
- Köpf, A., Kilcher, Y., von Rütte, D., Anagnostidis, S., Tam, Z.-R., Stevens, K., Barhoum, A., Duc, N. M., Stanley, O., Nagyfi, R., ES, S., Suri, S., Glushkov, D., Dantuluri, A., Maguire, A., Schuhmann, C., Nguyen, H., & Mattick, A.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2304.07327
- OpenAlex
- W4366341216
- Cited by
- 218
- References
- 0
- Citations by year
- 2026: 18 · 2025: 60 · 2024: 100 · 2023: 40
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- large language model alignment, human preferences, conversation corpus, quality ratings, reinforcement learning from human feedback, supervised fine-tuning, multilingual data, crowdsourced data collection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
