Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards

Ramé, Alexandre, Couairon, Guillaume, Shukor, Mustafa · arXiv · 2023