Gradient Descent - Podcast about AI and Data
LLM Fine-Tuning: RLHF vs DPO and Beyond
· 38 min · Temporada 1 Episodio 5
Notas del episodio
In this episode of Gradient Descent, we explore two competing approaches to fine-tuning LLMs: Reinforcement Learning with Human Feedback (RLHF) and Direct Preference Optimization (DPO). Dive into the mechanics of RLHF, its computational challenges, and how DPO simplifies the process by eliminating the need for a separate reward model. We also discuss supervised fine-tuning, emerging methods like Identity Preference Optimization (IPO) and Kahneman-Tversky Optimization (KTO), and their real-world applications in models like Llama 3 and Mistral. Learn practical LLM optimization strategies…
Web: podcasters.spotify.com
