Notas do episódio
I had an awesome time interviewing @IdanShenfeld and @jonashubotter from MIT and ETH Zurich about self-distillation.
this very promising post-training paradigm where the model acts as its own teacher by conditioning on environment feedback or demonstrations.
we cover the SDPO algo for reinforcement learning with rich feedback and SDFT for continual learning without forgetting along with many applications.
we dig into how it works, why it’s simpler and faster than GRPO, and where this is already showing up in production systems.
table of content:
0:00 - what is self distillation
2:50 - idan…
Site: yacinemahdid.com

