Show notes
In this episode:
• Distillation vs. Reinforcement Learning: Linda and Norris introduce the paper and discuss the differences between off-policy and on-policy distillation.
• The Math Behind the Magic: The hosts dive into the mathematical equivalence between on-policy distillation and KL-constrained dense RL.
• Extrapolating the Reward: Linda introduces Generalized OPD and the concept of reward extrapolation by setting the scaling factor above 1.
• Merging Experts and Reward Correction: They discuss merging multiple domain teachers and the strong-to-weak distillation setting, including the…
Website: mechanicaldreams.mechanicaldirk.com
