Přejít k obsahu
Global sound
Mechanical Dreams

Mechanical Dreams

Distillation Scaling Laws

· 19 min

Poznámky k epizodě

In this episode:
• Introduction to Distillation and Inference Costs: Linda and Norris introduce the paper and discuss why compute-optimal models are driving the need for smaller, distilled models to reduce inference costs.
• The Distillation Scaling Law and the Capacity Gap: The hosts break down the empirical scaling law discovered in the paper and explore the counter-intuitive capacity gap where overly capable teachers degrade student performance.
• Compute-Optimal Distillation Strategies: Linda explains the economic reality of distillation, detailing when it is actually more…

Web: mechanicaldreams.mechanicaldirk.com

Živě, poslední hodina: 12 992 skladeb na 6 589 stanicích v 146 zemích

Objevte svět prostřednictvím rádia

Další způsoby objevování (9)

Slyšeli jste něco pěkného? Uložte si to, nebo to otevřete rovnou ve Spotify.

★ Artist of the week The fastest riser on the radio this week
Your song here? From € 9 per week →
Radio RodzinaPokračovat v poslechu