Gå til indhold
Global sound
LLMs Research Podcast

LLMs Research Podcast

Your 70-Billion-Parameter Model Might Be 40% Wasted

· 12 min. · Sæson 1 Episode 7

Episodebeskrivelse

Your 70-Billion-Parameter Model Might Be 40% Wasted
Three papers from February 1–6, 2026 converge on a question the field has been avoiding since 2016: what if most transformer layers aren't doing compositional reasoning at all, but just averaging noise?
This video traces a decade of evidence, from Veit et al.'s original ensemble observation in ResNets through ShortGPT's layer pruning results and October 2025's formal proof, to three new papers that quantify the consequences. Inverse depth scaling shows loss improves as D to the negative 0.30, worse than one-over-n. TinyLoRA unlocks 91%…

Website: llmsresearch.substack.com

Live, seneste time: 8.281 sange på 3.766 stationer i 128 lande

Opdag verden gennem radioen

Flere måder at opdage på (9)

Hørt noget godt? Gem det, eller åbn det direkte i Spotify.

★ Artist of the week The fastest riser on the radio this week
Your song here? From € 9 per week →
Toker FMFortsæt med at lytte