Epizódjegyzetek
In this episode:
• Welcome and the 1/9th FLOP Miracle: Linda introduces Qwen3.8-Next and its massive efficiency gains, hooking Dr. Norris.
• Token Mixing with GDN and Sparse Attention: Discussing the Gated DeltaNet hybrid architecture and how Qwen Sparse Attention scales to long contexts.
• The Gated Residual (GR): Norris is skeptical about widening the residual stream, but Linda explains the gating mechanism and how it boosts stability.
• N-gram Embeddings & The Muon Optimizer: Exploring off-accelerator capacity scaling and why Muon needs special engineering like the Canzona framework.
•…
Weboldal: mechanicaldreams.mechanicaldirk.com

