Přejít k obsahu
Global sound
Neural intel Pod

Neural intel Pod

DeepSeek-V4: The Million-Token Efficiency Leap | Open Source SOTA

· 8 min

Poznámky k epizodě

DeepSeek-AI has just dropped the DeepSeek-V4 series, featuring a massive 1.6T parameter MoE model that natively supports a one-million-token context window. This isn't just about size; it's about a fundamental breakthrough in long-context efficiency, requiring only 10% of the KV cache compared to DeepSeek-V3. In this brief overview, we look at how the Pro and Flash models utilize Hybrid Attention (CSA and HCA) to break the quadratic complexity bottleneck.For a technical deep dive into the math behind the Manifold-Constrained Hyper-Connections (mHC) and the Muon optimizer that made this trillion-parameter training stable, check out our full podcast episode.Follow us on X/Twitter: @neuralintelorg
Visit our website: neuralintel.org

Web: podcasters.spotify.com

Živě, poslední hodina: 15 462 skladeb na 6 614 stanicích v 145 zemích

Objevte svět prostřednictvím rádia

Další způsoby objevování (9)

Slyšeli jste něco pěkného? Uložte si to, nebo to otevřete rovnou ve Spotify.

★ Artist of the week The fastest riser on the radio this week
Your song here? From € 9 per week →
France InterPokračovat v poslechu