Skip to content
Global sound
The Weight Update

The Weight Update

The Practitioner's Guide to TurboQuant

· 25 min

Show notes

KV cache compression on your own hardware: what works, what doesn't, and when to care.
Google's TurboQuant paper compresses KV cache to 3 bits per coordinate — 6x memory reduction, 8x faster inference, zero accuracy loss, no retraining required. This deep-dive walks through what it actually is, the three-layer compression stack, real benchmark results on a consumer RTX 4090, community implementations available today, the Hugging Face ecosystem integration, and a CTO decision framework for when this matters to your org. Companion to the LinkedIn article of the same name.
25 sources cited…

Website: podcasters.spotify.com

Live, last hour: 12,938 songs on 6,279 stations in 143 countries

Discover the world through radio

More ways to discover (9)

Heard something good? Save it, or open it straight away in Spotify.

★ Artist of the week The fastest riser on the radio this week
Your song here? From € 9 per week →
Radio 538 luisteren - Live!Popular right now