Note dell’episodio
In this episode:
• The Memory Wall and the 90 Percent Waste: Professor Norris and Linda introduce the episode's paper, discussing the GPU memory bottleneck caused by ultra-long context windows and the surprising observation about wasted Key-Value cache.
• Lookahead Sparse Attention: Linda explains the core concept of LSA, where the model proactively predicts and fetches only the critical context chunks instead of keeping everything in memory.
• Decoupled Training Magic: The hosts dive into the clever engineering behind the Neural Memory Indexer, highlighting how the authors trained it in…
Sito web: mechanicaldreams.mechanicaldirk.com

