Show notes
State space models like Mamba promised linear scaling and constant memory. They delivered on efficiency, but researchers kept hitting the same wall: ask Mamba to recall something specific from early in a long context, and performance drops.
Three papers at ICLR 2026 independently attacked this limitation. That convergence tells you how fundamental the problem is.
This podcast breaks down:
- Why Mamba's fixed-size state causes "lossy compression" of context
- How Mixture of Memories (MoM) adds multiple internal memory banks
- How Log-Linear Attention finds a middle ground between SSM and full…
Website: llmsresearch.substack.com

