Show notes
In 2023, Mamba promised to replace attention with elegant state-space math that scaled linearly with context. By 2024, the authors had rewritten the core algorithm to use matrix multiplications instead of scans. Their paper explains why:
“We restrict the SSM structure to allow efficient computation via matrix multiplications on modern hardware accelerators.”
The architecture changed to fit the hardware. The hardware did not budge.
This is not a story about hardware determinism. It is a story about convergent evolution under economic pressure. Over the past decade, Transformers and GPU…
Website: lambpetros.substack.com
