Frictionless with Logan Jastremski
Why Memory Is AI's Biggest Bottleneck with Vikram Sekar | EP 170
· 1 h 12 min · Episode 170
Show notes
Vikram Sekar, writer of Vik's Newsletter and co-host of Semi Doped, joins me to discuss AI agents, memory, and the hardware behind the data center buildout.
We break down why training and inference need different hardware, why copper becomes harder to use as GPUs move farther apart, and why the industry is turning to optical interconnects.
Vik explains the tradeoffs across SRAM, DRAM, high bandwidth memory, and storage. We discuss why useful work per token matters more than speed alone, how much context an agent needs, and why he sees memory as AI's biggest bottleneck.
Enjoy.
00:00…
Website: podcasters.spotify.com
