Přejít k obsahu
Global sound
Pop Goes the Stack

Pop Goes the Stack

Round Robin is Still Dumb: Load balancing has to grow up for AI inference

· 21 min · Epizoda 49

Poznámky k epizodě

Round robin isn’t just “not ideal” for LLM inference. According to recent scheduling research, it’s actively harmful, because it treats inference like stateless, interchangeable API traffic when it’s anything but. In this episode of Pop Goes the Stack, Lori MacVittie is joined by F5's Josh Mendoza, Principal Solutions Engineer, to break down why classic load-balancing assumptions fail under LLM workloads, and what to think about instead.

Josh walks through the evolution from early “spray and pray” distribution to smarter approaches that account for server load, workload type, and state…

Web: share.transistor.fm

Živě, poslední hodina: 8 146 skladeb na 3 669 stanicích v 128 zemích

Objevte svět prostřednictvím rádia

Další způsoby objevování (7)

Slyšeli jste něco pěkného? Uložte si to, nebo to otevřete rovnou ve Spotify.

★ Artist of the week The fastest riser on the radio this week
Your song here? From € 9 per week →
Planet RadioPokračovat v poslechu