Show notes
In the latest episode of the Cisco AI Insights Podcast, hosts Rafael Herrera and Sónia Marques welcome Cisco AI operations engineer James Tidd for a discussion on the world of small language models (SLMs) and the evolution of efficient AI inference. Together, they unravel the complexities behind “Fast Inference from Transformers via Speculative Decoding,” a groundbreaking paper from Google that explores how smaller draft models can speed up large language model predictions while maintaining accuracy.
James shares his hands-on experience experimenting with the technique, leveraging knowledge…
Website: podcasters.spotify.com

