Aller au contenu
Global sound
Super Data Science: ML & AI Podcast with Jon Krohn

Super Data Science: ML & AI Podcast with Jon Krohn

1025: Word Gravity: How Transformers Bend Space, with Dr. Luis Serrano

· 1 h 10 min

Notes d’épisode

In Episode #1025, Dr. Luis Serrano (Founder of Serrano Academy) joins Jon Krohn to explain the paper he co-authored on the curved spacetime of transformer architectures, in which attention stops being a lookup table and becomes something closer to gravity: words bend the space around them, and the embedding of "bank" visibly curves toward "river" as it travels through the layers of the network. In this episode, he recreates Eddington’s 1919 eclipse experiment inside a transformer, draws the line between an LLM workflow and an actual agent, explains why agent evaluation is a step harder than evaluating an essay, and gives the cleanest account of GRPO you will hear.

Additional materials: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://www.superdatascience.com/1025⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information.

In this episode you will learn:

(00:10:53) What changed, and what survived, between the two editions of Grokking Machine Learning

(00:27:48) Word gravity: how attention pulls "bank" toward "river"

(00:42:12) Why RAG is an LLM workflow rather than an agent

(00:51:23) The two-by-two that explains why GRPO powers reasoning models

En direct, dernière heure : 12 524 titres sur 5 873 stations dans 136 pays

Découvrez le monde par la radio

Plus de façons de découvrir (7)

Vous avez entendu quelque chose de bien ? Enregistrez-le, ou ouvrez-le directement dans Spotify.

★ Artiste de la semaine La plus forte progression à la radio cette semaine
Ton titre ici ? À partir de € 9 par semaine →
Radio Grand ParisContinuer l’écoute