Skip to content
Global sound
The Information Bottleneck

The Information Bottleneck

World Models | John Langford (Microsoft AI Labs)

· 1 h 5 min

Show notes

John Langford, one of the heads of Microsoft's AI Labs, the creator of Vowpal Wabbit, and a co-inventor of CAPTCHA, joins us to talk about world models. Transformers need orders of magnitude more data than humans to learn the same thing, and John argues a compact, implicit world model is how you close that gap. He explains why he's skeptical of JEPA-style objectives, why a transformer's KV cache is the Ptolemaic epicycle model of belief states, and what his Next Latent work does differently.
We also get into whether research still matters in the age of scale; open versus closed models…

Live, last hour: 7,631 songs on 2,953 stations in 118 countries

Discover the world through radio

More ways to discover (9)

Heard something good? Save it, or open it straight away in Spotify.

★ Artist of the week The fastest riser on the radio this week
Your song here? From € 9 per week →
Mole HoleContinue listening