Gå til innholdet
Global sound
learning with yacine show

learning with yacine show

Next-Latent Prediction Transformers Learn World Models feat. Jayden Teoh from MIT

· 2 t 8 min

Episodebeskrivelse

it’s very interesting to see where next-token transformers still fall short (like the path-star graph problems) while crushing problems that look even more daunting.I had the pleasure of chatting with jayden teoh from MIT who walks us through next-latent prediction (NextLat), a simple fix in the training objective he worked on when he was still an undergrad!the change is super tiny: -> bolt a small MLP onto a GPT and make it predict the transformer’s own next hidden state. to pull that off, the internal representation of the transformer needs to become a belief state, a compact summary of…

Nettsted: yacinemahdid.com

Live, siste time: 6 800 låter på 2 593 stasjoner i 112 land

Oppdag verden gjennom radioen

Flere måter å oppdage på (7)

Hørt noe fint? Lagre det, eller åpne det rett i Spotify.

★ Artist of the week The fastest riser on the radio this week
Your song here? From € 9 per week →
WEPALE RadioFortsett å lytte