Saltar al contenido
Global sound
Artificial Discourse

Artificial Discourse

Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization

· 12 min · Temporada 5 Episodio 8

Notas del episodio

This research explores whether transformers, a type of neural network architecture, can learn to reason implicitly over knowledge. The authors find that transformers can learn to reason implicitly, but only through a phenomenon called grokking, where training extends far beyond overfitting. The study investigates two reasoning types: composition and comparison. They find that while the transformers generalize well on in-distribution examples for both types, they struggle with out-of-distribution generalization for composition but succeed for comparison. Through mechanistic analysis of the…

Web: podcasters.spotify.com

En directo, última hora: 10.949 canciones en 4.624 emisoras de 125 países

Descubre el mundo a través de la radio

Más formas de descubrir (7)

¿Has oído algo bueno? Guárdalo o ábrelo directamente en Spotify.

★ Artista de la semana El que más sube en la radio esta semana
¿Tu canción aquí? Desde € 9 por semana →
C.R. - Jacqueline Du PreSeguir escuchando