Notas del episodio
sometime there is an idea that just makes other weird research results make so much more sense and sort of cascade to explain a whole bunch of phenomenon.
neural thicket from the great yulu gan is one such paper that I absolutely loved from start to finish.
the old mental model I had for pre-trained neural networked was that the pre-training regime made them land on a single good set of weight that was optimal for many tasks.
but after going through the paper "Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights" my view has been changed.
the better idea seems to be that…
Web: yacinemahdid.com

