Poznámky k epizodě
The Attention is All You Need paper started the LLM gold rush, but it initially came from Google scientists who only expected it to do language translation. As more data was used in training sets, and the number of parameters grew orders of magnitude it was less easy to understand what the model was doing.
Emergent Instruction Following
In Ars Technica from 2023 we have this summary of how the “emergent” AI behaviour was being seen popularly, in the press:
The biggest breakthrough came in the jump from GPT2 to GPT3 in 2020. GPT2 had about 1.5 billion parameters, which would easily fit in the…
Web: totallyathing.substack.com

