Notas do episódio
A fantastic research paper published in this month's Nature Computational Science suggests a solution may be in our midst for the incredible inefficiency in generative AI.
Large Language Models' (LLMs) transformer architecture requires the next token (generally part of a word) be predicted based on all the output tokens before it.
Power demands for this process are huge. Shuffling data between memory and processors isn't an easy pipeline, and when you need it to work quickly, those energy demands quickly stack up.
And in an AI arms race, where everyone wants bigger and better models…
Site: podcasters.spotify.com

