Skip to content
Global sound
Machine Learning Guide

Machine Learning Guide

MLG 033 Transformers

· 43 min · Season 1 Episode 54

Show notes

Links:

• Notes and resources at ocdevel.com/mlg/33
• 3Blue1Brown videos: https://3blue1brown.com/
• Try a walking desk stay healthy & sharp while you learn & code
• Try Descript audio/video editing with AI power-tools
Background & Motivation

• RNN Limitations: Sequential processing prevents full parallelization—even with attention tweaks—making them inefficient on modern hardware.
• Breakthrough: "Attention Is All You Need" replaced recurrence with self-attention, unlocking massive parallelism and scalability.
Core Architecture

• Layer Stack: Consists of alternating self-attention and…

Website: ocdevel.com

Live, last hour: 9,111 songs on 3,748 stations in 135 countries

Discover the world through radio

More ways to discover (9)

Heard something good? Save it, or open it straight away in Spotify.

Radio 538 luisteren - Live!Popular right now