What's inside the box?
The Transformer · Builder · 5 min · transformer, attention, llm · after Self-attention
A Transformer is a stack of identical layers. Each layer runs attention (who matters to whom) and then a small "think" step on every piece, and the stack repeats that dozens of times before producing odds for the next piece.
1 Guess first
2 Look
The same trick, twelve times
1 / 3itOne piece — "it" — enters the first layer carrying only what it is: a word and a position.
One piece changes colour with every ring: the same two moves, repeated, rewrite what the piece means here.
Attention lets each piece gather from the others; the "think" step (a small neural net, the MLP) works on what it gathered. Stack that a dozen times and a piece ends up carrying the whole sentence’s context — then the last layer turns it into odds.
4 You are the model
Put the four blocks in the order a piece meets them, then Check.
- Think (MLP)work on what attention found
- Next piece oddsthe loaded wheel
- Embeddingpiece → numbers
- Attentionwho matters to whom
× 12: the middle two repeat, layer after layer
Put the four blocks in the order a piece meets them.5 Check
6 Say it back
7 Mission · 2 min
Go back to the Journey, type your own sentence and scroll through the layers station slowly. Count the rings and watch the piece change colour.
Three quick ones
Drawn at random from this lesson's pool; different next time.
Done?
Go deeper
Sources
Published Oct 20, 2026 · updated Oct 20, 2026