Step 01 / 02

Build the block

Module map

A modern language model stacks many Transformer blocks. Each block is a familiar story: represent tokens, let them exchange context, transform those representations, then pass them on.

Walk the stages. Production blocks also include details like residual connections and normalization — important later, optional for the first mental model.

Tokens
Embeddings
Positional information
Self-attention
Feed-forward network
Output representation

Tokens

Start from tokenized input — the pieces the model can process.