Module 1 challenge

Build a Mental Model of an LLM

You have the pieces. Now assemble the full next-token story as an architecture: from raw text to the next generated token. When it works, we change one constraint and ask you to diagnose the failure like an engineer — not like a spectator.

1. Build the pipeline

Build the generation pipeline in order: Text → Tokenizer → Tokens → Embeddings → Transformer Blocks → Output Scores → Sampling → Next Token. This is the loop an autoregressive LLM runs again and again.

Text

Raw characters humans write

Tokenizer

Splits text into model-sized pieces

Tokens

Discrete units the model consumes

Embeddings

Meaning as numbers / vectors

Transformer Blocks

Attention + transforms for context

Output Scores

Logits over possible next tokens

Sampling

Choose one next token from the distribution

Next Token

Append to context — then often repeat

Keyboard alternative

Select a source node, then a target node to create an allowed connection.