Step 01 / 02

Attention within the same sequence

Module map

When Query, Key, and Value all come from the same sequence, we call it self-attention. The sequence models relationships within itself — pronouns to nouns, code identifiers to definitions, questions to earlier constraints.

That is why changing one word can ripple through the pattern of what attends to what. The matrix you are about to explore is a teaching visualization of that idea (not a claim about one exact production model).

Self-attention = “tokens looking at other tokens in the same input to build better context.”