Step 01 / 04

Same sequence, looking inward

Map

Last lesson: Query, Key, and Value are the search roles inside attention. When all three come from the same sequence, that is self-attention.

The programmer fixed the bug because it crashed.

“It” has to link to something earlier in this same sentence. Tap what “it” most likely refers to.

Tap what "it" refers to.

Tap the tokens that matter

Self-attention lives inside one sequence