Neura saysSelf-attention computes relationships using queries, keys, and values.
Each token produces a query, a key, and a value. A token's query is compared (dot product) against every token's key to get attention weights; those weights blend the values into the token's new representation. Intuitively: "what am I looking for (Q), what do I offer (K), and what content do I carry (V)". This QKV mechanism is the heart of the transformer.
Power-ups you unlock
Each token → query, key, value vectors
Query·Key → attention weights
Weights blend the Values
QKV is the core transformer operation
The Overfit Ogre attacks — common mistakes
Confusing the roles of Q, K, and V
Forgetting attention is computed for every token pair
Ignoring scaling/softmax in the math
Boss battleMap "what I want / what I offer / what I carry" to Q, K, V.
Example code
<!doctype html><html><head><meta charset="utf-8"></head>
<body style="background:#06040d;color:#e6e0ff;font-family:monospace;padding:20px"><pre>Q (what I seek) · K (what each offers) → weights
weights × V (content) → new representation</pre></body></html>