TL;DRMulti-head attention runs several attention operations in parallel to capture different relationships.
One attention "head" can only focus one way. Multi-head attention splits the representation into several heads, each learning its own Q/K/V projections — one head might track syntax, another long-range references, another local context. Their outputs are concatenated and combined. Multiple parallel perspectives are why transformers capture rich, varied relationships in a single layer.
Key points
Several attention heads run in parallel
Each head learns a different relationship
Outputs concatenated and combined
Captures varied patterns per layer
Common mistakes
Assuming one head suffices
Thinking heads are redundant copies
Ignoring head dimensionality trade-offs
Try it: Explain what different attention heads might specialize in.
Example code
<!doctype html><html><head><meta charset="utf-8"></head>
<body style="background:#06040d;color:#e6e0ff;font-family:monospace;padding:20px"><pre>head1: syntax · head2: long-range refs · head3: local
run in parallel → concat → richer representation</pre></body></html>