AI Seq2seq Tutorial: Encoder-Decoder

TL;DRSeq2seq uses an encoder to read input and a decoder to generate output — for translation.

Sequence-to-sequence models pair an encoder (compresses the input sequence into a context vector) with a decoder (generates the output sequence from it) — the original neural translation architecture. The bottleneck: cramming a whole sentence into one fixed vector lost information for long inputs. That bottleneck is exactly what attention was invented to fix.

Key points

Encoder reads input → context; decoder generates output
The original neural machine translation design
Fixed context vector = a bottleneck
Set the stage for attention

Common mistakes

Expecting good long-input performance from plain seq2seq
Confusing encoder and decoder roles
Missing why the bottleneck mattered

Try it: Explain the information bottleneck in vanilla seq2seq.

Example code

<!doctype html><html><head><meta charset="utf-8"></head>
<body style="background:#06040d;color:#e6e0ff;font-family:monospace;padding:20px"><pre>encoder: sentence → one context vector (bottleneck)
decoder: context vector → output words
long input → info lost → attention fixes it</pre></body></html>

Open the interactive lesson →

← Grus · Simpler Gated Recurrence Attention · The Seq2seq Fix →