AI Encoder-Decoder Tutorial: T5, Bart

TL;DREncoder-decoder models (T5, BART) pair both — strong for transformation tasks like translation.

These keep the full transformer: an encoder reads the input bidirectionally and a decoder generates the output attending to it. That makes them natural for input→output transformation: translation, summarization, T5's "everything is text-to-text" framing. They sit between pure encoders (understanding) and pure decoders (open generation), excelling when output is conditioned on a specific input.

Key points

Encoder reads input; decoder generates output
Great for transformation (translate, summarize)
T5: cast every task as text-to-text
Between encoder-only and decoder-only

Common mistakes

Using encoder-decoder for open-ended chat
Confusing it with decoder-only LLMs
Ignoring the conditioning on input

Try it: Pick the architecture family for translation vs free-form chat.

Example code

<!doctype html><html><head><meta charset="utf-8"></head>
<body style="background:#06040d;color:#e6e0ff;font-family:monospace;padding:20px"><pre>translate/summarize → encoder-decoder (T5, BART)
open chat → decoder-only (GPT)
understand/classify → encoder-only (BERT)</pre></body></html>

Open the interactive lesson →

← Decoder-Only · GPT Family Tokenization · Bpe · Wordpiece · Unigram →