TL;DRDecoder-only models (GPT) predict the next token left-to-right — the basis of chat LLMs.
GPT-style models use only the decoder with causal (masked) attention: each token sees only what came before, so they generate text one token at a time. Pretrained on next-token prediction over huge corpora, then instruction-tuned, they became the dominant LLM design — GPT, Claude, Llama, Gemini are all decoder-only. Generation, chat, code, reasoning all flow from this single objective.
Key points
Decoder-only, causal (left-to-right) attention
Generate one token at a time
Pretrained on next-token prediction
The dominant LLM architecture
Common mistakes
Expecting bidirectional context (it’s causal)
Thinking next-token prediction is "simple"/limited
Confusing pretraining with instruction-tuning
Try it: Explain why causal masking is required for generation.
Example code
<!doctype html><html><head><meta charset="utf-8"></head>
<body style="background:#06040d;color:#e6e0ff;font-family:monospace;padding:20px"><pre>token t sees only tokens < t (causal mask)
predict next → append → repeat
GPT/Claude/Llama all decoder-only</pre></body></html>