TL;DRAlignment is the meta-goal: making AI systems pursue what humans actually want.
Alignment asks how to ensure increasingly capable AI does what we intend — not just what we literally specified. It spans today's concrete work (RLHF, Constitutional AI, refusals) and longer-term concerns (a powerful system optimizing a mis-specified goal, deception, loss of oversight). The core difficulty: human values are hard to fully specify, and capable optimizers exploit any gap. It's the field's central open problem.
Key points
Make AI pursue what humans actually want
Spans RLHF/refusals → long-term oversight
Hard: values are underspecified
Capable optimizers exploit specification gaps
Common mistakes
Equating alignment with just "be polite"
Assuming specifying a goal = getting it
Ignoring the gap between intent and instruction
Try it: Explain why "optimize exactly what you specified" can go wrong.
Example code
<!doctype html><html><head><meta charset="utf-8"></head>
<body style="background:#06040d;color:#e6e0ff;font-family:monospace;padding:20px"><pre>you specify a proxy goal → optimizer maximizes it literally
gap between intent and instruction → misbehavior
alignment = closing that gap</pre></body></html>