Tensor TownISSUE #7 of 120

reinforcement learning · reward signals

NeuraVSThe Overfit Ogre

Neura saysReinforcement learning trains an agent by reward — it learns actions that maximize a score.

An agent takes actions in an environment, receives rewards, and learns a policy that maximizes long-term reward through trial and error. It powers game-playing (AlphaGo), robotics, and (as RLHF) LLM alignment. It's powerful for sequential decisions but sample-hungry and sensitive to how you design the reward — a bad reward yields clever, unintended behavior.

Power-ups you unlock

Agent acts, gets rewards, learns a policy
Maximizes long-term reward by trial and error
Games, robotics, RLHF for LLMs
Reward design is delicate (reward hacking)

The Overfit Ogre attacks — common mistakes

Poorly specified rewards → unintended behavior
Underestimating sample/compute cost
Assuming it suits problems without clear rewards

Boss battleExplain "reward hacking" with a one-line example.

Example code

<!doctype html><html><head><meta charset="utf-8"></head>
<body style="background:#06040d;color:#e6e0ff;font-family:monospace;padding:20px"><pre>agent → action → reward → better policy
reward hacking: reward "score" → agent loops a glitch
                instead of finishing the level</pre></body></html>

▶ Open the interactive comic issue

‹ Unsupervised Learning · Finding Structure Self-Supervised Learning · The Modern Shift ›