TL;DRGradient descent minimizes the loss by stepping downhill along its slope.
Training is optimization: compute the loss's gradient (slope) with respect to each weight, then nudge weights in the downhill direction. Repeat until the loss stops dropping. Stochastic/mini-batch variants use small data batches per step for speed. It's how virtually all neural nets learn — iteratively rolling downhill toward lower error.
Key points
Step weights downhill along the loss gradient
Repeat until loss plateaus
Mini-batch (SGD) for speed
The universal NN training method
Common mistakes
Learning rate too high (diverge) or too low (crawl)
Forgetting to shuffle batches
Expecting it to find the global minimum
Try it: Explain what the gradient tells the optimizer to do.
Example code
<!doctype html><html><head><meta charset="utf-8"></head>
<body style="background:#06040d;color:#e6e0ff;font-family:monospace;padding:20px"><pre>gradient = slope of loss vs weights
step downhill: w ← w − lr · gradient
repeat → loss decreases</pre></body></html>