Tensor TownISSUE #20 of 120

neural networks · the perceptron origin

NeuraVSThe Overfit Ogre

Neura saysA neural network stacks layers of simple units (neurons) that learn weighted combinations.

Inspired loosely by brains, a neural net has layers of neurons; each computes a weighted sum of its inputs plus a bias, then applies a nonlinear activation. Stacking layers lets the network learn increasingly abstract features. The original unit, the perceptron (1958), could only do linear separation — adding hidden layers and nonlinearity is what unlocked their power.

Power-ups you unlock

Layers of neurons: weighted sum + activation
Stacking learns abstract features
Perceptron (1958) was the origin
Hidden layers + nonlinearity unlock power

The Overfit Ogre attacks — common mistakes

Thinking neurons literally model the brain
Forgetting nonlinearity is essential
Assuming more layers always helps

Boss battleExplain why a single-layer perceptron can’t learn XOR.

Example code

<!doctype html><html><head><meta charset="utf-8"></head>
<body style="background:#06040d;color:#e6e0ff;font-family:monospace;padding:20px"><pre>neuron: out = activation(Σ wᵢxᵢ + b)
stack layers → learn abstract features
1 layer = linear only → can’t do XOR</pre></body></html>

▶ Open the interactive comic issue

‹ Knn · K-Nearest Neighbors Activation Functions · Relu · Sigmoid · Tanh ›