TL;DRAI runs on accelerators — GPUs and TPUs — whose memory and bandwidth gate model size.
Training and serving large models demand massive parallel compute. GPUs (NVIDIA dominant) and Google's TPUs provide it; the binding constraints are usually memory (can the weights fit?) and bandwidth (feeding the cores). Big models are sharded across many accelerators. This hardware reality drives cost, availability ("GPU shortage"), and why quantization/efficiency techniques matter so much.
Key points
GPUs/TPUs provide parallel compute
Memory + bandwidth are the real limits
Big models shard across many accelerators
Hardware drives cost and availability
Common mistakes
Ignoring memory limits when sizing a model
Assuming more FLOPs alone solves it (bandwidth matters)
Overlooking hardware cost/availability
Try it: Explain why "memory" often limits model size more than raw speed.
Example code
<!doctype html><html><head><meta charset="utf-8"></head>
<body style="background:#06040d;color:#e6e0ff;font-family:monospace;padding:20px"><pre>weights must FIT in accelerator memory
+ bandwidth to feed the cores
big model → shard across many GPUs/TPUs</pre></body></html>