TL;DRComputer use lets a model operate a desktop — clicking, typing, reading the screen.
The newest agent frontier: models that perceive a screen (via screenshots) and take UI actions — move the mouse, click, type, scroll — to complete tasks in real applications, including ones with no API. It generalizes automation beyond brittle scripts, but is slower, error-prone, and risky (it can take real actions), so it runs with confirmations, sandboxes, and human oversight for anything consequential.
Key points
Perceive the screen + take UI actions
Automates apps that have no API
More general than scripted automation
Slower, riskier → sandbox + oversight
Common mistakes
Granting unsupervised control of real systems
Expecting fast, flawless UI automation
No sandbox/confirmation for risky actions
Try it: Explain why computer use needs more guardrails than text generation.
Example code
<!doctype html><html><head><meta charset="utf-8"></head>
<body style="background:#06040d;color:#e6e0ff;font-family:monospace;padding:20px"><pre>screenshot → model decides → click/type/scroll
operates apps with no API
real actions → sandbox + confirmations</pre></body></html>