On-device models are getting good enough to leave the cloud behind
A new wave of small models runs entirely on your laptop or phone — fast, private, and free of an API meter.
For years the assumption was that useful AI had to live in someone else’s data center. That assumption is quietly collapsing.
The latest generation of small models — a few billion parameters, quantized to run on consumer hardware — is now good enough for summarization, drafting, classification and code completion without ever touching a network.
Private by default
Running locally flips the privacy story. Nothing leaves the device, there is no per-request cost, and the model keeps working on a plane or in a basement.
The tradeoff is capability: the biggest frontier models still win on the hardest reasoning tasks. But for the long tail of everyday jobs, “good enough and on my machine” is starting to beat “excellent and metered.”
