Running GPT-OSS locally: AI you own, not rent
I had Ollama installed on my laptop for about a year before a local model actually stuck. They were fun for a weekend, then forgotten. That changed when OpenAI released GPT‑OSS, its first open-weight models in more than five years, under an Apache 2.0 license. One command later I had the 20B model answering questions on a machine with 16 GB of memory, and I still use it.
The launch is old news now. What it showed is still true: a capable AI model runs on an ordinary laptop, with no cloud account and no data leaving the building.
Getting it running
- Install Ollama for Mac, Windows or Linux.
- Open a terminal and run
ollama run gpt-oss:20b. The first run downloads the model, about 14 GB. - Ask it something about your own business.
The 20B model is the one for a laptop. It runs in as little as 16 GB of memory. Its bigger sibling, gpt‑oss‑120b, needs a single 80 GB GPU, so for most people that one is a server decision.
What a local model is good for
Documents you would never paste into a chatbot. Contracts, payroll, patient notes. When the model runs on your machine, local means local.
Experiments. Try a prompt a hundred times, or a thousand, without watching a meter.
Small automations. Summarize a folder of notes, draft the weekly update, sort incoming tickets. A small box in the closet can do this overnight.
Learning. If you want to understand what these models can and can’t do, nothing beats having one you’re allowed to break.
What it is not
It is not a replacement for ChatGPT or Claude for most of my day. The hosted models are better and faster, and for ordinary writing, email and research that is what I use.
A local model is slower on a laptop, it knows nothing after its training date unless you give it documents, and you are the one keeping it updated. Those are fair trades for the sensitive work. They aren’t worth it for everything.
The point
AI is no longer only something you rent. For the sensitive ten percent of your work, you can own it.
If your team handles data that can’t go to a cloud service and you want to know what a local or private model could do for you, get in touch. I’m always glad to hear what people build with one.