The gap between open vs closed AI models has shrunk to roughly four months, according to a new Mozilla report. That matters because open models often cost a third of the price. So the real question stopped being “which one is better” and became “when is the free one enough?”
Why the open vs closed AI models debate just changed
For years the answer was simple. Closed models from Anthropic, OpenAI, and Google were the only ones worth using for anything serious. Open alternatives lagged on every benchmark that mattered, and “cheap but dumb” was a fair description.
That description is out of date.
Mozilla’s State of Open Source AI report (version 1.1) landed on September 15 with data through September 1. Its headline number: the best open-weight models now trail the best closed ones by 4.4 months. Epoch AI, an independent research group, puts the same gap at an even 4 months using its own index. Two separate measurements agreeing on anything in this space is genuinely rare.
What does “4.4 months behind” actually mean? The research nonprofit METR measures models by task length. Give a model a job, check how long a human expert needs to do it, and see which jobs the model finishes reliably half the time. Right now the best closed model can handle a 12-hour task. The best open model tops out around 7 hours.
Raffi Krikorian, Mozilla’s CTO, put it plainly: “If the open frontier can handle a seven-hour job, the closed frontier can handle a 12-hour one. In four months, the open model handles the 12-hour job, and the closed one handles something around 20.”
Both ladders keep climbing. The gap just isn’t much of a gap anymore.
The numbers behind the claim
Here’s the part that should make you check what you’re paying for.
On the Artificial Analysis Intelligence Index, Moonshot AI’s Kimi K3 sits three points behind the top closed model while costing about 60% of its list price. Against Anthropic’s Claude Fable 5 specifically, K3 is two points back at roughly 30% of the cost. On Epoch’s Capabilities Index, K3 scores 157 versus 162 for Fable 5 and Opus 5. Alibaba’s Qwen 3.8 Max and DeepSeek’s newest release sit right behind at 156 and 155. That’s the open vs closed AI models scoreboard as of this month.
Abstract index scores are boring, I know. So look at something practical instead. On the Terminal-Bench 2.1 coding board, Z.ai’s GLM-5.2 scored within one point of Claude Opus 4.7 while costing about one-fifth as much per completed task. I looked at a close cousin, Qwen 3.8, earlier this month and came away impressed for exactly this reason.
| Comparison (Mozilla report, Sept 2026) | Closed frontier | Best open-weight |
|---|---|---|
| Capability gap | Baseline | 4 to 4.4 months behind |
| Price (hosted API, list) | Full price | About 30% to 60% |
| Reliable task length (METR) | Up to ~12 hours | Up to ~7 hours |
| Top examples | Claude Fable 5, Opus 5 | Kimi K3, Qwen 3.8 Max, DeepSeek |
| Where it wins | Expert work, long docs, retrieval | Everyday tasks at a fraction of the cost |
The market has already voted with its tokens. Mozilla counted the top ten models by usage volume on OpenRouter, a marketplace that routes developer traffic, and found eight of them were open-weight. Seven were built by Chinese companies.
Still, hold on before you cancel anything. There’s a catch, and it’s a real one.
The sticker price problem
Kimi K3 lists at $15 per million output tokens. Mozilla found it effectively bills closer to $31 for equivalent work, because the model produces more tokens to reach the same answer. Researchers call it a verbosity tax. It quietly eats half the headline discount.
The second catch is hardware. “Open weights” means you can download the model. It does not mean you can run it. K3’s native checkpoint is about 1.56TB of memory. Mozilla’s own serving configuration calls for 64 or more accelerators. Almost nobody reading this has that sitting in a closet.
For most of us, “using open models” really means “using hosted open models,” and the price comparisons above are exactly that: API against API, at list price. That’s still a serious saving on everyday work. It’s just not the “AI for free forever” fantasy that the word “open” sometimes sells.
That’s the honest math of open vs closed AI models in 2026: cheaper, yes. Free, no.
A 3-question test for any task
Forget benchmarks for a minute. When you sit down to work, run the task through three quick questions.
1. How long would a skilled human need? Under a few hours? Open-weight models handle that band comfortably today. Multi-day expert work is where the closed frontier still lives.
2. What does it cost to be wrong? For drafts, summaries, classifications, and most coding help, a 1-to-3 point quality gap doesn’t show up in your results. For a client deliverable with your name on it, that gap might matter. Choose accordingly.
3. How many times will you run it? One-off task, who cares. But volume is where the 40-to-70% discount compounds into real money. A thousand small jobs a month on an open endpoint versus a closed one is the difference between a coffee budget and a phone bill.
DoorDash is already doing this at scale. The company routes routine work to Kimi and saves Fable 5 for the genuinely hard stuff. That’s the pattern worth copying, and you can run it at any size. Our model routing guide shows the cost-control setup step by step.
Where closed models still earn their money
Three zones, per Mozilla: expert professional work, high-intensity retrieval, and long context. On GDPval-AA v2, Fable 5 beats K3 by 92 Elo points, the widest gap on any benchmark in the report, concentrated in long-context fidelity and polish. Translation: long, messy, high-stakes documents are still closed-model territory.
Money agrees with the leaders too. Closed providers took 96% of model-layer revenue on OpenRouter in the period studied. People pay for the top of the market, and they pay a lot.
But revenue share measures where money flows. It doesn’t tell you where yours should go.
Route instead of picking a side
The open vs closed AI models decision was never really either/or. The smart setup is boring and effective: one open model as your default for everyday tasks, one closed model reserved for escalations, and a quarterly re-check because the gap resets with every release cycle.
If you want the mechanics, we have a guide to switching models without rebuilding your whole setup, plus the best free coding AI options if your budget is zero. Both pair well with the routing approach. Handy detail: on OpenRouter, open models like Kimi K3 and GLM-5.2 run through the same kind of API you’d use for closed ones, so no hardware required. For the full report, read Ars Technica’s coverage or Mozilla’s report page.
Takeaway
Mozilla’s report doesn’t declare closed models dead. It flips the default: start with open, pay for closed only when the task earns it. Take your three most repeated AI tasks this week, run them through the 3-question test, and cancel or downgrade one subscription if it passes. That’s real money back in your pocket.
