Want to run Qwen 3.8 locally without paying a cent for cloud APIs? Last weekend, a free open-source tool called Strata hit the front page of Hacker News with almost 700 upvotes, and it does exactly that on a regular gaming PC. There’s a catch or two, but the hardware bar is probably lower than you think.
What Strata actually is
Strata is a free inference engine that runs Qwen3.8-Flash-Next, a 125-billion-parameter mixture-of-experts model from the Qwen team. Normally, a model this size lives on rented servers with hundreds of gigabytes of memory. Strata squeezes it onto one PC by splitting the work across your whole machine: your graphics card, your RAM, your processor, even your SSD.
The project landed on GitHub on September 24. Eleven days later it passed 11,000 stars, and its launch thread on Hacker News pulled in 300-plus comments. For example, one user reported 124 tokens per second on an RTX 4090, which is faster than most people read. Meanwhile, the model itself passed 1.4 million downloads on Hugging Face, so the interest here is real, not hype. That combination is exactly why so many people want to run Qwen 3.8 locally right now.
Here’s the neat trick, in plain English. The model is a team of 24,576 small specialists called experts, and each word only needs about 10 of them. Therefore, Strata keeps the busy ones on your graphics card, all of them in RAM, and a lookup table on your SSD. A small helper model also guesses the next few words so the big one can check them in bulk, which makes answers arrive 1.6 to 1.8 times sooner.
Why run Qwen 3.8 locally instead of the API?
Fair question. Cloud AI is easier, and subscription fatigue is real. Still, running a model at home wins in three specific situations.
- Cost. After the 70 GB download, usage is unlimited and free. If you currently spend $20 to $200 a month on AI subscriptions, a capable local model pays for itself fast.
- Privacy. Nothing leaves your PC. Chats, documents, code: it all stays on your drive. For client work or anything sensitive, that’s a genuine feature.
- No rate limits. Batch-process 500 files at 3 AM if you want. Nobody throttles you.
And honestly? It just feels good. A model that used to need a data center, chatting away on the same machine you play games on.
The hardware you need
Before you get excited, check your specs. The requirements are specific, and RAM matters just as much as your graphics card.
- Graphics card: NVIDIA RTX 20, 30, 40, or 50 series, or an AMD Radeon RX 7900 XT/XTX, RX 7800 XT, RX 7700 XT, RX 9060 XT, or RX 9070. You need 12 GB of VRAM or more.
- RAM: 32 GB minimum. 64 GB runs every model size, and it’s the sweet spot.
- Disk: about 80 GB free, SSD strongly preferred.
- System: Windows 10 or 11, or Linux, with a current GPU driver.
Got a laptop with 8 GB of VRAM? Then this specific setup isn’t for you yet, and I’d skip it rather than fight it. Strata does list experimental support for older cards like the GTX 10 series and the Radeon VII, but expect a much bumpier ride there.
Pick the right model size
The same model comes in several compressed sizes, called quantizations. If you want the full background on what those letters mean, we have a whole GGUF vs GPTQ vs AWQ vs EXL2 breakdown. Short version: smaller sizes run faster but lose a little intelligence. The installer recommends one based on your RAM.
| Your RAM | Pick this | Why |
|---|---|---|
| 32 GB | Coder | The only size that fits; code-focused, reaching 91% of the full model’s SWE-bench score |
| 48 GB | IQ2_XS | Larger sizes won’t fit; Q2_0 if you want speed over quality |
| 64 GB | IQ2_XS | Recommended default; every size fits, so IQ3_S works if you prefer quality |
| 96 GB+ | IQ3_S | Room for the best sizes with everything else still open |
One warning from the community: the Coder version drops half of the model’s experts to fit 32 GB machines, and it’s noticeably weaker outside code tasks. So coders, enjoy. Writers and translators, pick a full-expert size instead.
How to run Qwen 3.8 locally with Strata, step by step
The whole setup is a wizard. No terminal skills required, no Docker, no compiling.
- Download Strata from the official GitHub repository and unzip it, or clone it with git.
- Run the installer. On Windows, double-click
START-HERE.bat. On Linux, run./setup.shin the folder. - Answer three questions. It asks which model size you want, how much context to keep, and whether the model should read pictures. Press Enter at each prompt and you get the recommended answer.
- Wait for the download. It’s about 70 GB. If it stops midway, run the installer again: it picks up where it left off.
- Wait out the first launch. Your PC may freeze up for 1-3 minutes while 35-55 GB loads into RAM. This is normal. Don’t close the window.
- Open the app. Your browser lands on
http://127.0.0.1:8080, with a chat interface and a live monitor of your GPU, CPU, and RAM.
Next time, run the same launcher: it starts right away and downloads nothing twice. There’s also an update script for later. And if the first-launch freeze makes you nervous, like it did me, set a timer before you touch anything.
Feeling lazy? Let an AI do it
This part made me laugh, then I immediately used it. Strata ships with a one-line prompt you paste into Claude Code, Cursor, or Copilot. It checks your hardware, picks the right model, installs everything, and starts it for you. The engine also exposes an MCP server, so your AI assistant can start and stop it later on request. Welcome to 2026, where the AI installs the AI.
Connect your other apps
Strata speaks both major API dialects, so your existing tools plug straight in.
- OpenAI-compatible apps: point them at
http://127.0.0.1:8080/v1. Any API key and any model name work. - Anthropic-style apps: use
http://127.0.0.1:8080/v1/messages. For Claude Code, setANTHROPIC_BASE_URL=http://127.0.0.1:8080.
Prefer a richer chat interface? Point SillyTavern, the free open-source chat app, at that same local address and you get a fully private setup with personas on top.
The catches nobody mentions
Every demo looks perfect. Here’s what actually bites, pulled from the docs and the Hacker News thread.
- The first message of a chat is slow. Strata reads your full first prompt at roughly one minute per 30,000 tokens. Follow-up messages start in seconds, but pasting a 50-page document means a coffee break first.
- One request at a time. By default, extra requests queue up. You can raise it to two parallel answers, though on a 12 GB card that slows each one down.
- AMD image reading is Linux-only for now. On Windows, AMD cards can’t process pictures yet.
- Port conflicts. If something else already lives on port 8080, Strata will complain. Usually that just means it’s already running, which is a good problem to have.
None of these are dealbreakers. However, knowing them upfront saves you an evening of confused Googling.
Real speeds you can expect
These are the project’s own benchmarks on an RTX 5070 (12 GB) with a Ryzen 5 7600 and 64 GB of RAM. “Writes” is how fast the answer streams out, and “reads” is how fast it digests your prompt.
| Model size | Writes answers | Reads your prompt |
|---|---|---|
| Q2_0 | 94 tokens/s | 2,650 tokens/s |
| IQ2_XS | 79 tokens/s | 2,090 tokens/s |
| IQ3_XXS | 62 tokens/s | 1,750 tokens/s |
| IQ3_S | 53 tokens/s | 1,620 tokens/s |
| Coder | 55 tokens/s | 2,180 tokens/s |
For context, 60 tokens per second is already faster than you can read. Even the slowest row here streams quicker than a person types. Bigger cards do better: the docs estimate 100-140 tokens per second on an RTX 3090’s 24 GB, and that community test I mentioned earlier hit 124 on a 4090. Curious how the model itself stacks up against the big names? We put it through its paces in our Qwen 3.8 review.
So, should you bother?
If you’ve got a 12 GB-plus graphics card and 32 GB of RAM sitting there, yes. You get to run Qwen 3.8 locally, unlimited and private, for the price of a 70 GB download and some patience. Check your specs, grab Strata from GitHub, and let the installer make the decisions for you. Worst case, you’re out an evening and some disk space. Best case, you cancel an AI subscription or two.
