Every support ticket that hits your inbox makes the same round trip: up to the cloud, through a big LLM, and back. Ollaya decision models flip that by running tiny classifiers on your own machine, answering typed questions in milliseconds. And no, nothing ever leaves your Wi-Fi.
What are Ollaya decision models, exactly?
Think of it as Ollama, but for a different kind of brain. Ollama runs chatbots that generate text token by token. Ollaya runs decision models that never generate text at all. You feed one a “state” (an email, a ticket, a JSON blob) plus typed questions, and it returns calibrated probabilities in a single forward pass.
The project behind these Ollaya decision models hit the Hacker News front page this week with nearly 400 points, and the pitch is simple: pull a model by name, serve it locally, ask structured questions, get answers in about 10 milliseconds. One binary, no API key, no metered bill.
It is an independent project, by the way. Not affiliated with Ollama. The name just tells you exactly what to expect from the workflow: ollaya run, ollaya pull, ollaya list all behave the way you already know.
Why milliseconds matter more than you think
Here’s the math nobody shows you when they demo a chatbot wrapper. A hosted “System 1” API like Jev answers in roughly 150 to 276 milliseconds per request once you include the network. That sounds fast. Now imagine a triage step sitting inside a loop that runs 10,000 times a day. (I covered that model in my Jev explainer, if you want the full backstory.)
Local changes the equation completely. On an RTX 4090, Ollaya answers five typed questions end to end through its HTTP API in about 10 milliseconds. No network hop, no rate limit, and no per-token bill. For automation builders, that’s the difference between a step you can sprinkle everywhere and a step you ration.
The latency comparison from Ollaya’s own benchmarks (fair warning: different setups, so read it as order-of-magnitude):
| Model or service | Median latency | What it is |
|---|---|---|
| laya:multilingual | 8.1 ms | 100+ languages, local |
| laya:en | 9.6 ms | English decision model, local |
| gliclass | 14.7 ms | Zero-shot, instruction-following, local |
| nli | 20.4 ms | Zero-shot NLI classifiers, local |
| decider:0.8b | 155 ms | Most accurate option, local |
| decider:2b | 190 ms | Larger decoder model, local |
| TypeSafe Jev (hosted API) | 236–276 ms | Network included, metered |
Notice the pattern. The encoder-based models answer before your browser tab finishes painting. Even the most accurate local option, the Qwen3.5-based decider, beats the hosted API on raw speed while running offline.
Install Ollaya decision models in five minutes
Getting Ollaya decision models running takes three commands. Seriously.
Step 1: run the installer
On Linux or an Apple Silicon Mac:
curl -fsSL https://ollaya.dev/install.sh | shOn Windows, you get a PowerShell one-liner (irm https://ollaya.dev/install.ps1 | iex), though NVIDIA GPU users should run the Linux command inside WSL 2 instead. There’s also a desktop app with a menu bar icon on macOS if you’d rather never touch a terminal again.
When an NVIDIA driver (R580 or newer) is detected, the installer quietly adds the CUDA runtime for you. No driver surgery.
Step 2: pull your first decision model
ollaya pull layaLaya is the default pick: a ModernBERT-large model (421M parameters) that detects the language of each request and routes it to either laya:en or laya:multilingual. The download stays small because Ollaya ships only 3 MB ONNX graphs; the actual weights stream straight from each author’s Hugging Face repository, pinned to a commit and verified by sha256. Nothing gets re-hosted or silently modified.
Step 3: ask your first typed questions
ollaya run laya --preset triage "I was charged twice for my subscription this month and want a refund."You get back something like this:
intent refund 1.00
is_urgent no 0.88
frustration 1.76 / 3 0.36
refund_requested yes 0.90
churn_risk no 0.61That’s five business decisions about one customer message, answered in under 10 milliseconds on a decent GPU. A chatbot would still be writing the word “Certainly” at that point.
The TypeSafe trick: swap one environment variable
This is my favorite part, because it’s rare to see compatibility done this cleanly. Ollaya speaks TypeSafe’s wire format. The endpoints (/v1/systemone, /v1/decisions, /v1/models) accept and return the exact same shapes as Jev’s hosted API.
So if you already built something on the official TypeSafe Python SDK (0.7.1 works unchanged), you don’t rewrite anything. You don’t even reinstall. Point the SDK at your own machine:
export TYPESAFE_BASE_URL=http://localhost:11435
export TYPESAFE_API_KEY=local
export TYPESAFE_DEFAULT_MODEL=layaDone. Same code, zero cloud calls, zero cost per request. If the experiment goes sideways, flip the variable back and you’re hosted again. It’s the least risky local-AI migration I’ve tested this year.
For agent builders, there’s an MCP server baked in: claude mcp add ollaya -- ollaya mcp exposes the models to Claude Code, Claude Desktop, and Cursor. Your coding agent gets a millisecond classifier for its own routing decisions. Pair that with the [MCP server examples already on this blog](https://logicandlayers.com/mcp-servers-examples-ready-to-use/) and you’ve got a solid local toolkit.
Which model should you pull first
Ollaya’s registry is bigger than the headline model. The short version:
- laya for speed and language routing. Start here.
- decider when accuracy matters more than latency. It tops the typed-decisions benchmark at 0.591 and reads text like a small decoder LLM, because it is one (Qwen3.5, 0.8B or 2B).
- gliclass when you don’t know your categories yet. It follows natural-language instructions, so you can test a new question without retraining anything.
- nli for zero-shot checks like “does this review contradict the refund policy?”
- qwen3guard as a safety filter, with built-in questions for safe, controversial, or unsafe content.
There’s also kev, a calibrated 0.8B model from Jared Palmer, and von, which scores every option at its own marker with an 8k-token context. The full list lives on Ollaya’s model search page.
One more power move: Modelfiles. Bake a question set into a custom model, then run it by name:
FROM laya
QUESTIONS ./triage.json
PARAMETER precision fp32Now ollaya run triage "..." carries your exact schema everywhere. Share it with your team and everyone classifies identically.
Three workflows worth setting up this weekend
First, inbox triage for solopreneurs. Forward your support address into a small script, ask laya five questions per message (intent, urgency, refund requested, churn risk, sentiment), and write the answers to a spreadsheet. Urgent rows float to the top; refund requests get a template reply. You’ll clear your inbox in a third of the time.
Second, cheaper lead scoring. If you run any kind of paid funnel, classify every inbound form submission locally before it touches a paid LLM. Only the warm leads (buying intent above 0.8, for example) get the expensive personalized follow-up. The rest get nurtured automatically.
Third, agent guardrails. Put qwen3guard or an NLI check in front of any automation that posts publicly. If a generated message fails the safety or policy check, it stops before anyone sees it. At 20 milliseconds per check, you can afford to verify everything.
If you’ve been curious about running models on your own hardware but your laptop can’t fit a 7B chat model, this is the gentle entry point. Plenty of people now run a local LLM on their phone, and a decision model is even tinier. Laya runs on hardware far humbler than a 4090, and the CPU path (fp32) is fully supported.
The honest catches
No free lunch here, so let’s be blunt about the limits.
Decision models classify. They don’t write. You won’t get a summary, a draft reply, or an explanation, just probabilities. Pair Ollaya with a real LLM when you need prose, and once you graduate to bigger local models, you’ll face the usual puzzle of picking between model formats on your own machine.
The headline latency assumes a GPU. On an older CPU, expect slower answers, though Ollaya decision models stay usable on CPUs because they’re small. Accuracy also depends on your data: the models are calibrated on their training distributions, so weird inputs deserve a validation week before you trust the numbers blindly.
And the project is brand new. Nearly 400 points on Hacker News is excitement, not a track record. The install worked cleanly for me, but pin your versions in production and watch the repo for a few weeks.
Should you switch from Jev’s hosted API?
Short answer: if you process high volumes of text and your workflow only needs routing or classification, yes, and the switch costs one environment variable. Jev already made this category cheap and easy; Ollaya decision models make it free, private, and roughly 20x faster per call. Keep the hosted API for anything that needs frontier-level language understanding. Route the boring, high-volume decisions to your own GPU.
If you only send a few hundred requests a month, the hosted API is still the simpler life. Don’t add infrastructure for the sake of it.
Your first move
Install Ollaya, pull laya, and run the triage preset against ten real emails from your own inbox. Five minutes, zero dollars, and you’ll know in one sitting whether millisecond local decisions belong in your stack. If the numbers surprise you, wire it into one automation and watch the API bill shrink.
