Someone Just Shipped 'Ollama for Jev.' Ollaya Runs Seven Decision Models Locally and Answers in 8ms Instead of Metering You by the Token
Follow-up to: September 24, 2026, "Someone Rebuilt TypeSafe AI's Jev in 25 Lines of Local Python."
Two days ago I wrote about a developer who reimplemented the core of TypeSafe AI's Jev classification API in 25 lines of local Python, as a proof of concept that the "decision model" category wasn't as defensible as its pricing implied. This week that proof of concept turned into an actual product. Ollaya is a packaged, open-source local server that runs seven decision models and speaks close enough to Jev's own API shape that TypeSafe's official Python SDK works against it, unchanged, once you point it at your own machine instead of TypeSafe's servers.
What's actually new here, versus Tuesday's hack
The 25-line version I covered on September 24 was a script: useful for proving a point, not something you'd put behind a production endpoint. Ollaya (available at ollaya.dev and on GitHub as ollaya-dev/ollaya, Apache-2.0 licensed) is a real local server. It hosts seven models under one roof: laya (fastest, multilingual), decider (most accurate of the set), nli (zero-shot classification), gliclass (instruction-following classification), qwen3guard (safety screening), kev (a LoRA-based model), and von (8k-token context for longer inputs). It exposes /v1/systemone and /v1/models endpoints that mirror TypeSafe's own API surface closely enough that you can swap your base URL and keep your existing integration code.
The numbers that make this a real decision, not a novelty
On a local RTX 4090, a five-question request through Ollaya completes in roughly 8 to 10 milliseconds, end to end. The hosted TypeSafe Jev API runs a 236 to 276 millisecond median for the same kind of request, most of that network overhead rather than actual model compute. On cost, TypeSafe's hosted Jev pricing runs around $40 per 1.5 billion tokens; Ollaya has no per-token fee and no metering at all, because you're running it on hardware you already own. For anyone doing high-volume classification (content moderation, intent routing, lead scoring, anything that fires on every request rather than occasionally), that's not a marginal savings. It's the difference between a line item that scales with your traffic and one that doesn't.
The tradeoff nobody should skip past
Here's the part that matters more than the benchmark numbers. Ollaya's own maintainer said it plainly on the Hacker News thread: the smaller models the project supports today are "well below Jev on harder queries." Bigger, more capable open models exist and are reportedly on Ollaya's roadmap, but they haven't shipped yet. One commenter on that same thread raised a fair point that could make this whole category moot: Ollama itself may eventually add native support for these decision-model architectures, which would remove the need for a dedicated tool like Ollaya at all. So this isn't "the hosted API is now obsolete." It's "a credible free alternative now exists for a meaningful slice of the workload, with an honest gap on the harder end."
There's also a benefit that has nothing to do with speed or price: zero data leaves your machine. If you're running classification over anything you'd rather not hand to a third party regardless of what it costs (user messages, internal documents, anything under an NDA), that's the strongest argument for Ollaya even before you look at the latency numbers.
What I'd actually do
If your classification workload is simple, routing tickets, flagging obvious spam, basic intent detection, swap to Ollaya now and keep the savings. Run it in parallel with your existing Jev calls for a week first, diff the outputs, and confirm the accuracy gap doesn't matter for your specific use case before you cut over fully. If your workload leans on harder, more nuanced classification where accuracy actually drives revenue or safety outcomes, stay on Jev's hosted models until Ollaya's larger models ship, and be upfront with your own users if you do decide to trade some accuracy for cost savings rather than quietly shipping a downgrade and hoping nobody notices.
Author
Lukas
@lukcombinator