A 4B Open Model Just Matched GPT-5.6 Sol on Search Accuracy at 100x Less Cost
A 4B Open Model Just Matched GPT-5.6 Sol on Search Accuracy at 100x Less Cost
On August 5, Neon and a post-training startup called Castform published a joint writeup showing a 4-billion-parameter open-weights model matching GPT-5.6 Sol's accuracy on agentic search, at roughly 100 times less cost per request. Not a benchmark slide from a frontier lab trying to sell you a subscription. A database company and a training-infrastructure startup, showing their work, with the pipeline public.
If you've shipped an AI-powered search box, a support bot, or a "chat with your docs" feature and defaulted to whatever frontier model your framework recommended, this is the number that should make you go check your API bill.
The gap Neon put in writing
Neon's post states the cost problem plainly: a typical multi-turn agentic search request against GPT-5.6 Sol takes more than 10 seconds and costs roughly $0.03 end to end. That's not one API call. Agentic search means the model plans a query, searches, reads results, decides it needs another search, and repeats until it has enough to answer. Every loop is another round trip to a frontier model, and the cost compounds with every iteration.
Small open-weights models running the same workflow are, per the post, roughly 100x cheaper per request. Out of the box, though, those models are worse at the task. They don't know when to search again, they cite the wrong passage, or they just guess. That gap between "cheap" and "good enough" is exactly what RL post-training is supposed to close, and Neon and Castform's post is a specific, dated case where it apparently did.
How the pipeline actually works
Castform's pitch is that most companies already have the raw material for this and don't realize it. You need three things to run reinforcement-learning post-training: a task, an environment for the model to act in, and a reward function that scores whether it did well.
For Castform's setup, the environment is a Postgres database on Neon, queried through Neon's Lakebase Search extensions for text and vector search. The task-generation step turns an existing document corpus (their example is internal travel-policy documentation) into synthetic question-answer pairs: a real sentence from a real document becomes the ground truth, and a question gets generated that the sentence answers.
The reward function then scores three things on every attempt: did the model retrieve the right source, did it cite the right chunk, and did it land on the right final answer. The model runs thousands of parallel rollouts against this setup, gets scored on each one, and gradually learns to search better. Neon's dynamic compute scaling is what makes the bursty, parallel-rollout workload affordable to run at all, since a training run might spike from near-zero database load to thousands of concurrent queries and back down between batches.
Why "beats GPT-5.6 Sol" needs an asterisk
The headline claim is real but narrow, and it's worth being precise about what "narrow" means here. This is a 4B model matching a frontier model's accuracy on one measured task: retrieval and citation against a specific corpus, graded by a specific reward function. It is not a claim that a 4B open model reasons as well as GPT-5.6 Sol, writes as well, or handles open-ended agentic work as well. RL post-training trades generality for cost on a task you can actually define and grade. That's the entire mechanism, and it's also the entire limitation.
If your product's AI feature is "answer questions using our docs," that's exactly the kind of narrow, gradable task this method is built for. If your product's AI feature is "have a useful conversation about anything," post-training a 4B model isn't going to get you there, and nothing in Neon's post claims otherwise.
What this actually changes for a solo operator
The interesting part isn't the specific 100x number, which is going to move around depending on your corpus and your reward function. It's that the infrastructure to do this yourself just got a lot more approachable. A year ago, RL post-training meant hiring or becoming an ML engineer: writing your own training loop, provisioning GPUs, building reward-scoring code from scratch. Castform is explicitly positioning itself as a self-serve product that handles the RL loop, the observability, and the infrastructure, so the part you actually do is write a reward function and point it at your data.
If you're running a support bot, a docs search feature, or an internal knowledge tool and every query round-trips to a frontier model, you're paying for generality you don't need. Your product doesn't need a model that can also write poetry or debug Rust. It needs a model that answers questions correctly against your specific corpus, which is precisely the case where a smaller, post-trained model can plausibly match the expensive one.
The honest take
I'd read this as a genuine signal, not a guarantee. Neon and Castform have an obvious incentive to present their own integration in the best possible light: this is a partner blog post, not an independent benchmark, and there's no third-party eval here to cross-check the numbers against. The corpus, the reward function, and the evaluation set all have to be built well for this to work, and getting a reward function right for anything beyond "did it cite the correct passage" is genuinely fiddly work.
What I'd actually do is treat this as a reason to run the experiment, not a reason to assume it's already solved. If you're spending real money on an AI search or support feature and your task is narrow enough to grade automatically, spend an afternoon scoping what a reward function for your product would even look like. If you can define "correct" clearly enough to grade it, you probably have a candidate for exactly this kind of cost cut. If your feature is genuinely open-ended, this method won't fix that for you yet, and no post-training pipeline changes that.
The broader pattern worth watching: the tooling gap between "frontier lab publishes a benchmark" and "you can replicate a version of this yourself by Friday" keeps shrinking. That's the actual story here, more than the 100x figure itself.
Author
Lukas
@lukcombinator