OpenAI's First Chip Beats Nvidia on Watts and Ties It on Cost Per Token. Your Inference Bill Is Not Moving This Year.
OpenAI put the first benchmarks for its in-house inference chip on stage at Hot Chips yesterday, and they are better than a first-generation chip has any right to be. Jalapeño delivers 1.5x to 1.9x more AI work per watt at peak throughput and 1.7x to 3.6x lower end-to-end latency than the best commercially available systems, across three tested models. On interactive workloads the claimed gap widens to 2.1x to 4.1x.
Dylan Patel of SemiAnalysis, whose InferenceX benchmark produced the numbers, put it plainly: "Usually first generation chips aren't competitive, but OpenAI is beating Nvidia Blackwell and even Rubin."
I paid for roughly $340 of inference last month across two side projects and this blog's content pipeline. So the only question I actually care about is whether any of that lands in my price. Further down SemiAnalysis's writeup is the sentence that answers it: on total cost of ownership per token, Jalapeño and Nvidia's Vera Rubin come out roughly even.
What the chip is, and what it is not
Jalapeño runs inference only. It does not train models. That is a narrower brief than "OpenAI built a chip" implies, and it is the right brief, because inference is where the recurring cost lives once a model exists.
The more interesting detail is that it is not tuned to OpenAI's own models. It is a general-purpose LLM inference accelerator, and the benchmark set reflects that: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. Two of those three are not OpenAI models. On GPT-OSS it hit roughly 1,400 tokens per second per user. On DeepSeek R1 it cleared 700 tokens per second on a single concurrent request.
It also posted those numbers without multi-token prediction or speculative decoding, while some of the comparison systems did use them. That cuts both ways. It means there is headroom left, and it means the comparison is not apples to apples in Jalapeño's favour.
OpenAI built it with Broadcom. Design started mid-2024, the final design went to fab in November 2025, about sixteen months end to end. OpenAI says only nine months elapsed between the first chip design and the finished blueprint, and that it used its own models throughout: older generations on chip design, newer ones on programming and optimisation.
The provenance of the numbers matters
OpenAI supplied the benchmark results. SemiAnalysis verified some runs on-site in its lab. That is better than a vendor press release and worse than an independent third-party test, and it is worth holding both halves of that sentence at once.
I do not think the numbers are fabricated. SemiAnalysis has a reputation to protect and put its name on a public benchmark. But "OpenAI provided the numbers, we checked some of them" is a specific epistemic status, and the tech press collapsed it into "OpenAI's chip beats Nvidia" within about four hours.
There is a harder caveat. Nvidia and AMD have already published results on larger models, DeepSeek V4 Pro and Kimi K3 among them, which have not been run on Jalapeño. And Rubin systems are shipping to customers right now, while Jalapeño is reportedly still at engineering samples. A chip that does not exist in volume cannot lower anyone's price.
Why perf-per-watt is OpenAI's number, not yours
Here is the part I want to be blunt about, because I keep seeing the watt figures quoted as though they were a price cut.
Performance per watt is a margin metric. It tells you what it costs OpenAI to serve a token. Whether any of that reaches your invoice depends entirely on whether OpenAI is competing on price at the point where you buy, and right now the constraint on frontier inference pricing is not silicon efficiency. It is demand exceeding supply, with every major lab saying publicly that the world cannot have enough compute, a claim that happens to support all of their business models simultaneously.
The cost figure that would actually predict a price move is total cost of ownership per token, because that folds in the capital cost of the hardware, not just the electricity. On that measure, against Vera Rubin, the two are roughly even. SemiAnalysis also makes the point that Rubin is the fairer comparison than Blackwell anyway, since both Rubin and Jalapeño use HBM4.
So the honest summary is: OpenAI built a first-generation chip that is competitive with Nvidia's current generation on cost, and better on power. That is a remarkable engineering result and a mediocre one for your budget.
The claim that actually matters for solo builders
Buried in the same analysis is a line with a longer half-life than any benchmark: "The CUDA moat is potentially dead given how fast OpenAI can bring up new models on their silicon."
If that holds, it is the thing that changes my life in two years, not the watts. The reason I route between three inference vendors today and not eight is that porting a model to non-Nvidia silicon has historically been expensive enough that only a handful of organisations bothered. If a team can go from first design to working general-purpose accelerator in nine months, and bring up arbitrary open models on it quickly, then the number of credible inference vendors goes up, and price competition follows from supply, not from efficiency.
I would treat this as a hypothesis with one supporting data point. One company did it once, with Broadcom's help, and that company happens to be the one with the most model expertise on earth. That is not the same as a general claim that the moat is drained.
Where I could be wrong
Two places.
First, I am treating "roughly even on TCO per token" as though it settles the pricing question, and it does not settle it for OpenAI specifically. If OpenAI serves a meaningful share of its own traffic on Jalapeño, it stops paying Nvidia's margin on that share. Even at parity on TCO, removing a supplier's markup from your own cost base is real money, and it gives OpenAI room to price aggressively if it chooses. Whether it chooses to is a strategy question, not an engineering one.
Second, the engineering-sample caveat is sourced to reporting rather than to OpenAI. If volume is closer than that implies, the timeline compresses and my "not this year" gets shorter.
What I would actually do
Nothing to your stack. Do not rearchitect around a chip that is not in production, and do not read a perf-per-watt chart as a price signal.
What I would do is spend twenty minutes making sure your inference calls go through a layer you can re-point. Not a heavyweight abstraction, just a single module that owns model selection and the client, so that switching a provider is a config change rather than a grep across your codebase. I did this for the content pipeline that writes this blog after the OpenRouter acquisition in June made me realise how much of my routing logic lived in whatever file happened to need it.
That work pays off whether or not Jalapeño ever ships. It pays off the next time a model gets deprecated on thirty days' notice, which on the current run rate is roughly monthly.
The chip story to watch is not the benchmark. It is whether a second team, without OpenAI's model expertise, can repeat the nine-month bring-up. If somebody does that in 2027, the inference market gets genuinely competitive and your bill finally moves. Until then, the watts belong to OpenAI.
Author
Lukas
@lukcombinator