OpenAI Just Taped Out Its Own Inference Chip in Nine Months. You're Not Building Silicon — But Here's What Jalapeño Does to Your Token Bill.

On June 24, OpenAI and Broadcom unveiled Jalapeño, OpenAI's first custom-built inference chip. It reached tape-out in roughly nine months, it's slated to start deploying by the end of 2026, and it's the first of a multi-generation hardware platform the two companies are building together with Celestica handling the boards and systems. OpenAI says early testing shows performance-per-watt substantially better than current state-of-the-art parts.

If you run a small product on someone else's models, none of that ships you anything this quarter. But it tells you which way your biggest variable cost is pointed, and that's worth ten minutes of your attention.

What actually got announced

OpenAI designed the chip around its own models (the kernels, the serving systems, the way its products actually call inference) rather than buying a general-purpose accelerator and bending workloads to fit. Broadcom did the silicon implementation, networking, and connectivity; Celestica does board, rack, and system assembly. The partnership itself was only made public in October 2025, so a nine-month path to tape-out is fast, and OpenAI says it used its own models to speed up parts of the design work.

The part to anchor on: this is an inference chip, not a training chip. Inference is the cost you pay every time a user hits your product. Training is OpenAI's cost. By building its own inference silicon, OpenAI is going after the recurring cost of serving you and everyone else, and trying to stop paying Nvidia's margin on every token it generates.

Why every frontier lab suddenly wants its own chip

There are three reasons, and none of them are vanity.

The first is margin. Nvidia's gross margins on data-center GPUs have been extraordinary, and every dollar of that margin is a tax the labs pay to serve inference. At OpenAI's scale, designing your own part (even after eating the engineering cost) pencils out if it shaves enough off perf-per-watt.

The second is supply. If your entire product depends on getting allocation from one vendor's fab schedule, you don't control your own roadmap. Owning the design is how you stop competing with every other lab for the same chips.

The third is fit. A chip built for exactly the models you serve can drop the overhead that a general-purpose GPU carries to be good at everything. That's where the perf-per-watt claim comes from.

This is the same vertical-integration move Apple made with its own silicon and Amazon made with Graviton and Inferentia. When a cost gets big enough and predictable enough, you stop renting it.

What this means for your token bill, and what it doesn't

Here's the honest read, and it's less exciting than the headline.

Jalapeño does not lower your costs in 2026. It's deploying late this year inside OpenAI's own infrastructure, and the savings, if they materialize the way OpenAI hopes, show up first as fatter margins for OpenAI, not as a price cut for you. Price cuts come later, and only if competition forces them. The thing to take from this isn't "cheaper tokens are here." It's "the people who set token prices are now spending billions to make serving cheaper, which means the long-run direction of inference pricing is down."

That trajectory has been real for a while. The cost per token for a given capability has fallen hard, year over year, as models get more efficient and hardware improves. Custom inference silicon at the labs is another push in that direction. If you're making a multi-year bet on a product whose unit economics depend on inference, you should assume the per-token cost of today's capability keeps dropping, and that the capability you can afford at a fixed budget keeps rising.

What you should not do is re-architect anything around a chip that hasn't shipped. I've watched people make infrastructure decisions on the strength of a keynote, and it's a reliable way to waste a month. Jalapeño is a roadmap, not a product you can call.

What I'd actually do

Two things, both boring, both within reach this weekend.

First, make a future price drop a config change instead of a migration. If your inference path is wired so that switching models or providers means touching one adapter and a couple of environment variables, you capture every price cut and efficiency gain automatically. If it's hardcoded across your codebase, you'll watch cheaper options arrive and not be able to use them without a rewrite. The labs are building the supply side of a price war. Your job is to be positioned to take the winnings.

Second, instrument your cost per action now, not your cost per token. Tokens are the input; what matters to your business is what it costs to serve one user one useful result. If you don't have that number, you can't tell whether a future price drop actually helps your margin or just gets eaten by your prompt bloat. Most solo products I've seen waste more money on sloppy context than they'd ever save by switching chips.

The chip announcement is OpenAI's problem to execute. The only part that's yours is staying liquid enough to benefit when the cost curve bends, and it's bending toward you.

Author

Lukas

@lukcombinator

Sources

• OpenAI and Broadcom unveil an LLM-optimized inference chip — OpenAI
• OpenAI unveils its first custom chip, built by Broadcom — TechCrunch
• OpenAI unveils first chip as part of Broadcom deal in effort to 'build the full stack' — CNBC
• Broadcom and OpenAI unveil custom-built Jalapeño inference processor — Tom's Hardware