2,100 tokens per second. That is the throughput figure that Cerebras hardware is already delivering through cloud APIs[1], and it is the benchmark OpenAI’s Jalapeño chip is designed to challenge on OpenAI’s own infrastructure[2]. For most advice firms, a token-per-second figure means nothing on its own. But the economics behind it matter considerably.

The short version: OpenAI has built its first proprietary inference chip. It is fast, it is purpose-built for running large models at scale, and it is designed to reduce the cost of each query. If it performs as the early benchmarks suggest, the per-token price of using OpenAI’s models through the API will fall. For regulated advice firms that have been sitting on the edge of committing to AI in their client processes, cheaper and faster inference changes the calculation.

This is an analysis, not a how-to. My judgement is at the end.

What the Jalapeño chip actually does

Inference is the step where a trained AI model generates an answer. Training a model is expensive and happens once. Inference happens every time a user sends a query, drafts a letter, or runs a document through a pipeline. At scale, inference is where the cost accumulates.

OpenAI’s chip is designed specifically for this step. By building its own silicon rather than buying from Nvidia or running on generic cloud GPU clusters, OpenAI gains two things: control over the cost structure, and control over the throughput. The chip is already showing benchmark performance that puts it in the same range as the fastest alternatives available[2].

The context here is a wider trend. Major AI vendors are moving toward vertical integration because token costs are becoming the primary constraint on profitability[3]. When you are selling access to a model by the million tokens, the hardware running it is the margin. OpenAI’s move is rational commercial strategy, and it mirrors what Google has done with its TPU programme and what Amazon is doing with Trainium.

For firms using AI, the direct consequence is that OpenAI’s API pricing will come under downward pressure. OpenAI has already previewed an Ultrafast mode delivering up to 750 tokens per second using Cerebras hardware[4], and batch processing discounts of around 50% are already available for non-time-sensitive workloads[5]. The Jalapeño chip, if it delivers at the benchmark performance claimed, extends that trajectory.

What this means for the cost case

The numbers that have been circulating put current cloud inference for capable models at roughly $0.88 per million tokens at the lower end of the market[6]. That sounds negligible until you model a firm running document summarisation, suitability letter drafting, and research synthesis across fifty client reviews a month. At that volume, per-token pricing is a real budget line.

The more relevant frame for most advice firms is not the absolute cost per token but the threshold at which private hosting becomes worth considering. One analysis puts the break-even for running a local model against a cloud API at around 5,000 complex queries or 50,000 simple queries per day[7]. Almost no IFA or small wealth manager runs anywhere near that volume. Which means that, for most firms below 50 advisers, the Jalapeño chip matters primarily as a market signal that cloud API costs will fall, not as a prompt to invest in on-premise hardware.

Local models shift the cost structure from per-token to fixed. You pay for compute once and marginal inference is near-zero[8]. That is genuinely attractive from a data-control perspective, particularly for firms handling client data with GDPR obligations. But the total cost of ownership for self-hosted AI is significantly higher than the compute alone, once maintenance, model updates, and support are included[9]. For most advice firms, the honest answer is that cloud, at falling prices, will remain the better option for the foreseeable future.

The data-control question this does not resolve

Cheaper cloud inference does not solve the question of where client data goes. OpenAI’s API, even with data-processing agreements in place, routes queries through OpenAI’s infrastructure. For many FCA-regulated firms, particularly those with Consumer Duty obligations around client data handling, that is a governance consideration that does not go away because the price drops.

The cost case for AI in advice firms is improving. The data-governance case requires a separate answer.

This is where the Jalapeño story intersects with a broader shift: the emergence of capable open-source local models as a genuine alternative. NVIDIA’s Nemotron release signals that open-source models on local infrastructure are becoming credible for professional services workloads[10], particularly structured tasks like document classification, compliance checks, and entity extraction. These are tasks that do not require frontier model capability but do require data to stay on-premise.

The practical framework that is emerging is hybrid. Route sensitive or structured queries locally, and send complex reasoning and generative tasks to a capable cloud model. The Jalapeño chip makes the cloud leg of that architecture cheaper and faster. It does not change the logic for the local leg.

The agent cost problem

One area where inference economics directly affects what a firm can build is agentic AI: systems that run continuously, check for triggers, process incoming documents, or monitor for compliance flags without being explicitly invoked each time.

These systems consume tokens constantly, not just when a user asks a question. At current pricing, running a background agent across a working week for a firm of thirty advisers is a real budget item. If Jalapeño-era pricing brings inference costs down by a meaningful fraction, some agent architectures that are currently marginal become viable.

The governance question for any agent system is not the cost, though. It is oversight. Agents that make decisions or draft regulated documents require human review before those outputs are acted on. Cheaper inference does not change that. Any deployment of agentic AI in suitability, KYC, or client-facing processes remains a human-in-the-loop requirement under the FCA’s operational resilience and Consumer Duty frameworks. The economics improve; the accountability does not shift.

My judgement

OpenAI’s move into proprietary inference silicon is significant, and it will reduce the cost of using its models. The Jalapeño chip is not a product advice firms will buy or configure. It is a supply-chain development that will show up as lower API prices and faster response times over the next twelve to eighteen months.

The firms most likely to benefit are those that have already decided to use OpenAI’s API for non-sensitive workflow automation and are held back primarily by cost. For them, waiting for pricing to fall before scaling volume is a reasonable position.

The firms that are holding back because of data-control concerns will not find that those concerns are addressed by a chip announcement. For them, the more relevant development is the maturation of capable open-source models that can run on-premise without sending client data anywhere.

The broader pattern I expect to hold: inference costs will continue to fall across the market as vertical integration matures, hardware efficiency improves, and competition between OpenAI, Google, and Amazon intensifies. Firms that treat AI cost as a fixed constraint are likely to find that it is not. Firms that treat data governance as a variable they can defer are likely to find that regulators disagree.

This article is for informational purposes only and does not constitute regulated financial advice or a compliance opinion. Consult a qualified compliance professional for advice specific to your firm.

This article does not constitute legal advice. Data protection obligations vary by circumstance and jurisdiction. Consult a qualified solicitor or data protection adviser for advice specific to your firm.

Sources

[1] van Riel, Z. (2026) ‘Cerebras on AWS Bedrock: Fastest AI Inference’, AI Engineer Blog. Available at: https://zenvanriel.com/ai-engineer-blog/cerebras-aws-bedrock-fastest-ai-inference

[2] TechCrunch (2026) ‘OpenAI’s Jalapeño chip is built for fast inference at scale, benchmarks show’, 25 August 2026. Available at: https://techcrunch.com/2026/08/25/openais-jalapeno-chip-is-built-for-fast-inference-at-scale-benchmarks-show/

[3] OpenAI (2026) ‘OpenAI and Broadcom: Jalapeño inference chip’, OpenAI Blog, 3 July 2026. Available at: https://openai.com/index/openai-broadcom-jalapeno-inference-chip

[4] OpenAI (2026) ‘Previewing Ultrafast’, OpenAI Blog, 15 August 2026. Available at: https://openai.com/index/previewing-ultrafast

[5] Digital Applied (2026) ‘Bulk LLM job engineering: batching, idempotency and QA’, Digital Applied Blog, 2 August 2026. Available at: https://www.digitalapplied.com/blog/bulk-llm-job-engineering-batching-idempotency-qa-2026

[6] Digital Applied (2026) ‘Local AI workstation economics: running costs vs cloud 2026’, Digital Applied Blog, 30 June 2026. Available at: https://www.digitalapplied.com/blog/local-ai-workstation-economics-running-costs-vs-cloud-2026

[7] van Riel, Z. (2026) ‘Local vs Cloud LLM: Complete Decision Guide for AI Engineers’, AI Engineer Blog, 15 August 2026. Available at: https://zenvanriel.com/ai-engineer-blog/local-vs-cloud-llm-decision-guide

[8] van Riel, Z. (2026) ‘Local AI for agencies: protecting client data’, AI Engineer Blog, 31 July 2026. Available at: https://zenvanriel.com/ai-engineer-blog/local-ai-for-agencies-protecting-client-data

[9] Mavik Labs and Spheron (2026) ‘Document AI automation for SMB back office: build vs buy’, Digital Applied. Available at: https://www.digitalapplied.com/blog/document-ai-automation-smb-back-office-build-vs-buy-2026

[10] NVIDIA (2026) ‘Local AI and open-source models: agents and Nemotron’, NVIDIA Blog, 24 August 2026. Available at: https://blogs.nvidia.com/blog/local-ai-open-source-models-agents-nemotron/