Cerebras hardware already delivers around 2,100 tokens per second through cloud APIs[1], and OpenAI’s Jalapeño chip is designed to compete with that on OpenAI’s own infrastructure[2]. A tokens-per-second figure means little to most advice firms by itself. The economics behind it matter more.
In short, OpenAI has built its first in-house inference chip. It’s fast, built for running large models at scale, and meant to cut the cost of each query. If it performs as early benchmarks suggest, the per-token price of OpenAI’s API will fall. For advice firms still deciding whether to put AI into client processes, cheaper and faster inference shifts the sums, though it settles less than the headlines imply.
What the Jalapeño chip does
Inference is the step where a trained model produces an answer. Training is expensive and happens once. Inference happens every time someone sends a query, drafts a letter or runs a document through a pipeline, so at scale it’s where the cost builds up.
OpenAI’s chip is built for that step. Making its own silicon instead of buying from Nvidia or renting generic cloud GPUs gives OpenAI control over both the cost and the throughput. Early benchmarks put it in the same range as the fastest alternatives[2].
It’s part of a wider trend. AI vendors are moving to own more of their stack because token costs are becoming the main limit on profitability[3]. When you sell model access by the million tokens, the hardware is your margin. It’s a rational commercial move, and it follows what Google has done with TPUs and Amazon with Trainium.
For firms using AI, the direct effect is downward pressure on OpenAI’s API prices. OpenAI has already previewed an Ultrafast mode running up to 750 tokens per second on Cerebras hardware[4], and batch discounts of around 50% are already available for work that isn’t time-sensitive[5]. Jalapeño continues that direction, if it delivers.
What it means for the cost case
Figures in circulation put cloud inference for capable models at roughly $0.88 per million tokens at the cheap end of the market[6]. That sounds negligible until you model a firm running document summaries, suitability drafting and research across fifty client reviews a month. At that volume it’s a real budget line.
The more useful question for most advice firms is when private hosting starts to make sense. One analysis puts the break-even for a local model against a cloud API at around 5,000 complex or 50,000 simple queries a day[7]. Almost no IFA or small wealth manager gets near that. For most firms under 50 advisers, then, Jalapeño matters as a sign that cloud prices will fall, not as a reason to buy hardware.
Local models change the cost from per-token to fixed: you pay for the compute and each extra query costs almost nothing[8]. That’s attractive for data control, especially for firms with GDPR obligations on client data. But once maintenance, model updates and support are added, self-hosting costs a good deal more than the hardware alone[9]. For most advice firms, cloud at falling prices will stay the better choice for the foreseeable future. (Our comparison of AI providers covers the cloud options side by side.)
The data-control question it doesn’t answer
Cheaper inference doesn’t change where client data goes. Even with a data processing agreement in place, queries to OpenAI’s API run through OpenAI’s infrastructure. For an FCA-regulated firm with UK GDPR obligations, that governance question stays exactly where it was when the price drops.
The cost case for AI in advice firms is getting better. The data-governance case still needs its own answer.
That’s where Jalapeño meets a separate trend: capable open-source models running locally. NVIDIA’s Nemotron release suggests open models on local infrastructure are becoming workable for professional services[10], especially structured tasks like document classification, compliance checks and pulling names and figures out of documents. Those tasks don’t need frontier models, but they do need the data to stay in-house.
The approach taking shape is hybrid: run sensitive or structured work locally, and send complex reasoning and generation to a capable cloud model. Jalapeño makes the cloud half cheaper and faster. It doesn’t change the reasons for the local half.
The agent cost problem
Inference prices matter most for agentic AI: systems that run continuously, watching for triggers, processing incoming documents or monitoring for compliance flags without anyone asking each time.
Those systems use tokens all the time, not just when someone asks a question. At today’s prices, a background agent running across a working week for a thirty-adviser firm is a real cost. If inference gets meaningfully cheaper, some agent designs that are marginal now become worth building.
Cost isn’t the governance question for agents, though. Oversight is. Agents that make decisions or draft regulated documents need human review before their output is acted on, and cheaper inference changes none of that. In suitability, KYC and client-facing work, the firm stays accountable for what the agent produces.
My judgement
OpenAI’s move into its own inference chips matters, and it will bring the cost of its models down. It isn’t something advice firms will buy or configure. It’s a supply-chain change that will show up as lower API prices and faster responses over the next twelve to eighteen months.
The firms that gain most are the ones already using OpenAI’s API for non-sensitive automation and held back mainly by cost. For them, waiting for prices to fall before scaling up is reasonable. Firms held back by data-control concerns won’t find those concerns answered by a chip. Capable open models running in-house are the development that matters for them.
I expect inference costs to keep falling across the market as vendors own more of their hardware and competition between OpenAI, Google and Amazon gets fiercer. Firms that treat AI cost as fixed will find it isn’t. Firms that treat data governance as something to deal with later will find their regulator sees it differently.
This article is for informational purposes only and does not constitute regulated financial advice or a compliance opinion. Consult a qualified compliance professional for advice specific to your firm.
This article does not constitute legal advice. Data protection obligations vary by circumstance and jurisdiction. Consult a qualified solicitor or data protection adviser for advice specific to your firm.
Sources
[1] van Riel, Z. (2026) ‘Cerebras on AWS Bedrock: Fastest AI Inference’, AI Engineer Blog. Available at: https://zenvanriel.com/ai-engineer-blog/cerebras-aws-bedrock-fastest-ai-inference
[2] TechCrunch (2026) ‘OpenAI’s Jalapeño chip is built for fast inference at scale, benchmarks show’, 25 August 2026. Available at: https://techcrunch.com/2026/08/25/openais-jalapeno-chip-is-built-for-fast-inference-at-scale-benchmarks-show/
[3] OpenAI (2026) ‘OpenAI and Broadcom: Jalapeño inference chip’, OpenAI Blog, 3 July 2026. Available at: https://openai.com/index/openai-broadcom-jalapeno-inference-chip
[4] OpenAI (2026) ‘Previewing Ultrafast’, OpenAI Blog, 15 August 2026. Available at: https://openai.com/index/previewing-ultrafast
[5] Digital Applied (2026) ‘Bulk LLM job engineering: batching, idempotency and QA’, Digital Applied Blog, 2 August 2026. Available at: https://www.digitalapplied.com/blog/bulk-llm-job-engineering-batching-idempotency-qa-2026
[6] Digital Applied (2026) ‘Local AI workstation economics: running costs vs cloud 2026’, Digital Applied Blog, 30 June 2026. Available at: https://www.digitalapplied.com/blog/local-ai-workstation-economics-running-costs-vs-cloud-2026
[7] van Riel, Z. (2026) ‘Local vs Cloud LLM: Complete Decision Guide for AI Engineers’, AI Engineer Blog, 15 August 2026. Available at: https://zenvanriel.com/ai-engineer-blog/local-vs-cloud-llm-decision-guide
[8] van Riel, Z. (2026) ‘Local AI for agencies: protecting client data’, AI Engineer Blog, 31 July 2026. Available at: https://zenvanriel.com/ai-engineer-blog/local-ai-for-agencies-protecting-client-data
[9] Mavik Labs and Spheron (2026) ‘Document AI automation for SMB back office: build vs buy’, Digital Applied. Available at: https://www.digitalapplied.com/blog/document-ai-automation-smb-back-office-build-vs-buy-2026
[10] NVIDIA (2026) ‘Local AI and open-source models: agents and Nemotron’, NVIDIA Blog, 24 August 2026. Available at: https://blogs.nvidia.com/blog/local-ai-open-source-models-agents-nemotron/