OpenAI has built a custom inference chip in partnership with Broadcom, codenamed Jalapeno[1]. It is designed specifically to run large language models at scale, and it marks the moment OpenAI stopped being purely a software company and started becoming a hardware company too. Meta is doing the same thing with its MTIA chips[2]. Google has been building its own TPUs for years. Nvidia’s position as the sole infrastructure supplier to the AI industry is, for the first time, genuinely contested.

None of this matters to your advice firm today. But it will matter in two to three years, and understanding why is worth fifteen minutes of your time.

What custom silicon actually is

Custom silicon means a chip designed for one specific job rather than general-purpose computation. Nvidia’s GPUs are powerful but general: they can run games, render video, train AI models, and run inference. A custom inference chip does one thing: serve AI model outputs at scale, and it does so more cheaply and efficiently than a general-purpose GPU.

For the companies building these chips, the motive is straightforward: running large AI models is extraordinarily expensive, and owning the compute layer removes a significant cost from the income statement. For the rest of the industry, the downstream effect is that AI inference costs will fall, probably materially, over the next few years.

Why falling inference costs matter to a firm your size

The cost of running an AI query is what determines whether AI automation makes commercial sense at the volume your firm operates. At current GPU pricing, some automated workflows cost more to run than they save in staff time, particularly at low volumes. As custom silicon drives inference costs down, the breakeven point shifts. Workflows that are marginally uneconomic today may be straightforwardly cost-effective by 2028.

There is a concrete illustration of this dynamic already. LLM routing, directing queries to lower-cost models where a high-capability model is not needed, can reduce per-query costs by around 57%, from roughly $42,000 to $18,000 per million queries at scale[3]. Custom silicon will push both of those numbers down further. The firms that have instrumented their AI use now, and know what they are spending and why, will be best positioned to capture those savings when they arrive.

The firms that have mapped their AI use before costs fall will capture the savings. The firms that haven’t will simply pay less for the same opaque spend.

What it means for vendor concentration and data handling

There is a less comfortable implication that is worth stating plainly, particularly for firms operating under UK GDPR.

When OpenAI runs its models on its own custom silicon, the entire inference stack (the chip, the data centre, the model weights, the query processing) sits within a single vertically integrated company. That has real consequences for where your client data goes when it passes through an AI system, and for how much visibility you have over it.

Under UK GDPR, if you are using an AI tool that processes personal data about your clients, you need a lawful basis, a data processing agreement with the provider, and a clear understanding of where that data is processed and retained. Vertical integration at the infrastructure layer does not break any of those obligations, but it does concentrate them. Instead of a chain of sub-processors you can audit separately, you have one entity that controls the full stack. Whether that makes things simpler or harder to manage depends on how clearly the provider documents its data handling, and how thoroughly you have read that documentation.

The practical question for your firm is not whether OpenAI building a chip is a problem. It is whether you know, for each AI tool your firm currently uses: where client data goes, how long it is retained, whether it is used for model training, and whether your data processing agreement reflects the actual infrastructure the provider now operates on.

Most small advice firms I speak to cannot answer all four of those questions confidently. That is the gap worth closing.

What to do with this, right now

The custom silicon story will develop over years. The data governance work has a shorter horizon.

First, audit your current AI tool use. List every AI tool anyone in your firm uses, even informally. Include ChatGPT, Copilot, dictation tools, any AI-assisted CRM features, and anything your paraplanners or administrators have started using on their own initiative. You cannot govern what you have not mapped.

Second, check your data processing agreements. For any tool that touches client personal data, confirm you have a current DPA in place that names the provider as a data processor, specifies the processing purpose, and covers the sub-processors they use. If your DPA was signed two years ago and the provider has since moved to new infrastructure, it may not reflect current reality.

Third, apply a simple test before adding any new AI tool. Does this tool process personal data about clients? If yes: what is the lawful basis, where is the data processed, and is there a DPA? These are not complicated questions, but they need to be asked before the tool is in use, not after.

Fourth, distinguish between tools that use your data for training and tools that do not. Enterprise tiers of most major platforms offer a contractual commitment that your data will not be used to train the underlying model. Consumer-tier or free accounts typically do not carry that protection. Make sure the people in your firm who are using AI tools are on the right tier.

None of this requires a formal compliance project. It requires honesty and about a day’s work to get through properly.

The longer arc

The Jalapeno chip is a signal of something bigger: AI infrastructure is being rebuilt at scale by companies with the capital to own it end-to-end. Baseten, an AI inference startup, recently raised $1.5 billion at a $13 billion valuation[4], a figure that illustrates where serious money thinks the structural value in AI sits. It is not in the application layer that most SMEs interact with. It is in the compute layer underneath.

That matters for vendor risk. The application-layer tools your firm uses (the AI writing assistants, the document review tools, the client-facing chatbots) are built on top of infrastructure that is being consolidated by a small number of very large players. Some of those application vendors will be acquired, pivoted, or discontinued as that consolidation continues. Building workflows that depend critically on a single application-layer vendor is a risk that deserves at least the same attention as any other supplier concentration risk in your business continuity planning.

The firms that will handle this well are not the ones that have the most sophisticated AI strategy. They are the ones that have taken the time to understand what they are actually using, and to make sure those tools are governed with the same care they apply to any other part of their regulated business.

If you want to think through where your firm’s AI use currently sits relative to that standard, a discovery call with Cordrey Consulting is a reasonable place to start.

This article is for informational purposes only and does not constitute regulated financial advice or a compliance opinion. Consult a qualified compliance professional for advice specific to your firm.

Sources

[1] OpenAI, ‘OpenAI and Broadcom introduce Jalapeno, OpenAI’s first AI inference chip’, OpenAI, 25 June 2026. Available at: https://openai.com/index/openai-broadcom-jalapeno-inference-chip/

[2] Meta AI, ‘Scaling AI chips for billions with MTIA’, Meta AI Blog, 12 June 2026. Available at: https://ai.meta.com/blog/meta-mtia-scale-ai-chips-for-billions

[3] Zen van Riel, ‘Top 5 LLM routing techniques’ (2026), Zen van Riel, AI Engineer Blog. [Vendor-sourced. Cost figures cited as illustrative of routing economics at scale.]

[4] TechCrunch, ‘AI inference startup Baseten reportedly raising $1.5B months after its last mega-round’, TechCrunch, 18 June 2026. Available at: https://techcrunch.com/2026/06/18/ai-inference-startup-baseten-reportedly-raising-1-5b-months-after-its-last-mega-round/