OpenAI has built a custom inference chip with Broadcom, codenamed Jalapeno[1]. It’s designed to run large language models at scale, and it’s the point where OpenAI stopped being only a software company. Meta is doing the same with its MTIA chips[2], and Google has made its own TPUs for years. For the first time, Nvidia’s position as the main supplier of AI hardware is being contested.

None of this affects your advice firm today. It will in two or three years, and the reasons are worth a few minutes.

What a custom AI chip is

A custom chip is designed for one job rather than general computing. Nvidia’s GPUs are powerful but general: they run games, render video, train models and run inference. A custom inference chip does one thing, serving AI model output at scale, and does it more cheaply than a general-purpose GPU.

For the companies building them, the motive is simple. Running large models is very expensive, and owning the hardware takes a big cost off the books. For everyone else, the knock-on effect should be that the cost of running AI falls, probably by a lot, over the next few years.

Cheaper inference changes which automations pay for themselves

The cost of each AI request decides whether an automation makes commercial sense at your firm’s volume. At today’s GPU prices, some automated workflows cost more to run than they save in staff time, especially at low volume. As custom chips bring inference costs down, the break-even point moves, and workflows that are marginal today may be clearly worth it by 2028.

You can already see the effect in a smaller way. LLM routing, which sends each request to a cheaper model when the expensive one isn’t needed, can cut costs by around 57 per cent at scale, from roughly $42,000 to $18,000 per million queries according to one engineering blog’s figures[3]. Custom silicon should push both numbers lower. Firms that already track what they spend on AI and why will be the ones that notice the savings when they arrive.

The firms that have mapped their AI use before costs fall will capture the savings. The ones that haven’t will simply pay less for spending they still can’t see.

One company holding more of the stack

There’s a less comfortable side to this for firms working under UK GDPR.

When OpenAI runs its models on its own chips, the whole inference stack (chip, data centre, model weights, request processing) sits inside one vertically integrated company. That affects where your client data goes when it passes through an AI system, and how much you can see of it.

If you use an AI tool that processes clients’ personal data, UK GDPR requires a lawful basis, a data processing agreement with the provider, and a clear picture of where the data is processed and kept. Vertical integration doesn’t break any of that, but it does concentrate it. Instead of a chain of sub-processors you can check one by one, one company controls the lot. Whether that’s easier or harder to manage depends on how clearly the provider documents its data handling, and how carefully you’ve read it.

The chip itself isn’t the problem. The question is whether you know, for every AI tool your firm uses, where client data goes, how long it’s kept, whether it’s used to train models, and whether your data processing agreement still describes the infrastructure the provider runs on. Enterprise tiers of most major platforms commit contractually not to train on your data. Free and consumer accounts usually don’t, and that’s the tier anyone who signed up on their own initiative is likely to be on.

Where the money in AI is going

Jalapeno is part of a bigger pattern: AI infrastructure is being rebuilt by companies with the capital to own all of it. TechCrunch reported in June that Baseten, an AI inference startup, was raising $1.5 billion at a $13 billion valuation[4]. Serious money thinks the value sits in the compute layer, not the application layer most small firms deal with.

That matters for supplier risk. The application tools your firm uses (AI writing assistants, document review tools, client-facing chatbots) sit on infrastructure being consolidated by a handful of very large companies. Some of those application vendors will be bought, change direction or close as that happens.

My read

Cheaper AI is coming, and for small regulated firms it will make a lot of currently marginal automation worth doing. The cost of that is concentration: fewer companies holding more of your clients’ data, and a layer of application vendors that won’t all survive. My view is that the firms that benefit will be the ones that treat each AI tool as a supplier like any other, with a current DPA, a known data location and an exit plan, rather than the ones with the most ambitious AI strategy. Building a workflow that depends entirely on one application vendor is a business continuity risk, and it should be written down as one.

This article is for informational purposes only and does not constitute regulated financial advice or a compliance opinion. Consult a qualified compliance professional for advice specific to your firm.

Sources

[1] OpenAI, ‘OpenAI and Broadcom introduce Jalapeno, OpenAI’s first AI inference chip’, OpenAI, 25 June 2026. Available at: https://openai.com/index/openai-broadcom-jalapeno-inference-chip/

[2] Meta AI, ‘Scaling AI chips for billions with MTIA’, Meta AI Blog, 12 June 2026. Available at: https://ai.meta.com/blog/meta-mtia-scale-ai-chips-for-billions

[3] Zen van Riel, ‘Top 5 LLM routing techniques’ (2026), Zen van Riel, AI Engineer Blog. [Vendor-sourced. Cost figures cited as illustrative of routing economics at scale.]

[4] TechCrunch, ‘AI inference startup Baseten reportedly raising $1.5B months after its last mega-round’, TechCrunch, 18 June 2026. Available at: https://techcrunch.com/2026/06/18/ai-inference-startup-baseten-reportedly-raising-1-5b-months-after-its-last-mega-round/