“We ran a pilot last year. Everyone said it was impressive. Nobody uses it.” It’s a familiar story in advice firms. The demo worked, the technology was fine, and the project is dead anyway.
Most of the time that isn’t a technology problem. It’s a sequencing problem, and it’s predictable once you know what to look for.
The pattern that kills AI pilots
A typical stalled pilot goes like this. A financial advice firm decides to use AI. Someone senior champions it and a working group forms. Before anything is built, the firm invests in the things that feel responsible: a governance framework, a data infrastructure review, an approval workflow, a vendor assessment process. Months go by. When the automation work finally starts, it has to satisfy a committee set up to evaluate something much bigger than what’s being proposed. The pilot gets built in a sandbox, real users arrive late, and the workflow it’s meant to improve carries on as before alongside it.
Then the pilot ends, the committee reviews it, the demo looks fine, and nobody takes the step of changing the actual workflow. The project stalls and someone quietly archives the project page.
The cause is that the pilot was built next to reality rather than inside it. Showing that something could work isn’t the same as showing it survives contact with real users, real edge cases and real time pressure. Plenty of things look promising in principle and fall over in a live operation.
A governance framework won’t save a pilot that was never part of a real workflow.
The real test of a project is whether people are still using it six months later. No amount of executive sponsorship makes people keep using something that doesn’t help them, and advisers already have more tools than they want. If the AI doesn’t make their day easier, they’ll drop it the moment the sponsor stops checking.
What UK regulators are watching
The regulatory picture doesn’t make this easier. The FCA and Bank of England’s joint statement this month signalled that AI governance for UK-regulated firms is moving from voluntary good practice towards something firmer, and it framed frontier AI as creating cyber vulnerabilities faster than defences can keep up. That’s no reason to stop. It is a reason to be precise about what you’re building and why.
The firms most at risk are the ones that built governance frameworks with no production use to govern. They end up with machinery for managing the risk of automation that doesn’t exist yet, and when the automation arrives, the approval load is so heavy that nobody can move fast enough to learn anything.
A three-part test for the first production build
The most underrated decision in any automation programme is what to automate first. Get it wrong and you can spend months learning nothing.
A first production build has to pass three tests.
Narrow enough to finish in six to eight weeks. Anything longer builds in the conditions for stalling. The point of the timeline is that it’s too short for the firm to reorganise around the pilot, so real decisions get made with incomplete information, which is what production is like.
Important enough that the firm would notice if it disappeared. Most firms skip this one, and it’s why pilots die. If the automation handles something the firm cares about, someone owns the outcome. If it handles something peripheral, nobody does. Suitability letter drafting, post-meeting notes and client data reconciliation before reviews are things a firm notices. A dashboard nobody opens isn’t.
Built into the real workflow. The pilot can’t sit in a sandbox while the normal process carries on. Real users have to use it, real data has to go through it, and real pressure has to test it. That’s the difference between a demo and a production system, and there’s no shortcut.
Where an advice firm should start
The L1/L2/L3 split helps here. Level 1 (education) covers what you can fix with a prompt or a template inside a tool you already pay for. Level 2 (integration) connects two or three tools you already have, using something like Make or Power Automate. Level 3 (custom build) is real engineering, and should wait until something has worked at Level 1 or 2.
A lot of firms that think they need Level 3 turn out to need Level 1 or 2. Before commissioning a build, ask whether a simpler version of the fix already exists in your current tools.
A sensible Level 2 first build for an advice firm might take a completed fact-find, pull the relevant client data from the back-office system and draft the structure of the suitability letter for a paraplanner to review and finish. Not a fully automated letter, but a structured draft that removes the blank-page work for each client. It can be done in six weeks, the firm would notice if it went, and it sits in the paraplanning workflow from day one.
Governance gets much easier once something like that is running. You’re no longer managing hypothetical risk. You’re managing a real system with real output, which gives you something concrete to audit, refine and expand.
My read
I don’t think the pilot-to-production gap is inevitable. It’s what you get when governance is designed before there’s anything to govern, and pilots are built where they can’t fail. Even the most mature AI systems still go wrong in ways that are hard to predict, so internal AI in an advice firm will too. The answer is to put something narrow into the real workflow early, keep a person reviewing its output, and record every correction. That record becomes the governance framework, one that fits what the firm actually runs rather than what it imagined it might. Firms that do it the other way round will keep producing impressive demos that nobody uses.