“We want our own AI” is one of the most common things a firm says to someone in my line of work, and one of the most expensively misunderstood. It can mean four different projects, and the most expensive costs vastly more than the cheapest. Here’s what each one fixes, what it costs, and where most firms should stop.
| Rung | What it fixes | Typical cost | Most firms should |
|---|---|---|---|
| 1. Careful prompting | Tone, format, rules | A day of thought | Always start here |
| 2. RAG | “It doesn’t know our stuff” | A real build, modest running | Stop here, happily |
| 3. Fine-tuning | Consistent style/format at volume | Cheap runs, costly datasets | Use for specific niches |
| 4. From scratch | Almost nothing a business has | Lab budgets | Never |
Rung one: careful prompting
Careful prompting means a written brief with good examples, your terminology and your red lines, kept up to date like any other document. Most of the time this is what “our own AI” really needs: the model behaves your way because it’s told to, and you can check it does on every call. It costs about a day of thought. Every project further up starts here anyway, and plenty never need to go further. The internal assistant playbook shows how it’s done.
Rung two: RAG, your knowledge attached
Retrieval-augmented generation finds the relevant passages in your own documents and hands them to the model when it answers. Answers come from your knowledge base, they’re current, they can cite their source, and the system refuses when nothing matches. This is the right rung for what most firms want: a bot that knows your policies, products and history without making them up. It’s how grounded customer-facing chatbots are built, and keeping it up to date means filing a document, not retraining anything. The cost is a real build, modest running costs, and the discipline of keeping the source documents accurate.
Rung three: fine-tuning, behaviour adjusted
Fine-tuning retrains an existing model on your examples so that behaviour you used to prompt for becomes the default: house style held exactly, a fixed output format at high volume, or a narrow classification done cheaply by a small model. The frontier labs and cloud platforms sell it as a hosted service, and open-weight models can be fine-tuned with efficient adapter methods, giving you a specialised model you can run privately. Before you start, you need hundreds of clean examples, a test you trust, and evidence that RAG and prompting have failed, because fine-tuning isn’t for teaching facts. If you have all three, it’s a good buy. If you don’t, it’s an expensive way to avoid writing a better prompt.
Rung four: training from scratch
Building a model from raw data is work for AI labs with lab budgets, and for a business the case almost never adds up. An adapted open-weight model beats a small custom one on cost, capability and upkeep in nearly every commercial situation. If a supplier proposes it, ask what fine-tuning failed to deliver, and listen carefully to the answer.
Which rung fits your problem
Start from the symptom and work down:
- Wrong tone or format → rung one, then rung three at volume
- Doesn’t know our stuff, or makes it up → rung two, almost always
- Too expensive per call at scale → a smaller model plus rung three
- Data can’t leave the building → rung two or three on open weights you host
No symptom points to rung four.