By Laas NuryThe numbers on AI pilots are not encouraging. Gartner predicted in 2024 that at least 30% of generative AI projects would be abandoned after the proof of concept by the end of 2025, citing poor data quality, weak risk controls, rising costs and unclear business value. MIT's much-discussed 2025 study went further: only around 5% of custom AI tools in companies made it into production, with brittle workflows and poor fit with day-to-day operations among the main causes. The exact figure has been debated. The pattern has not.
The good news is that the reasons are predictable, and almost none of them is about the AI itself. This article sets out why pilots stall, how to design one that is meant to go live, and a go/no-go checklist for the step from demo to daily use.
Why pilots stall
1. The pilot was a demo, not a first version
It ran on a clean sample of data, outside the real systems, with someone copying the results into the ERP by hand. It looked good. But nothing of it can be put into production: the connections, the exception handling and the logging still have to be built, and that is most of the work.
2. Nobody owns the process
The pilot was run by IT or an innovation team. The people who run the process day to day watched from a distance. When the pilot ends, nobody on the business side is responsible for making it part of the work, so it quietly stops.
3. Success was never defined
"Let's see what AI can do" is not a goal. Without an agreed measure (accuracy on real cases, time saved, errors avoided), the end of a pilot becomes a discussion about impressions, and discussions about impressions rarely end in a budget.
4. The exceptions were left for later
The pilot handled the 80% of cases that are straightforward. The 20% that are not (the unreadable scan, the customer with a special price, the order that covers two deliveries) were set aside. In production those exceptions are the job. If there is no route for them, the people doing the work stop trusting the system.
5. The running costs appear at the end
During the pilot, nobody looked at what it costs to run at full volume, to host it where the data must stay, or to keep it maintained. When the numbers arrive, the business case no longer adds up.
Design the pilot as version one
A pilot that is meant to go live looks different from the start:
- It runs in the real process, on real incoming cases, next to the people who do the work today. First in the background, comparing its answers with theirs.
- It connects to the real systems, even if only read-only at first. The connection built for the pilot is the connection used in production.
- It has a business owner: the person responsible for the process, who decides on rules, exceptions and when it is good enough.
- It has an exception route from day one: every case the AI cannot handle goes to a named person, with the reason.
- It logs everything, so that at the end you can show what it did, not just what people remember.
- It has a running-cost estimate at expected volume, agreed before it starts.
This costs a little more up front. It saves the rebuild that most pilots need afterwards, which is exactly the step where most of them die.
What a pilot should prove
Agree these before it starts, and measure them the same way you measure the current process:
| Question | How to measure it |
|---|---|
| Does it get it right? | Accuracy on real cases, checked against what a person decided |
| Does it save time? | Time per case, including the time to check the AI's work |
| What do its mistakes cost? | The type and impact of errors, not just how many |
| Can the exceptions be handled? | Share of cases that go to a person, and how long they wait |
| Will people use it? | Whether the team relies on it by the end, or works around it |
| What does it cost to run? | Cost per case at full volume, including hosting and support |
The go/no-go checklist
At the end of the pilot, go live when you can say yes to all of these:
- The accuracy on real cases meets the level agreed at the start.
- Every exception has a route to a person, and the queue is manageable.
- The connections to the systems are the ones production will use.
- There is a full log of what the AI read, decided and changed.
- Anything that cannot be undone needs human approval.
- There is a way to stop it and roll back.
- The business owner signs off, not only IT.
- The running cost at full volume fits the business case.
- The people using it have had an explanation of how it works and where it goes wrong.
- It is in the AI register, with its level and owner.
If one of these is a no, the answer is not "stop". It is usually "two more weeks, on this specific point".
Going live in steps
Production does not have to mean full autonomy on day one. The safest route:
- Background: the AI runs alongside the team without acting, for a few weeks.
- Prepare: the AI prepares the work in the system; a person approves each case.
- Act within limits: the AI acts on its own below agreed limits and sends the rest to a person.
Each step is a decision based on the numbers from the step before. We describe this ladder in more detail in AI agents for business.
The short version
Pilots stall because they are built to impress, not to run. Build the first version in the real process, with the real systems, a business owner, an exception route and an agreed measure of success, and the step to production becomes a formality instead of a second project. That is also how we work: we stay until what we build runs in the daily operation, not until the demo is done.
