Most AI projects fail for one reason: nobody defined, before the money moved, what success was supposed to look like.
I see this pattern constantly. Every wave of business technology I’ve worked through across a quarter-century of running IT projects has produced its share of confident spending and vague results. AI just raises the stakes. The pitch behind it is bigger than almost anything that’s come before but the proof asked for afterwards seems thinner.
The measurement gap AI exposes
If you push a business owner on what an AI project is actually meant to deliver, the answer tends to stay soft: time saved, productivity gained, or some version of “it’ll make us more efficient.” Push further and ask for one figure, tied to one process, checked before the work started and again after it finished, and the room goes quiet.
Every capital technology purchase I’ve reviewed across my career has needed this same discipline: agree the gain in advance, then check it against what actually happened once the thing is running. AI tends to get a pass on that discipline, purely because the technology still feels new enough to excuse skipping the usual rigour. Confident demos get nods around the boardroom table but months later, once someone finally asks whether a given project earned its keep, there’s often no baseline left to compare against, and no figure anyone actually agreed to be judged by.
The signs a project has skipped that step are consistent, and are worth checking your own pipeline against:
- Nobody checked what the process cost before the AI went live
- Success gets described in impressions, not figures
- No one is named as responsible for reporting back on the result
- The success metric changes partway through, right as the original one starts looking weak
The three numbers every AI project needs
Before any project gets budget, three figures need to exist.
First, a genuine baseline: what the process costs right now, expressed in hours, error rate, or whatever unit the pain actually shows up in.
Second, a real target: what that figure should become once the project completes, set as something specific rather than a general sense of “better.”
Third, a date: when the business will actually sit down and check the two against each other, rather than leaving the question open-ended.
If you can’t produce all three, there’s no project yet, only a piece of software with a price tag attached.
The three numbers change shape from one process to the next. A booking system’s version of this test looks nothing like an invoice-processing tool’s, and neither looks like a content-drafting assistant’s, but the underlying requirement never moves.
I hold every capital IT decision brought to me to the same standard, and have done so for my whole career. AI needs the same ordinary discipline any spend of this size would get, applied at the point of decision rather than months into delivery. Our IT strategy advice work exists precisely to build that alignment in before a contract gets signed.
Why the wrong person often grades the result
There’s a second problem sitting underneath the numbers: the question of who gets to decide whether the numbers were hit.
More often than not, the person reporting on an AI project’s success is the same person who pushed for it to get funded in the first place. In other words, the project’s champions tend to mark their own homework.
This is simply how most AI projects come together. Someone spots the opportunity, sits through the vendor pitches, wins the argument for budget, and ends up owning delivery. By the time anyone’s asking for results, the project is too far gone to back out of.
Compare that to how you’d treat a piece of capital equipment. Nobody would sign off on whether it paid for itself based solely on the word of whoever chose to buy it.
Yet AI pilots routinely get exactly that treatment, because the person who championed the tool is usually also the person closest to the numbers. The stakes are higher here than with most capital spend, because success with AI is genuinely harder to pin down than a server’s uptime or a piece of equipment’s output, which leaves more room for the same person to shape both the target and the verdict on whether it was met.
Because their own judgement is on the line alongside the project’s survival, this creates a huge temptation to, to put it kindly, leans toward the more flattering account.
The actual remedy is structural: separate the person who owns the number from the person who chose the vendor, so whoever signs off the result has no stake in the original decision.
Building that separation in costs almost nothing up front, and in my experience it’s far cheaper than the alternative, which usually only surfaces months later, when a number everyone trusted turns out not to hold up.
What the evidence says about where AI pays off
None of this is guesswork. MIT’s NANDA research initiative examined more than three hundred publicly disclosed enterprise AI projects worldwide and found that ninety-five percent of the organisations behind them saw no measurable return on investment (ROI) at all. The same research found that sales and marketing teams absorb most of the AI budget on offer, even though back-office automation, consistently the most overlooked candidate, produced the strongest returns.
A survey of UK business leaders from KPMG lands on a related finding from a different angle. Confidence in measuring AI’s productivity benefits runs high. That confidence collapses once the benefit being claimed is strategic rather than immediate, with barely one in seven leaders trusting their own figures on anything longer-term. Put the two studies side by side and a consistent gap appears. Businesses are reasonably good at measuring the obvious wins, and poor, sometimes entirely absent, at measuring the wins that actually decide whether a project was worth doing.
KPMG data shows that a majority of UK leaders say they’d carry on investing in AI even without a clean ROI figure to point to, treating it as a strategic bet rather than an entry on the profit and loss (P&L) statement.
That’s a defensible position for some businesses and a warning sign for others. This article’s aim is narrower than settling which camp is right: making sure whichever position a business holds is one it chose deliberately, rather than drifted into.
Building the discipline in before the money moves
There’s a further implication buried in the MIT findings, separate from the three-number test itself, and it concerns where AI budget actually lands, such as which function received the investment this year, and can anyone explain why that was the case.
- Was the decision driven by visibility, or by a genuine read on where the return was likely to be largest?
- Has anyone actually tested whether a duller, back-office process would outperform the one currently getting the demo and the budget?
On any capital project, budget allocation is the first thing I’d look at, well before anything else, and AI doesn’t warrant an exception to that habit. Agreeing the target figure ahead of spending is one check. Confirming the spend is actually landing where the evidence points it will deliver the best value is a separate, second check. Both are straightforward in principle, but both get skipped constantly anyway. Unfortunately, a persuasive demo beats a spreadsheet almost every time in the room where the decision gets made.
Strip away the AI framing and this becomes an ordinary question of buying discipline, the kind any business would apply to any significant technology purchase, at a moment when the sheer volume of AI pitches makes it unusually tempting to wave that discipline aside.
Our managed IT services team builds this kind of accountability into a client’s technology decisions as a matter of course, and our IT cost optimisation work exists specifically for the businesses that suspect, correctly, their AI spend isn’t landing where the evidence says it should.
New AI pitches, new demos, and new reasons to feel behind aren’t going away any time soon. The businesses that come out ahead will already know who’s writing down the three numbers, and who’s checking those numbers against someone other than whoever picked the vendor.
What would your business find if it asked those questions about whichever AI project is closest to getting signed off right now?
Frequently Asked Questions
1. How do we actually measure ROI on an AI project?
Agree three figures before the project gets signed off: what the process costs today, what it should cost once the AI is live, and the date you’ll check the two against each other. Without all three in place, there’s no defined outcome to measure against, only a piece of software with a budget attached.
2. Who should be responsible for reporting whether an AI project actually worked?
Not the person who championed the tool and won the budget for it, where that can be avoided. Because that person’s own judgement is bound up with the project’s, it’s structurally sounder to have someone with no stake in the original vendor decision sign off the final result.
3. Is it true that most AI projects fail to deliver a return?
MIT’s NANDA research initiative reviewed more than three hundred publicly disclosed enterprise AI projects worldwide and found that ninety-five percent of the organisations behind them saw no measurable return on investment. The same research found the strongest returns actually came from back-office automation, the function that typically receives the least attention and budget.
4. Should we still invest in AI if we can’t produce a clean ROI figure?
That depends on the business. KPMG’s own data shows a genuine split, with many UK leaders saying they’d continue investing in AI as a strategic bet even without a clear return figure to point to, and that’s a defensible position for some businesses. What matters is that it’s a deliberate choice rather than something drifted into by default.
5. Where should AI budget actually go?
Check whether the choice was driven by visibility, such as a customer-facing chatbot everyone can see in a demo, or by a genuine read on where the return is likely to be largest. MIT’s research found that sales and marketing functions absorb most AI budget despite back-office automation typically delivering stronger returns.
You must be logged in to post a comment.