In July 2024, Gartner predicted that at least 30% of generative AI (artificial intelligence) projects would be abandoned after proof of concept by the end of 2025. The reasons it gave were poor data quality, inadequate risk controls, escalating costs and unclear business value. None of those four is a problem with the model.
A year later, MIT's NANDA initiative published The GenAI Divide: State of AI in Business 2025. It was based on 150 interviews with leaders, a survey of 350 employees and an analysis of 300 public AI deployments. Its headline finding, widely reported, was that only about 5% of pilots were producing rapid revenue gains, while the rest showed little or no measurable effect on profit and loss.
Both studies have limits. The MIT sample is modest, and "measurable effect on profit and loss" is a demanding test to apply to a pilot. Gartner's figure is a forecast. But they agree on where the problem sits, and that agreement is more useful than either number.
What the studies say goes wrong
- The tool never joins the workflow. MIT's authors put the gap down to integration and a "learning gap" in organisations and tools, not to model quality. A pilot that runs beside the real process proves the technology works. It does not prove the business will change how it works.
- Budgets go where results are easiest to show, not where they are largest. According to the MIT report, more than half of generative AI budgets go to sales and marketing tools, yet the biggest returns it found were in back-office work, such as cutting spend on outsourcing and agencies.
- Building everything in-house. The report found that tools bought from specialist vendors, and partnerships, succeeded about two-thirds of the time. Internal builds succeeded about one-third as often.
- No one owns the risk. Gartner lists inadequate risk controls as one of four reasons projects are dropped. Without agreed controls, a pilot cannot be approved for wider use, however well it performs.
What this means for the next pilot
Choose the process before the tool. Name the process the pilot will change, the people who run it, and the measure that will move if it works. If that measure is cost, say which cost line, and agree how it will be counted before the pilot starts.
Plan the integration as part of the pilot, not after it. That means the data the tool needs, the system it has to write back to, and the step in the process it replaces. A pilot that skips these steps tends to produce a good demonstration and no change.
Settle the controls early. Decide who signs off the model's use, what a person must check, and how errors are caught and corrected. These are the questions a risk committee will ask before wider rollout, and answering them late is a common reason pilots sit unused.
Look harder at the back office. If the MIT findings hold, the clearest returns are in finance, procurement, claims and operations, where the cost base is known and the baseline is easy to measure.
The question to ask
Before approving another pilot, ask what will have changed in the business if it succeeds, and who will be responsible for making that change. If nobody can answer, the pilot is a technology test. That can be worth doing, but it should not be expected to show up in the accounts.
Sources
- Gartner prediction on generative AI projects abandoned after proof of concept, reported by Intelligent CIO, 5 August 2024
- Fortune, "MIT report: 95% of generative AI pilots at companies are failing", 18 August 2025 (coverage of MIT NANDA, The GenAI Divide: State of AI in Business 2025)
Related services and industries
Published by Midvale Consulting. Views are general in nature and are not advice for any specific situation.
