Analyst firm Gartner thinks at least half of all generative AI projects "will overrun their budgeted costs due to poor architectural choices and lack of operational know-how," and most organizations that try to build custom models "will abandon their efforts due to costs, complexity and technical debt in their deployments."
Yes, and this matches decades software project failuresuccess studies from the Standish Group.
The success rate of projects has held steady forever:
That chart ends 11 years ago, but I haven't heard a lot of reports that the numbers have changed much over the years...case in point the opening quote!
In IT and software, very few projects are successful on the iron triangle of budget, schedule, and quality1.
You can take this to mean that expectations were unrealistic, or that there is just genuine failure. I favor the first. I'm more of a "I'm not late to this meeting, it was just scheduled at the wrong time" kind of guy.
There's some kind of Jevons Paradox thing here. Each time we optimize how we make software, we then take on more challenging and difficult tasks, likely causing setbacks again. To me, this is what accounts for the low success rate.
If you don't want to do new things and try to do better, you could get those success numbers up probably by just continuing to do what works.
But, you add in a new technology, and while you're figuring it out, it feels like you're failing.
Back in the digital transformation days, we'd be clever and say: you're not failing, you're learning.
🔗 Most generative AI and custom model projects will be a bust: Gartner
- I think of quality as more than "bug free." It also includes "does the software solve our problem, are the features done well," e.g., did we get something useful? ↩