Why Most AI Projects Fail to Deliver Business Value

A thought leadership piece for Technology Magazine by Alan Jacobson, Chief Data & Analytics Officer at Alteryx

MIT research revealing that 95% of enterprise Gen AI projects fail to generate significant business value has become a frequently cited data point in the discussions around a potential “AI bubble” and growing scepticism about the effectiveness of LLMs.

Squaring this finding with the reality that billions of people now use LLMs in their everyday lives presents a paradox.

While LLMs have clearly transformed the consumer web browsing experience, that impact has yet to translate into broad enterprise value.

This gap should be seen as a lag rather than an absolute incompatibility, especially given that the factors holding back enterprise value are increasingly understood and diagnosable.

The woes of enterprise-scale transformation 

Anybody who’s ever been involved in rolling out a new technology in an enterprise environment will attest to the fact that enterprise transformation is rarely straightforward.

Back in 2017, for example, a Gartner analyst claimed that as many as 85% of big data projects at the time were failing. 

The underlying challenges remain consistent today and are especially relevant to enterprise-scale rollout of LLMs. Change is difficult in general, but it is particularly complex in enterprise environments where strong risk aversion can slow adoption.

Successful transformation often hinges on effective employee education and change management. 

Experience in execution also matters: the MIT study showed that Gen AI project success rates double when external experts are involved.

To navigate these hurdles, enterprise change initiatives must be guided by a deep understanding of the business context and a clear strategy for how technology can drive meaningful outcomes. This principle holds true for AI today. 

Looking beyond headlines to adapt AI’s rollout

The worst response enterprises could have to the MIT finding on project failures is to pull back from Gen AI or scale down investments. The real opportunity lies in the insights the report provides.

A key takeaway is that stalled Gen AI projects often stem less from the technology itself and more from workforce readiness-employees’ understanding of how and where to apply LLMs to improve workflows and outputs. 

The report also highlights the challenge of inherent variability in model outputs, which can make achieving consistent result difficult. 

The first point demonstrates how Gen AI rollout amounts to much more than putting the right technology into the right hands.

In my experience, effective initiatives are those that are complemented by education for the workforce on the fundamentals of working with data and the inner workings of LLMs.

This doesn’t mean upskilling every employee to become a data scientist – but it does require organisations to think beyond one-off training and technical step-by-step guidance to offer a mix of education on the fundamentals of working with data and AI, alongside practical training to apply these principles effectively. 

This approach fosters an internal culture of data and AI literacy where Gen AI is far more likely to be deployed by employees in a targeted manner rather than as a silver bullet solution for every problem.

I also fully agree with the MIT study’s emphasis on careful use case selection. Enterprises must align their strategies with the unique capabilities of LLMs, which differ fundamentally from traditional enterprise technologies. This includes breaking old habits shaped by working with “code”-based solutions, where outputs are repeatable and consistent. 

For example, we’re accustomed to writing a formula that calculates the sum of revenue across all regions; once verified, we can rerun it each month with confidence that it will yield the same result.

LLMs, however, behave differently. Verifying their initial output does not guarantee the same correct result on subsequent runs.

This illustrates a challenge that can affect many use cases. With the right expertise, the LLM can be configured to minimise variability, the use case can be adjusted to reduce risk or a more suitable use case can be chosen to improve the odds of success.

These complex considerations are why many enterprises are complementing internal training with the appointment of experienced change agents who possess deep knowledge of AI to guide the transformation process. 

Make or break

Enterprises that take a thoughtful, strategic approach to AI rollouts are best positioned to move beyond the hype and achieve meaningful impact. 

By doing what’s necessary to deploy Gen AI effectively, they can ensure they’re not left behind as applications mature across their industry.

A gap between technology investment and ROI is common with emerging innovations. 

In the case of LLMs, however, that gap is already beginning to close, thanks to the technology’s capabilities and the tangible results delivered by organisations that know how to leverage it effectively.

The challenge for enterprises serious about AI success is clear: strive to join the small but exceptionally successful group realising millions of dollars in ROI from LLM deployments, while avoiding the pitfalls that derail most projects.