A Gartner, Inc. survey of 451 senior technology leaders in the second quarter of 2024, found that 57% of CIOs said they are tasked with leading an AI strategy in their enterprise. However, four emerging challenges are making it difficult for CIOs to deliver value with AI.
“Because of the relentless innovation happening in the tech vendor race, CIOs feel like they are always living the hype, while the reality of their AI outcomes race – how tough it is to get value – makes it feel like they are also in the trough,” said Mary Mesaglio, Distinguished VP Analyst at Gartner.
“However, CIOs can set the pace in their AI outcomes race,” said Hung LeHong, Distinguished VP Analyst and Gartner Fellow. “If you have modest AI ambitions, in an industry that isn’t being remastered by AI yet, you can afford to go at a more measured pace. This is an AI-steady pace. For those organizations with bigger AI ambitions, or in an industry that’s being reinvented by AI, the pace will be faster. This is an AI-accelerated pace. Whether you’re moving at an AI-steady or AI-accelerated pace, you have to deliver value and outcomes.”
During the opening keynote of Gartner IT Symposium/Xpo, which is taking place here through Thursday, Gartner analysts explained to the audience of over 8,000 CIOs and IT executives how to overcome four emerging challenges to deliver value from AI safely and at scale.
The Business Benefits of Using AI Don’t Always Materialize
To generate business value with generative AI (GenAI), people must consistently use GenAI tools in their workflow. In a second quarter 2024 Gartner survey of over 5,000 digital workers in the U.S., UK, India, Australia and China, employees said that they saved an average of 3.6 hours per week by using GenAI. But not all employees get the same degree of benefit from using GenAI.
“Here’s the real challenge with AI productivity,” said LeHong. “Productivity gains from GenAI are not equally distributed. Gains vary by employee, not just because of their personal interest and levels of adoptions, but according to complexity of job and level of experience.”
AI-accelerated organizations are also looking at benefits beyond productivity – operations and process-level improvements, such as automating key business processes or redesigning roles to work with chatbots; and business-level, game-changing improvements, such as outcomes that create new revenue streams or redesign the enterprise value proposition.
“In these cases, CIOs should manage AI benefits like a portfolio. Determine the size of your bet in each benefit area, and manage risks and rewards across this portfolio,” said Mesaglio.
The Cost of AI Can Quickly Spiral Out of Control
More than 90% of CIOs said that managing cost limits their ability to get value from AI for their enterprise, according to a Gartner survey of over 300 CIOs in June and July 2024. In fact, Gartner believes that cost is as big an AI risk as security or hallucinations.
If CIOs don’t understand how their GenAI costs scale, Gartner estimates that they could make a 500%-1,000% error in their cost calculations.
“As a CIO, you need to understand your AI bill,” said LeHong. “You must understand the cost components and pricing model options, and you need to know how to reduce these costs and negotiate with vendors. CIOs should create proofs of concept that test how costs will scale, not just how the technology works.”
With AI and data proliferating everywhere in the enterprise, AI and data are no longer centralized assets that IT directly controls. The Gartner survey of over 300 CIOs found that on average, only 35% of their AI capabilities will be built by their IT teams. This means that new approaches are needed to manage and protect data access and govern AI inputs and outputs and safely deliver AI value.
“This is where the concept of a ‘tech sandwich’ comes in,” said LeHong in describing the AI tech stack of the future (see Figure 1). “On the bottom of the sandwich is all the data and AI from IT, typically centralized. On the top is all the data and AI coming from everywhere, typically decentralized. And the middle contains the trust, risk, and security management (TRiSM) technologies that make it all safe. It’s what you need to create to accommodate AI and data coming from everywhere.”
Figure 1: Gartner AI Technology Sandwich
Source: Gartner (October 2024)
Using AI Can Both Positively and Negatively impact Employees’ Performance and Well-Being
Some employees may feel a strong affinity for AI. Others may feel threatened or resentful. These intense reactions to AI can lead to unintended behavioral outcomes that negatively impact employees’ work performance, such as jealousy of those using AI and overdependence on AI tools.
However, few organizations are actively managing these behavioral outcomes. In the June/July Gartner survey, only 20% of CIOs said they focus on mitigating potential negative impacts of GenAI on employee well-being.
“Most enterprises aren’t curious enough about how AI makes their employees feel. This matters because AI can lead to all sorts of unintended behavioral outcomes,” said Mesaglio. “The critical point is that if you use change management to manage this, be intentional about who owns which behavioral outcomes. Organizations must manage behavioral outcomes with the same rigor as technology and business outcomes.”