The Rush to AI: Now the Hard Part

When OpenAI introduced ChatGPT in March 2023, the large language model (LLM) grabbed the attention of enterprises around the world. Generative AI was the crowning achievement of three decades of AI development, and senior executives didn’t want their company to look as if it were lagging behind. They wanted a new app or service that could highlight the company’s AI commitment. For the rest of the year, IT teams had to identify, prioritize, and choose which projects would be part of their company’s initial AI wave.

And now, the hard part: IT teams are undertaking the challenging work of actually developing useful, accurate, and attention-grabbing AI apps. At a number of recent roundtables sponsored by CIO, executives were talking about the difficulties and issues that arise with AI projects. And there are many.

From those discussions, some common sticking points for building and delivering AI solutions emerged:

• Hallucinations: Many CIOs are worried about what happens when AI is wrong. Most roundtable participants said that poor or incorrect responses are not unusual in their pilot programs, and therefore they are restricting AI systems’ ability to interact directly with customers. Instead, many AI systems are only assisting human staff, who serve as a firewall against giving bad answers or advice to customers.
• Data problems: IT executives knew that going into their AI projects, much of the data that would be used by any resulting AI app was not ready for that task. Data held by the enterprise tended to have different formats and much of it had to be extracted from legacy stores. AI often demands real-time data or at least timely data feeds, creating another problem. And data that was integrated from various sources had to work well together. As pilot projects progressed, additional data issues surfaced. Disparate datasets might have quite different timing, making it hard to deliver a single reality. The thorniest issues of all concerned are compliance, governance, and privacy. Keeping the systems closed is the best way to satisfy all those sorts of requirements, but that limits the amount of data available to inform the AI. And the privacy issue does not really have a clear answer right now because the regimes that produced the privacy statutes never anticipated anything like generative AI.
• Infrastructure: The AI solutions being developed in enterprises require much more costly computing than other cloud instances, causing notable sticker shock.

In short, IT executives are waking up to the realities of developing AI-enabled tools, but executive management is still putting pressure on them to deliver enterprise AI. The CIO roundtables make it clear that the enterprises that are best able to complete valid AI projects will have an important head start. But for that to happen, IT vendors will need to address those sticking points to help keep the momentum going.