When generative AI first emerged, many wondered whether it would kill internet search as we know it. The large language models (LLM) behind the technology have revolutionized how we interact with information, making it easier to ask questions in a conversational and intuitive way and retrieve answers. However, search is still here and, in many ways, more important than ever. So why hasn’t generative AI supplanted it?
As the initial excitement about generative AI faded, it became clear that LLMs possess limitations and risks that would make any enterprise hesitate to use them. Generative AI tends to hallucinate, confidently responding to queries with false answers. In response to a query, an LLM excels at predicting the next best word or sentence according to its training data. LLMs are adept at mimicking human language, but they lack the ability to verify the accuracy of their responses.
Another challenge is relevance and timeliness. Though trained on vast amounts of data, generative models are limited by their cut-off date, meaning they can only generate responses based on outdated information. This can lead to outdated responses and introduces risks of inaccuracy, bias, and privacy violations. For example, some of these tools have been trained on publicly available data, which may not be compliant with regulations such as GDPR.
Then there is the limitation of company data. An LLM’s training does not include a company’s proprietary information and as a result does not draw from it when responding to queries. Enterprises produce new information at a rate far exceeding the speed of AI model training. Training models takes a lot of time and money, and most businesses lack the time and resources to train models every time they need to refresh their content and information. But in enterprise use cases, up-to-date information is essential for customer interactions and employee enablement.
Enterprises now recognize that generative AI’s effectiveness depends on the accuracy and relevance of the information it retrieves—making search a vital component.
While many businesses are eager to make generative AI work for them, enterprises consistently report that retrieval is the most challenging aspect of this. Enterprises face big obstacles in retrieval, such as finding the right data, outdated content, and fragmented sources.
Recently, retrieval-augmented generation (RAG) has emerged as the most effective solution to these challenges in enterprise AI. RAG combines a retrieval system with a generative model, improving the relevance and accuracy of responses by sourcing up-to-date information from reliable sources. Grounding the model in a reliable external source means an AI model can generate accurate answers with less risk of hallucinating.
With a RAG approach, the system’s effectiveness depends more on the quality of the underlying retrieval infrastructure. The model draws from a source of truth provided by the business in order to respond to queries. A search system identifies and retrieves the most relevant information to the query and user from an external source, such as a knowledge base. The system then sends the query with the additional information to the LLM, which generates the best answer to the question.
Search is the critical component to the retrieval part of RAG, providing multiple advantages, such as greater control over the LLM’s output, enhanced accuracy and relevance, reduced hallucinations, more up-to-date information, source citations, and secured content access—all while being more cost-effective than retraining or fine-tuning LLMs with new information.
Moreover, the same search system can be used within an enterprise to provide information to all generative AI applications, such as generative answering, chatbots, agents, or copilots. No need to copy the data everywhere—all these applications can use the search system to access the whole enterprise knowledge effectively and securely to generate accurate, relevant, and up-to-date answers.
The introduction of generative AI into the mainstream has been a true breakthrough. Enterprises will want to rely on the strength of the generative model, which is its understanding and mastery of language, but they should not depend on its knowledge. Search is the critical component to accessing relevant, timely information that grounds generative AI to produce more accurate responses.
Rather than killing search, generative AI has made it even more essential. Search will remain necessary to making generative AI useful to enterprises, albeit in an evolving form, such as expanding increasingly into different modalities of images, video, and audio.
Discover how a robust search foundation can elevate your AI strategy. Connect with Coveo to explore how quality retrieval can enhance your enterprise’s generative AI success.
Author: Sebastien Paquet, VP of Machine Learning, Coveo
About the author:
Sébastien Paquet is VP of ML at Coveo. He obtained his PhD in Artificial Intelligence before working as a computer consultant for eight years, mainly on research and development projects in the field of military intelligence. He worked on several types of technologies related to knowledge management and decision making, natural language processing, machine learning, operations research and automated reasoning.
Since 2014, Paquet has been leading the machine learning team at Coveo, developing algorithms to improve information retrieval, recommendations and personalization. His recent interests and work are in the areas of machine learning, large-scale usage data analysis and natural language processing.