TL;DR: consider both information retrieval and generative capabilities as you build your AI architecture
I’ve been sitting on this idea for a couple of years now, dismissing it as something both patently obvious and gratuitously contrarian. Recent conversations about it with several very accomplished knowledge engineers compel me to share it now.
As the acronym implies, GAR turns RAG on its head. RAG (retrieval augmented generation) puts documents, knowledge bases, and other sources into AI workflows, typically vectorizing them as embedding models that LLMs can work with, to generate better-informed answers. The shortcomings in this architectural decision, I think, are 1) the loss of precision that comes as soon as you’re working with a vectorized version of the source material and 2) delegating knowledge work to a probabilistic system. RAG can definitely improve gen AI results, but it’s not the only possible solution to the issues (typically accuracy) that it addresses.
Rather than such an LLM-centric architecture, GAR (generative augmented retrieval) takes a retrieval-centric approach. It leverages the conversational interfaces and vast learning of LLMs to turn natural language and other inputs into queries (SQL, SPARQL, Cypher, etc.), the results of which are retrieved as-is from an authoritative knowledge source (relational database, graph database, knowledge graph, etc.). I’m aware of the technical challenges in implementing this, but working on those problems seems to me as worthy of our efforts as all of the cognitive resources we’ve invested over the last three years in learning how to write better prompts.
This idea first occurred to me when Denny Vrandečić observed in his keynote at the 2023 Knowledge Graph Conference that the cost of retrieving a known fact from a search engine was about a tenth the cost of generating an (often-inaccurate) answer with an LLM. Many others have since pointed out the huge difference in computing costs between generating language to describe a fact and retrieving the fact itself from a knowledge source.
To think about this another way, When you’ve taken the time to capture a fact, contextualize it in a schema, annotate it with metadata, and store it in an enterprise system, is the task of answering questions about that fact best handled by a probabilistic system?
This wasn’t a particularly innovative idea three years ago (1, 2, 3), and it’s certainly not novel now (1, 2, 3, 4, 5, 6). But I’m surprised that this kind of AI-architecture approach isn’t getting more attention, especially with the growing understanding of the limitations of generative AI as well as its substantial costs (computational, financial, environmental, and human).
I know from talking fairly extensively with AI engineers at FAANG companies and big enterprises that there are non-trivial obstacles to implementing GAR solutions. Still, as enterprises design their AI infrastructures and grapple with the trust and explainability issues that generative AI has created, it seems like a good idea to work with as big a toolkit as possible.
