An LLM is not an enterprise system of record
Structured retrieval provides exact facts. Vector RAG retrieves relevant knowledge. Ontology defines what enterprise concepts mean. A knowledge graph connects those concepts. Graph RAG retrieves the relationships relevant to the question. Deterministic rules protect critical business decisions. The LLM brings all of this context together and turns it into a useful customer-service response.
A large language model contains broad, learned knowledge from its training data. It does not automatically know today’s enterprise transactions, a specific customer’s contract terms, the latest policy revisions, an invoice’s payment status, or anything sitting in an internal SAP or CRM system. Ask it about a real invoice and, left alone, it will either say it doesn’t know, or, worse, guess.
Retrieval-Augmented Generation closes that gap: before the model answers, the system retrieves relevant knowledge and hands it to the model as context.
Different knowledge should be retrieved differently
Rank passages by similarity to the question rather than exact keyword match. A question about a return policy retrieves the passages of policy text closest in meaning, even if the exact words differ.
Semantic retrieval filtered first by structured metadata: distributor tier, region, or policy type, so the search only ranks the chunks that could actually apply.
Semantic retrieval plus a separate traditional keyword index, useful when exact terms (a SKU, a policy name) need to match as much as meaning does. This lab's single TF-IDF index already blends term overlap into its similarity score, but there is no separate keyword-index layer alongside it.
Retrieve exact information from structured sources, like an invoice amount, a quantity, or a date, by lookup, not by similarity.
Retrieve knowledge through relationships between entities: a distributor connects to a contract, which connects to a policy.
The LLM chooses and combines multiple retrieval mechanisms depending on the question, rather than always running the same fixed retrieval step.
| Remove this | What goes wrong |
|---|---|
| Structured retrieval | The model invents invoice amounts, dates, and quantities instead of looking them up. |
| Knowledge graph | The wrong tier's policy gets applied. Similarity picks a plausible-sounding policy, not the contracted one. |
| Rules engine | The model can approve a return that fails the eligibility rules, or state a refund amount it never actually calculated. |
| Vector RAG | The model paraphrases policy from training-data memory instead of citing the actual, current policy text. |
An ontology defines the important things in a business and how those things relate to one another. It gives business meaning to otherwise disconnected records.
In this lab: Dallas Power Distributors → Contract C-004 → Gold tier → Gold Distributor Return Policy, and AGM-100 → AGM Battery Warranty Policy. See it live on the Graph tab of the evidence panel.
This lab intentionally simplifies a real enterprise architecture. In production, the equivalent retrieval layer might reach into S/4HANA, Salesforce, SharePoint, a contracts repository, a data warehouse, and a real graph database.