The Context Layer: What to Build, What to Skip, and Where to Start

TL;DR:

• If you connect a model straight to your warehouse, it gets the answer wrong about 80% of the time on real enterprise schemas. The fix is a context layer: the data architecture between your raw tables and the AI reading them.
• Build it in stages: a written inventory of the questions worth answering, then a few entity models, then gold models with a semantic layer on top. The inventory is the step most teams skip, and the one that matters most.
• Hold off on knowledge graphs, streaming, fine-tuned models, data meshes, and multi-agent orchestration until a real use case demands each. That is where most teams over-build.

Leadership wants AI agents that can answer questions about the business. The vendor pitch is that you connect a model to your warehouse and the answers start flowing. The production reality is that this approach produces wrong answers roughly 80% of the time on real enterprise schemas.

That number comes from Spider 2.0, the enterprise-grade text-to-SQL benchmark released at ICLR 2025. Top agentic approaches using frontier models solved only about 21% of tasks on schemas that look like what most companies have in production: hundreds of columns, ambiguous naming, business logic buried in transformation code. Promethium’s analysis of production deployments corroborates this, reporting 10 to 31% accuracy on real enterprise data versus the 85 to 90% that vendor demos suggest. Uber’s internal text-to-SQL system, one of the most mature in the world, showed only about 50% overlap with ground-truth tables before heavy investment in context engineering.

The gap between demo accuracy and production accuracy is explained almost entirely by context: the data architecture between your raw tables and the AI consuming them.

This is the problem the industry has started calling the “context layer,” a framing Benn Stancil popularized and Andreessen Horowitz elevated into an investment thesis. Their framing is correct: the first wave of enterprise AI agents failed because they lacked proper business context, and context needs to be treated as first-class architectural infrastructure rather than an afterthought.

The question that matters to the person writing the check is more specific: which components do I need, which can I skip, and in what order should I build them? That is what this post addresses.

Architecture follows from value mechanism

Which subset of context-layer components you build is downstream of which value mechanisms you are building toward. We covered the framework in A Map of Where AI Creates Value in Your Business: thirteen mechanism categories spanning internal value (how AI changes how you operate) and external value (how AI changes what you sell). If you have not done that mapping, do it before reading further. The rest of this post assumes you have identified two or three primary mechanisms and want to know which architectural components serve them.

The reason that mapping is the prerequisite: two companies with identical warehouses often need very different context-layer architectures. A B2B SaaS company whose primary mechanism is cognitive labor augmentation needs a semantic layer and gold models, not a knowledge graph or vector database. A professional services firm whose primary mechanism is knowledge capture needs retrieval over documents and projects, with the warehouse as a secondary surface. Starting with the architecture rather than the mechanism is how companies build expensive infrastructure that does not serve the use cases that would create the most value.

What the context layer contains

The context layer is the set of components that sit between raw data and the systems consuming it. Its job is to make data usable: queryable by machines, trustworthy for decisions, governed for compliance, and accessible to the right people at the right time.

The core components:

• Entity models: resolved definitions of what “customer,” “order,” “product,” and other core business objects mean across systems.
• Gold models: pre-joined, pre-aggregated structures that map to the questions your business asks most often.
• Semantic layer: named metrics and dimensions exposed through a stable query interface.
• Metadata catalog: what data exists, where, who owns it, and how fresh it is.
• Retrieval infrastructure: how AI agents and humans find and access the right data at the right time.
• Governance controls: who can see what, what audit trail exists, and what happens when something goes wrong.

Every major vendor offers some version of these components. Snowflake has Cortex Analyst backed by Semantic Views. Databricks has Genie backed by Unity Catalog Metric Views. MotherDuck has Agent Skills backed by metadata-rich tables and embedding columns. Omni has Blobby backed by its workbook-driven semantic model. dbt Labs has the Semantic Layer backed by MetricFlow. Cube, AtScale, and LookML each take a different architectural approach. The Open Semantic Interchange initiative, backed by Snowflake, dbt Labs, Salesforce, and dozens of other participants, is working to make semantic definitions portable across these tools.

Almost no company needs all of these components, and building them all at once is the most reliable way to get none of them working. The right sequencing depends on your specific situation. What follows is a staged framework for figuring that out.

Figure 1: Context Layer Decision Flow. The linear path is shared by every company; the branches diverge based on observed failure modes after Stage 2, not on ambition.

Stage 0: The inventory

Every successful deployment for which we have evidence started with someone doing the unglamorous work of writing down what they had and what they were trying to do. Amazon’s RRDA team, ClickHouse’s internal DWAINE, Uber’s QueryGPT, and LinkedIn’s SQL Bot all describe a scoping phase that preceded any technical build. Skipping this stage is the single most common failure pattern.

The inventory has six pieces:

1. A catalog of your data assets: if you have a data warehouse, this means every meaningful table, with row counts, refresh cadence, ownership, and a one-sentence description. If your data lives in source systems without a warehouse (Salesforce, HubSpot, QuickBooks, spreadsheets), the catalog is a list of those systems, what each contains, and how they relate to each other.
2. An audit of active reports and dashboards: which questions each one answers, who uses it, and how often. Half the reports in most companies’ inventories answer questions nobody is asking anymore.
3. A collection of 100 to 200 real questions people have asked about data: mine these from Slack, Jira, email, and meeting notes over the last quarter. They become your evaluation set, the ground truth for whether your AI agent is answering the questions that matter. Repeated questions have the highest ROI to automate.
4. A triage of those questions by current answering difficulty: which can someone answer in under ten minutes today, and which require deep investigation. The under-ten-minute questions are your first target.
5. A map of source systems, data flows, and identity resolution: “we have a dim_customers table” tells you little; “we have a dim_customers table built from Salesforce as the system of record, with NetSuite mapping handled via a coalesce on email and a fuzzy match on company name” tells you whether your entity foundation is real or aspirational. If you do not yet have a warehouse, this map matters more, not less: it documents which systems hold which pieces of the truth and where they overlap or contradict each other.
6. An accounting of data quality: if you have automated data tests, this is the failure rate. If you do not, the accounting is qualitative: how often does Finance disagree with Product about basic numbers, how often is a report pulled because someone noticed a mistake, and how often does someone say “I do not trust that number” without being able to explain why. If your reports are wrong twice a week, your AI agent will be wrong twice a week, with the additional problem that it presents the wrong answer confidently.

From those six pieces, extract four lists that will guide the rest of the build:

• The high-value questions list: the twenty questions an agent should be able to answer, ranked by how often they are asked and how much time they consume today. This is the evaluation set that determines whether your investment is working.
• The high-leverage decisions list: the ten decisions that, if automated or augmented at scale, would meaningfully change the trajectory of the business. Examples include pricing adjustments based on real-time competitive data, customer churn interventions triggered by usage patterns, and inventory rebalancing across locations based on demand signals. Identifying these early gives you a target for what the architecture ultimately needs to support.
• The guardrails list: the questions and actions an agent should refuse to answer or execute, or should answer only with explicit caveats. Examples include revenue recognition questions during an audit, headcount planning numbers that are not finalized, customer data with specific access controls, and any action that could create legal or financial liability if taken on the wrong entity.
• The target users list: who will ask these questions and what action they will take on the answer. An executive who glances at a summary dashboard has different accuracy requirements than an analyst who will build a model on the output.

The deliverable from Stage 0 is a written document, ideally five to fifteen pages, that makes these lists explicit. If you cannot articulate them, you are not ready to build anything yet. If this sounds like the kind of structured assessment your team does not have capacity to run, a data assessment engagement compresses the work with an external team.

The investment is two to six weeks for an internal team, depending on the size and complexity of your data environment. The dollar cost is effectively zero. The opportunity cost of skipping it is approximately the entire project budget.

Synthesis without a target is just summarization

A pattern is showing up in vendor pitches and AI-platform manifestos: put agents at the center of every system, give them access to everything, and let them autonomously curate organizational knowledge into a unified context store. One vendor pitches a repo of agent-generated entity briefs. Another pitches a markdown directory that compresses each project, customer, and product line into a paragraph. The pitch is appealing. The flaw is structural, not something a better model will fix.

The distinction that matters is between summarization and synthesis. Summarization takes a larger thing and makes it smaller; by definition it loses information, and the only question is which information. Synthesis takes a defined lens on a source and constructs a representation that preserves what is relevant to that lens. Synthesis is the operation worth investing in for AI consumption. But synthesis is only useful when the lens is real. Without a defined target (the question set that determines what counts as relevant), synthesis collapses back into summarization, and an agent-curated knowledge base becomes a smoothed-over caricature of the business that answers nothing well.

This is why the Stage 0 inventory is doing more than inventory work. The high-value questions list, the high-leverage decisions list, and the target users list are the lenses that decide what your context layer is for. They tell you which entities to model, which gold models to build, which metrics to define, and what shape your retrieval surface needs. If you cannot articulate the questions, no amount of autonomous curation will produce something useful; the pragmatic move is to leave the information in the source systems and let agents retrieve what they need at query time. If you can articulate the questions, you do not need autonomous curation. You need infrastructure shaped to those questions, which is what the rest of this post lays out.

BI first, then AI

Most companies should build BI on top of the context layer before AI. The humans who consume BI become the validators of whether the entity models, gold models, and metric definitions are trustworthy; by the time an agent goes live, the most contested metric definitions have already been negotiated and the most common questions have known shapes. Skipping the BI step means writing ground-truth answers from scratch and discovering, three months in, that Finance and Product disagree about what counts as revenue. For teams that must deploy AI without a BI step, build a benchmark of high-value questions paired with accepted answers before any modeling work, and treat that benchmark as the bar any new architectural component must clear.

Stage 1: Entity models that resolve ambiguity

Once you know what you are building toward, the next step is to build the entities your agent will reason about. This precedes the semantic layer. It is the warehouse modeling pattern that makes a semantic layer useful.

The premise, as Preset explains in their introduction to entity-centric data modeling, is that the agent should query things (Customer, Order, Product) rather than facts scattered across a star schema: one row per entity, all known attributes consolidated, with pre-computed derived attributes (lifetime value, current segment, days since last activity) attached.

Most companies should start with two to four entities, not their entire conceptual model. Pick the ones that show up most often in the target questions from your inventory. For a B2B SaaS company, that typically means Customer (spanning Salesforce, billing, product usage, and support data) and Subscription (with current state, change history, and renewal information). For an ecommerce company, it usually means Customer and Order. For a manufacturer, it might be Product, Plant, and Work Order. The point is to be ruthless about scope.

Each entity table should consolidate attributes from every source system that contributes to that entity, with explicit handling of cases where systems disagree. When billing and CRM hold different addresses for the same customer, or when support and product disagree on which subscription tier is current, pick a precedence rule (e.g., billing wins for financial fields, CRM wins for relationship fields) and document it. Stage 1 does not require fully resolved entities; it requires the precedence rules to be explicit so downstream agent errors trace back to a documented decision rather than an unknown reconciliation.

Each entity table should also carry IDs that link back to every source system (Salesforce account ID, NetSuite customer ID, Stripe customer ID, Zendesk organization ID). Those IDs become critical in later stages, when the agent needs to hand off from the data warehouse to an action in a system of record.

With well-modeled entities and rich documentation, single-entity lookups work reliably at Stage 1. “What is Acme Corp’s current MRR?” works because there is one Customer entity with MRR pre-computed. Two-entity joins work when the relationship is obvious. Multi-entity reasoning, deep temporal analysis, and conceptual questions about metrics (“what counts as active?”) still fail, and that is expected.

The investment is four to eight weeks for a small data team. The lift available at Stage 1 is significant when the modeling work is done carefully: LinkedIn’s published research found that schema-only access to well-modeled tables produced correct-or-close-to-correct text-to-SQL responses 9% of the time, but adding rich metadata (descriptions, sample values, examples) on top moved that figure to 48%.

Stage 2: Gold models and the semantic layer

Gold models are where entity tables get combined into structures that answer the questions on your inventory list. If your inventory shows that “MRR by segment by month” is a question your finance team asks twelve times a quarter, you build a gold model with MRR pre-computed by segment by month, with every relevant dimension already attached. The agent selects from a model that is already shaped like the answer, rather than generating SQL that joins five tables and aggregates on the fly.

Three observations from companies that have done this well:

1. Most questions concentrate around five to ten gold models, not fifty. The long tail exists, but the volume of asks is heavily weighted toward a small number of analytical patterns: a revenue model, a customer activity model, a product usage model, a sales pipeline model, and a few cohort and retention structures. Build those well before expanding.

2. Turn the Stage 1 IDs into deep URLs and carry them on the gold models. The gold model that summarizes customer account health should expose the Salesforce account URL, the Zendesk organization URL, and the billing system customer URL as columns. When the agent answers a question and the user wants to take action, the action lives in a source system; deep URLs eliminate a handoff that would otherwise require a separate integration layer.

3. This is where the semantic layer earns its place. A gold model is a denormalized table; a semantic layer turns its columns and aggregations into named, governed metrics and dimensions that the agent can call deterministically. Without the semantic layer, every question becomes a generative-SQL gamble. With it, the failure mode changes from “wrong number” to “I cannot answer that,” which is the difference between a system you can put in front of executives and one you cannot.

The dbt Labs benchmark from April 2026 is the cleanest evidence. With Claude Sonnet 4.6 querying through the dbt Semantic Layer, accuracy on covered questions hit 98.2%. With the same model doing text-to-SQL on the same underlying tables, accuracy was 90% on covered questions but only 64.5% on the full question set. The failure-mode difference matters more than the accuracy gap: text-to-SQL returned plausible wrong numbers. The Semantic Layer returned correct answers or refused to answer. For anything going to a board deck, an auditor, or a customer dashboard, that distinction is everything. We covered why that distinction matters at the infrastructure level in a previous post.

Google’s internal testing found a similar pattern: Looker’s semantic layer reduced errors in generative AI natural-language queries by as much as two-thirds. Snowflake’s Cortex Analyst reports roughly 2x the accuracy of single-shot GPT-4o when grounded in semantic views. The pattern is consistent across vendors: the semantic layer is what turns “technically accessible” into “reliably usable.”

The semantic layer tool choice matters less than its existence. Snowflake Semantic Views for Snowflake-native shops. dbt Semantic Layer for portability. Cube or AtScale for multi-platform or embedded use cases. LookML if you are already in the Google ecosystem. The bigger cost is organizational, not technical. The fight to get to one canonical definition per metric is harder than the YAML configuration.

What the agent can reliably do at Stage 2: answer the questions on your inventory list with high accuracy and consistent metric definitions. Finance, Product, and Sales all see the same number for revenue because revenue is defined once. The agent can hand off to systems of record through linked URLs and IDs. This is the threshold where you can put the agent in front of business users.

What still fails: questions the semantic layer does not cover, questions about entities the agent cannot disambiguate (“which Acme are you asking about?”), and questions whose answers live in unstructured data.

The investment is two to four months for a small team. Most of that time goes to organizational alignment on metric definitions, not to the technical build.

Stage 3: Where the path branches

Up to this point, every company should follow roughly the same path. After Stage 2, the right architecture depends on what value you are creating. Four branches cover the space. Most companies will eventually need elements from more than one. The point is to pick the one that returns value fastest and build it first.

Branch A: Internal analytical agent. You are a mid-market company with a data team, or working with an extended data team, and your business users want to ask questions of the warehouse, build charts, or assemble shareable data apps without filing tickets. Tools like MotherDuck Dives let analysts and even business users move from a question to a deployable, AI-assisted data app in a single environment. The questions are mostly about metrics, trends, and business performance. You do not need the agent to take actions, pull from documents, or serve customers directly.

This branch is the simplest and the most common. Stage 2 is largely your destination. Invest in growing gold model coverage as new questions emerge. Invest in evaluation infrastructure so you can measure accuracy and improve it over time. Invest in user feedback loops so verified queries become few-shot examples for the next generation of asks. Resist the temptation to build anything more complex.

The integration path for this branch runs through MCP. As of April 2026, every major warehouse exposes an MCP server that lets agents query data through a governed interface: Snowflake’s managed MCP server, Databricks via Unity Catalog, MotherDuck’s MCP and Agent Skills, and community-maintained servers for BigQuery and PostgreSQL. The semantic layer sits on top; MCP is how the agent reaches it.

A SaaS company on this branch builds a Slack bot or a chat widget inside their BI tool, gets to 80% user-rated satisfaction within six months, and stops. ClickHouse’s internal DWAINE deployment follows this pattern: it handles roughly 70% of warehouse questions and reduced analyst workload by 50 to 70%.

The trap is over-engineering. Every vendor in the space wants to sell you a knowledge graph, a federated context layer, or a multi-agent orchestration platform. For an internal analytical agent serving a single domain or two, you almost certainly do not need any of that. Databricks’ own solutions architects recommend starting with 5 to 10 tables and fewer than 50 columns per Genie Space. That constraint is the point: tight scope is what makes agent accuracy achievable.

Branch B: Bridging structured and unstructured data. You are a company where most questions cannot be fully answered from the warehouse alone. A professional services firm CEO asks “how is the Acme renewal going” and the answer requires the warehouse (current MRR, usage trends, support volume), the CRM (deal stage, last activity, AE notes), the ticketing system (open issues, recent escalations), and possibly the contract itself (renewal terms, pricing protections).

This is where most B2B companies and customer-facing organizations end up, and it is where companies most commonly get the architecture wrong.

Three architectural options exist, and the cost differences between them are significant:

1. Relational database search: if your unstructured data is already summarized or referenced in your warehouse or CRM (meeting notes in Salesforce, ticket summaries in Zendesk, contract metadata in your CLM), you can search those fields without any dedicated vector infrastructure. PostgreSQL’s tsvector, MotherDuck’s full-text search and embedding columns, the FTS capabilities now built into Snowflake and BigQuery, or even a well-indexed LIKE query against a summary field can cover a surprising number of “find me the relevant context” questions. This is the right starting point when most of your unstructured data has a clean structured anchor in the warehouse.

2. Metadata-link architecture: your gold models carry not just metric values but also URLs and IDs that point at the systems of record. The customer health gold model contains the Salesforce account ID, the most recent Gong meeting URL, the open Zendesk ticket IDs, and the contract document ID. The agent gets the structured answer from the warehouse and uses those metadata links to fetch unstructured context just-in-time via MCP. This works when the unstructured data is anchored to a structured entity, which is true for most CRM, ticketing, and contract questions.

3. Full search-based architecture: the agent has access to multiple retrieval systems (warehouse query, CRM search, ticket search, document search) and orchestrates across them at query time. This is necessary when the question is fundamentally about discovering information without a structured anchor, like “what are customers saying about our pricing in support tickets this quarter?” That kind of question requires hybrid retrieval: vector embeddings, BM25 keyword matching, and cross-encoder reranking.

In practice, most companies on this branch end up with all three. Relational search handles quick context lookups. Metadata links cover the core 80% of questions where the warehouse has the anchor. Full hybrid retrieval is reserved for the long tail where no anchor exists. The mistake is jumping straight to the third option, which adds six-figure annual costs (whether through dedicated search infrastructure like vector databases and rerankers, or through additional warehouse compute if you run search in-warehouse) and meaningful operational overhead that most use cases do not require.

The metadata-link approach can be illustrated with a biotech example. A clinical operations team needs to understand trial enrollment across sites. The gold model carries enrollment counts, screen failure rates, and site activation dates from the warehouse, plus links to the Clinical Trial Management System (CTMS) site record, the most recent monitoring visit report, and the relevant protocol amendment. The agent answers the structured question from the warehouse and links to the unstructured context for follow-up. No vector database is required.

Branch C: Customer-facing embedded analytics. You are a SaaS company building analytics into your product, and your customers need to ask questions of their own data.

Before building an agent for this, make the case that you cannot use an embedded dashboard or an off-the-shelf tool. If your customers need a fixed set of metrics presented clearly with filtering and drill-down, an embedded BI platform is simpler, cheaper, and more reliable. Tools like Omni, Metabase Embedded, and Lightdash Embedded handle multi-tenancy, caching, and role-based access out of the box, and products like Omni’s Blobby can summarize dashboard data in natural language without a custom agent. For most customer-facing analytics use cases, this is the right answer.

An agent earns its place in customer-facing analytics only when the question space is too wide for pre-built dashboards to cover, when customers need to ask novel questions that a dashboard designer could not anticipate, or when the answers require synthesizing data across multiple views in ways that are specific to the customer’s context. A marketplace platform where sellers need to ask “why did my conversion rate drop last week compared to similar sellers in my category” is a different problem than “show me my sales by month,” and the first question is where an agent adds value that a dashboard cannot.

Three architectural requirements that internal agents can defer become non-negotiable on this branch:

1. Tenant isolation enforced at the data layer, not through prompt instructions: the system querying data on behalf of Customer A must be architecturally prevented from accessing Customer B’s data, the same way a row-level security policy in a database prevents one user from reading another’s rows. Cube’s trusted-proxy architecture and Looker’s access filters enforce this at the query layer. Anything that depends on prompt-level tenant filtering, telling the model “only return results for customer X,” is a data breach waiting to happen.

2. Tight latency budgets: internal agents can take ten seconds to answer because analysts are willing to wait. Customer-facing agents need to feel responsive, which means caching, materialization, and often pre-aggregation at the customer level. Semantic layers earn their keep here because metric definitions are deterministic and cacheable.

3. Non-negotiable accuracy on customer-visible metrics: a wrong revenue number in an internal Slack bot is a message away from being corrected. A wrong revenue number in a customer-facing dashboard is a support ticket and possibly a churn signal. Branch C companies almost always end up with a tight semantic layer scoped to their core analytics surface, with text-to-SQL deliberately disabled for customer-facing use cases. Microsoft’s own documentation is explicit that without proper semantic model preparation, Copilot “mainly produces low-quality and inaccurate outputs that might be incorrect or even misleading.” For Branch C, the flexibility of unconstrained text-to-SQL is not worth the risk.

The investment on this branch is significant: embedded analytics SDKs, tenant isolation infrastructure, caching layers, and dedicated reliability engineering. The requirements are different enough from internal-agent architecture that it is usually cleaner to build for Branch C from the start rather than repurposing what you built for Branch A.

Branch D: Operational agents that take actions. Read-only value is captured. Users are now asking the agent to do things: cancel these orders, reassign these accounts, trigger this renewal campaign, update these records in the CRM.

This is the most expensive branch and the most complex to build safely. Four requirements compound on top of everything built so far.

Identity resolution becomes non-negotiable. Taking an action on the wrong entity is structurally worse than answering a question about the wrong entity. The Forrester Consulting Total Economic Impact study via Reltio reports 366% ROI and $13M NPV over three years for modern master data management (MDM) programs at scale, and those numbers reflect the cost of operating without one: duplicated records, misrouted actions, conflicting updates. RudderStack’s analysis invokes the “1:10:100 rule” of data quality, which holds that the cost of resolving an entity mismatch grows by roughly an order of magnitude at each layer it propagates through (data layer to application layer to business outcome). If you have not invested in dedicated identity resolution or MDM, you cannot safely cross the threshold into action-taking agents.

Authorization must be enforced structurally at the action service, not at the prompt. The action service knows who the requesting user is, what permissions they have, and what the audit trail should look like. The agent is an interface, not an authority.

Reversibility design is required. What happens if the agent calls cancel twice? What if an action partially fails? What can be undone, by whom, with what authorization? These questions need answers before the first action goes live.

High-stakes actions require human-in-the-loop checkpoints: the agent proposes, a human approves, and automation comes later, selectively, after enough operational data exists to know which actions are safe to execute without review.

Palantir’s AIP and Foundry are the most visible implementation of this pattern. Their Ontology provides a structure where agents take actions through well-defined objects with explicit constraints, permissions, and audit trails. Tampa General Hospital’s deployment is the most detailed published clinical case study: an 83% reduction in patient placement time, a 30% drop in mean length of stay for sepsis patients, and a roughly 50% reduction in documentation time for the more than 500 physicians using AIP-powered ambient listening. The cost reflects the complexity: multi-million-dollar annual commitments, eighteen-to-thirty-six-month implementation timelines, and dedicated platform engineering teams. That cost model works for government agencies and large enterprises with clear operational ROI. It does not work for most mid-market companies, which is why this branch should be deferred until a specific use case justifies the investment.

Short of a Palantir-scale commitment, the major warehouse platforms now provide agent-governance primitives that cover part of this pattern. Snowflake gives each agent its own auditable identity, scopes which tools an agent can call, and supports multi-party approval for sensitive operations. Databricks’ Unity AI Gateway evaluates every agent tool call at runtime and returns allow, deny, or a request for user consent, across both internal and external agents. Microsoft’s Entra Agent ID and Salesforce’s Agentforce offer comparable controls. These handle the authorization, approval, and audit requirements above. They do not resolve entity identity, design reversibility, or build the action service into your systems of record, which remains the harder and more expensive part of this branch.

The use cases that justify Branch D share a common profile: high-volume operational decisions that follow consistent patterns and have measurable cost-of-delay, such as:

• Claims processing and adjudication in insurance
• Fraud investigation and case management in financial services
• Supply allocation and rebalancing in manufacturing
• Clinical documentation and order entry in healthcare
• Fleet routing and dispatch in logistics

If you cannot point at a use case with that profile, you do not need this branch yet.

What else you probably do not need yet

Each component below is legitimate technology with a defensible use case, and a common source of premature investment in companies that have not yet reached the scale where it pays off. Each has a specific trigger that makes it worth building. Until that trigger fires, money and engineering time are better spent deepening the foundational stages.

Real-time streaming infrastructure: most analytical use cases, which comprise the vast majority of early AI agent workloads, are fine with data that is minutes or hours old. Streaming (Kafka, Flink, Materialize) is justified for operational use cases with strict latency requirements, such as fraud detection, real-time pricing, fleet routing, or observability alerting. If your primary AI use case is “help the CFO understand revenue trends,” you do not need a streaming pipeline. You need a well-modeled warehouse that refreshes on a sensible cadence. The trigger: a production use case where stale data has a measurable cost per minute.

Knowledge graphs: CIO Magazine reports a 78% accuracy improvement when LinkedIn augmented customer service with knowledge-graph-backed retrieval, and Lettria’s production GraphRAG case study with Qdrant and Neo4j reports a 20 to 25% accuracy uplift over pure vector retrieval across verticals like finance, aerospace, pharmaceuticals, and legal. Those are real numbers from organizations with massive entity relationship complexity, where multi-hop traversal (customer to account to subsidiary to transaction to counterparty) is the primary query pattern. For a company where the typical question is “what is our churn rate by segment,” a knowledge graph adds cost and complexity without addressing the bottleneck. The trigger: your questions routinely require traversing three or more entity relationships, and the join paths are not stable enough to pre-model in gold tables.

Custom fine-tuned models: semantic layers and retrieval-augmented generation get you to 90-100% accuracy on covered questions. Fine-tuning addresses the last increment and costs an order of magnitude more to build and maintain. Unless you have a domain where public language models systematically fail (rare biomedical terminology, proprietary financial ontologies, highly specialized legal reasoning), RAG with a good context layer is the right architecture. Harvey AI represents the exception: in partnership with OpenAI they added roughly ten billion tokens of US case law to a custom-trained model because general-purpose legal reasoning required it. Most companies are not Harvey. The trigger: RAG with a well-built context layer demonstrably fails on your domain-specific questions, and you have the training data to fix it.

Enterprise data mesh: Zhamak Dehghani’s four principles (domain ownership, data as product, self-serve infrastructure, federated computational governance) solve coordination problems at very large scale. If you have fewer than ten data-producing teams, the overhead of domain-specific infrastructure, data contracts, and federated governance exceeds the benefit. A well-run central data team with a monolith warehouse is faster, cheaper, and easier to govern for organizations under five hundred people. The trigger: domain teams are blocked waiting for a central data team, and the central team cannot keep up with the volume of requests across distinct business domains.

Multi-agent orchestration platforms: a supervisor agent coordinating multiple sub-agents across domains sounds architecturally elegant. For most organizations at Stages 0 through 2, a single agent with access to a well-built semantic layer via MCP handles the work. Anthropic’s own guidance on building effective agents is clear: start with the simplest architecture that works. The trigger: a single agent demonstrably fails because the task requires coordinating across domains with conflicting context windows or tool sets.

Sequencing

1. Spend a month on the inventory. Resist the temptation to start building before it is done. The investment is small and the failure rate of skipped-inventory projects approaches 100%.

2. Pick two to four entities and model them well, with rich documentation and clear identity logic. Two months.

3. Build five to ten gold models for the questions on your inventory list, with semantic layer definitions on top, and link them to systems of record through metadata columns. Three months. You are now in production for Branch A use cases.

4. Pause. Look at what users are asking and where the agent is failing. Let the failure mode determine the next investment:

   1. More gold model coverage: if the agent cannot answer common questions.
   2. Better entity resolution: if the agent confuses similar entities.
   3. Unstructured retrieval: starting with relational search or metadata links, if answers require context outside the warehouse.
   4. Customer-facing isolation: if the agent needs to serve external users.
   5. Operational action-taking: if the business case for write-back is clear.

5. Repeat step 4. The companies that get this right are the ones who match their architecture to their failure modes, build at the level of complexity their use cases require, and resist the temptation to build for use cases they do not yet have.

Where to start

The technology for every component described in this post exists today. Semantic layers are mature. Entity modeling patterns are well-understood. Retrieval infrastructure has been production-hardened by companies processing billions of queries. MCP provides a standardized integration surface. The components are available. The question is which ones your business needs and in what order.

The companies that capture the most value from their data are not the ones with the most sophisticated architectures. They are the ones who know which questions matter, build the minimum infrastructure to answer those questions reliably, and expand only when production usage reveals the next gap.

The first step costs nothing: write down what you have, what you are trying to do, and for whom. If you have done that work and want help turning the inventory into a sequenced plan, or if you are partway through a build and the accuracy is not where it needs to be, start a conversation with us. We will tell you what we think you need, and what we think you do not.

Sources

• Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows. Lei et al., ICLR 2025. Source for the roughly 21% top-agent accuracy on enterprise schemas.
• Enterprise Text-to-SQL: What Accuracy Benchmarks Really Mean for Your Organization. Promethium. Source for the 10 to 31% production accuracy versus the 85 to 90% vendor marketing claims.
• QueryGPT: Natural Language to SQL Using Generative AI. Uber Engineering. Source for the Table Overlap evaluation framework underlying the 50% baseline figure before context engineering.
• The Context Layer. Benn Stancil, August 2025. Source for the term “context layer” entering industry vocabulary.
• Your Data Agents Need Context. Jason Cui and Jennifer Li, Andreessen Horowitz, March 2026. Source for context as an investment thesis.
• Text-to-SQL for Enterprise Data Analytics. Chen, Bundele, et al., LinkedIn (KDD ’25). Source for the 9% to 48% accuracy lift from rich metadata on well-modeled tables.
• Practical Text-to-SQL for Data Analytics. LinkedIn Engineering, December 2024. Source for SQL Bot’s roughly 95% user-rated accuracy and the multi-strategy implementation pattern.
• Building a Data Platform for Agents. ClickHouse Engineering. Source for DWAINE handling roughly 70% of warehouse questions and a 50 to 70% analyst workload reduction.
• Build a Conversational Data Assistant: Text-to-SQL with Amazon Bedrock Agents. AWS Machine Learning Blog. Source for Amazon’s RRDA team scoping pattern.
• Semantic Layer vs. Text-to-SQL Benchmark. dbt Labs, April 2026. Source for Claude Sonnet 4.6 reaching 98.2% via Semantic Layer versus 90% (covered questions) and 64.5% (full set) via text-to-SQL.
• Cortex Analyst Text-to-SQL Accuracy. Snowflake Engineering Blog. Source for “nearly 2x as accurate” versus single-shot GPT-4o.
• Gemini in Looker Deep Dive. Google Cloud Blog. Source for Looker’s semantic layer reducing gen-AI query errors by as much as two-thirds in internal tests.
• Knowledge Graphs: The Missing Link in Enterprise AI. CIO Magazine. Source for LinkedIn’s 78% accuracy improvement combining RAG with a knowledge graph in customer service.
• GraphRAG: How Lettria Unlocked 20% Accuracy Gains with Qdrant and Neo4j. Qdrant. Source for the 20 to 25% accuracy uplift of GraphRAG over pure vector retrieval across finance, aerospace, pharmaceutical, and legal verticals.
• Contextual Retrieval. Anthropic, September 2024. Source for the hybrid retrieval pattern combining vector embeddings, BM25, and reranking.
• Building Effective Agents. Anthropic, December 2024. Source for the guidance to start with the simplest architecture that works.
• Use Copilot with Semantic Models in Power BI. Microsoft Learn. Source for the verbatim Microsoft warning that Copilot “mainly produces low-quality and inaccurate outputs that might be incorrect or even misleading” without proper semantic model preparation.
• Customizing Models for Legal Professionals. OpenAI. Source for Harvey’s custom-trained case-law model with roughly ten billion tokens of legal text.
• Forrester Total Economic Impact ROI Calculator. Reltio. Source for the Forrester TEI study reporting 366% ROI and $13M NPV over three years for modern MDM programs at scale.
• What is Entity Resolution?. RudderStack. Source for invoking the 1:10:100 rule of data quality across the data, application, and business-outcome layers.
• Tampa General Hospital Selects Palantir’s AI Software. Tampa General Hospital, June 2024. Source for the 83% reduction in patient placement time, 30% sepsis length-of-stay reduction, and 50% documentation time reduction figures.
• Connecting AI to Decisions with the Palantir Ontology. Palantir. Source for the Ontology-based action-taking pattern.
• Agentic AI Security: Snowflake’s Data-Model-Agent Framework. Snowflake, 2026. Source for per-agent auditable identity, per-agent tool permissions, and multi-party approval for sensitive agent operations.
• Governing AI Agents at Scale with Unity Catalog. Databricks, 2026. Source for Unity AI Gateway evaluating every agent tool call at runtime with allow, deny, or user-consent outcomes across internal and external agents.
• Open Semantic Interchange. Snowflake. Source for the cross-vendor semantic portability initiative.
• Best Practices for AI/BI Genie Spaces on Databricks. Databricks SME Engineering. Source for the 5-to-10 tables and fewer than 50 columns scoping guidance.
• Introducing Entity-Centric Data Modeling for Analytics. Preset. Source for the entity-centric modeling pattern.
• Data Mesh Principles and Logical Architecture. Zhamak Dehghani, martinfowler.com. Source for the four data-mesh principles.
• LLM Evals: Everything You Need to Know. Hamel Husain. Source for evaluation-infrastructure guidance.