Finally, the market has realized that data and analytics agents are essentially useless without the correct context.
This is a translation of an article originally published by a16z. Original article: https://www.a16z.news/p/your-data-agents-need-context
Recently, in the circles of data and AI agents, the Context Layer and Context Graph have become unavoidable topics. Chat with any organization working on data and AI, and within five minutes, the conversation will inevitably turn to context.
This is normal. Over the past year, the market has finally figured out one thing: Data and analytics agents are essentially useless without the correct context.
They cannot break down ambiguous questions, understand business definitions, or effectively reason across disparate data sources.
This is not the fault of the agents. The modern data stack has evolved over a decade, moving from fragmented data sources to centralized data and cleaned definitions, which is a good thing. However, centralization has never been perfect, introducing significant chaos in the process. The general evolution path is as follows:
1. The Rise of the Modern Data Stack
We previously discussed this topic with Tristan Handy from dbt and wrote about it in our reference architecture articles. Over the past decade, data architecture has undergone transformation in ingestion, transformation, warehousing, and storage, with the goal of centralizing data so people can use it quickly and easily. The ideal vision is: data is cleaned up, teams write SQL to extract data from the data warehouse, create charts and dashboards, and enable the entire organization to use business intelligence.
2. The Agent Boom
Entering 2025 from 2024, LLM capabilities have grown stronger, and almost every organization wants to build agents on top of their existing data stacks. We have previously discussed how to define agents. From an organizational perspective, the natural attraction of doing more with less time, improving efficiency, has pulled everyone towards agent-based workflows. Companies began building chatbots to "chat with your data" and customer service agents. This trend happened both bottom-up and top-down. Developers wanted to use the latest and most impressive LLM capabilities, while management pressured for AI implementation to increase automation and reduce costs.
3. Hitting the Wall
The optimism did not last long. It soon became clear that most of these efforts failed. Organizations deployed agents and hit a wall. MIT published its famous "State of Business AI 2025" report, stating that AI deployments "mostly fail due to fragile workflows, lack of contextual learning capabilities, and disconnection from daily operations."
A key reason for poor agent performance is the lack of appropriate data context. Enterprise data today remains extremely fragmented and chaotic. Data agents struggle to answer even simple questions like "What was the revenue growth last quarter?" because they face various architectures spanning structured and unstructured data.
The vision of "fully self-service analytics" from years ago was not realized. And it seems the vision for data agents is following the same path.
The Context Problem: Far Beyond Text-to-SQL
Why was that initial wave of agent deployment so difficult?
Initially, many thought the problem lay with the models, specifically their data reasoning and SQL code generation capabilities. The general idea was: the model receives a natural language query, reasons over existing data systems, generates corresponding SQL code in the traditional BI manner, retrieves the correct data, and answers the question. If the model failed or was inaccurate, it was just poor SQL writing by the model, and it would improve over time.
This is not entirely wrong. Model capabilities in code generation and mathematical reasoning have indeed improved significantly, but they still lag behind in data aspects, as evidenced by SQL benchmarks like Spider 2.0 and Bird Bench. Model capabilities have indeed leaped forward, but we quickly realized the problem went far beyond text-to-SQL.
Let's break down the revenue growth example in more detail:
1. Assume a data agent is built within an organization. It uses a modern foundational model, connects to all required data sources, and has a nice interface for internal users to ask data questions. 2. A query arrives: "What was the revenue growth last quarter?" This seems like a very simple question. Usually, one could answer it by glancing at a Looker or Tableau dashboard; it should be easy for a highly intelligent agent, right? 3. Challenge 1: How does the agent know how "revenue" and "quarter" are defined in this organization?
Revenue is actually a business definition, not hard-coded in the data warehouse or pipelines. Does the user want run-rate revenue or ARR? Fiscal quarter definitions can vary completely between organizations; the same "quarter" in a different company might cover three entirely different months. What time window should be looked at?4. Fortunately, the head of the data platform steps in: "We built a semantic layer specifically to solve this; the revenue definition is inside." The agent should be able to ingest all semantic layers as context. This sounds promising. But after the team flipped through a few YAML files, they discovered these files were updated by a data team member who left last year, the BI tools no longer call them, and they do not include two new product lines launched later. The agent has no idea how revenue is actually calculated today. 5. To bypass this obstacle, someone manually hard-coded the definitions of revenue and time windows. The data agent continued running but soon hit Challenge 2: Where are the correct data sources? Which ones are the true sources of truth?
Raw data is scattered across multiple tables and data warehouses. The finance team uses thefct_revenuetable, which might be correct. But the data team also built materialized views:mv_revenue_monthlyandmv_customer_mrrare both sitting there.
Clearly, a data agent needs a continuously updated knowledge base containing business definitions and data source information to overcome these hurdles.
The Emergence of the Context Layer
The crux of the problem is here: Agents are not given the appropriate business context, preventing them from answering even the most basic questions. This reflects a larger gap. Building automated AI systems within an organization requires a continuously updated and maintained context. This context must understand how the enterprise operates, how data systems are organized, and carry the tribal knowledge that ties everything together.
This has given rise to the Context Layer. Many names have emerged in today's discussions: Context OS, Context Engine, Context Data Layer, Ontology, etc. The underlying concept is the same: connect all the chaotic enterprise data and add a context layer on top that helps agents understand business logic, encapsulating it so agents can use it.
Deja Vu in Context Management
Let's pause for a moment... Does what we are saying sound too similar to the Semantic Layer?
There are indeed similarities. But for agent workflows to truly move towards autonomy, they need more than what current semantic layers can provide.
In the traditional BI context, semantic layers excel at handling specific metric definitions like revenue, churn rate, and ARPU. However, they are usually manually constructed by data teams using very specific syntax, written through dedicated layers like LookML, and directly connected to BI tools like Looker.
The modern data context layer should be a superset of what traditional semantic layers cover.
Specific metric definitions can certainly be hard-coded, but for a modern context layer to ensure agent autonomy, it must include more: canonical entities, identity resolution, specific instructions to decode tribal knowledge, appropriate governance guidelines, and more.
This article focuses primarily on the data context connecting traditional recording systems. Another equally important and overlapping opportunity is capturing organizational decision logic and workflow logic, enabling the creation of truly multi-purpose agents rooted in the organization's entire data and decision context.
Connecting Everything
Based on our recent conversations with clients and understanding of requirements, here is what a modern context layer coupled with an agentized data system should look like, step by step:
1. Access the Right Data
The first thing is to ensure all the correct data is accessible. This is fundamental. Ideally, organizations should be using some form of modern data stack, achieving a degree of unification through a lakehouse architecture. Even so, it must be ensured that the agent can access all the data it needs, which may extend beyond what is already in the warehouse and operational applications. Tribal knowledge沉淀 in internal systems, GDrive, Slack, etc., all count.
2. Automated Context Construction
Once data is accessible, the next step is to start building the context layer. The benefit of using LLMs is that the initial context collection work can be largely automated. The focus should be on high-signal context. For example, reviewing historical query logs can efficiently identify the most cited tables and common joins. Data modeling tools like dbt or LookML can provide clear definitions for business metrics.
3. Human Refinement
Automated context construction may cover most of the corpus but cannot piece together the complete picture. The idea of letting the agent collect all internal knowledge on its own is tempting, but some of the most important context is implicit, conditional, and historically contingent, existing only within the team's tribal knowledge.
Human input provides those final critical connections that make true agent automation possible. For example: "CRM data: for all new North American deals after 2025, look at Affinity; for global leads before that, look at Salesforce."
With this, the context layer can become a multi-dimensional corpus where code and natural language coexist, capturing everything the agent might need. Just as developers can set .cursorrules files to guide agents and control output behavior, data practitioners can maintain their own rules and guidelines.
4. Agent Integration
Once the context layer is built, expose it to the agent for real-time accessibility. This is typically done via API or MCP.
5. Self-Updating Context Stream
The system is built, but data systems are never static, and neither should the context layer. Upstream data sources and formats may change, and personnel may have custom instructions they want to add, delete, or modify based on changing business needs. If a data agent provides incorrect data that needs correction, the correction should naturally be fed back into the context layer. In this way, the context layer becomes a living, continuously evolving corpus.
From this entire process, one thing becomes clear: Building a competent data agent is by no means easy. There are challenges in data infrastructure and engineering on the technical side, and challenges in collecting tribal knowledge on the human side; the two challenges are intertwined.
The OpenAI team recently published an excellent article detailing their own internal data agent creation process. It is written transparently, with a detailed and elegant implementation, but it also illustrates how long the road is to get to that point. Palantir has a long history of building ontologies for organizations,理清ing clear context from chaotic data, and has built a major business on this.
Market Trends
All of the above naturally opens a window for external solutions. Realistically, not every enterprise can (or should) build this internally. Various solutions are already entering the market.
We believe we are still in the early stages, but here is a high-level market map of the emerging solutions:
Let's look at them by category:
Data Gravity Platforms
Platforms like Databricks and Snowflake have gone through the entire process of data ingestion, transformation, and storage, possessing strong data gravity effects. They are already working on AI data analysis products, such as Databricks Genie and Snowflake Cortex Analyst, built on top of data warehouses, using foundational models for text-to-SQL, allowing users to query data using natural language. These platforms do not yet have particularly mature context layer functionalities, but they support lightweight semantic modeling. The path of introducing context layer capabilities into the platform through acquisition or internal development is feasible.
Existing "AI Data Analyst Companies"
A batch of companies has emerged that use AI to let customers "chat with data." After struggling in the market, many companies have realized that the key to building good data agents actually lies in constructing a context layer. Consequently, some companies have made data context construction a core part of their products.
Emerging Specialized Context Layer Companies
A new category has emerged, building context layers from scratch. They must traverse the entire path described above: ingesting data, collecting tribal knowledge, etc. And for every new customer, they have to go through this process again.
Looking Forward
We are currently at an interesting juncture. The problem of lacking context has been identified, but the work of building solutions is still in a very early stage.
The future is promising. Perhaps the vision of truly self-service analytics can finally be fully realized. BI, data analytics, and data science can be truly transformed by AI.
Of course, many questions remain open. Where will this context layer reside? Can it exist in multiple places simultaneously? Will it become a standalone product?