226: The Eye of context (The Dungeon of martech architecture, part 2)

Phil: Let's say you have a new AI system running on the fresh new data warehouse that you built, like, how would you know if the output was wrong? Jason: The moment you ask the simple follow-up questions, like, what data drove this decision? The logic started to thin out. Jason: The failure is essentially believable nonsense.

Danielle: It's about making sure it's not hallucinating. So guess what? Everyone's focusing on. Their data.

Tiankai: AI with bad data is just bad AI. I call it the comeback of data quality Austin: Unstructured data is not cool if it has to eventually be sent into a structured format, and you don't have some concept of doing that. Lindsay: We started by populating context from across the org into Google Drive folders. Converting, uh, the materials into markdown files.

Keith: Make sure your data is as consistent, tagged, really clear, understandable definitions of what the different data means in your organization. In This Episode - Phil: What's up, folks? Welcome to our four-part series of crawling through the dungeon of martech architecture. You've arrived at part two, The Eye of Context.

We're gonna cover today why AI produces believable nonsense, And to combat this, we'll unpack what data quality actually means, Phil: Why context engineering is more than just prompt engineering, why the industry built the wrong semantic layer in 2012, Phil: And how to fight context rot and fragmentation. All that and a bunch more stuff after a quick word from two of our awesome partners Sponsor GrowthLoop - Sponsor: GrowthBench - Welcome back - Phil: Welcome back to the dungeon of martech architecture.

You've arrived at part two. If this is your starting point, I encourage you to check out part one, where we cleared the first floor's two bosses in two forms, the false truth king in the CRM and the export hydra that spread it everywhere. Uh, that said, if you already have a data warehouse, you might actually be able to start right here in part two. So, you know, today is The Eye of Context.

Context is the word of 2026 according to Scott Brinker, and the layout on this floor is really interesting. So let's, uh, let's make our descent onto floor two here. FLOOR 2: THE EYE OF CONTEXT - Phil: Entering floor two, the Eye of Context. So the layout on the second floor of the dungeon of martech architecture actually looks pretty fancy.

You're getting there and it looks cozy, it's modern. The whole place is lined with mirrors. There's actually mirrors everywhere. It's a bit weird.

Um, and it's also a bit creepy because, like, once you look a little closer at the reflections, you notice that some of the details are a bit off. The boss on this floor is low-key danger that sneaks up on way too many teams. It's not like the big flashy monsters from the past two floors. Um, so let's, let's dive in here.

Let's say you have a new AI system running on the fresh new data warehouse that you built, and initially it looks really good. every output is delivered with impeccable confidence, maybe a bit too confident. So the next step is asking yourself, like, how would you know if the output was wrong? There's a lot of obvious hallucinations that you probably catch when you chat with GPT or Claude, like totally inventing stuff, especially when it's your own niche of expertise, you're a subject matter expert.

You're just like, "This is wrong. What are you talking about?" I'm so sorry. You're completely right.

I'm talking about the details though, the less obvious hallucinations, the kind of wrong that passes that first glance check. We've all seen that viral post on r/analytics, uh, a couple months ago about a company that found out AI was making up analytics data for three months. Um, you know, whether this post is from a real story or not, a lot of comments are just like this, "There's no way. This is, this is fake.

It's not real." Um, but some versions of this are happening inside companies today. There's just no denying that. like I've talked to technical marketing leaders that have greenlit agentic tools at their startups before the data definitions were settled.

Uh, one of them called it believable nonsense, and that term kinda stuck with me, like really explains the dangers of the hallucinations and how they can kind of sneak up on you. This floor's failure is designed to look like success until you dig into the details, look under the hood. Why AI Produces Believable Nonsense - Phil: So let's look at the origins of the hallucination oracle boss. Phil: Humans are wired to trust smooth, confident talkers.

It's actually baked into our evolution and how our brains develop from infancy. There's a bunch of studies on babies and brain scans that show this is an innate thing that kicks in really early. A person who sounds certain usually knows something, right? Like, LLMs and AI systems are actually breaking this calibration.

They produce fluency without really producing correctness. And at first glance, the output sounds right for structural reasons, um, and, and once something sounds right, we engage with it differently. We're forwarding it to coworkers. We build on it.

We present it to stakeholders. We don't have context to really question it. you know, what made it dangerous wasn't really like the obvious hallucination that we all see or used to with LLMs. Obvious hallucinations, easy.

Um, you kind of reject it, you move on. Jason: That's Jason Dobbs, head of marketing and GTM engineering at Kumo. He greenlit agentic analytics and predictive workflows at his startup before the team had settled on shared definitions. I mean, I think what, what we were seeing looked polished enough to be operational.

Um, at first glance, you know, the scores looked precise, the summary sounded coherent. Uh, the recommendations felt data backed. But the moment you ask, like the simple follow-up questions, like, uh, and that's why explainability is so important. Like, okay, well why did you choose this account?

Or, um, why is that happening now? Or what data drove this decision? Um, you know, the logic started to thin out. So that's really the layer, the, that you need to, to be able to, to dig into.

Um, so, you know, the lesson for me wasn't like the data was bad or the warehouse was bad, or the model was bad. Uh, what's, what, what was wrong is we were trying to automate ambiguity. Um, you were, we were asking a AI to solve for confusion that we hadn't yet ourselves solved for, uh, internally. Um, and, you know, and once you do that, you kind of enter the danger zone because, you know, the, the failure is essentially believable nonsense.

Uh, and that, and that's dangerous because people trust it. Phil: So this is some pretty scary stuff, right? Like when this believable nonsense gets trusted long enough to make it into campaigns, decisions, board decks. The good news is that we already have a weapon in our inventory that's perfect to slay this boss.

Uh, we shaped it in the last episode. It's the data warehouse. It houses data obviously, uh, and data is how we defeat the believable nonsense of this boss. Um, but we need to enhance it BOSS BATTLE: The Hallucination Oracle - Phil: B-b-b-boss battle.

The hallucination oracle. The boss on this floor is the hallucination oracle, the ultimate creator of believable nonsense. Output that sounds right, passes the first look, makes it into decks, and gets trusted before anyone dives under the hood and asks follow-up questions. So let's check our inventory because we have three potions here to enhance the data in our data warehouse to conquer this, uh, hallucination oracle boss.

So the first thing in our inventory is data quality, context engineering, and the semantic layer. So we're gonna cover all three of those in depth, but just a little preview. Data quality, clean, governed foundation stops the agent from lying about what the warehouse contains. Context engineering supplies the right meaning at the right time, the gap between governed warehouse and a trustworthy agent.

And the semantic layer represents domain concepts, relationships, constraints, definitions, So we're gonna cover all three of those here. Uh, but let's pull out the first one, data quality. Data Quality: When Agents Read Your Messy Data - Phil: Good old data quality. If you work in ops or data role, you've pitched a data quality project before, and you've had it deprioritized many times.

Wah, wah, wah. AI changes that, though. AI has surfaced bad data problems and made fixing them feel way more urgent than before. For the first time in most organizations' histories, data cleanup became something executives actually wanted to fund and talk about.

What a time to be alive, folks. It's almost Danielle: Everyone wants to leverage this technology. It's about making sure it's not hallucinating. So guess what?

Everyone's focusing on. Their data. Phil: That's Danielle Balestra, fractional marketing technology executive. Danielle: So I said, this is great 'cause this is gonna have amazing downstream impacts for us in marketing because guess what?

People are gonna actually start cleaning their databases. It's gonna be amazing. So, um, from our standpoint, I just literally had a call earlier with a, um, person on my team and he's building out, um, a chat agent to help verify, um, some information as we're building press releases. And it's pretty amazing that we have this, um, in-house.

That we are playing around in a secure environment and that we can leverage our internal data systems, um, to bring the, AI agents in to help us with, um, confirming or validating stuff as we're, as we're creating the copy. So, and again, we have amazing colleagues who are very interested in this and they wanna leverage it. Um, so. For us, it's about putting the safeguards in place, understand what the technology can do or what it can do wrong to look for those.

Um, and again, it's validate. Don't just like take it, throw it on a plate and be like, here it is, but like, make sure it really is working. Do a lot of testing, do a lot of, you know, probing adjustments. Um, and again, our biggest focus right now is data.

We're like working through cleaning and making our data accurate so that. These agents can do what they need to when we build them out. Tiankai: Before, um, data quality was not sexy. Everyone was focusing on, everyone complained about it, but no one wanted to take care of it.

Right? Phil: That's Tiankai Feng, data and AI strategy director at Thoughtworks and the author of Humanizing Data Strategy. Tiankai: I call it the comeback of data quality And now that AI is there and everyone realizes that, uh, AI with bad data is just bad AI, right? All of a sudden the focus is much bigger.

It's like we are bad. Data quality is hindering all of our innovation here. And, um, if we cannot compete with the rest of the world in AI, then what are we doing? This needs to be the highest priority all of a sudden.

And that puts a lot of pressure on it. Right. And, uh, going back to the basics and actually focusing on data quality again, is the right move, I would say. Right.

I have to say that, that also a lot of, especially in the gen AI space, right, there's a lot of pre-trained models. That only needs the context of your organizational data and doesn't need to be trained completely by your own organizational data. So, um, there's um, always a certain impact that you need to deal with. But even then, still, right, you want to tailor it to your organization that that means you need to provide it in the high quality contextual data that you have for any AI model to probably work.

So, yeah, absolutely. Um, it's, uh, now that everyone wants to do it, we all get to do data quality again. So good for us. Phil: So the comeback is real.

A lot of people are feeling this, and for most organizations and rev ops and marketing ops teams, it's long overdue. Lou: One thing that will never change is the garbage in garbage out component of this, right? Which is if your data foundation is garbage. So too will your AI.

Phil: That's Lorenzo Mello, director of product marketing at Snowflake Lou: You can have a beautiful UI, you can have the right kind of prompts in there, but the insights that's going to surface are going to be directly linked to to what you have on the back end. And, uh, that’s why we believe so much in that component of data quality and really think that the first step to getting the proper CDP downstream is getting that customer 360 properly built. But that can't be a buzzword that requires the quality data.

Same thing with You can run MMM But if you're sitting on a poorly constructed foundation of campaign intelligence, You're probably going to be spending your advertising dollars poorly, right? Um, and so I totally agree with that, and I think that's going to be one of the biggest helps, uh, or components that Gen AI is going to help, I think, the ecosystem with. Phil: All right. So what does data quality actually mean though?

Like everyone talks about it. No one explains really how do you tackle fixing the data. For me, it's a bunch of different things, but I kind of boil it down to five things. So first one is records are accurate and de-duplicated.

De-duplication, the bane of existence for marketing ops folks, like ID resolution, de-duplication, you've got that down pat. Um, number two is field definitions are agreed upon across every GTM team. Everyone has the same understanding if you're using MQLs, RIP, but everyone knows what an MQL is across GTM teams. Uh, data pipelines have a known schedule and someone owns different pipelines.

When something looks wrong, you can trace it back. There's auditability back to the source. And two people asking the same question gets the same answer, and that answer has an author. So that's the like bare minimum, if you will.

A lot of folks have different perspectives on this, but, uh, let's hear from someone at OpenAI Keith: If you're going to try to do anything involving some version of agentic orchestration, you have to be really confident in what I'll call the less sexy elements of any deployment, right? Phil: That's Keith Jones, who runs one of the GTM systems teams at OpenAI, and he's built and broken more martech stacks than most, and he's actually stating the prerequisite for us here Keith: All the things that.

If we are all being honest, we know we need to do and we probably could do better, but especially if you're in hypergrowth or early stage, you might count them more of a, a luxury than a, than a necessity. Right. And I'm talking about things like having really clear, understandable definitions of what the different data means in your organization. Having a consistent level of syntax.

For that data, right? When I say X equals Y, I need to say that every single time. And then in addition to that, you need to make sure your data is as consistent, tagged, categorized, codified as much as possible because again, the, the agents you know, aren't there yet where they can. Fill in some of the gaps that the human brain is capable of doing today, right?

And so you've gotta make these things more mechanical in nature, if you will. Um, the models will get better, they'll be able to make more and more educated guesses or informed decisions as as we go. But, um, where we're at today is that you've gotta have your stuff in order and if you want to deploy anything. Phil: And let's hear from Austin Hay Austin: Unstructured data is really cool.

But guess what? Unstructured data is not cool if it has to eventually be sent into a structured format, and you don't have some concept of doing that. Phil: Austin is a martech, RevTech, and GTM systems advisor and AI builder, writer, ex-founder. Guy's got a bunch of stuff going on.

Uh, such an awesome interview. Also had him twice on the podcast. Uh, but he talked about the tension between new capabilities and old foundations Austin: And so the emphasis is not to be like the old guy in the room, like, you know, wagging my finger or shaking my stick at, um, at agentic experiences. It's just to say that, like, I think a lot of companies right now are innovating at the boundary of what's possible, which is really cool.

But there will, there will be some time to catch up before. Um, before that's fully realized. And I think that, you know, one of the stances that we take is that in the meantime, the original, um, kind of data structures of CDPs and CRMs are really valuable. And people are still gonna be operating in that way.

The only case I can see where that's not true is like if you know, AI progress is so rapidly in the next five years that computers can effectively design their own APIs and send data without needing any human intervention. And sure, you don't need any data structures at all, because you could actually just store everything in a data table and talk to your LLM or talk to your agent and say, Hey, I want you to give these types of data to this endpoint. And as long as both endpoints have an LLM or an agent on either side of the end point, being able to decipher and structure it.

But to me, that seems really far fetched and very far off. It seems like more than a three year adventure. So in the meantime, my, um, my belief is that, you know, some of the best companies are maintaining what's good about old data structures while allowing and importing, um, the ideas from the past. You know, from agentic experiences.

And just to give you an example, it's like, you know, and clarify we have a schema that you can have on contacts, but we automatically update fields for you based on on unstructured data. So we allow you to define the structure, but we're trying to take the action out of it by, you know, Updating that schema for you automatically, you know, and for people who use Salesforce, it's like, what happens if Salesforce updated itself based on your emails and calls? You know, it's like you still define the schema.

You're in control. You're structuring the data. You're giving context to the CRM and saying, Hey, these are the things I care about. But then you don't have to do the manual work of just like going and updating fields because it's listening for those fields then and trying to sort things into the right groups and update them.

So that's an example where I think like, Old where old world, uh, schema and like new world automation, i.e., AI Intelligence is can be really, really powerful. Phil: Getting the right foundation requires a lot more than just documentation.

Ana: The data template is the language that the CDP understand that that's the only language that the CDP understands. Phil: Anna Moreau, who's managed data infrastructure at Stanley Black & Decker, built an enforcement mechanism for global enterprise. She called it the data template, the formal definition of what the CDP would accept, and she built the consequences directly into her workflow Ana: So if it doesn't come formatted in that language, we won't be able to see it. The CDP won't understand it.

That doesn't mean that the data is lost. It's not going to make its way to the Centralized or unified profile for the end user, right? So with that and by sharing that responsibility and by Going through the data template with our business stakeholders, the marketers, the e commerce managers, that was really what made a big, big difference because then they became owners of that data collection and that data activation. And then they understand that if they set up the form right at the beginning and they use the data template, labels that we have defined in the data template, then the data will flow automatically from the CDP or to the CDP, from the data source to the CDP.

And that made a huge difference because with that they really understood that it pays to pay attention to some of those details, I also worked with them to make sure that they understood that the data template is a living document, Phil: So we've got forms and data sources that match the template structure and it - that flowed automatically into the unified customer profile within, you know, hours. Data that didn't match it got stuck. It didn't pollute the system. It required someone to go back to the source, reformat manually, upload through a separate process, and Anna built that friction intentionally, right?

Like, teams that paid attention at the beginning got clean automatic data. Teams that ignored it and were just going too fast had to take ever, uh, ex-extra work, and the behavior for them hopefully changed the next time they wanted to do it. So let's go back to our first potion, uh, data quality, and it's, gonna enhance the weapon that we wield to weaken our boss here, but it doesn't finish it. That's why we need our second potion, context engineering.

It's built exactly for that gap, Context Engineering: What It Is and Why It's Not the Same as Prompt Engineering - Phil: So let's dive into it. Phil: Clean data is obviously a prereq here. Like teams that fix their schemas, tighten their pipelines, eliminate duplicates, often find the AI still doesn't work reliably all the time. It's not just about clean data.

The real problem is what the agent does with that data. So in my research in context engineering and preparation for, uh, this episode, I came across a person called Chris Lema, He wrote a really cool article where he's describing a pattern he sees in nearly every organization that runs into this wall. So you've got like a team investing weeks into the retrieval layer. Um, some teams call this RAG, the RAG pipeline, getting the right documents, the right CRM data, the right knowledge base, and gets the context pipeline working beautifully.

Then the agent still produces believable nonsense, and the team blames the data. They add more documents, more context. They tweak the RAG pipeline. They adjust the chunking strategy, how quickly it can get fed into the LLM.

None of it helps because the data was never the problem. Chris explains that like most AI context failures happen after retrieval. The agent has the right information, it just does the wrong thing with it, and until teams can name the specific way it's going wrong, they keep fixing the wrong layer. So a lot of this shows up tactically as RAG retrieval augmented generation, but RAG is just the mechanism, not the strategy with what you're doing after.

The strategy is deciding which context deserves to be retrieved when and for which agent under which constraints. The, the quote that I love from Chris's article is, uh, "The data isn't the problem. What you're doing with it is." Sponsor: MoEngage - Sponsor: Knak - Context Eng vs Prompt Eng - Phil: Part of this is like unpacking the difference between context engineering and, and prompt engineering.

For me, like the context engineering piece of this controls what the AI knows. Prompt engineering controls like how you actually interact with it, how you talk with it. Um, we teased out some of the stuff from Scott Brinker in our first floor episode. Uh, but we obviously can't talk about context engineering without talking about Scott's State of Martech 2026 report, co-authored with, uh, Frans Riemersma.

It's not just Scott. Um, but you know, it's the word of the industry in 2026, context, and it applies directly to what's on this floor, obviously. So the report is arguing that the difference between AI that generates plausible output and AI that creates meaningful value is context. It's the difference between send an email and send the right email to this customer at this moment with awareness of what they've already done, what they're trying to accomplish, what we promised them, what we're allowed to say, and what the brand should sound like while doing it Boom.

Sounds easy. The report describes three kinds of contexts that have to come together before the sentence becomes even possible. So, um, the first three that, um, Scott and Frans unpack is customer context, company context, and systems context. So we'll put those up on the screen for folks watching on YouTube here.

But, uh, customer context is the customer situation, intent, history, moments that matter. The company context is your goal, strategy, brand, you know, OKRs, um, whatever processes, capabilities, and governance. And systems context is what your stack can actually access, connect to, deliver. It's the deep customer insight into your data warehouse and the sharp brand strategy in your, in your Google Doc maybe.

Where all three of these context layers are converging, is what the report is calling golden context. It's almost like the ultimate achievement, uh, for our floor here. Um, but if value engineering is identifying the value, context engineering is making it actionable, and together they're what the 2026 martech architecture actually centers on. So this is really interesting stuff because you need to kinda think about these layers of context as different things at different times of different use cases versus just this dumping ground of the context layer.

Um, one thing that kinda complicates things a little bit here is ID resolution. Like the customer context, the situation intent history is a bit more complicated than that. Every- anyone that's worked in, you know, data deduplication knows this. this awesome article from Sonal Goyal, the founder of Zing and the author of Learning From Data.

Uh, her newsletter is on Substack. We'll link out to it here. But she's got a piece of the context problem that keeps getting skipped, and she talks about most organizations still don't have clean customer identities. In the AI era, the - that's the gap between the golden context and the garbage context according to her.

So most stacks feed AI three different versions of the same customer before the model ever runs. in the CRM, let's say, you know, our customer is Miriam Dom, and Miriam is, uh, in the CRM under miriam.dom@cashmeregoatfarms.com.

In the e-commerce platform, she's prepotente.mom@gmail.com with three purchases this quarter. In the support system, she's a different record.

She's miriam_d with five open tickets and urgent issues. And in the marketing automation platform, she signed up for the newsletter under miriam.dom+donotemailme@cashmeregoatfarms.com.

So bunch of different records. Every system has a different stranger in it, and none of them know that they're actually looking at the same Miriam Dom, the same customer. The quote I love from Sonal in her article is, "When a customer exists as three separate records across your systems, you don't have customer context, you have customer fragments." So let's talk about what this context bundle requires, and, and we'll go back to this element of fragmentation here around context.

But, um, Jason Dobbs, he describes what the layer of our context bundle has to include before any agent gets real authority over a workflow Jason: The fix the data may, may be, uh, not the best label. Uh, you know, it kind of, it kind of makes this effort sound like some mythic authoritarian quest for the Holy Grail. Uh, you know, it's the one quest we all seek out for, but we never quite actually seem to make it to the end. Uh, you know, regardless of, of kind of how good we are.

Uh, you know, I think the reality is that. The warehouse is often where the richest relational signal already lives, or, you know, breakout kind of wherever. I mean, it's different for different organizations or wherever your core, uh, data sets live. Um, so the, you know, the issue isn't usually that the business has zero signal.

Uh, I think the issue is that teams jump from, Hey, here's a big pile of our data to, uh, let's throw this agent on top that should now be able to like, act on any scenario without defining that layer in between. Right. So I think it's not really like how do we clean everything? Um, it's more like, what minimum context and control does a workflow actually need to become reliable?

So that includes, you know, things like what are the definitions, you know, what is the source of truth? Uh, what's the freshness of the data? What's, what is the permission that we're actually giving for this? Um, you know, defining those thresholds and then like figuring out like where does human judgment still belong?

you know, how prompt engineering can make something smart. I think context engineering is really what makes it dependable. Phil: Before any agent gets real authority or power over a workflow, Jason is arguing that it needs a context bundle, and the first three items are a prerequisite that data quality has to answer before context engineering starts. We talked about this a little bit, but like some of the data quality stuff here is shared definitions that all GTM folks agree on, trusted access to the right records with the right joins, and then a named owner who's accountable when something goes wrong.

Um, the two layers of context engineering here in that context bundle are clear authority boundaries on what the system can and can't do without human review, and then an evaluatidown path to validate outputs against historical examples before anything goes live. Think of it the same way you'd onboard a new hire. Like, here's what we mean. Here's what you have access to.

Here's when you escalate stuff. Lindsay: Zapier is actively working on and thinking through how we bring context from all of our tools into that context layer that is naturally staying up to date. Phil: That's Lindsay Roethlisberger, director of GTM innovation at Zapier, and she actually built this layer from scratch for her go-to-market organization at Zapier. Her team started with shared definitions.

The RevOps analytics lead went through every key term, documented it so agents would actually have something to work with Lindsay: Other things that we're doing, um, internally in go-to-market to keep context fresh. Um, so like I said, we have the clear owners of the different layers of context, um, personal, team, departmental. And then the way I think about it is there's, like, the slow layer of context of like, the things I mentioned earlier about that don't change. So your company strategy, your ideal customer profile, um, your data definitions and things like that.

So we put a lot of time, I will just say, side note, we put a lot of time into context around the data definitions. I'd say that is, like, such an important place to start. We had our RevOps person, um, who works specifically on analytics really go through and build a lot of context into lead definitions, opportunity definitions. If someone asks for a lead report, make sure you're asking them, are they asking for an MQL report?

Are they asking for, you know, something else? So I will side note that, um, very important to start there. Another thing I'm thinking about is how you potentially tier different levels of context. So for example, um, you're gonna want an agent to know that if you make a pricing change, that's a really big deal.

Whereas if someone has a random idea about an experiment they wanna run, maybe that's not weighted quite as equally in terms of, like, how we want agents to think about the context. So those are other things I'm thinking about is, like, creating some sort of, like, tiering system. Again, I'm a very operational person, so I, I, I think in process and systems. Um, but we'll see.

Like, there are so many new tools coming out, and, like, agents are getting better and better and better. Um, so it's definitely keeping all of us on our toes. Phil: So her team pretty much organized the shared brain at Zapier into two main layers, and I loved how Lindsay broke this down. There's the slow-moving layer, company strategy, ICP, data definitions, playbooks, whatever, and it requires deep upfront investment, changes infrequently.

Like, sometimes it will, positioning messages, we all know ICP changes. But then there's a fast-moving layer. These are things like daily decisions, routing changes, experiments that were finished, anything sitting in Slack threads or meeting notes. All that is part of most teams haven't solved yet.

It's that, like, fast-moving layer of context. Um, if you - we look back at, like, the layers that Scott had for his context, like this one would fall into the company context. Some of it would be in the systems context also. But these are like all the fast decisions there.

And the slow layer can be built in a sprint, but the fast layer is where, uh, we have something called context rot and fragmentation. So the State of Martech 2026 describes context as operating in six pace layers, so very similar to Lindsay's approach on this. Um, but they each have their own decay rate, and it's worth putting up here on the screen. Uh, for folks listening on audio, we've got moment context.

This changes by the second. So this is like your customer's real-time state, their current query. The second is session context, and this is like minutes and hours, the actions they've taken, the conversations that, that they've consumed. Number three is journey context.

This is days and weeks, like buying stages, engagement. For long enterprise B2B, uh, segments, this journey context is even longer. Relationship context is months and quarters. This is like account history, preferences, LTV.

And then we've got company context. This shifts over quarters, years. And then the last one is market context. This is like industry structure, macro regulations, and stuff like that.

The more granular and timely the layer, the faster it loses value in a way, right? Like an intense signal from a morning browsing session may be pretty worthless by the afternoon if the customer's moved on or simply lost interest. Uh, but the report calls this context distribution as a problem. Real-time signals that can't reach the right system before it decays is worthless regardless of where it lives.

The part that ruffles everything is that AI accelerates this like oscillation across all these layers. It lets systems respond faster in the moment, but only if the slower layers are stable, accessible, and aligned underneath it Example, a real-time agent that knows the current query but nothing about the customer relationship, the company's commitments, or the governance boundaries is just reactive. Fast without grounding is how an agent produces customers who are genuinely angry about an experience that looked personalized on the outside.

So this is where Scott and Frans talk about golden context. According to the report, it's when the fast layers and the slow layers are moving together, immediate enough to be relevant, grounded enough to be trusted, and that two-layer structure is the right architecture for any org trying to manage context at this level. And so our second potion, context engineering, closes the gap between what the warehouse contains, what the agent understands. There is a third potion here that's worth, um, considering in, before our battle.

Why the Industry Built the Wrong Semantic Layer in 2012 - Phil: This is the semantic layer, and it's the layer the industry actually got wrong 15 years ago. Let's take it out. Let's look at that one Phil: So Scott Brinker, uh, again, uh, we, we name Scott a lot. He's almost like the, the game guide here.

But he, uh, n-not just his, like, State of Martech report for 2026, he also co-authored a, a March 2026 report on AI-era architecture. He did that with Databricks. Um, he's describing the organizational consequences, skipping the work around the semantic layer. Without a shared semantic layer, every agent becomes its own island of interpretation.

Uh, you can have perfect warehousing, clean data, well-scoped prompts, but still end up with three agents taking three different actions based on contradictory, uh, contradictory assumptions about what customer or pipeline counts for and what this quarter's, uh, ICP actually is. The shared definition problem is organizational, and it's nothing new for RevOps teams that are balancing marketing and sales definitions. Um, each team encodes meaning into the data that they own, and the agents inherit those local dialects, if you will.

David Chen is the managing director at Deloitte Digital. He's one of my favorite voices in martech. I had the pleasure of interviewing him in the earlier, uh, years of the podcast when I was doing the debate between package and composable CDP. Uh, we should probably have him on since I'm always reading his blog.

Uh, but at Deloitte, he runs their CDP and marketing transformation practice, and he wrote a detailed response to the same report that Scott did with Databricks. His piece argues that Scott's composable canvas vision is directionally right, but the friction points blocking it are organizational, not technical. So he names four main things. The first one is decision paralysis.

When every component is swappable, teams freeze because nothing forces a choice. The second one is semantic misalignment. Internally, sales, marketing, RevOps, all these GTM teams, they can't agree on what a customer, what a pipeline, what a qualified lead actually means. And externally, 15,000-plus SaaS vendors encode their own incompatible versions of those definitions.

The third one is false novelty. This is, like, the new architecture that may be more flexible expressions of what already existed, not the paradigm shift that it's being sold as. And the last one is governance overload, like AI-generated custom software that's creating accountability and maintenance problems that most teams aren't set up to handle today. So the semantic misalignment point is really important to this upcoming boss battle here.

My favorite quote from David's article is, "We spent the last two decades solving system integration, and we're about to spend the next two decades failing at semantic design. Book it." So his conclusion, the constraint is agreement. The technology is the easier half.

Let's talk about knowledge architecture and the bet that our industry got wrong in 2012. The deeper issue is one that took 15 years to surface. The industry built the wrong kind of meaning infrastructure. I'm a big fan of the Metadata Weekly newsletter, recently rebranded to Context and Chaos.

The author behind it is Jessica Talisman. She's a semantic engineer and information architect. in '26, January 2026, in her article titled Ontologies, Context Graphs, and the Semantic Layer: What AI Actually Needs in 2026, she's arguing that in 2012, there were two bets that were placed simultaneously. One gave us consistent dashboards.

The other, uh, gave us drug discovery AI and reasoning systems that intelligence agencies trust with life or death decisions. So there was, like, that BI industry bet The technology was LookML and metrics layers. The goal was to define metrics once, govern centrally, and it produced, you know, good-looking dashboards for agents. It answered what X is, what an MQL is, what are, uh, how we calculate MRR.

Um, but in healthcare and life sciences, we had this new technology for formal ontologies and a knowledge graph. And the goal for this was representing domain concepts and relationships and constraints, and it produced drug discovery AI, intelligence agency, reasoning systems. It basically allowed agents to answer why X was allowed to happen. So two different things, and the assumption behind the metrics bet was that consistent calculation would produce consistent meaning, but it turned out to be wrong.

Knowing that revenue is calculated as sum order total where order status equals completed doesn't tell you why revenue dropped in Q3, which customers are at risk, or what to do about it. Semantic layers modeled metrics, but they failed to represent an organizational's reality, the c- the company context. The distinction matters for agents in a way that it never really did for dashboards that just cared about metrics. The semantic layer answered what is X, like what is our MQL?

What is our MRR? It modeled metrics and calculations, and it was built for dashboards. The limitation is that consistent calculation doesn't create consistent meaning. And so on the other side of that, we have the context graph.

The context graph answers why X was allowed to happen, and it models concepts, relationships, constraints, and inference rules, and it's built for agents and AI. And, you know, the limitations here are that the layer the industry skipped building in the first place is the context graph. So, let's unpack why that matters for martech architecture, what she means exactly. Without this second kind of infrastructure, decision traces stay trapped in Slack threads and email chains, and institutional memory doesn't have a system that can be indexed, and no agent or LLMs can read from it.

Uh, in her article, like one of the cool quotes is, "The era of metrics-based thinking is coming to a close. AI systems need to understand what your domain is, the concepts, the relationships, the constraints, the inference rules. It's a knowledge architecture problem." The teams who end up doing this well in the next decade are gonna spend more time on meaning design than on pipeline engineering.

The pipeline is necessary obviously, but the meaning is what makes it useful for agents and AI. So the practical starting point here is less intimidating than the terminology implies. Um, like, uh, you could pick just one entity in your agents to act on, like maybe customers or, you know, the opportunity object, and you define five things about it: what it is, what states it can be in, what relationships it has with other entities, what decisions get made about it, and who has the authority to change those decisions.

So when you write that down, the document is kind of the seed of the context graph. Most teams have never done this manual exercise for even like one entity, so it's a really cool way to think about it, and doing it for one forces every assumption about that entity into the open, and those assumptions are usually where the contradictions between teams start to happen and pop How Context Rot and Fragmentation Break AI Agent Performance - Phil: All right, so let's just build a context bundle.

It sounds way easier than it is in practice though. So phase two of our hallucination oracle boss on this floor is less visible and more structurally damaging, and that's context rot. It produces answers that were right when the context was current, but it degraded silently as the organization changed around it, and the slow layer of context stayed frozen, and the fast layer just kept moving and changing, and we couldn't keep up with it. The gap accumulates invisibly until an agent acts on assumptions that the team stopped believing months ago.

Fragmentation is the other piece of this. It's like the spatial version. The right context exists across stack pieces, but there's no single system holding all of it in one spot. Each of these context pieces are accurate, but the synthesis never happens.

We don't have a good summary of all of it. Uh, a governance decision in s- in a Slack thread or like a campaign exception in a Google Doc, an ICP update in a deck that no one can find. The agent acts on whatever it reaches first, and it doesn't always look at everything and have a good understanding of everything. Building the shared context layer before our boss on this level scales is really important, and so the four questions above are the diagnostic that you can't really answer all of them if the agent cannot either.

All right, let's unpack those two failure modes for context, context rot and fragmentation. In the same article I referenced earlier, Chris Lema has cataloged a bunch of different failure modes when it comes to context, and the two that stood out to me as the most important for martech architecture and potentially the most expensive ones are context rot and context fragmentation. So context rot is early in a long interaction or multi-session workflow. Maybe you're in Claude Code or some other type of agent, and the agent receives critical context, and it's usually as more input accumulates, that early signal, it gets buried under volume, and it stops like surfacing when it actually matters.

The agent still technically has the context and the information, but it no longer has the ability to surface it when it matters for you most. The second one is context fragmentation. The agent has all the relevant pieces distributed across its input, but it fails to synthesize them into a coherent picture. Each fact is accurate on its own.

The agent can retrieve any individual context piece. It never connects them into the insights that would actually be useful, though, and Chris specifically calls this the most costly of the nine failures that he's cataloged So in our case, both failures have the same structural cause, context treated as an infinite resource when it really isn't. And these models keep improving, sure, but context windows will keep expanding, and that doesn't really fix the structural problem that we have here.

Um, one way to think about it, like an impactful, uh, series of questions about the LLMs are, are they getting the right context? Is it surfaced when it matters most? And can it synthesize into a coherent picture? And is it fresh?

BOSS BATTLE: Rotten Context Mage - Phil: B-b-b-boss battle. Rotten context mage Phil: The second version of our boss on this floor is context rot and fragmentation in the form of our rotten context mage. We actually don't need anything new for this battle. As ops people, most of us have an innate knack for organization.

Let's check our inventory. There's two things that we've got that we can use for this boss battle: organized context brain and decision provenance. Let's unpack both of these. How to Build a Shared Context Layer for AI Agents - Phil: So how do we build a shared context layer for AI agents or an organized context brain?

Lindsay: Context rot and fragmentation come from context that was never organized in the first place. Lindsay's team at Zapier had built a shared brain in Google Drive as, like, the first step to this. So we built a shared brain in, um, Google Drive to start. Um, and that's sort of evolved since then, but our initial version was a Google Drive, and it's very structured folders.

So company d- you know, company level, uh, department level, team level, and working group level. Um, so we started by populating context from across the org into those, um, Google Drive folders. And, um, first, I will say converting, uh, the materials in those folders into markdown files. So, uh, that was sort of our first iteration of, like, building the shared brain.

And so, um, I helped sort of facilitate that from the go-to-market side, Phil: So Daryl actually asked Lindsay, like, how long did it take? And she said it was a, a sprint that took four weeks. Most of the work was converting documentation, uh, the organization already had into format agents could actually use and read more easily, not writing new content from scratch specifically. Um, and I feel like Lindsay is working through these problems at Zapier from the operational side, right?

And, like, her team started being deliberate about closing out Slack threads with explicit decisions and documenting what was agreed in meetings, not just action items, but, like, the actual reasoning behind them. Um, and the, the joke that she mentions here kinda captures what it actually requires Lindsay: Um, but then the other layer of context that I think folks are still really figuring out how to utilize in the best way is- Just all the day-to-day conversations, like we were mentioning before, that happen.

So ideas, experiments that have launched, um, changes to lead routing or changes to territories or, you know, anything that's sort of happening day to day in the organization. If you can figure out how to harness that, then you can have agents become very powerful because they can connect the dots from the data layer to why. Um, and so that context often is living in humans' brains or in conversations or in meetings. So what we're doing about that is starting to be really, um, particular about documenting, um, sharing decisions, like at the end of a Slack thread, closing it out, like, "Here's what was decided."

At an end of a meeting, making it really clear what decisions were made and which were not. Like, my partner is sort of in a similar field, and we joke like, "Are you gonna have to start saying at the end of meetings, like, 'Let the record show that...'" You know? Like, in a way that an agent can understand, because we are seeing them get better and better at sort of sussing out, um, things that happen, but it's not perfect yet.

And so do you wanna rely on that for updating your full context layer? Maybe not necessarily just yet. Um, so we're taking stepping stones, and the thing that I'm thinking about is, like, those decisions, documenting, putting things in public. Um, an idea that I have, um, that I think we're gonna try out with a small team and go to market at Zapier is having, um, an agent that actually collects all that context from, like, Slack, um, from meeting notes, and puts it into a decision log at the end of the week.

So we can see, like, here are all the decisions that were made or things that were discussed, and I would call that, like, um, you know, a first step. And then having a human be able to review that and say, like, "Okay, this is an update we wanna make. This is not," and having agents then update the context layers. Um, that's a way I'm thinking about it.

Phil: So the slow context layer in this case here, strategy, ICP definitions, it's a documentation problem. And the fast layer, routing changes, precision, uh, pricing decisions, experiment outcomes, those are decision provenance problems, and both have to be solved for an agent to work from anything more than a frozen snapshot of what the organization used to believe. There's one more component the context layer needs, and it is a new class of company data in decision provenance, introduced in 2018: the reasoning chains and approval histories that explain why consequential choices were made.

So example questions are like, why did the team hold this campaign in the first place? What service incident justified this exception? What precedent, precedents did leadership invoke when the policy changed? Like in a world where agents are taking actions faster than humans can document them, that institutional memory has to live somewhere the next agent or the next human can read from it.

Without it, the context layer has no memory of what it actually decided and no way to audit whether an agent's action was consistent with that intent Dael Williamson, who's EMEA Field CTO at Databricks, and he described the structural risk that's underneath all of this. "Without standardized semantic layers and programming interfaces, organizations are risking isolated pools of intelligence, replicating data silos of the past in a new, harder-to-see form. The data silos took 15 years to dismantle.

The intelligence silos are forming now, one proprietary model at a time." So decision provenance sounds way fancier than it is. It sounds like a sci-fi term almost. But the minimum viable version is a shared document that's updated weekly with one entry per consequential decision.

What was decided, who made the call, what alternatives were considered, and what evidence was used. It's almost like an “I told you so” ledger for people that were against a certain decision. The document is what an agent reads instead of reconstructing logic from Slack threads or disconnected meeting notes. once the habit exists, like the tooling to automate the collection becomes a bit easier.

Uh, Lindsay has this like agent-driven decision log at Zapier, who we heard from earlier in this episode. This is like the automated version of this. Like start with the manual version first, um, but her, her version is basically like an, an agent that, that finds those decisions and starts cataloging them for you. we, we covered a bunch of different ways to, to compete, like the fragmentation and context rot here.

Uh, but obviously, you know, there - this goes really deeper, uh, and for anyone that wants to, uh, take a deeper step into like solving context rot and fragmentation, uh, beyond some of the approaches we covered here, there's, uh, I'll link out to this in the show notes, the, the prompting guides context engineering section. it, it covers the architectural pattern in more detail, like layering context across system, task, tool, and memory levels. Uh, you can also find ways of adjusting it dynamically as task complexity and execution history changes.

Anyways, the core principle is that context works best when it's delivered just before the agent needs it in that like just-in-time window. Uh, for more research-oriented angle on coherent synthesis across multiple sources, I'll link out to this one as well. It's the NJUNLP context synthesis project on GitHub. It explores generating synthetic background context from short instruction answer pairs so that you can train models to handle longer multi-source inputs without losing that coherence between them.

It's super fascinating. But anyways, I'll, I'll link out to those there. Testing Whether Your Context Layer Works - Phil: As we're kinda like battling the, the context boss here and like the data quality elements here, there's a quick fun diagnostic to whether an AI system has been built on a stable context foundation or a shaky one. Um, let's hear from István Mészáros, the founder and CEO of Mitzu, who built warehouse-native analytics at scale, was featured in a, a few parts of our first episode in this crawl.

imagine you are in a meeting with the, like in the boardroom, so a board meeting or something like that. And there's a question popping up. How many sales items we had like last month. István: Can you type it in the chat box and like that chat box and like some number come up and it was like, okay, do we trust this?

But did it do something? Uh, actually I have a really good, uh, Turing test. Let's say, let's go with the Turing test. Ask the AI the same question twice.

If they, if it gives you the same answer, that's most likely okay. Uh, for me, I still, even with newer GPT models, I still very often get for the same question, two different answers if I, for example, I ask like a SQL type of question, like, how would you solve this in this data warehouse? This is my data, this is my table. How would you calculate something?

And it often still creates syntax errors. So I, it, it is getting there, but it's not there yet. Phil: So if the answer changes depending on who is asking the question and when, the system is generating a plausible response, maybe not reasoning from evidence, uh, in a board meeting, that could be really embarrassing. In a production agent stack, it's a different category of problem.

The fix is building the context layer that makes outputs traceable in the first place. István adds, like, a description of what that looks like when it István: One more thing I would like to highlight here is that the end-to-end traceability of why we see a number on the chart is, is possible with this type of technology. Uh, for example, via us, we just present the SQL query that we generate. You can copy paste it, actually, you can copy paste your BI tool if you want to have it in your, like, in your static dashboards.

Uh, the same query. You can create the same chart in the BI tool that is the same as what we built or what we show. And this end-to-end traceability is the one that is making it not a black box anymore. It's, it's an open horizon, essentially.

You can look into it how the things work. You can see the joins. If we use joins, you can see all the other things that we do in the SQL and data analyst, like marketing ops. People can review it line by line.

And often our customer calls, we do it. We, we do review line by line, the follow-up query: why things happen. What is the, how do we calculate conversion rate? How do we calculate time to convert metrics?

Um, all sort of things like that. We, we can like review and, and talk about it. And they might have like a different opinion about like how to calculate retention for example. Um, but again, that's a good discussion to have because now we can finally actually talk about this.

Phil: That's the difference between a context layer and a black box. One produces an answer, the other one gives you an answer that you can actually audit and trace back. Boom. We've made it through our second floor.

NEW ACHIEVEMENT: The Meaning Layer Is Live - Phil: New achievement: The Meaning Layer Is Live. Phil: Data quality stops agents from lying about what the data warehouse contains. Phil: Context engineering closes the gap between governed data and what the agent understands. Our semantic layer represents what our domain actually means, the concepts, relationships, constraints, and inference rules, and context rot and fragmentation are defeated by building and maintaining that shared layer before agents run on top of it.

What you have underneath this floor is agents that interpret that data with the right context, the shared definitions that hold across every system, and a meaning layer that carries into the next floor. So we are two floors down, almost four boss forms defeated. Um, yeah, it's been super fun.