The B2B Podcast Index
Index
All categories
MarketingSalesSaaSFinanceHROpsLeadershipCustomer SuccessAI & DataProductStartups & FoundersRevOpsEngineering & DevTools
MethodologySubmit
Best of:MarketingSalesSaaSFinanceHROpsLeadershipCustomer SuccessAI & DataProductStartups & FoundersRevOpsEngineering & DevTools
An independent project byFame
SearchBest episodesGuestsInsightsMethodologySubmit a podcast
Index/AI & Data/Don't Panic! It's Just Data
Don't Panic! It's Just Data artwork

How Enterprise Data Architecture Must Evolve for the Age of AI

Don't Panic! It's Just Data · 2026-06-15 · 32 min

0:00--:--

Key moments - from our scoring

Substance score

46 / 100

Five dimensions, 20 points each

Insight Density9 / 20
Originality8 / 20
Guest Caliber13 / 20
Specificity & Evidence9 / 20
Conversational Craft7 / 20

The enterprise data infrastructure landscape is undergoing a fundamental shift driven by agentic AI adoption, yet most organizations are forcing AI agents onto architectures originally built for traditional applications. Karthik Raghanathan draws historical parallels - from relational databases through NoSQL to cloud-native systems - to show how each era required architectural reimagining. Today's critical gap stems from three architectural problems: agents must model and build workflows on the fly across multiple databases without human review, creating inefficiency and non-determinism; knowledge bases and memory are treated as siloed systems when they should flow together like they do in human cognition; and organizations must simultaneously optimize for massive scale and tiny, numerous concurrent operations. Yugabyte's new Mekko product directly addresses these gaps by enabling agents to share decision traces and reasoning pathways rather than just outputs, reducing token burn and improving multi-agent collaboration. This infrastructure shift mirrors fundamental shifts from the mid-70s (traditional databases), mid-90s (open-source databases), mid-2000s (NoSQL for scale), and mid-2010s (cloud-native systems) - each solving for reliability, scalability, and operational efficiency in their era.

Key takeaways

  • →Agentic applications require agents to share decision traces and reasoning pathways with other agents, not just final outputs, to avoid redundant token consumption and improve answer quality.
  • →Knowledge and memory must be treated as interconnected systems that flow into each other, not as separate silos, mirroring how humans learn and operationalize information.
  • →Current multi-database architectures force agents into inefficient, non-deterministic data modeling decisions made on the fly without human review, breaking down data infrastructure beneath agentic workflows.
  • →Context sharing and lineage tracking across vector stores, graphs, and relational systems is essential for agents to understand how facts were derived and improve collective outcomes.
  • →YugabyteDB makes PostgreSQL resilient and scalable for cloud environments by building reliable systems on inherently unreliable cloud substrates through distributed architecture.

Guests

Karthik Raghanathan

Topics in this episode

Agentic AIVector databasesMulti-agent systemsPostgreSQLCloud-native infrastructureGraph databasesYugabyteYugabyteDBMekkoApache Cassandra

Questions this episode answers

What is Mekko and how does it help agents work together?

Mekko is a data infrastructure engine that enables agents to share context, decision traces, and reasoning pathways with each other. It allows memories to be harvested into collective knowledge bases with lineage tracking, so agents understand not just facts but how those facts were derived, reducing redundant processing and improving collaboration.

Why do current multi-database architectures fail for agentic AI applications?

Current approaches force agents to model workflows across relational, vector, and graph databases on the fly without human review. Since agents are non-deterministic and lack full human context, this creates inefficient, partial data modeling decisions repeated thousands of times rather than establishing repeatable primitives like traditional databases did.

How does YugabyteDB differ from traditional PostgreSQL for enterprise use?

YugabyteDB takes PostgreSQL and makes it enterprise-grade, cloud-native, and resilient by distributing it across multiple nodes for scalability, availability, and failure tolerance while maintaining PostgreSQL's familiar API on unreliable cloud substrates.

What is the difference between knowledge and memory in agentic systems?

Knowledge is permanent, factual storage - information synthesized and validated over time. Memory is the short-term ingestion of conversations and experiences. Agents should cycle between them: reading becomes memory, memory is digested and promoted to knowledge, and knowledge enables more meaningful interactions that create new memories.

Why is context so critical when scaling AI workflows?

Context enables higher-quality answers because it includes decision traces and reasoning pathways, allowing agents to build on prior work rather than solving problems in silos. It also controls costs by preventing redundant token consumption when multiple agents independently solve the same problems.

What our scoring noted

Our reviewer’s read on each dimension, with quotes from the episode.

Insight Density

9 / 20

There are a few genuinely interesting ideas - the knowledge/memory duality for agents, the problem of non-deterministic LLM data modeling creating downstream inefficiencies, and the concept of decision traces - but they are buried under a lengthy database history recap and extended vendor pitch for MeKo. The ratio of novel-per-minute to filler is low.

agents and like LLMs in specific are notoriously non deterministic. If you run two times, they give you different answers and they're not exactly the experts in how to do the best Data modeling
you're burning tokens to get a different agent to guess why you would have come up with something instead of just sharing what you did

Originality

8 / 20

The knowledge-versus-memory framing applied to agentic infrastructure is moderately fresh, and the decision-trace argument has some novelty, but the episode repeatedly falls back on recycled takes: 'AI is only as good as the data,' 'silos are bad,' and a well-worn database genealogy tour. The host even coins 'context is the new oil' and claims it as original.

context is the new oil. You heard it here first
your AI is only as good as the data

Guest Caliber

13 / 20

Karthik has genuine practitioner depth - co-building Apache Cassandra at Facebook, working on HBase and RocksDB, co-founding Nutanix before Yugabyte - making him a credible infrastructure operator. However, the conversation functions largely as a product launch vehicle for MeKo, which caps how much raw operator wisdom surfaces.

my co founder and I and a bunch of others, we ended up building Apache, Cassandra and open sourced it. We worked on HBase, we worked with a team that built RocksDB
I was at Nutanix, a company that builds distributed block stores

Specificity & Evidence

9 / 20

There are a handful of concrete anchors - Paramount Plus scaling for Champions League finals, four named internal agents, a rough '5 to 10 million per year' savings figure - but customer metrics are unattributed and approximate, no adoption or benchmark data is provided for MeKo, and most claims about efficiency gains and cost control remain hand-wavy.

we have customers, um, that are using us. Like for example, a Paramount plus, like one of their workloads is they need to be able to adapt on the fly
typically I think the number comes to about you know, 5 to 10 million per year is what we're able to save in a lot of cases

Conversational Craft

7 / 20

The host asks one worthwhile follow-up (pressing deeper on knowledge vs. memory) but otherwise lobs softballs, never challenges product claims, and inserts filler asides ('context is the new oil, you heard it here first') that break momentum. The interview functions as an unchallenged press release for a new product launch.

Can you dive a little bit further into this idea of knowledge versus memory? I think it's fascinating. It's not. I don't hear that a lot
Well, this has been great stuff. This is a wrap, though. We're out of time here

Conversation analysis

Computed from the transcript - who did the talking, and the words that came up most.

Share of words spoken

  • Speaker B89%
  • Speaker A11%

Most-used words

data41knowledge32agentic27agents25infrastructure23databases20different15problem15back15context15memory15scale13first13bunch12example12together12

Episode notes

Most enterprises believe they have a data problem. In reality, it is an architecture problem in disguise, and the rise of agentic AI is making that distinction impossible to ignore. That is the central argument Karthik Ranganathan , CEO of Yugabyte , makes in this episode of Don’t Panic! It’s Just Data , hosted by Scott Taylor of MetaMeta Consulting. The conversation traces 30 years of infrastructure evolution in 30 minutes. From Oracle’s dominance as the monolithic backbone of enterprise applications, to the NoSQL revolution of the mid-2000s, and the cloud-native era of the 2010s, it builds toward the rise of agentic AI. In this new phase, systems do not just store and retrieve data; they act on it autonomously. “The challenges of current architectures under pressure are no longer theoretical. Agentic systems expose every seam, every silo, every bottleneck you've been quietly managing around,” says Ranganathan . Why Current Architectures Crack Under Agentic Pressure Taylor opens the episode by asking the question many enterprise data leaders are quietly asking: Are today’s architectures actually fit for AI workloads? Raghanathan’s answer is measured.

Full transcript

32 min

Transcribed and scored by The B2B Podcast Index.

Speaker A: Foreign. Hello and welcome to Don't Panic. It's Just Data, the podcast where we explore how data and emerging technologies are shaping enterprise business. My name is Scott Taylor, the Data Whisperer and I'm a full time content creator focused on helping organizations craft a business successful narrative to gain executive support, stakeholder engagement and ongoing funding for enterprise data management efforts at and I'm pleased to be your host. Today, enterprises are racing to adopt AI agents and autonomous systems. But many are discovering that their data infrastructure was never built for this level of complexity. Instead of unified intelligence, organizations are managing fragmented architectures, disconnected vector stores and siloed workflows. Can't we ever get rid of silos? Well, to help us through this all, I am pleased to bring on Karthik Raghanathan, CEO and co founder of Yugabyte. He will discuss how data infrastructure is evolving for the age of agentic AI, why current architectures are starting to crack under pressure, and what comes next. Karthik, help us. Welcome to the show.

Speaker B: Thank you. Thanks Scott. Thanks for having me on. Thanks everybody that tuned in. Thanks for listening.

Speaker A: Absolutely. So before we dive in too deep, can you give us a little background on yourself and also a bit about Yugabyte?

Speaker B: Absolutely. So on myself. I'm a technologist at heart and uh, before we started Yugabyte, my co founder and I, we started Yugabyte about 10 years ago. I was at Nutanix, a company that builds distributed block stores. So think of it as the storage or the disk layer that runs in a private cloud in a distributed manner, keeping data resilient and scalable. And before that at Meta, I mean I still refer to it as Facebook but it's know meta and that's the company and that's the company where like you know, we ended up building my team, my co founder and I and a bunch of others, we ended up building Apache, Cassandra and open sourced it. We worked on HBase, we worked with a team that built RocksDB. We worked on yet another system internally called Zippy DB, which is all about scale and resilience and availability for modern workloads. So yes, a lot of data oriented background. And you know what Yugabyte does, surprise, surprise, is you know, resilient and scalable data. Right, right. And this time it's in the form of um, APIs that are very easy to understand like Postgres. So it makes Postgres resilient and scalable and cloud native.

Speaker A: Well, a great background. So. But when you think about kind of the origin Story here. What led you to focus on the future of enterprise data infrastructure and agentic AI?

Speaker B: Absolutely. So it's just an interesting transition in the history of databases. Right. If I'm to paint the broad strokes. Right. First they were the original databases. They were the first databases and databases accelerated application development. It was as simple as that. And people didn't have to reinvent primitives that they needed over and over again. And once you put it into the database, it's a lot simpler, it's a lot cleaner. Everybody understands how these primitives work. They're very repeatable. And from the original traditional relational databases like your Oracles and so on, and you then had the open source variants of those databases like MySQL and Postgres, especially when the integrated Internet came up. Right. For the LAMP stock, for example, a lot of small, apparently low value applications, which then became very high value over time and high scale. And the traditional commercial models didn't work. So it was open source and it was supported a different way. Right. Fast forward to about the mid 2000s. So this was like in the mid 90s and the mid 2000s, you had the NoSQL databases come up when there was the need for scale. And Mongo specifically about mobile applications. Right. How do you make that simpler? So scale and mobile applications. And we were a part of the NoSQL wave with, you know, Cassandra and HBase and a lot of these type of databases. Right. So and that was a time when a single node couldn't handle everything. You needed multiple nodes for resilience, availability, scale, all of those kind of things. Then I think in the mid-2000, 2010s, which is when Yugabyte was born, around 2016, was the acceleration to cloud. Right. Like how can you move everything to the cloud and run in cloud? Native cloud is a noisy and unreliable substrate. Not because anyone did anything wrong, it's just, uh, it's cheaper and it's at scale and therefore you do expect failures. Right. And so it was about how do you build reliable systems on top of, you know, a substrate that is economic, yet not as reliable. Right. So that was our original flagship product, YugabyteDB. And that's what it does, it puts Postgres and makes it enterprise grade and cloud native on top of any cloud environment. Right. Private or public. Now in the so and, and by the way, this was mid 70s to 80s, then mid 90s, then mid 2000s,

Speaker A: bringing us all the way up here. Yeah, beautiful.

Speaker B: Now we're in the mid-2020s. Right. And so, you know, the newer thing is around agentic and agentic applications and smarter applications and, and you know, if you look at it, a similar set of challenges have arisen. I mean they always are different in the flavor, but you know, the nature of the challenge is very similar. It starts to pose itself in the form of how do you give flexible, uh, reliable, resilient and scalable systems for agentic applications? Right. And that's a big gap. And what we have thanks to Postgres and its extreme success over the last decade is that, you know, we have all of these different data models that you can put on top of postgres like relational and uh, vector and graph and more. And with yugabyte DB in specific, making Postgres resilient and scalable, you now have the NoSQL flavors inside there as well. You can always scale it and you can keep it high. But there's still a gap to bridging, to going from what this substrate is to what the agentic applications need. And we got into Agentic because people kept asking us, is your query layer optimal for the different pools of data? Which it is. But it was really about the intention of what was being done across these different layers, not inside any one of them. And to understand that we need a data infrastructure that makes it repeatable across the layers, not inside a layer. Right. That's the data view of it. Obviously from the agentic view they're just trying to get some of these repeatable patterns, which is what databases did for applications in the first place to actually work for Agentic. And they're different. So that's the gap we're filling with the, with the newer product that we just announced. So that's kind of the evolution and how we got in.

Speaker A: Right? Yeah, wonderful perspective there, pulling the whole thing together. And NoSQL, I love to ask people this. The no and NoSQL stands for.

Speaker B: Well it's not only SQL.

Speaker A: Not only. Right. So no equals yes in the data space. No wonder they don't understand this in the business.

Speaker B: But I think there's a dichotomy there. I think there's a dichotomy which is very interesting. NoSQL actually, it's just my perspective, it's SQL gave some really strong and powerful, uh, primitives from the language perspective. NoSQL gave powerful primitives from the scale and resilience of the non functional requirements perspective, but took away a lot of the primitives from the SQL. And we were a part of that problem because we thought it's way too complicated. People don't want that complexity. Let's give them something simple. And then when people got used to the simple but scalable and resilient, they were like, no, no, we want all of that other stuff back. And then when you looked at, okay, bringing all of that other stuff back, well, the whole architecture needs to change. So.

Speaker A: And speaking of architecture, I mean there's a lot of conversation around in the enterprise space around this thing that some people are calling an architecture war. So what's your perspective on that?

Speaker B: Yeah, this is a great question. Right. So today the de facto of when people are trying to implement agentic applications is let's take agentic stuff and put it on top of known databases, right? And so they say like, you know, relational solve problem. There's postgres, there's graph, there's vector, there's all of these things and there's a multiple class of databases inside each. Just pick one and then shove it in there. You have data infrastructure and why don't we just build agentic on top? And that's kind of what a lot of people are doing and of course trying to improve these databases to kind of work in the agentic way. I'll give you an API based way of spinning it up and make it cheaper and so on. Right. But there are three fundamental problems with this architecture. The first one is that agents are not. We are asking agents to do the modeling and the building of the application and the workflows on the fly by doing this because they have to take high level concepts of what an agentic application wants to do, which is around give me a decision, give me a pathway to make a decision and in fact use that decision and take the next step of actually doing what it means. Rather than give me a way to build an app to put it on top of multiple different databases and for me to review if it actually works. We've kind of abstracted that whole thing away. Right? So when you're going decision oriented, nobody wants to come in and, and review what's happening beneath the hood, right? That's the first problem. And the problem with the way it shows up is it's not one, but it's thousands of different repetitions of those workflows. And the way you're mapping it into the data stores are done on the fly. And agents and like LLMs in specific are notoriously non deterministic. If you run two times, they give you different answers and they're not exactly the experts in how to do the best Data modeling across along with some inputs from humans who cannot translate their entire mental context because it's very difficult to write everything, you know, inside on a piece of paper. It's just not possible. So, so like, like if we asked ourselves, what did you learn in life? You can't really answer that question in the context of something. Yes. But it's always partial. Right? So and agents don't understand partial and they're non deterministic and you put all this together, it becomes terribly inefficient. Right. So that's the first one which is moving towards decisions and workflows. Very specifically agentic are now breaking down the data infrastructure silo below. The second thing that I'd say is that there is like even inside the agentic, uh, world we have two silos. We have the data and knowledge base and all of that, that's one side and then the other side is conversations and memories. That's another side. But if you take a parallel to agents being like humans, because they are and they learn like humans, we do not. Like, for example, your knowledge base is like a library or a computer with stuff like the Internet. You go and refer and you learn new things. And your mem, when you start to operationalize it, you learn other smaller, newer things in real time and you have to put it all together to make a useful system. But uh, we're treating them as silos. What I read on the Internet and what I learn in doing something, I can never match them, map them, use them together. Right. That causes a problem and that causes a problem that the two silos have to grow independent of each other and the outcome is very, very suboptimal. Right. So that's the second one. Right. And then the third one is that, you know, it's kind of a uh, war of extremes when it comes to data. On one side is this knowledge based thing which is highly scalable. On the other side is this agentic thing where there's a lot of agents that are doing tiny stuff and they all add up to be a lot. But it's not one thing, it's many things. Right. And not to mention the speed at which it's going. You need to know where did this data come from? Where did that information that come. So you need to track lineage, effectiveness, utility along with how do you optimize your, your data infrastructure for large and small simultaneously. Right. So I'd say these are the three things that are three of the many things, but three things at the top level, which is agentic, primitives, you know, having to be very efficiently mapped. The second one being knowledge and memory being treated as silos, but they're not. And you know, the third thing being you have a war of extremes where you have scale and multitenancy, extreme multi tendency being mushed together. Right. So you have to solve for all of these. And that causes the architecture to shift quite a bit for agentic compared to traditional, you know.

Speaker A: Can you dive a little bit further into this idea of knowledge versus memory? I think it's fascinating. It's not. I don't hear that a lot. So I'd love for you to just kind of give me a little more around that.

Speaker B: Absolutely. So maybe we'll just use humans as examples. Right?

Speaker A: So good example. Yeah.

Speaker B: We're all human and we have to make agents work like us because, you know, we know a way that works and there's no need to find another way that works because we already have one. So you can think of knowledge is as a fact, right? So knowledge is like, for example, let's say we're a team, right? You and I, and we're working on something, knowledge and, and let's say we're working together to go do, to achieve a, uh, greater purpose and impact, something that we want to achieve together. So I take, I say I'm going to work on this portion, Scott, and you're going to work on the other portions, what you say. And so we go and go read about our independent portions, right? That's knowledge. We now have learned a lot. We have knowledge now. We want to apply this back into a product that we want to build together or a project that we want to do together. We need to bring back the, not the entire knowledge. Like we might have read each for a week and come back. We don't want to spend a week transferring the same information. We need to get the gist of it across along with our assumptions and what, what we think is the value and our synthesis of it and exchange that and kind of iterate on it until we find a good thing to build. And then we say these are the portions we're going to build. And then we go back, right? And then we work on it. And as we build this thing, we will now find, oh, wait, this assumption doesn't quite work. And that thing has to change a little bit. And this one is very hard to do. And we have to keep coming back. We have to work in these iterations, right? So I call knowledge, the research we did and the stuff we stored in our brains and memory is closer to, you know, when we, you know, did all these things and we read it, we have to keep it in our brains. What is it that we have just talked about, what is it? So that becomes memory. So when you read it, it becomes memory. Or when you talk and ingest something, it becomes memory. When you digest it and you know, what is fact and useful versus what is not, the useful factual stuff becomes knowledge, right? So it goes in the cycle where you read it, you put it in memory, you process it and you shove it into knowledge. Now you have more knowledge, you can have more meaningful conversations which will give you more memories or conversations or read more. Whatever it is you do, it'll give you more memories which you digest more, you put it more in the knowledge base. So it just keeps going in a cycle, right? So knowledge is a permanent store of facts. Memory is, you know, the ingestion of what happened and the short term thing

Speaker A: that you process and you've got some applications working on the right Meco, isn't that it? Memory, knowledge, that's the brand name there. Can you just give us a little quick idea of what that does?

Speaker B: Absolutely. So Mikko, as you said, me is for memory and KO is for knowledge. And it really captures the interplay between the two sides. As I said, one of the architectural war pillars was the distinction, the hard line between memory and knowledge. And it's not the case in reality. Right? Like as humans, we, we read, we remember, we then operationalize and we might forget, but we can always go and refer to knowledge to go get it back because we cannot remember everything, but we have a working set that we work with and we keep making progress through that because it's the assumptions, the principles that we remember. Agents need to work in a similar fashion and they don't have data infrastructure that lets them cross this boundary. Memories are stored as memories, knowledge is stored as knowledge. We work on improving the contextual quality of a, uh, knowledge base independent of improving the contextual quality of the memories you have. But they have to flow between each other, right? The memories have to get harvested into the knowledge base so that you pull out facts and useful stuff and put it there. And like us, like humans, sometimes bad, bad knowledge gets in like, you know, some wrong. You read something in the Internet which is wrong and but you thought it was right and it gets corrected and, and then so you can ask like, hey, where did that. Oh, that came from that location and maybe I shouldn't trust that location or maybe that location, I need to go inform m them to go fix it or something. Right. So it's very similar here. And that means you need to know where it came from and trace it and understand the context in which this information was produced. And maybe it needs a little bit of if conditions to say these are the cases when it holds true, and otherwise not so. Mikko is this engine at the data infrastructure level that enables you to share the context with other others. That's the first thing that's required. Because if you cannot share, there's no point because you can do it by yourself, but you're not achieving anything as a team. So it's a data infrastructure for agents that can work together as a team, agents and humans. And secondly, it, it is able to take these conversations, extract relevant memories, and then, you know, extract and, and then find out which of these memories are actually relevant to the project at hand and promote them to knowledge and collective memories. So, so instead of one person remembering, it's like a hive mind that can remember and promote the knowledge and you can see where it came from. And that way the impact is much better.

Speaker A: So what are some of the other operational challenges though, that are created by fragmentation, especially in these, as you're talking about these multi agent environments?

Speaker B: I mean, let's start with simple things that we want to model, right? Okay, so let's say you and I are working on this project and we are able, like, how would we do it as humans first, right? So you'd put down some notes to say, here's what I, I kind of figured out, I'd do the same. And then we'd come back and we'd start talking about, here are the points. And then you'd say, a point number two is, you know, we should talk about, you know, this particular, or this is what I found. And immediately I'd go like, how do you find that? Because I found something similar or I found something different. That's what I'm thinking. But I want to know how you found it, Right? Now imagine a world where you're not allowed to tell me how you found it. You can only give me the output. So now I'm going to be staring at your list and saying, so Scott said that, and he must have found this by doing something. And I think it's this thing. And now let me propose this instead. And I now propose something. Now you're going to look at what I propose and say, hm, he looked at what I said and must have thought this. And we're never going to converge. In fact, we're Going to go way offline, right? It's way off base. And that's what agents are doing today, right. They don't have an infrastructure where they can look at the reasoning from each other. Right? That's number one. Now let's make it a little worse, right? That's what it is with the agentic world. It's not that I'm staring at the bullet you wrote and I'm thinking how you got it. I'm actually paying somebody to figure out what, why you wrote what you wrote because you're burning tokens along the way, right? So, so we're burning tokens to get a different agent to guess why you would have come up with something instead of just sharing what you did. Right. So and sharing what you did actually helps the other reason. So this is the gap or this is what I need to fix in my reasoning. And that can get make it a lot more efficient, Right. So that's what Mako enables. It enables the decision traces. The uh, decision trace, meaning what was the conversation and the set of processing that led to a particular fact being extracted. That along with, you know, the complex problem of how do you share context? So you can share a bunch of databases, but how do you make sense of the fact that a single conversation lives in one particular location and is tied to like, you know, vector and graph and other aspects for it to get processed into memory, which then got promoted to a vector and a storage piece for knowledge? You will not be able to see this chain, you'll just be able to see a bunch of data. Right? And so the agent looking at it is like, okay, I see a bunch of data, I'm going to use a bunch of data instead of talking about this is the fact. And this was the chain that led to it being derived, right? So it kind of gives you a flavor. Now that's one agentic workflow, or like one is how do you share with a different agent? Very simple question. And then how do you reason? Or how do you look at the reason behind a fact getting derived? And how do I actually use my collective memory? How do I use my collective knowledge to improve results and more and more and more, right? So there's multiple different such agentic workflows that we all have to like, think about and make them efficient and make them, you know, high quality and making

Speaker A: sure everybody understands that. Context. Context is so hot these days. Context is the new oil. You heard it here first. Uh, but it is, in all seriousness, it's really critical. You know, when you think about infrastructure design. Why is that so critical when you're starting to scale some of these AI workflows?

Speaker B: Yeah, that's a great question. So ultimately when you think about what you want to do from agentic workloads, you want to derive like you know, a couple of outcomes. You basically want higher quality answers, right? So that means that the, if I asked a question, if you give me a better answer, I can do more with it than if you give me a really bad answer. I now have to work on improving that answer.

Speaker A: Right?

Speaker B: And if each of us gets a bad answer, then we all work on uh, uh, put most of our efforts in improving it and we're all doing it in silos and not reusing it. So that's the first problem. And the second problem is, you know, let's kind of not break the bank doing it. Right. I got to make sure that the output and the value we give is proportional to, and, and the uh, and therefore the money that you're able to derive from the value produced is kind of in line with the amount spent, not like the amount spent is 10 times more for the value generated. It's kind of pointless. Right. So these are the two things at the highest level and when you look at what really breaks it underneath, right. From an infrastructure point of view, firstly, there is no vehicle to do this in a pre optimized manner. That means you're kind of bushwhacking your way into it on the fly using agents in humans. And uh, any level of inefficiency starts to get amplified both in terms of poor quality of context as well as, you know, the cost economics.

Speaker A: Both.

Speaker B: Right. Both start to get dinged. Poor quality of context in the sense we still need a lot of humans in the loop every now and then to improve the system. Otherwise we're all out of a job and thankfully we're not there yet. But yeah, so increasing this quality of context means you should be able to reason and look at what goes on and you should be able to attribute, I got a higher quality from this location, these things. We're not able to do that with the current infrastructure today. No one thinks that way. And uh, the second one is actually the token burn. And the infrastructure efficiency problem is far worse. Agents, you can easily hit a million agents in a company, very easy because there's a number of projects and there's a bunch of people working on each project and each person uses a bunch of different agents to do it. So when you multiply all that out, it's a big Number it gets out of control really quick out of control. And if you're spinning up databases on the fly and, and be that they're tiny databases, fine. And everybody starts putting this kind of context and then working on improving it in parallel, then the cost economics of that get really out of control. Right. And so what Mikko does is, takes a more opinionated stance. It says that it's going to map all of these workflows into the most efficient infrastructure that you can do and it's going to give you context attribution for when the context drops in quality. Well, we can't improve it, but you can go look at it very quickly and see why and keep all of this in like the best possible cost control. Right. So that's kind of the approach. And so it allows the sharing without the cost burden and the improving, whether it's by human or agent in the loop.

Speaker A: There's still a gap though, in a lot of enterprises between AI experimentation and getting really tangible business value. So I'd love to hear any anecdote you have from your customers where you really help them get some productivity and some business gains.

Speaker B: Absolutely, absolutely. So we have answers from both customers and ourselves. And we look at ourselves. I mean, in the world of AI, right? Like you can always find somebody struggling with a specific problem. So I think it's more important to read the market as a whole and it's like, and we do it using ourselves also, how are we as technologists trying to solve this problem for ourselves? So I'll talk about more than specific customers. Like, we have a lot of customers where we're able to, for example, cut their uh, ability to scale, manage infrastructure, cut their ability to like, uh, their cost of uptime, like the, the, the impact of downtime straight by half, right? Like it just goes down by half immediately and then keep improving from there, for example.

Speaker A: Right.

Speaker B: And so when AI based systems start taking decisions, that amounts to millions of dollars in a year, right. So typically I think the number comes to about you know, 5 to 10 million per year is what we're able to save in a lot of cases. Right. So, um, and similarly the cost of scaling, right. Like we're able to just take that away and make it variable scale. Like so we have customers, um, that are using us. Like for example, a Paramount plus, like one of their workloads is they need to be able to adapt on the fly. They get a big event like a Champions League final, which has happened a few weeks back, that requires a much larger audience to serve the infrastructure and the agent systems underneath. Two, compared to a regular day like that, people are consuming Paramount Plus. And if you're going to, for example Oscars and Grammys and you know, the AFCs, each one is a different size. You have to keep going up, down, up, down. So this kind of scaling, it becomes very critical to be able to solve. And with agentic systems you don't get much time to scale so you have to be able to do it all the time. Right? So, so that's one example. But, but let me step back. I think your question was not just uh, one of how do we help them simplify, manage their infrastructure. It's also about the business value. Right? So let me take that to a step higher. Right. We have at Yugabyte 4 agents we rely a lot on.

Speaker A: Right?

Speaker B: And a phrase I often like to say is it's not the companies with the most agents, it's the companies with the most agents that can work as a team that will win. So are, uh, the. Just uh, even a few agents is enough. Right. So we have uh, four agents. We do use, you know, code agents for coding a lot. Right. So I mean, I'm ignoring that because it's not an agent we build for ourselves to improve efficiency. But we have four agents that we use a lot. Right. The first one is uh, called Hagen. It does customer support and sentiment. Second one is Growth Vector. It helps our go to market teams work in a combined fashion. The third one is Voyager. It helps our customers modernize their workloads from traditional to modern. And the fourth is Perf Advisor, which is like an agentic DBA or a troubleshooting thing. It goes and understands why systems fail very quickly, finds the RCA and more importantly takes those findings and then goes all the way to the developer building the code to say, look, if you write that code you're probably going to get screwed down the road. Right. So these are systems that we apply Both inside, outside, etc.

Speaker A: Right.

Speaker B: And I can tell you about deriving business value for these firsthand. Right. The problem is the dynamic nature of the data. It's why we ended up building makeover. Right. If you take, for uh, let me take any one of these as an example. Let me take for example, um, support. Let's say that we have a support person one that's really killing it and we have a, another support person that's like, okay, can we answer why support person one is killing it and why support person two is not able to rise up to that level? If you Ask them. Humans have notoriously short memories. I mean as in they will remember what happened in the last two days or a week, but they don't know what happened the last three months or a year. It's just an impression, right? A high level impression. It's not specific. Now with something like mako ah attached to an agent, we're actually able to go look at what are the specific scenarios where support person one did better and how can we bring that knowledge and memory into a collective learning and give it to the you know, Support Person's Person 2's agent in order for them to be able to rise up to that level and are uh, there specific workflows that they're doing that we're able to, you know, just beneath the hood just push to agent two, support person two to keep raising their level of impact. Right. It's very simple. But if you cannot go extract that knowledge memory and do that as a team and push it into this, it's not going to happen because you're then back to asking them like give me an output of what were the challenges and they give you something and then you don't know why they wrote it and you don't know if that was a simple or back to the same problem. Right. It happens everywhere though, every single place. So similarly, like our uh, perf advisor, right is another great example. The team actually counts it a success only if they're able to attach to find, troubleshoot, suggest uh, to a customer an improvement or an issue before they're able to tell us and the customer accepts it. So all the way, it has to go through all the way. Right. And we're starting to do a bunch of suggestions a week across so many customers. So obviously it's an exciting time but we have to be able to learn if you're seeing a new issue that you're not able to catch and there's something that comes from the outside, it has to go into the whole system. How do you do that? If people are using this system to debug it and you know they kind of find 80% mark and then the last 20% they're like oh, I know the problem.

Speaker A: Bottom line, what do enterprise leaders need to rethink about data architecture?

Speaker B: Yeah, I think the um, the, the architecture wars that we talked about, I think they're the most important, I'd say from the bottom up. Like the first thing is, you know, there's a, you know there's a way to drive agentic applications to impact and there's Another group that's actually treating the infrastructure for this as a bunch of databases, right? So I think that's a big disconnect because a bunch of databases is just data, it's not information, right? So, and how do you connect them, use them? So I think that becomes super critical. So we should stop treating AI infrastructure as just a bunch of tools, uh, point tools that are shoved together and somehow they're working. So I think that's important. The second one is that I think it's important to realize that, you know, our issue right now in the world is not models. Our models are very capable and they're getting better every day, right? The issue is really the data infrastructure and our models, and we have to be, you know, aware of that. It's the data, that your AI is only as good as the data. And. But also with the realization that models keep getting better. So design for portability, model portability, just portability across the board, right? So you should be able to go from one model to another, one tool, tool chain to another. Like, you know, just inside Yugabyte. We keep having debates every week about, is it, you know, cloud code or Cursor or, you know, or, or OpenAI codecs, all three coding agents and which one is better. And I swear, everybody swears by one or the other and it keeps changing every week, right? So I think our solution is, you know, what, have it have a attachment code to it and keep switching every week, doesn't matter until you figure out the one that works. So we're in that kind of a situation. So it's important to deal with portability. And thirdly, one of the most important but overlooked aspects is the entire decision trace, right? It's not just a sequence of which queries ran where it's like, what, what was the question? What, what did the user say? What did the LLM decide to do and how did it come up with that decision and what did it actually do and what were the results? So if you're not able to see these things, you cannot go back and say, why did my context degrade in quality and my output is terrible and my cost is going up. I want to reason about it. You have to be able to do it methodically, right? So, so I'd say, right, these are the three things, like, you know, not a, not a bunch of tools. It's more holistic than that. The second one is, you know, model portability, tool portability, like, you know, the toolkit portability is super important. And thirdly, data prominence and all of that. Right. So to shrink it. Right. Like don't build on top of existing databases and data infrastructures. Just think about the outcomes and then derive the data infrastructure that's needed. Right. And it got to be agentic first. Right. It cannot be like know agents on top of what we have.

Speaker A: Yeah. It feels, when I think about what you're saying, it feels like they should start thinking about data architecture strategically.

Speaker B: Yes. Or agent, as opposed to take agent take on top of what they've made as a strategic decision today. Right. So.

Speaker A: Yeah.

Speaker B: And it's also why we ended up building a new thing. We already had a database with all these other kinds of databases that the market is pushing towards. And, uh, it's still. There's a gap. Right. So that's really the story.

Speaker A: Well, this has been great stuff. This is a wrap, though. We're out of time here. So for those of you listening, actually parts of this episode, Scott, script and the way we work together was done with the help of Mako. So we used. We. We drank our own champagne or ate our own dog food or whatever the latest way to say that is. Anyway, so thank you, Mako. Thank you so much, Karthick, for joining us. And thank you everybody for listening. For further information on what we've been talking about, please head over to Yugabyte.com. we'll be back next week with another episode of our series. Until then, make sure you subscribe to this podcast wherever your pods are cast, and follow the conversation on our socials at, uh, em360tech and on X and LinkedIn. And for more great daily content, head over to em360tech.com. So this is Scott Taylor, the data whisperer, reminding you that good decisions made on bad data are just bad decisions you don't know about yet. So we'll see you next time. Cheers.

Related episodes across the Index

Other episodes covering the same guests and topics, from across The B2B Podcast Index.

  • Decision Logic: The Difference Between an Answer and a DecisionThe AI Forecast · on Agentic AI87 / 100
  • KYA Won't Always Protect You. The Real Risk Is the Swarm!Fintech Conversations & Insights with Efi Pylarinou · on Agentic AI86 / 100
  • Agentic AI in Sales: What Business Leaders Need to KnowScaling with AI · on Agentic AI86 / 100
  • Is RAG Dead? The Pioneer Who Invented AI's Memory Layer Answers - with Douwe Kiela, Co-Founder, Contextual AI {ICYMI}Making Data Simple · on Vector databases86 / 100
  • EP284 Closest Alligator to the Canoe: How Transforming SOC Became P0 for Lloyds BankCloud Security Podcast by Google · on Agentic AI85 / 100
  • AI Is Ready for Government. Is Government Ready?The So What from BCG · on Agentic AI84 / 100

More from Don't Panic! It's Just Data

All episodes →
  • No Use Case, No Value: Why Managing AI Use Cases is Key to Demonstrating Business Value
  • Why Enterprise RAG Systems Still Produce AI Hallucinations
  • Is “Frankenstein Data” Slowing Down AI Transformation in Insurance Enterprises?
  • Can Your MDM Strategy Survive the Shift to Real-Time AI Decision-Making?
  • Why Is the Semantic Layer Critical for Data Governance, Compliance, and AI at Scale?
Explore the best B2B AI & Data podcasts →
All Don't Panic! It's Just Data episodes →