Unstructured Data in an AI World

Speaker A: Foreign.

Speaker B: Good evening wherever you are, and welcome back to the Enterprise AI Show. I'm your host, Brian Grace Lee. And today we're going to dive into a topic that I feel like we talk about a lot, but we really haven't dove into a lot when that is, data. And really looking at sort of the complexities of what's going on these days in, uh, unstructured data, in file data, in managing capacity, in managing, you know, how much, how much time are you spending making sure that your data is AI ready? How are you dealing with making sure that as AI agents are beginning to just, you know, hammer your inference environments, that the right data for those agents is made available? And it's, and it's, and it's fast and readily available. And we're going to dig into all sorts of topics about data, data economics and other things right after the break. Today's show is sponsored by Nasuni. There's a growing gap in AI right now between what's possible in theory and what successfully works at scale inside an enterprise. The difference comes down to unstructured file data. Many AI, uh, initiatives struggle because the file data they depend on is scattered, unstructured and disconnected from where and how work actually happens. Nasuni changes that. It brings your unstructured file data into a single secure foundation so AI, both generative and agentic, can access it with the context, governance and performance it needs in production. Bring AI to where your unstructured data lives. See what it takes to activate your data for AI and request a demo@nasuni.com AI. Today's show is sponsored by sharegate. You're out of time. Copilot needs to be deployed asap, but your tenant really isn't ready for it. You years of data, permissions and users lurk in its shadows, ready to be exposed by AI. Sharegate Protect sees it all so you can find the exposure risks, fix them Fast, and deploy AI with confidence. Microsoft 365 Governance. They've got this. Learn more at. Ah, sharegate.com protect. And we're back. And folks, you know, we spend a lot of time on this show, obviously talking about models and all the things that are going on in terms of, you know, some of the things that, that you're directly interacting with in terms of your AI, uh, experience, whether it's an agent or a chatbot or other types of things. But the reality is we probably don't spend enough time talking about the data that's going into those models. The data that your business is using to help customize and train those models. And, you know, it's one of those things that if we're not talking about it enough, we're not figuring out what are the best practices, what are the things that people need to be aware of. And so today's going to be exciting. We're really excited to have Jerry Carter, who is CTO at nasuni, joining us today. Jerry, welcome to the show. Great to have you on.

Speaker A: Hey, thanks, Brian. It's great to be here.

Speaker B: Uh, I'm excited to sort of dive into all things data. You guys are obviously right at the forefront of helping customers store data, organize data, make it available. But before we dive into that, give us a little bit about your background. You've obviously been working on a lot of interesting things, but then also, where are you focused today? Primarily at dsuni in your role, obviously overseeing a lot of things.

Speaker A: Yeah, yeah. So it's, it's, uh, a, it's a fun, it's a fun job and it's a fun time to be in the industry. So I, I started off in open source, uh, you know, about 15, 20 years ago, whenever it was. And then that led me through, uh, interoperability and sort of bridging between, like, how to make systems work together. I worked on a project called Samba, kind of back in the day when, you know, Windows NT was a new thing and Solaris was the dominant Unix. And so I got involved in storage kind of through that activity and then kind of through that journey, I ended up at Nasuni. And what really attracted me was this idea of collaboration, this idea of kind of bringing data to where customers need it, kind of at the edge, but being able to manage it in a global file system space, which was just when I was in enterprise storage and I ran several on prem enterprise storage products for large companies. It was challenges that we couldn't solve. And so, and as soon you kind of had that opportunity and then, you know, being kind of a plumber and sort of on the infrastructure side, understanding that there's just a lot of history, there's a lot of intelligence, there's a lot of business experience that's kind of in that operational file layer and being able to kind of open that up for AI workflows for AI agents, and then helping companies and customers be able to figure out how to solve some of the problems that are facing them, um, kind of now, as we're going through this next platform disruption.

Speaker B: So, yeah, yeah, no good stuff. It's, it's. And yeah, we love kind of learning where people Kind of came from, because so many of these things, you know, they, they build upon each other, new things come along. You, you obviously sort of start to see patterns that look like previous patterns. So, uh, good stuff. So, you know, like I mentioned in the, in sort of the opening, you know, we talk a lot about models on this show. Uh, I don't know that we spend enough time, you know, digging into sort of the data behind them. What are you seeing? So we're moving into this era where, you know, again, data never sort of falls out of grace in terms of being really, really important. But now it's, you know, it's data quality, it's, you, uh, know, data hygiene. It's, you know, how quickly can we keep it up to date? How do you, you know, you've been around, you've been around data and storage and so forth for a while. Like how are you sort of framing it in your mind as to what's different with data these days? How are customers thinking about it differently? What's, what's sort of the bigger picture that you see as that intersection between sort of data and where it's feeding into models and agents and other things.

Speaker A: I mean, I think it really just boils down to context. And the way I kind of always think about it is really good context leads to better answers. Right? Mediocre context leads to plausible answers. And I think that's where we run into a lot of challenges and troubles, is where plausible answers, uh, sort of eat at the trust of, well, the model told me something and it didn't really work out. And so to me, data equates to context and context drives AI. When we talk to our customers and we're sort of looking and thinking about kind of the data retention needs that they have or sort of how they're thinking about, well, I have to manage capacity and we have kind of a lot of customers that are dealing with kind of the hardware refresh cycle and supply chain issues at the moment. And they're going, well, I've got to sort of manage like how much data do I need to store? Do I need to keep data that was 10 years old? Do I need to keep data that was only five years? Like, is it a policy based retention or is it, you know, is it a value based retention? And what the people that I talk to, kind of our customers, they're saying, look, I've got to the point, I really don't want to delete anything because I don't know when it's going to be valuable. But at the same time, I don't really know. I don't really know what all is there. So I need kind of some way to be able to kind of manage the unstructured data. And that's really the area where I've always lived. I grew up in nas. We dealt with file protocols. And so it's. It's all about, you know, the PowerPoints or the CAD drawings or the Word documents. Documents or the text files or whatever they are. But they represent sort of the knowledge in the history of the business. And businesses are becoming kind of increasingly protective of that and don't want to delete it. They want it to be available. Maybe they don't want it on, like, the flash tier that they have because that's more expensive, but they want it to be accessible when they need it. So when they're thinking about kind of how that data reflects to their business. There's a time where we talked about data being like the new oil of transformation or new oil of the business. So somebody actually said, well, it's not really oil, it's more like air. And it's kind of this place where it's becoming so valuable that they just want a place to be able to make sure that they can get to it, that it's managed, that it's protected, that, um, their employees can see what they have permission to see, but can't see what they don't. Like you don't have leakage in the data. That's kind of another thing that comes up as well. So a lot of interesting spaces, and it's a hard problem for them to solve because of the kind of massive amounts of unstructured data that they have and that the business continues to generate.

Speaker B: Right, right. Yeah. And it's an, um. Unstructured has always been one of those ones, like you said, the structured stuff, the stuff that's in data warehouses and other sites of things, those things tend to be like, well, they're tied to big databases, hence they're tied to really critical systems. And then the unstructured stuff is always a little bit of like, well, we're not totally sure. You, uh, know, we know it's been generated, we know it's important, but how important is it as we're moving into, you know, this sort of AI era where, again, volume of data is really beneficial, especially if you're training things, you're looking for trends, you're looking for, uh, uniqueness. What do you find as people are, again, kind of reevaluating their data, trying to Figure out is it AI ready? Are there things we have to do to prepare it? Like are there new unexpected things that they're finding or are there, you know, with this new era, are there new things that you know, they weren't doing in previous, uh, NAS errors file, you know, unstructured data errors that sort of new and unique.

Speaker A: Yeah, I think there's probably a couple of things that come up. You know, one is permission auditing, right? It's, it's kind of, we've always been a little bit fast and loose and it's like well, if customers can't, you know, if, if my employee can't find the data then you know, I want to have like an auditing system for the permissions. But you know, if they can't find it, like it's, it's, it's not going to be that big of a deal. But AI can find everything. And so there's this sort of thing at scale and I think even you've mentioned it in the past, if you're not good at managing your data to begin with, AI is going to absolutely break it. And so there's this kind of scaling problem that comes with the amount of data that you have and how quickly that data can be exposed to your end users. So you have to have some way of auditing, centrally managing, having not only policies but enforcement about who can see what data and who can access that data. You know, and I think the second thing, and I've sort of run sort of a lit product development for large scale out systems and so uh, you had the storage administrators that would just be honest and say look, I actually don't know what the application developers or the application owners are actually doing. Like, I don't, I don't actually know what data is there because it's being generated by you know, thousands of applications. And so there's this question of well, you know, if the data that I'm going to expose to AI, does it have, you know, PII data, does it have employee comp data? Like what, what's actually there? So I think it's, it's being able to, to come up with some sort of management layer over, of uh, on top of that file, layer on top of that unstructured data and know that not only is what's there should be exposed, but it's only getting exposed to kind of the right people. So I, I think it's kind of the, it's the context, the contents and then the permissions that are, are uh, probably some of the bigger challenges that our customers are kind of bringing up when we talk to them and reasons why, you know, we try to help, you know, you obviously try to help them solve that because it's, it's a big problem.

Speaker B: Right, right. Well, and I can imagine that agents and their, you know, their identity or lack of identity is also going to, going to create some new, new wrinkles in there as well. I'm curious, you know, you've, you've been through a lot of these evolutions. You know, I've been through a number of these. It feels like on one hand, you know, whether it was like, you know, we went through sort of virtualizations and virtual data centers, we dealt with cloud, we're now dealing with AI. You know, these, these trends want to bring together a bunch of technologies but also a bunch of teams. Where are you seeing kind of your piece of the data puzzle right around files and unstructured in the context of discussions around rag and metadata. And uh, do you have to interact with data science teams? Is it still primarily sort of infrastructure teams? What do those interactions look like today? Because again, it's a bunch of things coming together that kind of cross what used to be somewhat traditional silos.

Speaker A: Yeah. And I guess to say what history doesn't repeat, but it does rhyme. And you're right, there's a lot of similarities that kind of come up in this, uh, you know, before it was you deal with the storage administrators and maybe they had to coordinate with the application owners to kind of understand sort of what's going on. You know, now you have storage budget that's actually being sort of moved to pay for AI budget. But at the same time data is, is being sucked, storage is actually being sort of pulled into kind of the AI, the, the AI strategies. So you have not only kind of storage administration, but they're sort of operating with kind of the, the data science teams, the chief data officers. It becomes kind of a strategic part of like the CIO's agenda, maybe even the CEO's agenda. So you've got this, this breadth of kind of the operators that run the infra, the strategic decision makers having to align in terms of where that investment goes and where it's actually kind of the priority. So it's, it's again, it's not just about like being able to sort of store the data and sort of maintain it, but it's about being able to really open up the business experience and intelligence that is inside of it, which becomes an outcome based, an outcome based conversation for like just you know, how do we actually automate these processes? Like what does that mean? How do we deploy these agents? How do the, how we do agents actually get access to the data that they need and only the data that they need? So those, those kind of Personas are really changing and I think the people that have to come to, together to the table to sort of think about like how do we solve this problem together? That the, just the community of people involved in those decisions, those strategies, kind of those roadmaps within businesses are changing because the decision makers are just so varied at this point.

Speaker B: Yeah, a lot of times we're, you know, we're hearing discussions about uh, you know, a term or a phrase that's often used, AI governance. And, and to a certain extent, if you, if you boil that down to, you know, some of the technologies, I mean there are things that, that are sort of wrapped around the models themselves. So, so guardrails and certain kind of harnesses and so forth. What is that? You know, when you hear AI governance and you're, you're thinking about, I mean you talked about sort of management layers on top of, you know, where data stored and how it's stored. Like how do you see that interaction between what the folks who are talking about AI governance and what's ultimately you uh, know, sort of just data governance in general. Where do you see the intersections or where do you see them being, you know, somewhat different things.

Speaker A: Yeah, so you're right. I mean data governance is a huge part of it because again, going back to the original statement, like you know, good context, good output and good outcomes, you know, bad context or mediocre context, mediocre, uh, ochre outcomes. So it's, you know, data governance is a key part of that. But data governance sort of boils down to one kind of can I trust my data that I have? And then is my data being exposed to agents in a way that is not leaking data that they shouldn't see? So that's kind of one layer and you sort of build up the stack from there. I think there's another area of, and you touched on it previously, identity, the agent's identity. Is it operating sort of as an independent identity within the system or is it operating on behalf of the user? And so where, where are the permissions from the data governance layer being enforced? Are they pre query, are they post query? Like how do you actually design that? So obviously if you do it pre query, then you're only returning data to that agent or to that user that they have the ability to see you're not trying to filter it out kind of after the, the fact, which is you have sort of, you know, the indexes that are built on sort of large corpuses of data and traditional rag like you don't really know, so you're kind of guessing should they see this, should they not? And really on the AI governance, there's the policy piece, there's like the orchestration pieces like where do they run, how much access do they have? You know, how do you, you know, how do you sort of think about sharing of agents kind of across an organization, how do they get registered? So there's, there's a lot of aspects to it. So in, in some cases, yeah, data feeds, AI and there's a governance piece to both. But I do think there's, there's a lot of, there's a lot of evolving, a lot of evolving sort of thoughts and sort of experiences and knowledge that's kind of coming out on, on both sides.

Speaker B: Yeah, I want to shift gears a little bit.

Speaker A: Yeah.

Speaker B: You know, if you're talking about AI these days and you're not, you know, and agents aren't coming up as part of the conversation, you know, you're kind of wondering, oh, okay, what, what are we missing out on, you know, in the inference world? So, uh, you know, kind of reframe this. So you know, agents now have a tendency because they're not sort of one off chat bots and they're not human in the loop. There's a lot more autonomous work that goes on. You know, we see the byproduct of that being uh, you know, huge, you know, much m. Much higher usage against things like inference and so potentially higher GPU interactions and usage. What are you seeing or you know, are you yet seeing, you know, changes as to how data is accessed when, when agents are involved? Obviously, you know, there's, we've, we've always had sort of tiering now with, you know, with storage where you've got like flash tiers and other stuff. But like, are you starting to see agents as, you know, a way it's accessing data change that are people having to sort of rethink and redesign their, you know, their tiering architectures around access to data for performance?

Speaker A: Uh, I mean I think it's always been, it's always been a challenge to know what the working set of data that you need. And this is even sort of in a pre AI world, you were always trying to figure out like what's my hot data and what's my you know, near line data. What's my cold data? And so there's been various systems of trying to solve that, some policy based, you know, I think for what we see and I think the conversations that we're having, you know, this notion of I need my data to be where my agents are running and I need to make sure that the data that's being accessed is, is near line or is in that sort of fast tier, but I need all of the other data to be available. So it's kind of on a, on a, on demand driven. Now there's another aspect of this where it's kind of, you know, difficult to sort of predict. So you have to be able to, you have to be able to falter or, or cache that data bringing into the cache sort of when the agents are needing it. So certainly, you know, we're seeing and, and what we do, and this is one of the things I love about the technology is we have caching devices that we can place at the edge that sort of pull that data to where the customer needs land speeds and all of the sort of large corpus of data, the global file system is stored within the hyperscalers environment. But then we also have kind of other technologies as I think about where agents are actually going to be running on prem, even on laptops as we get higher GPUs sort of like at the edge and sort of the far edge. Like how do we actually get data to that far edge? And we have another sort of kind of client based technology uh, that makes that available because I do think there's this continuum of, of sort of where the workloads reside and we've got customers that are collecting data on oil rigs or on. We talked to, I talked to a customer several weeks ago that was actually like doing the mapping of like channels in the Boston harbor to like understand the depth. And they're collecting a lot of data like on that ship, but then they need to do analysis of that data. They need to get it off the ship without having to pull the ship into shore. So this data movement, whether it's, we're talking about from a tiering standpoint or just a locality standpoint, like where it's being operated on is starting to change because that data needs to be available sort of wherever, wherever that analysis is going to be done. It doesn't need to be sent to a data center or sent to the cloud. You just have to sort of pull it to where it is. And that caching or that data mobility capability is something I think everybody's going to have to find a way to solve.

Speaker B: Yeah, yeah. And I guess the good news is that that problem has been going on now for a while now. So if you've been solving it so far, you're probably already moving in that right direction. I'm curious, what do the economic conversations you're having these days look like? You know, are they, are they a lot different than they were five years ago? Sort of pre AI, Are you seeing, you know, are you having to sort of, you know, think about talking about gigabytes and terabytes and tiers and relating those to tokens somehow? What does the economic conversation look like in the data world these days?

Speaker A: I mean, I think there's, I think there's two dimensions on the economic side of the conversation. You know, one is, I mean, what data should I keep around? Like, how do, how do I, you know, it's that. And that's simply just the economics of storage. I think the other side, like you mentioned, is that is the economics of tokens. And what is the efficiency of my queries? Like how am I not sort of kind of issuing the same thing over and over and over? And so I think, you know, both of those are sort of, both of those are coming up in sort of the economics. On one side, you know, you want to be able to maintain everything because it's going to feed your AI. On the other side you want to think about, well, what is the appropriate model for the tasks that I'm trying to do. So this routing layer in terms of sort of pre query optimization and how do I think, you know, is this sort of, you know, like a very small kind of micro model? Is this a domain specific model? Is this more of a foundation model that I need? And I think those are the two places where the economics are really coming up. And um, the storage economics has been around the token economics very, very different. It's kind of more of a utility, like how do I optimize my cloud costs? Which is kind of what we were trying to figure out in the day. I think the difference is there's a big difference between leaving a VM that's running in cloud environment for weeks and then getting a bill versus you've got somebody that's just issuing ineffective queries over and over and over and you get a bill within a week that it kind of eats up your entire budget. So I think both of those are ones that companies and our customers are really looking into and really spend a lot of time thinking about.

Speaker B: Yeah, I'll, I'll, I'll end with one, one, one final question, and I feel like this is, this is, this is fair game for, for anybody who's cto. What, uh, what, what sort of trends or what sort of things should be people, you know, be looking for, you know, from you guys in this space? Like, what are some of the technology trends that you're keeping your, your finger on the pulse for that might be right around the corner that people should be, you know, either exploring or, you know, you know, you're seeing a, uh, problem on the horizon.

Speaker A: Yeah, I think there's two things that I'm really looking at and it's pretty fascinating to see where it goes. I think one is agent orchestration layers in terms of kind of how they get onboarded, how they get registered, how they get sort of into that control plane. And I think the other one, Brian, is just, uh, how much of the inferencing is going to move to the edge. I think as we get more compute sort of in, in edge devices, laptop devices, you know, how much of that is going to be kind of local, local model execution? Kind of going back to the question about economics. Like, you know, are local models going to be a way that people try to kind of address some of the economics, particularly in software development lifecycles? I think that's kind of another place where I spend a lot of time thinking about how, how the future of software development is going to look like in another 12 months. Those are, I think, some of the big areas I'm focused on.

Speaker B: Yeah, well, and I think the local models and the locality of models, I think, really, uh, ties back to something we talk about all the time, which is we've never had a market in which the vast majority of everything ran on the most expensive hardware from sort of a single vendor. So we still haven't really unlocked that multi vendor, multi price point sort of GPU or accelerator. And I think that's going to have a direct correlation to the location challenge that you guys have on the data side of things. So. Yep, absolutely, Jerry, good stuff. I could sit here and ask you questions for hours and hours, but you're a busy man. I want to be conscious of that time. Thank you so much for all of this. Uh, you know, like I said, we really need to make sure that we're keeping track of what's going on in the data side as much as we are with models and agents, because the three of them are so intertwined. We really appreciate you giving us some insight into what you're seeing in the world, the challenges that unstructured data has. And folks with that, again, I want to thank Jerry for his time. I want to thank the folks from Nasuni for being been a great partner these last few months. And uh, with that, we're going to wrap it up and we will talk to you next week.

Speaker A: Thanks for listening. Check us out@theenterpriseaishow.com for past shows, newsletters

Speaker B: and all things enterprise AI.