Navigating AI Risks with Trevor Horwitz from TrustNet

Speaker A: Welcome to the podcast. As a listener, you're invited to claim your comp pass for our next live full day Software Oasis virtual AI event with the world's top B2B AI experts. By attending, you'll join our engaged community of founders and executives from over 200,000 leading B2B global organizations. AI won't take your place, but uh, a competitor leveraging it to drive results will learn the latest in AI first Thursday of every month. Tap green button@softwareoasis.com or use show notes link to secure your past.

Speaker B: So imagine starting your tech journey with a Commodore VIC 20. That's exactly how Trevor Horowitz, founder of Trustnet, began his path in technology. Growing up in Johannesburg with mentors like his uncle and father, he, he was driven by a fascination for cybersecurity's impact on society. And you know, by the end of this episode you'll understand exactly why. Finding product market fit and embedding trust in early stage companies is key. We'll unpack Trevor's journey from hustling for those first customers to making AI systems accountable and secure. I am your host, Michael Bernsweig and this is B B2B Automation Spotlight.

Speaker A: For a lot of the different individuals listening in this week, I know a lot of um, a lot have had their own personal journeys in business and entrepreneurship getting to where they are. But uh, hopefully we can share uh, a little bit of your journey prior to founding trustnet and help a few individuals out there along the path to uh, success on their own.

Speaker C: Thank you Michael. I think my journey goes back to uh, my formative years growing up in South Africa, uh, Johannesburg where I was born, ah, I got involved in technology very early, very fortunate to have good mentors, um, including family members, my uncle and my father who were kind of early adopters and early pushed me really to get involved in technology. So I started off very modestly in just messing around with Commodore, uh, VIC20 as a throwback, if anyone remembers that, um, and then kind of evolved that into pursuing that as a career in college, um, and then later on getting involved in a lot of professional services organizations early on, um, I did a bunch of entrepreneurial things along the way that are not, uh, directly in the cyber security assurance space, uh, involved in some helping build and scale some businesses, um, and then eventually decided to come back to my roots and get back involved in technology. So it was just kind of like an evolution of, uh, ideas. But what really attracted me to the cyber security field, um, you know, 20 years ago was just how the, the impact that it would have on humanity. And um, I, I really saw this as being a foundational aspect of how businesses would operate and society would operate, uh, for generations. And the idea of being able to contribute in a way that would uh, make society safer and really protect us from the bad guys. So that really appealed to me. Uh, guess being um, uh, a lover of law and order and understanding and appreciating the impact that criminal aspects can have and destroy lives really appealed to me and that was, kind of attracted

Speaker D: me to the profession.

Speaker C: I really had a purpose as well, besides just a love of the technology in and of itself.

Speaker A: Interesting. And uh, yes, uh, I think a lot of uh, individuals remember that early Commodore, if you, if you were depending uh, upon which side of the tracks you grew up on, you either had the 64 or the Commodore Vic 20. And uh, if you were really doing well, you, instead of having a tape backup, you had uh, you had a floppy backup. So in interesting early days, but in, in your journey, um, were there. There's some pivotal moments at which you uh, you know, really saw, you know, or learned from, you know, along the way that, that kind of shaped the direction, you know, maybe some successes or failures that you had along the way.

Speaker C: Uh, sure. Uh, I think a lot of the foundational um, journey was shaped by other people like mentors and having good mentors. Um, and I find over time you learn from everyone and if you keep that mindset open, sometimes it's not learning necessarily positive things, but you can still learn negative things. So I've had mentors that have encouraged and helped mentor me and experiences that have been very positive and also negative experiences. So I think what's healthy for entrepreneurs and people who are going through a journey like this is to appreciate that every experience, positive and negative, is something to be taken away from that along the way. I've had um, definitely had uh, successes and had failures. Uh, the failures came. A lot of it came around through uh, technology that was too early or technology that was poorly designed, uh, in being involved in startups that didn't work out, um, startups that were not innovative, that were doing the same thing as everyone else, late to the market, couldn't uh, find market traction. One of the things I think that's always driven me to appreciate if someone is involved in a startup is ultimately a lot of it has come down to finding that product market fit. And it sounds kind of obvious, but product market fit is really what's been a bedrock. And if you can find that happy, uh, Niche where you have product market fits and people appreciate your product, I think that's when the revenue will come, the sales will come, everything else will start to kind of click and it's uh, it's definitely the leading edge to be able to get that uh, those first few sales under your belt. I've been fortunate to be able to connect with some large companies early on in some of the entrepreneurial things I've done and, and um, having those reference customers has certainly helped accelerate us ah and get other customers on board because they had the confidence in what we do.

Speaker A: So a lot of times um, going from zero to one that very early, you know, finding those first customers uh, that are actually willing to vote with their dollars and pay, what was that like? How did you find that first handful of customers for Trustne and what learning lesson did you have there?

Speaker C: A lot of it was just hustle meaning just getting out there and grinding and um, connecting with people, uh, kind of old school stuff, just one on one communication. Who do you know? I've got this cool technology, we've got this cool service, this is what we're trying to do. Ironically it's what Trustnet actually offers in terms of the assurance services that we do today, uh, that actually has been a big driver for other businesses. So we, we work with a lot of founders and a lot of early stage companies as well as mid size and large organizations. A lot of early stage companies are trying to find their traction and in order to find their traction they need to have uh, they need to have some trust and having third party assurance services like a SOC 2, a PCI assessment or uh, other regulatory compliance frameworks that they're adhering to really often will be the bedrock for um, enabling them to get in the front door with uh, other organizations. So that's been a big driver I think for at least for our customers. Uh, for us it was again primary. Just hustle and connecting with a lot of people in that early stage.

Speaker A: Yeah and a lot of times it's providing you always hear the uh, 10x uh value but you know, helping your customers, customers achieve something greater with the Trustnet solution than they could on their own. That's obviously adding many, many times the value. Um, so can you share a little bit about the solution overall and the types of organizations using the Trustnet solution and how it fits into their business?

Speaker C: Well we are primarily a uh compliance driven organization meaning we providing assurance services. So again things like SOC, uh two independent assessments, PCI, DSS, uh things like GDPR, HIPAA assessments, high trust assessments, ISO 27001. So our customers are leveraging the, what we provide in terms of audits and assurance services to gain more market traction, to secure their customer base and to provide the confidence. And you know, some organizations, in other words, our customers, customers will look at assurance as being a checkbox and they'll ask our customers, do you have a sock too? And they'll say yes or no. So it's just kind of a, you know, check off that box. But others will take it seriously. They'll look at the sock too, and they'll look at the quality of the report, what data is inside there, where are the risks? And that's really where we shine. Because we're not a cookie cutter organization. We don't just, uh, kind of tick and tie and provide assurance services that don't really have any foundation to it. So it becomes a reputational thing. When our clients are using our services, do they get a report and an assessment that can stand up under scrutiny? And again, it's just another.

Speaker A: Yeah, and I think so many organizations fall into that trap and I'm sure you've seen so many arrive on your doorstep after they've had an incident. But you know, the organizations that maybe that's the wake up call, but you know, for the ones that take it seriously and really dive in with both feet and not only get compliant, but maintain compliance and really, um, you know, bake it into their DNA of their organization, you know, obviously they're seeing not only the results in terms of, you know, clients and customer and reputation, but also, you know, learning along the way, you know, all of the different, uh, things that they need to have on their radar that they may not have before.

Speaker C: Yeah, that's, uh, it's a great point. Um, one of the things that I find really interesting just around trust in and of itself is the idea that it is something that's really hard to define. Uh, when you're in business. You know, what does trust actually mean to your customers? But you certainly know it when you lose it, when there is a data breach, when there is, uh, an incident that affects your customers and the data they have, or the ability for you to provide services, uh, and availability of your services and customers are relying on you. And when you lose that trust, it's very, very hard to gain back. There's also a concept that a lot of, um, IT professionals recognize and that's this technological deficit. So you're building up this debt in a way, this technological debt where you're not investing, the organization's not investing enough in their security and their assurance. And over time it just accumulates and eventually something breaks and that's the time when you're, that death will come back to bite you. So for the organizations that are maintaining their security and continue to invest it and putting in the effort, performing controls, ah, getting those assurance reports, they'll land up in the long term better off than organizations that are just flying by the seat of their pants.

Speaker A: You're almost in a way protecting your future revenue by protecting your clients when you think of it in that way. So if you were, you know, uh, you've worked with so many organizations over the years, if you were to, and I know it's hard to boil it down to a few points, but if you were to, you know, boil it down to a few points of any executives that are listening into this conversation, what's the starting point? What are the few things that they have to get right?

Speaker C: Uh, the starting point for most organizations is just understanding what does your technological landscape look like. And that's the starting point. So understanding what technologies exist and then doing a risk assessment. So understanding, okay, so all these technologies, these systems we have in place, what do they mean? Which are the ones that are critical for us to maintain, which ones are customer facing, which ones are internal facing? So doing a risk assessment to me is like the starting point and understanding the, you know, that not all systems are, have the same level of risk. You have high, medium and low risk and understanding where you need to apply your controls, you don't necessarily have to have the same level of controls and maybe even shouldn't across the board, you know, focus on those systems that are very high risk in terms of the controls that are, must have um, and some of the controls and the scrutiny over some systems that are low risk, you know, don't, don't spend the money, don't spend the resources on that. In reality, most organizations want to have kind of a, protect the moat type of mentality where they protect everything with the same level of controls. Uh, but a sophisticated organization, particularly one that's um, getting to the medium or large size, will appreciate that they're going to have kind of zone security where there's certain systems that are going to be in high risk areas that are, ah, well protected and in low risk. But the starting point always comes back down to identify what systems you have and then start with the risk assessments.

Speaker A: And uh, as we're wrapping up for organizations that are in the mid market enterprise space, you know, Looking into different changes that have happened related to AI. Are there certain areas related to AI that you're excited about? As we approach, uh, 2026 into 2027,

Speaker C: I am very excited about the advent of AR because of the ability to automate so many processes that's going to free up so many resources. A lot of the first use cases for AI are getting rid of a lot of the drudgery of, um, how business need to operate and being able to automate these things and being able to do it at scale. And that's very exciting. Um, I just think we're at a point now in history that is, um, you know, generations will look back and we'll talk about this era and the start of it. Uh, the flip side of that is it's also pretty frightening because of the speed, uh, the technology is incredible. The speed of change is amazing. I was just at a conference a couple of weeks ago and the presenters were saying, hey, I'm sorry, it didn't include, you know, this aspect or that aspect because my slides had to be in like, uh, three months before the conference started. Well, so much change in three months in AI that sometimes they completely missed the point. Things had moved so fast. So that's the exciting part and it's also the part, uh, that's a scary part too.

Speaker A: Yeah. In the early days of the Internet they would say six months on the Internet is like a decade in any other industry. But I think with AI that pace has only accelerated. So interesting.

Speaker B: Trevor highlighted how crucial finding product market fit is for startups to succeed. Now he's going to take us through the hidden depths of AI in business systems and why it's vital to know where AI is embedded. This understanding ties directly back to the importance of accountability and managing AI risks.

Speaker C: We're going to discuss is essentially talking about AI without safeguards and how businesses can manage risk in the age of AI driven systems. Uh, so a little blurb about myself, uh, to start off with Trevor Horowitz, as Michael mentioned, I'm the founder and CISO for trustnet. I've uh, been involved in cybersecurity, risk management and compliance for over 20 years. And uh, there's a lot of acronyms behind my name. Some of them are actually true. So, um, let's move into a little bit about Trustnet. Uh, Trustnet is an advisory company providing end to end security and compliance solutions. Uh, we do a lot of work around SOC assessments, ISO 27001, PCI, DSS and of course now moving into helping uh, clients and uh, assure their AI systems as well with many industry standards and frameworks. So our main topic today is really talking about AI. Um, and one of the realities for a lot of organizations is not really understanding or appreciating necessarily how AI is already in their business. Today. AI has been embedded into many systems. Your CRM system, your HR system, finance, SaaS, platforms, and often this is introduced through third party vendors and they're offering. So it's not necessarily something that organizations have top of mind when they think about AI is because a, uh, lot of it's taking place in the background and certainly most of the, excuse me, infrastructure players on the back end are using AI in a heavy fashion. Another aspect is this kind of stealth or shadow AI, which is AI being used by employees and team members and they may not be even authorized, but they could be using it on the back end. Maybe they have a personal account reality, uh, and the takeaway from this slide really is that many organizations just don't realize how deep it goes and AI is really not waiting for any approvals. That's reality today. So a couple of ways that AI is actually shaping our uh, business, it just doesn't automate. It's actually making decisions at scale. And what that means is that the AIs that are, have flawed recommendations are resulting in bad decisions and those bad decisions are just being accelerated, being made faster. We have a lot of bias embedded into business processes because of AI systems as well. And that's largely to the a function of the type of LLM they're using or other language models to process the data. So there's very confident sounding uh, solutions coming out, a lot of AI tools, but oftentimes there's misleading results. And where this really hits hardest again is on some of the key things like hiring decisions and financial planning and forecasting, you do things that have a serious impact on the organization. Customer communications and experience is another area. There's many instances of AI really going rogue and going off the rails, so to speak. So the bottom line of this uh, particular aspect here is that there's no.

Speaker B: I'd love to hear how companies can begin identifying where AI is already integrated in their operations. Can you walk us through that initial step?

Speaker C: Uh, ownership, there's no accountability and you have uncontrolled risk and that's something we obviously want to try and avoid. So what fails first? Firstly, what fails first is that lack of ownership. If there's no ownership of AI systems wherever they are in your organization, then there's obviously going to be no accountability. Another big fail is the fact that a lot of organizations do not have a full inventory of where AI is being used. And that's a huge big gap in terms of just visibility. We have a lot of blind trust also in AI generated outputs. The classic failures of a lawyer who used an AI to generate an argument and the AI essentially hallucinated and made up case law. Um, and obviously that lawyer got into a lot of trouble for using AI in that particular fashion. But that's a humorous anecdote. But sometimes there are even more consequences for organizations who are using AI where you just have blind trust in them. Another big issue we see around the board in organizations that are deploying AI is really, they're not looking at this from a risk standpoint. There's no risk classification or controls. Uh, so what does it mean when you're using generative AI or agentic AI and how's it being deployed in the organization? AI is also obviously being introduced through a lot of our vendors and that for most organizations we don't have a lot of visibility into that. So there's a supply chain issue there as well in terms of AI being embedded in some underlying systems. So AI failures really start with governance, not necessarily with technology, where the mind shift really happens. And this is like a critical function for organizations to understand. Both the technical and non technical people can appreciate this. The way we've traditionally thought about IT systems is they follow defined paths. So we process data, we have a defined logic, we have a if then and kind of uh, scenario where systems are essentially deterministic and they produce predictable outcomes. And that's been a bedrock of IT systems for many years. In fact, a lot of the controls that we have around technology have been about that particular factor, ensuring that we have predictable outcomes. Where AI creates this mind shift is we have a new paradigm where decisions aren't being shaped and the decisions themselves are essentially non deterministic. Meaning if we put in one plus one today it may equal two to a system, an AI system, but tomorrow it could be 2.5. And the reason for that is that AI themselves are non deterministic and they're continually evolving and learning. A big part of LLMs and other AI systems in terms of their behavior is determined based on the input criteria. And again, the input criteria is constantly being influenced based on uh, new learnings and evolving structures. And that obviously creates a uh, complete different paradigm. And with that paradigm it creates new opportunities for us to different type of safeguards.

Speaker B: Tell me a bit about the decision making process within organizations. When it comes to deploying AI, who should really be owning these decisions?

Speaker C: And different type of guardrails that need to be in place in order to deploy AI in a way that's going to be measurable and controlled. So AI risk first and foremost is a business risk. And this is something that needs to be really hammered home in an organization. Now in AI governance, where does this really matter? Well, there's four questions we should be asking ourselves whenever AI is being deployed and for most businesses, again it's already here, uh, who owns the decisions? So who are the decision makers in the organization who actually deciding whether to deploy AI and where it's being embedded? Where is it being used? You know, in is this third party systems or is this internal systems? Are we using agentic AI? Are we using generative AI, uh, or using machine learning? And machine learning is actually something that's been around for many, many years. A lot of organizations have been uh, familiar with that. But again this is a new paradigm now on generative AI and uh, genti. So it's really been a game changer. Third question we should be asking ourselves is what controls exist? What safeguards do we currently have in place today? And the fourth question is how do we prove that? Can we prove this? So really we want to build a foundation and there's three pillars to that foundation. One is accountability. The second one is visibility and the third one is evidence. Now if we don't have any proof of these controls and we don't have any proof of these foundations, then we really have no defense for ourselves. And through an audit for third parties, for regulators. Uh, so having proof is one of those key characteristics that needs to be also be embedded within the environment. So here are the five controls that matter the most. Number one, we need ownership. There needs to be very clear accountability for outcomes for the organization. So when we have these systems that are non deterministic, can we assign ownership for that particular, uh, decision and ensure that the owner of that a human being is actually accountable for the results from that system? Second thing we need is a lot more risk awareness. And this is not getting security awareness training for your employees. This is about focused on AI and the use of AI. So it's a new type of risk awareness. But not all AI use cases carry the same risk. If we're talking about for example a chatbot, um, with limited capabilities that may have an implication, reputational impact on the organization if it goes rogue. But compare that to a major financial decision where AI is Embedded that could completely steer an organization on a different track, could approve a purchase order or a payment. Uh, then we're talking about obviously a completely different size and scale of risk. The third aspect of controls is about lifecycle management. Like traditional systems, AI also needs continuous monitoring, also needs to be tested on a regular basis and needs to be updated. And those AI, those updates are things like ensuring that you have the right security controls in place. Uh, security is always evolving. There's always new controls coming out, systems are being improved.

Speaker B: Um, I was hoping you could walk us through the risks of AI model corruption. How do bad actors exploit these systems and what can be done about it?

Speaker C: So the lifecycle management becomes just as important as it was in traditional systems and more so in AI. The fourth aspect that we need to have in place is this idea of data integrity and we need to know the sources. Um, there's a concept in AI of data poisoning where essentially the model gets corrupted by a malicious actor who would inject malicious data into a particular AI model and would corrupt, ah, it with the intent of providing some advantage to the, to the nefarious actor. So we need to understand what are the limitations of these, uh, AI systems and understand the controls around the data inputs. And the last thing, and probably the most important, is we need this continuous human oversight. Humans need to be accountable for decisions. Now there's two ways that we can have accountability there. One is what's called humans in the loop and also humans on the loop, a slightly different variation just in terms of the level of autonomy that's given to AI. But both of those controls are super important and those are terminology that everyone should understand going forward. So bottom line from this slide for me is really just looking at this overall picture and understanding that more controls are not necessarily better. We just need to have the right controls and make sure that those controls are actually working effectively. All right, so how do we get started? If you're brand new to this and you're just getting started in terms of understanding AI environment, number one step I would recommend is really start off, ah, identifying where AI is already in use in the organization and also where you're planning to use it. Uh, so start looking at that and identify and create an inventory of systems that are particularly at risk. You also need to assign clear ownership for every single use case, uh, across the organization. So I have someone who is essentially going to be responsible for the overall oversight of that AI and often it's the owner of that particular application. So if you're talking about a Financial application, maybe your cfo, your head of finance, talking about a sales application, that may be the head of sales, the VP of sales ultimately is accountable for that system, but also embed some technology people in that to tag team them. Um, one of the challenges of a technology like AI is it is poorly understood. It's poorly understood by lay people, for sure. Even within the industry itself, that's being poorly understood. But because it's so new and it's, it's rapidly evolving, the third thing to get started would be to prioritize what are your high impact in your high risk areas. Obviously you want to put more attention onto those systems because they have higher likelihood of going haywire. Um, so put a lot more attention into those systems and then kind of work your way down. We want to also have the very defined and simple enforceable rules around AI. Uh, and the reason I'm saying start off with simple is because we don't want to have too many guardrails and too many things in place right off the bat because they probably will not be implemented correctly. So start off small and then build from there. We also want to be able to monitor your outcomes, not just the output.

Speaker B: Could you explain the potential implications of, uh, generative AI outputs? What should organizations be considering as they move forward with AI puts?

Speaker C: So not just looking at whatever data's been spit out from an AI, as in the case of a generative AI system, but also look, okay, what are the implications? How's that output actually being used and what are the implications for that going forward? And lastly, look at documenting your decisions and your controls. So having a good risk, uh, assessment process in place, having those, uh, defined, simple enforceable rules, having those also well defined and documented will help, uh, a long way. So start small, build your controls, and then you can scale with confidence. Now the last point I just want to hammer, um, a little bit more. And one of the things I think is underappreciated is why do we need these controls in the first place? And the way I like to look at these controls is that these are kind of like the parachute on a ship. Let's say we're coming from space and we need to descend at a slow speed. Now we can go very fast without those, uh, parachutes. But, but of course, if we do that, we have a high likelihood of crashing into the ocean and potentially losing the ship. The safeguards, the controls, the parachute is what allows us to do that. With confidence, we can go as fast as we need to and descend as quickly as we need to, but knowing that we have a parachute that can slow us down and provide those guardrails and that's really what comes along with AI systems. Just in general, having these uh, guardrails and having these controls will allow you to frankly to go faster. So some quick wrap up uh, for, for this presentation. Just some key takeaways here. Firstly, just understand AI is already in your business. The AI is, is in the business here today. It's here to stay. It's not going away. This is a rapidly evolving technology, probably the fastest evolution of technological change I've seen in my lifetime. And m certainly for a lot of other practitioners as well, the risk here is that we have unmanaged use of these tools. They're super useful, they're very powerful. But with unmanaged risk, we're causing potentially a lot of damage to our organization going forward. And AI, uh, ultimately is a business risk. It starts off as a business risk and then can obviously evolve as a technological risk as well. So my action plan for if anyone is going through an exercise and looking at this, start off with the visibility. Identify all the places where the technology exists, assign ownership for uh, each of the different technologies, focus on your high impact and your high risk use cases and then build the simple repeatable controls and you can scale that over time. And control ultimately is what's going to enable you to scale these AI systems for the long term. Okay, and that's it. Thank you, Michael.

Speaker A: Fantastic. And coming up in just a few minutes, Trevor will be on our next Q and A panel. So if you have questions for Trevor, type them into the Q and A box that you see in front of you and we'll do our best to get to as many of those as we can. During the live Q and A session,

Speaker B: Trevor just laid out how AI governance is essential to managing business risk. This is the live panel from our Software Oasis AI Summit where real audience questions are tackled. These questions will delve deeper into how businesses can effectively manage AI integration and accountability. Uh, the presentation made it clear that without knowing where AI is embedded, organizations face uncontrolled risks. I'm guessing a lot of you are now wondering how to take inventory of AI systems and establish accountability. And that's exactly, exactly what this audience brought to the table in the panel discussion.

Speaker A: I'd like to welcome everyone to our second panel of the event today. For everyone that's been with us since 8am, I hope you're enjoying the event so far. Uh, this next panel, uh, proves to be a great, uh, one we're actually, um, joined by all of the speakers you've heard over the last couple of hours, and we have, uh, quite a few different, um, questions that have come in. So let me, uh, jump right to the top. I want to start by introducing the panel. We, uh, have Mike McNeil. He's the CEO over at Fleet Device Management. Welcome, Mike.

Speaker D: Hi.

Speaker E: Thanks for having me.

Speaker A: Matt Brown, he is the founder and CEO over at We're Four. Welcome, Matt.

Speaker F: Good to be here.

Speaker A: Adnan Masoud, he's PhD and chief AI architect. Uh, I want to welcome you, Adnan.

Speaker G: Thank you. Glad to be here.

Speaker A: And, uh, Joanna Henkel, she is the director, AI and Automation Solutions over at Toshiba Global Commerce Solutions. Welcome, Joanna.

Speaker H: Thank you. Hello, everyone.

Speaker A: And Adnan is with ust. I just want to know to make sure we recognize that. And Trevor is the CISO over at Trustnet. Welcome, Trevor.

Speaker D: Thank you, Michael. Pleasure to be here.

Speaker A: So, why don't we just start at the top? Uh, let me see. It looks like the first questions came in for Joanna. And, um, let me just jump in here. Uh, the first question comes to us from Megan from Allentown, Pennsylvania, right here in the US And Joanna. Um, Megan is asking, when your AI product scales from a few pilot stores to thousands, what specific signals tell you that distance from real users is starting to hurt the experience, and it's time to step back in?

Speaker H: That's a very good question, and I will not claim to know the exact. I think that's part of the trick here. Um, you will hear feedback from your users that I think the first signals are, uh, when it really doesn't line up with your data, which seems very counterintuitive. You think data is king. If we have all of the usage metrics, we know exactly how the models are working, how can the data lie? But your users are the ones that are touching it every day. They're feeling it every day, and they're giving you that qualitative feedback that sometimes the data can overlook. So in some ways, it's those biggest gaps with your real power users when you need to go look and question, are you measuring the right thing? Um, that's kind of where we've really found. It's that moment you're like, oh, huh, huh. I don't think that's right. But I trust that person, I trust that user. I trust that voice. And that's when you really need to dig in.

Speaker A: Fantastic. Um, Trevor, the next question came in for you. Um, it's, uh, right here in the US From Danielle who is just outside of Dayton, Ohio. And Danielle is asking for organizations with no clear AI inventory today. What's the most practical first step you recommend to discover where AI is already embedded in, in systems and vendors.

Speaker D: Great question. Uh, I would start with the users, ask them what to uh, do some surveys about what tools they're using as a starting point, and then it'd be like an initial guide. Uh, then look at your, your vendors and uh, also send out surveys to

Speaker C: them where they're embedding AI into their

Speaker D: products, into their services. Um, have that, has it been included in any SLAs or contracts with regards to privacy? Uh, maybe the legal team as well would be a, uh, team to consult with as well. Yeah, let's start though with those two areas.

Speaker A: Sounds like a good foundational, uh, first step. Um, Matt, the next, um, question came in for you and I'm going to spin the globe a little bit and include some of our international, uh, attendees. Uh, this is from Hugo, who is in Coimbra, ah, Portugal. And Hugo, uh, is asking for manufacturers tempted to let AI handle everything. How do you explain where you deliberately keep humans in the loop so quality, traceability and audits stay rock solid?

Speaker F: Well, yeah, that's interesting. I would say one of the challenges, I would say with AI currently is, uh, this idea of hallucinating. When you're thinking about traceability and manufacturing a product you don't want to have happen as something hallucinates that some ingredient or component went into your product when you made it. So if you're thinking about using AI in a manufacturing space, probably a good place for humans to play would be in the compliance side. So maybe okay to have AI produce or formulate products, but have humans do a secondary or quality check at every stage of the process so that you have that human eyes on the, on the stages and you have the confidence to be able to say, okay, this is exactly what into my. What went into my product and somebody signed off on it. It's all good.

Speaker A: Adnan. Um, this question came in from a few different attendees. Uh, so I'm going to pick one. Uh, this one is from Daniel, who is in Galway, Ireland. And Daniel is asking for leaders designing AI governance. How can your three layer model, technical, operational and regulatory that you spoke about be turned into a concrete checklist they can execute against this year?

Speaker G: Very specific question. Great question, by the way. I think it's um, when you are thinking about the models, especially when it comes to governance, you probably want to think about the loop of what your Use case is what kind of governance framework you are planning to use. So for instance, if it's nsu, uh, the EU AI act or NIST RMF framework, or it's your own internal compliance thing, if you're using ISO 42001 based on that, what controls you need to put in place, that's essentially the mental model you will start with. And now based on that you can create checklist. And essentially a checklist gets created for you based on how your use case applies to that. So if you think about a concrete checklist that comes out of your mental model of how the risk triage is happening, some of these use cases will have a high risk. So that checklist will be much longer. Some of this will be actually like I'm going to summarize a Word document like okay fine, in that case your list will be just really small. Does it have Phi Pii data? There is no one checklist, there's no one size fit. All essentially will depend on the use case, the regulation you are following and the controls you have to put in place. I hope that answers your question.

Speaker D: And Michael, if I could just add on to Adam, another, another idea would uh, be to start off and do uh, overall risk assessment. And that risk assessment should guide where the uh, where the focus should be. So that includes doing things like uh, identifying what the assets are, look at the risk factors associated with each of those different assets. Who do they belong to, the nature of the process that's being, that's taking place. And that can be a great starting point to focus and then to kind of add on to what Ednan said. Then use your frameworks as the basis for determining what control should be in place to mitigate the risks associated with that risk assessment.

Speaker A: The next question came in from Mike, uh, for Mike and this is from Carlos who's in uh, just outside of um, looks like right in Waukesha, Wisconsin. Um, and Carlos is asking how do you help admins who grew up on GUI tools get comfortable approving git based policy changes without needing to become full time developers?

Speaker E: Yeah, really good question. I think a lot of people hear git and they think of that time where they tried to follow a tutorial or they did some googling and they got these confusing terminal commands and they got their repo all in a tangle and they got merge conflicts somehow they didn't even mean to. Right. Um, I think it's gotten a bad reputation because understandably it's been a low level tool. Um, and I think there's sometimes a little bit too much, uh, pushing people out and trying to say the only real way to do this is on the command line. Um, but what I've seen, partly I learned this teaching people to code years ago, is that you can take someone who's a barista and you can sit down with them for a couple hours and you can get them competent in the GitHub UI. Um, you can get to where they can understand that the output from the CICD job. I mean, does it say error or not? Right. If it says error, you have to do something and then it'll usually tell you what to do. Um, at least in Fleet, that's something that we really obsess over, is making that experience meaningful. Right. Um, so what I've seen is people go to one of the, you know, we do. I'll plug it. We have certification workshops. They're free, you can have dinner there. And we do them locally. About 10, 15 people usually. Um, and a lot of folks there learn for the very large first time to use Git. Um, we've had people with 30 year old IT estates, um, mostly windows, come, uh, in, sit down, figure out how to do the basics in terms of, I can be a participant in this. I can approve things, I can see if they failed. I can be like, hey, you should change this. I can see the diff, um, there. It's really just, uh, going and using plot or using another tool to basically get it to create the changes for you. Because even if you don't know how to do the exact YAML structure, um, that's where I think just the tooling is so, so important. Um, but, uh, yeah, uh, confidence is hard to come by in that world. But it is really, it has gotten a lot easier than if you tried git maybe 10 years ago. GitHub Desktop makes it incredibly easy. Um, and I recommend giving it a go.

Speaker D: So should we be introducing barista experience into all of our job descriptions?

Speaker E: You know, I do a fine video interview every time and they. One of the questions is, hey, have you ever had a customer service position? Because it is always nice.

Speaker H: I think your point about confidence is important too, that everyone likes what they're comfortable with or they're already an expert. And when it's time to go and learn something new, you just have to get rid of that mystique and the unfamiliar familiarity, give them that entry point, give them some confidence, and then they can run with it. And sounds like you're doing a lot of those things with those workshops, Joanna.

Speaker A: Um, the next question actually came in for you, so I'll just jump right in. Um, this is from Carlos, who's in Mobile, Alabama. And, um, let's see here. Carlos, I'm hearing a little bit of feedback. I don't know if someone might, might need, uh, to be muted. Um, how do you keep your engineering and innovation teams in regular contact with store associates and managers so feedback reaches you before it's filtered through layers of dashboards and reports?

Speaker H: That's a great question. Definitely a challenge. Um, my team that I lead, we're in the product management space, and that really is our role that we are that connection between the users and the needs and that true voice of the customer. And coming back to our engineers, um, we absolutely sometimes will have those engineers talk to those users as well. Um, but I think it goes back to the point of, of never forgetting that your users are important. Um, you can have different layers and roles to communicate, but thing is, just don't trust the dashboards always. We always go out, we speak to our customers, we, you know, in our profession and read the retail world as well. There's a lot of people out there, and everyone in our world will go talk to all of those users, whether they want to talk to us or not. Um, we reach out to our customers. And so I think even back to the switching tools and confidence, it's just do it. Just go have those conversations. Make sure that you build those relationships with those people and are constantly reaching out. Um, and that will also then make them comfortable with us so they know, hey, if I have something, I'm not going to hold it within. They know how to reach us and find us and how to give that feedback. Um, and, you know, we always have to work very closely. Product and technology, engineering, innovation teams. There's always those feedback loops because those engineers, if they don't understand what we do about the users and about the problems being solved, then they are limited in their ability. Right. They're problem solvers. They want to know what the things they're building are actually doing. So it's always really important for us to keep our teams really at, uh, the forefront, understanding it, uh, and bringing it back every day and talking to those engineers.

Speaker A: Great advice. And I think a lot of the different, uh, executives that are at the event today, uh, realize that that's important. No matter what area of tech, uh, you're responsible for, you know, getting that firsthand. User advice and feedback is key. Um, Trevor, the next question came in for you, and it's from, uh, Josh, who's in Peoria, Peoria, Illinois and Jos is asking, uh, you stressed in your presentation that AI failures start with governance, not tech. What's the very first ownership decision a leadership team should make to change that trajectory?

Speaker D: Your Honor, I would like to take the fifth.

Speaker A: I love.

Speaker D: That's a great question, uh, uh, looking at governance and what's the first decision point? And uh, it goes back to executive leadership, uh, a lot of education, uh, particularly around the non deterministic nature of AI. And I think that's something that needs to be really pounded when it comes to educating the sea level, uh, because you're traditional way about thinking of systems where you have a purely deterministic output based on a consistent uh, set of criteria changes the nature of how we need to address risks and allowing and enabling the C level and executive level to understand this is the nature of the risks associated with introducing AI into environments. And the panel had made uh, comment earlier about the hallucination and that occurs within AI systems. And that's a risk factor that not a lot of executives appreciate, uh, because this technology is so new. Clearly over time we will see systems become more and more accurate and there'll be less hallucination and there'll be more accuracy. Most likely will never eliminate that. And that's something that really needs to be factored in when it comes to understanding and appreciating the governance factor

Speaker A: makes good sense.

Speaker G: Matt M. Um, I wanted to add on top of that that people are now getting more comfortable with the latent space. So there's a latent space and then deterministic space. So I feel like more executives are getting because of the whole frontier capabilities becoming a mainstream thing. I see. And I think as engineers, as scientists, as leaders, we have to push the envelope a little bit more to bring in those capabilities because otherwise we lose on our big frontier capability edge. That uh, capability overhang is there us to really exploit. And I think that alpha really resides between that difference between where the organizations are and where the capability of these models are. I'm here on a job site with Tim, who owns his own electrical contracting business.

Speaker C: Three employees and two work trucks.

Speaker G: Tim traded up to Geico Commercial Auto Insurance. We're positively here where he needs us most.

Speaker F: They sure are.

Speaker G: With step by step help on all his insurance needs. All for shockingly low rates.

Speaker F: Shockingly low, huh?

Speaker G: Just a little bit of electrician humor.

Speaker H: Do you get it?

Speaker B: I got it.

Speaker G: You know, it feels like we have a real connection.

Speaker C: All right, I'll stop, get a commercial

Speaker A: auto insurance quote today@geico.com and see how much you could save.

Speaker B: It feels good to Geico.

Speaker A: Matt, the next question came in for you from Tyler who's just outside of Boise, Idaho, uh, here in the US And Tyler is asking for a mid sized food or cosmetics manufacturer such as ourselves. Which single process do you suggest they apply your AI assisted test to and control point approach to first see quick value?

Speaker F: Uh, yeah, great, great process question. So it's, it's going to be unique to every company and every type of product that you make. But I would say in general, um, the ones that people tend to skip or forget the most. So in other words get AI to jump in and enforce discipline about the most tedious tests, the ones that require the most uh, reminders or memory to make sure you do them. Computers uh, are great at reminding you repetitive tedious tasks. So if you start there, that helps your team reduce the tedium of their job and they might start to see more value and, and it kind of replicates. So once I get the AI doing my hardest work then I start to see I can use this in other places and really start to enjoy the process more. So I would say start there.

Speaker A: Adnan, um, a question for you. Uh, this one from the U.S. this is from Dustin who's in, let's see, Fresno, California. And Dustin is asking for teams used to classic software development life cycles. What's the smallest first step you recommend to start operating in, in an AI development life cycle without overwhelming them or their team?

Speaker G: I guess the smallest one would be to get a startup that cloud or codex depending on the poison of your choice. Um, I feel like we are in a, in an area where our sales folks, our HR folks are building applications. It's completely democratizing that. It's become such. That's a great thing. It's not a bad thing. It's a great thing that people are talking to the databases. Their natural language has become the way we communicate with machines. So the first smallest step you can take instead of you know, starting to read up on how to do bash client or how to do um, like create algorithms around this, start just by doing just like you do today with your chat GPTs. So as you build upon that foundation of using plot code or using ChatGPT, uh, Codex, Codex or even like just get cursor installed and start playing with the three models, right? It will give you the foundational element of what you need and from there you can build upon like how do I prompt better, how do I create A better harness. How do I actually get repeatable results? How do I create emails? But if, you know, you start with all of that, that barrier to entry is not there anymore. You can start building applications today. So utilize that opportunity and start, you know, with small cursor Godex or uh, a quad and then that will get you, I guess 90% there. That I think that inertia, if you overcome that inertia, then the sky's the limit.

Speaker A: Mike, this next question comes to us from Olivia, who's in Sioux Falls, South Dakota. And I think this is something that uh, absolutely applies to many of the enterprises in attendance today with many um, systems, um, in their arsenal. So what Olivia is asking is you mentioned patching in hours while fleets still patch in months. What combination of live queries and automation do you see actually making that shift a reality in practice?

Speaker C: Yeah.

Speaker E: So if you're in an organization that has a patch SLA today, there's likely some corner of machines that you've prioritized to be the most important. Right? Like your critical assets or ones that are in a particular regulated environment. Um, when you go out and look at what you're doing today to achieve that patching, um, goal or really to even maybe don't get, maybe not getting there, maybe it's happening once a month and you want to kind of do it faster as a security initiative, um, I think the key is kind of first scope, scope, where you get the biggest bang for your buck. Um, or where you're already doing the work so you can immediately start saving some productive hours from like, you know, humans that are already doing, um, is really easy to start just auto patching kind of everything that you wouldn't have spent time on before because if you have like a thousand apps off the shelf, you can just kind of turn it on. Uh, and it's great. The other side of that coin is the user experience. So I think that like when you're trying to think of like how do I go and achieve this in a smart way. I think usually you're thinking about it on a per app basis and you're thinking about what's the most risk. So I probably don't want to kill the browser. Right. First of all, Chrome has a great built in piece of functionality where you can set ah, a date when a little thing pops up in the top right corner and says like ready for update. Um, and there's some managed UI there, you don't need to go build anything. Um, I've also seen people use umon and Go even, because you can do that for any app and you can say, wait till the user doesn't have it open and then try to do the update. Um, so you can achieve that with scripts. Um, you can use live queries. In this case, if you're going to do automations, there aren't really live queries, they're in a policy. But you can write a policy that says, hey, when this process stops running. Um, right. Basically, um, you can use as well if you have a. This, uh, is kind of like, I guess going a little bit further. But if you wanted to sort of see more from a supply chain perspective as well, beyond just vulnerabilities, like, hey, like every time a particular version of an app, any app really gets installed, go ahead and write that to the data lake. So also from like a security perspective, you can have in your SOC records of all the software installs that occurred on the computer and you can get that from your edr, depending on your organization. Um, what team owns that can matter too. So I would kind of think about it from first principles and just actually kind of to um, Matt's point from before, like, focus on where you're going to save the most, get the most value first and then apply it. There's um, on the, on the fleet side, one thing we're doing that's kind of unique is six times a day we're looking at all of the new patches that came through. Um, we just, I think, had another record patch Tuesday, not at Fleet. I just mean on Earth.

Speaker G: Right.

Speaker E: In terms of just stuff that had to be patched. So, uh, one, one that you can kind of benefit from. There is, you know, that everything's basically been vetted, um, at least six times a day.

Speaker A: Joanna, uh, next question came in for you from Priya, who's in Rochester, New York. And Priya is asking, when stakeholders in a large retailer all want different AI features, how do you protect the core use cases that made the product successful in the first place from getting diluted?

Speaker H: This is at the core of every product manager's life. I think that frankly it's not different than that topic for non AI use cases. You're always having to understand a wide variety of needs and uh, figure out talking about the biggest bang for the buck. It's the same thing here that you look at. Where are the common needs, what is the biggest problem to solve, what is the most urgent problem to solve? And every single time is that, uh, use case, is that problem something that fits into my vision to what my Portfolio is to what our core space is and a, a very important capability to have in there is the ability to redirect. Sometimes you have to not say no, but say, well this is the solution belongs here and this is how we solve your other problem in another way. Um, I think when you have a lot of different stakeholders at uh, retailers wanting, asking for different things, if you can really get into it, they all still have the same core problem they're trying to solve. So you can always also look at, well maybe you don't give them exactly what they're asking for because it's not in line. You know, it might dilute the core product. But if you can deeply understand that problem, you can at least look at, well, where can this solution and where can our core use cases or updates still help address that core problem? Um, so I think that's always, that's always a way to reroute and make sure you are addressing the need even when you're not able to do exactly what the ask is.

Speaker A: I'm going to spin the globe and grab a question from another part of the world. So let's see, we have um, Sophia who's uh, joining us from Cork, Ireland and this is for Trevor. Sophia is asking make sure I get this right. For Companies already juggling SoC2, ISO 27001 and PCI, how do you layer AI specific controls on top without overwhelming your existing compliance program?

Speaker D: Oh, nice question. I think a lot of uh, this comes down to firstly understanding the nature of uh, what's changed overall in the cybersecurity realm with AI and the fundamentals haven't dramatically changed. Things like um, access controls and logging and monitoring and governance overall hasn't really changed. Those principles remain the same. What's changed is the uh, scale and speed. Uh, those have dramatically changed. Um, so when it comes to adding on new uh, criteria, you need to have some ability to map controls across different frameworks. And if you're really dealing with things like ISO and SoC2 and PCI, there are controls that are common across boundaries. Now not all of the controls are uh, unified and they're not all in sync. So you may have controls for one particular requirement, say for PCR that has some strict requirements and then maybe for a SOC2 they have less strict requirements. Uh, SOC2s are more based on objectives, whereas uh, PCI as an example is based on specific requirements. PSAL will actually tell you how many characters at a minimum you have to have in your password. SOC2 does not say that, it just says you need to have appropriate controls. So when you layer on the um, AI specific controls on top of that, you need some ability to map them across those boundaries. Um, I think again the things that have changed are not so much the fundamentals, it's more of the edge cases. And back to my earlier point about the non deterministic nature of AI, um, that's an area where a lot of the frameworks don't actually address them directly, particularly the ones that you mentioned. And that's where the real work has to happen in mapping those controls across those boundaries.

Speaker A: Matt, this next question comes for you and um, I love this question. I think a lot of individuals um, in the audience can probably identify with this. This one is from Marcus, who's in Des Moines, Iowa and I've actually been to Des Moines. Shout out to everyone there, um, how do you reassure cautious clients, clients that using AI for test definitions and control points won't leak formulas or sensitive specs into external models?

Speaker F: Yeah, this is something we think uh, about struggle with and worry about quite a bit. Um, that is a key concern for anybody that makes a product. How do I keep my competitors or just my special knowledge about that product from leaking out of my company? Um, one of the things we're doing in terms of using AI is making sure we're um, putting guardrails around it wherever we can. You know, checking terms of service with Gemini or Claude, um, is first step. Second step is putting guardrails in with your prompts and your back end programming to make sure that you're not giving it access to stuff it should not have. And also uh, letting people turn it on or off is important too. So I think a lot of there's a Russian software to use it to uh, build features with it, uh, to give it to users. A lot of that could be really good. I do worry sometimes about, you know, who's really paying attention to what data is going where and who's getting that data and where's it landing. So um, I think for our customers we're, we're conscious about giving them choices when it comes to AI. AI. Some people are less concerned about their formulas, others are more concerned. Um, so I think giving them the option that to go traditional manual way or AI way is going to be our approach to that for the near future. Uh, this stuff changes fast. However, um, and I think eventually there will be some, I don't know if it's going to be case law or something, but there will be some situations with AI that are going to Start to I think define data privacy and protection and who owns what. And I think that is yet to come. Unfortunately.

Speaker A: Yeah, really, really tough challenges and I think that's why everyone's here today. There's so many, yeah, so many real, real questions.

Speaker D: So if I could add just one other comment.

Speaker A: Yeah, absolutely.

Speaker D: Uh, I think this, this is an area we see in the cyber security realm, uh, that's been going on for some time. The, a lot of organizations have a big tech deficit in terms of not taking data classification seriously. Data classification has been, you know, the redheaded stepchild for a long, long time. No one likes to do it. Uh, it's a, it's a difficult process and for companies that haven't done that appropriately, you know, when it comes to the type of controls that Matt's talking about becomes extremely difficult because you don't really even know what it, where is your sensitive data, what does it look like and what data is not sensitive. Is this private? Is this confidential? What does that classification even look like? Um, and that's an area again that's going to need a lot of attention as they start to roll out more and more tools that have access to extremely sensitive data. Just identification process itself is uh, a major leap.

Speaker A: As we're coming to the end of this panel, I'm going to send this last question out to Adnan. Uh, um, this question is from Aaron who's in Lincoln, Nebraska. Aaron's asking, in regulated industries like health care and finance, how do you balance rapid AI feature delivery with the fairness, transparency and human alignment checks you described in your talk,

Speaker G: in one word, risk tiering to what Trevor was talking about earlier? It's like if you tear these things down in this different risk tiers, if you classify them, if you have hopefully you have a good data foundation and you have risk tiering built in there, which means if you are working with a regulated industry, majority of the risk comes from the processes and data. Those are the two major areas. So if your business process you are trying to automate or trying to make it a smarter, try to make it better and more efficient and is uh, a low risk process. It doesn't have to be a low impact process but it's a low risk process then you can easily expedite that and go through that much faster as compared to if there's ah, something which requires like 50 different guardrails to go through and it's a high risk process, then it will take longer because of the human in the loop and uh, essentially just going through the Governance process is usually painful. Right. The bureaucracy associated with. So what I have found uh, working with different leaders is that they try to stratify that between different tiers the risk tiering will help you really exploit the things much faster. Try to still have the ROI associated with that. So um, it doesn't mean that low risk doesn't mean trivial use cases because triviality wouldn't get you buy in from stakeholders. So make sure they are actually value generation use cases and you can stream, streamline them and make them fast track. As long as you don't lose the value, there will be friction when it comes to the governance specific use cases and that's by design. Um, and for that you can automate a lot of it. Um, I think that's the shortest answer I can give on this topic.

Speaker A: There you go. Well, as we're wrapping up uh, Aaron, uh, next month our summit is on governance, uh, risk and compliance. So that might be a great, great uh, day to attend. Any which way. I'd like to um, let everyone know that if you would like to reach out to any of the speakers you've heard on this panel or throughout the event, up at the top of the site you're going to see a big orange button. Just press that button, we'll get you connected. And um, Mike, Matt, Adnan, Joanna, Trevor. I'd like to thank everyone for the fantastic presentations today and for taking the time out to answer a few of the questions that we had come in from the audience. And I think that was uh, was a great uh, great panel.

Speaker H: Thank you so much, very much.

Speaker E: Thanks Michael.

Speaker B: All right, so a few key takeaways from today. First, knowing where AI is embedded in your operations is non negotiable. Second, establishing clear ownership and accountability prevents uncontrolled risks. Lastly, Trevor showed us that product market fit is all about trust and assurance. Thanks so much to Trevor for sharing his insights. If you're interested in learning more, connect with him on LinkedIn. And if you haven't already, come join us over@software Oasis.com that's where all of this really comes together. If you enjoyed the episode, consider subscribing or leaving a review. I'm um, Michael Bernzweig signing off warmly. Until next time,

Speaker A: thanks for tuning in to the Software Oasis Podcast network. With more than 200,000 listeners each month, you're invited to join our engaged community of B2B founders and executives at our next live full day Software Oasis Virtual AI event with the world's top B2B AI experts. Reserve your compass today. Get ahead with real AI strategies. Join the first Thursday of every month. Tap green button@software Oasis.com or use show Notes link to claim your spot today.

Speaker F: Best thing that's ever happened to you financially.

Speaker C: Go easy. Sold my car on Carvana. Amazing offer, really. I hit 200 on a scratcher. Did the scratcher come to your house and hand you a check?

Speaker B: No.

Speaker C: How many scratchers did you hit to get that? I hit a button on Carvana.com once.

Speaker F: Okay, that's fair.

Speaker C: It's like the lottery, except you always win. Not like the lottery at all, actually. Exactly. Inexplicably good offers worth bragging about. Sell your car today on Carvana.

Speaker F: Pick up.

Speaker E: Fees may apply.