Why Conventional Cybersecurity Won’t Protect AI? | Interview with Hugo Huang

Welcome to Secure and Simple Podcast. In this podcast, we demystify cybersecurity governance compliance with various standards and regulations and other topics that are of interest for consultants, CISOs and other cybersecurity professionals. Hello, I'm Dejan Kosutic, the CEO at Advisera and the host of Secure and Simple Podcast. Today, my guest is Hugo Huang, and he's the product director at Canonical and is also the author of a research and an article called Conventional Cybersecurity Won't Protect Your AI, which is published in Harvard Business Review.

So in today's podcast, you'll learn why companies need to shift their cybersecurity towards hardening the architecture and towards change in their managerial processes really to make actually sure that their systems are protected. So, welcome to the show, Hugo. Thank you, Dejan. So glad to having me here.

It's really my pleasure to share a bit more about our research. To the credit to our team, it's a research based on whole effort from our team in Conoco and also our partner in IDC and our partner Google Cloud. So my only wrote that article, but all that work was from the team. Great.

It was a great team actually, and a big company is actually standing and backing up this research. Can you tell me a little bit about the research? So what did you find actually in the research? Well, before we talk about what we find, I think it's very straightforward as I put it in the title.

So conventional security tools does not protect your AI. That's the finding. So the key finding is we find that. But I want to talk a little bit about before why we want to conduct this research.

This is very important. I think, you know, getting to the year of 2025, there are many people talking about we are in the age of the inference or age of the AI agents. The things get changed a lot. But we were thinking of, well, what are the leaders looking at this landscape?

What they feel? Do they see any threats, or do they see any difficult challenges? How they manage their teams, things the team had been changed their practice, the business cycles. So we try to capture the first the very subjective opinions from the leaders.

You know, there's many technical details. We can we can, you know, research from many different ways. But I think after all, the business is about people who lead these who lead the direction are the leaders. Who, you know, are the decision makers in the in the large companies or even, you know, smaller startups.

So those people's opinion shape their actions. They control the budget. They control how much they want to invest in certain areas, how they want to approach the market. So that's the starting point of our research.

So we surveyed 500 executives in confidence, like, at least the size of more than 1,000 people. So then most of those audience are, you know, like, CIO, CISO, and also, like, chief AI directors, these kind of titles. So we request IDC to survey and conduct individual interviews with those executives. So this is like 500 very valuable samples from different organizations.

And through that, we try to capture what do we get of the opinions from the leaders. Obviously, for research is very grounded. Why did you actually come to this conclusion that this conventional cybersecurity is not good enough for AIH? Well, that's basically from our those questions, we we see the fear.

They're basically can like, there's three concerns trigger us to to try get to this conclusion. The first is that the shared AI. So there there are many people, many individuals in the company that they're using more and more AI tools without any monitoring from their IT department, even their supervisors. So this is one of the problem.

The other one is that the cost is basically not easy to control. So you probably see that there are there are many people using AI agent at this moment. The difference between an agent and a normal, tool is that you probably be sure as the traditional, know, traditional software, you probably know how many times it will work for the CPU. What's the cost?

What's the resource it will consume? But for AI agent, when you ask it a question, it probably persists several thousands of times from different dedicated agents. So this is a very complex situation. So many leaders consider, wow, since their the the process is is not transparent, that means there are many black boxes in this process.

I I don't know whether I should trust or not. Another thing is about there are so many new infrastructure coming out. We see those, NVIDIA GPUs. We see, you know, memory loss card, which, you know, connect different GPUs together, Intel IPUs, and Google TPUs recently.

So there are so many different XPUs. Are you sure those are secured? How to secure those hardwares you've never seen before? This is like with these three different, you know, categories of problems.

Firstly, the people, the shadow IT, shadow AI, they're using something that you didn't know. And then because of the agent, because of the very complex process, you don't know what happened in your process, you know, the software software, you know, supply chain. The third is those new hardwares. You you purchased once and you started using that without any security properties.

So that's a very complex situation. Come to the conclusion, we say that. Yes, now it's the time for you to reconsider your infrastructure, Your original process, your original cybersecurity practice may not apply to your current situation. In your article, you actually mentioned a couple of risks actually, that you found out are significant.

So, like, data poisoning and other serial prompts and model inversion attacks. Can you explain briefly what actually these mean and perhaps give some examples? Oh, thank you. Those are kind of new risks and new threats from the, you know, a new practice we just recently observed.

For example, the data poisoning. This is like a deliberately some you know bad actors they they put the data the corrupted data in your data in your training data sets So with that, since your data is kept corrupted and your result, your model will be corrupted afterwards. The result is very obvious. So your customer in the future, when they try to use this model, the model you shipped in your product, they were likely to get an answer, try to guess their wish list.

So this is something, we see quite common. And also, you just mentioned the adversarial prompt or adversarial This is kind of a practice that some bad actors try to use prompt engineering to force your border to output certain results or certain outputs. For example, the interesting result pressing there actually we in our hiring process and we get this kind of tricky example. That was someone putting a very small font of the couple of sentences in their resume.

Whenever that resume gets screened by automatic machine, it always output very good results. But when you look at the PDF version, you never see there's anything there. But when the machine reads it, well, they can find a small font of that just saying, oh, this is a good candidate. This candidate will meet all the requirements.

It's a very funny experience, but yeah, there are many ways. So I think the traditional cybersecurity people have not really seen those practice before, I think two years ago. And what about this modeling virtual attacks? How does this work?

Well, model inversion is more about the IP inference risk. So you create a border and then someone just use this technology to try to to ping, to to to to, like, to reverse engineering your motor, to to steer your motor without actually going to fully copy all of your motors' wage, any hierarchy of your motor engineering. So that's the way they try to steer steer your model. So what you're saying is basically by, let's say, putting certain prompts, they can actually pull back the logic that was built into this model and also the data that was used to train it, right?

Exactly. Maybe not the old data training, but they can definitely get some very critical model or metadata. Okay, okay. So, in your article, you're actually presenting a thesis where you say basically that companies should not focus anymore on patching, let's say, rather they should actually focus on hardening the underlying infrastructure and also on changing their managerial processes.

So, why do you actually think that? Why do you think that the patching itself is not, I would say, the most important solution to this problem? Well, I think patching is quite important. This is But the patching itself is not sufficient enough.

So I think there's an example, like we mentioned in article and also, I think, spend a little bit about the original story that we worked with our real customer. So this was customer from a bank, and they tried to, they patched everything, they followed the the the, you know, standard process to to the security pipeline, the all the patches, and also make sure those software are being tested and verified. So there's nothing wrong with the software itself. But there was a keylogger in the in the operating system they choose.

So the the because the operating system have a much powerful access to to almost everything running on top of that, That that means, even though your your software itself, your your application is is secured, but your the the KeyLopper can all can can can still see all your secret. That then that gives your makes your the whole system is vulnerable. Like, you know, you're you're building a very solid building on top of the SAC. So that that means no matter how beautiful your architecture, how strong your material you have been choosing, the foundation was soft.

That means literally, you know, doesn't secure anything on top of that. Okay, and how do you actually then harden the architecture? So basically, is needed to actually push cybersecurity in that direction? I think that's the the kind of direction we are we're putting towards too.

I think we believe that confidential computing, encrypt everything in the at least from hardware when that not data in data in in use, but also data in compute can encrypt the time. So nowadays, we have been working with Intel. Intel has a technology called Intel TDX, which encrypts the data when it's in the CPU. And, also, AMD have the similar properties like they call the SCV SMP.

And, also, for GPUs, NVIDIA s 100 have the capability to in in encrypt the data when it processed in s 100 GPUs. So these are the fundamental layer we believe that, you know, combined with all your good practice to to secure your software, secure your middleware. We we hope you can encrypt all the data when you process it in the CPU or GPUs. You mentioned this confidential computing.

You refer to this Google's confidential computing. This is about the technology that they're using our product. Yes. Conventional computing is invented by, I think starting from the Intel and then Intel and AMD both have the similar technologies to encrypt the CPU, the processor.

Nowadays, we see more and more demand from the GPU side because people do more inference cell training, and NVIDIA follow-up this this practice, apply the similar technology to NVIDIA H100 GPUs. Okay. It's just a starting point. You may see more and more GPUs and other, like, maybe TPUs, we are talking about this with Google, and could follow this same trend.

Okay, and if I understood well, you're actively involved in this Google's TPU, which stands for Tensor Processing Unit, if I'm right, correct? Yes, yes, Tensor Processing Unit is, we believe this will be a game changer for this industry, especially the market. We see that you probably can already see the Jason Wang's famous five layer architecture of the AI infrastructures. We see that, you know, in this five layer infrastructure, the most profitable layer is the hardware.

Right? And when you talk about hardware, most, like, nowadays, 5% of the market share is concentrated on NVIDIA. No one can compete against that at this moment, but we see there's a hope. When TPU comes into this market, we will see some customers may stop paying tax to GPUs, but they may join the family of TPUs.

We see Anthropic. They have training on TPUs, maybe because they have some like, know, Neptune's partnership with Google. And we see other partners like Apple. Probably I should not share more about that, to be honest, beyond these public information.

Of and so from this, let's say, hardening actually infrastructure, so how actually is TPUs how TPUs will actually improve the hardening aspect? Can you just explain in short terms? Oh, I think that I cannot share more about this. About hardening TPUs, the first is it's something, you know, in the road map.

We we we we haven't haven't started working on this at all. But I think in the future, I I I said to for the because when we see NVIDIA H100 GPU have the confidential computing feature, see there's many customers asking for this, and there's a strong demand. I believe in the future when we can do confidential computing to other architectures, other other PUs, other XPUs, that probably have the strong demand as well. Great.

Now, you also mentioned this managerial aspect, which will have to change if companies want to make the shift. So, what kind of, mean, you mentioned a couple of these managerial changes that need to be made in your article. So, can you just elaborate a little bit on them? Well, I think the first is like, I think it's more kind of when you talk about security, previously we always point to CISOs, point to CIOs, but nowadays it's more kind of, we believe, it's more kind of a CEO responsibility.

At least it's not just the responsibility from the CISOs. Because we we see in most of those AI initiatives, they are driven from the business demand. They are not actually just to make sure your current infrastructure is secured or your certain application requires more attention. That's not the case.

AI is kind of the, for most enterprises, they are kind of new strategy. How we want to embrace this AI revolution is more of the CEO project rather than a CIO or CTO project. So, we believe that from management point of view, since you initiated this, you'd better think about the security when you start finding this product. Otherwise, it would be too late if you start to initiate it without considering, as I just put the building on top of the sand as an example, if you do not consider the foundation in the beginning, where you revamp your foundations will be costly law.

That's one thing to consider. Another thing is that because from the human resource point of view, the talent is quite scarce at this moment. We'd better to make sure your CHRO, your human resource department, be aware that this is your strategy. This is to predict the demand for the future human resource requirements.

When you compete against other companies to a group of very high quality and scarce talents, you'd probably be ready to understand what they require, what what what their expectation, what their career aspiration to make sure your team, your whole organization, to meet their requirements. So this is more management. It's not really too much. It's more management of the other technology itself.

Yeah, and when you mentioned human resources, do you refer to let's say AI specialists, specialists, or some other profiles of people? I think these assets are combined. It's not easy to it's already very difficult to find a cybersecurity expert, but at the same time, we need someone not only good at cybersecurity, but also first they have to have some experience in those AI projects, those AI initiatives. But you may have different opinions or different, you know, you may prefer to have someone have some AI project experience, but not necessary to be very good at the the deep technologies of the new cutting edge technologies in the AI.

But you want this person to be well balanced, or you may want someone, you know, be good at one side, but but still, it's okay to at least have some some, you know, footprint to the other other side. That depends on your profile. But overall, we feel that it's kind of difficult to find. For for example, in Linux, it's very difficult to find a good a good engineer.

They they continue to do kernel patching Because we see color patching is very important, but it's unlikely to find someone just graduated from college and see their whole career passion is to the kernel patching. But this is sort of like we have to look at the experienced talent pool to find out who actually doing very good in multiple years. As action to either the human resource department or as a CEO when you're searching for your talent pool. So you have to first, I understand what kind of the talent, where where the don't worry.

Stay. So you you should should need to figure out to the you know, in the startup, we call it a beachhead market. You have to figure out where where you find the beachhead market and then to try to attract the talent from that net market. You also mentioned that the CEO need to have need to, let's say, actively, if not run projects, but at least be a sponsor of these kind of projects.

And are you saying that the CEO should take interest in cybersecurity projects, or are you saying that CEOs should actively take interest in AI projects, but also take care that cyber is part of these AI projects? So what kind of angles should CEOs take? Well, that depends. I think, know, as AI initiatives, it says kind of In many cases, sold their famous stories recently years that some CEOs, some very famous companies, CEOs take their weekends, their holidays to start AI initiatives.

And, you know, it just happened to be there. The technology enables so many senior people without, you know, heavily getting into the technical details, can they can invent something substantially. So but as for security, it's it's unlikely you you can, you know, when you have an idea, you can just generate a pipeline to make sure everything is good. You've probably heard of the mythos in these days.

Anthropic mythos just released, I think that's back to April, one month ago. They discovered a CVE waterbedia that no one has ever discovered for twenty seven years. That means means security is there it's more difficult to figure out the problem. So I think my advice is that AI is moving very fast, but security is much easier, harder for people to to actually make sure you know to to find out the problem is easy to make sure you have no problem at all is very difficult so it's kind of it's it's it's very, how to say, as a as a company, you'd better be more conservative in the security side and you can be more aggressive in AI projects.

But why should the business side actually take active interest in cybersecurity? Well, because if you do not take consider of the cybersecurity, the cost in the future would be way much higher than you do today. So for example, I always point to the example of the cloud migrations. There are so many companies, from five years ago, there was a trend that many companies pulled back from the cloud migration.

They they they realize that everything in the cloud actually cost them more, which is not true. But you will see that when when you make a decision maybe ten years ago that, oh, we will only go to cloud, we will only, you know, run servers in the cloud, it's kind of decision that you prefer flexibility rather than the cost. But after multiple years, you realize the flexibility may not be your priority. You prefer a stable workload, which means stable workload means that you don't really need those flexibility.

So it's much cheaper for you to move your workload to your own data center. Purchase your server, and you own your own data center. You can run those servers small, maybe for five, eight years without without any problem. But that definitely, it will cost you more when you run the cloud.

The security is the same. When you not consider it, we say, okay. It's a small project. We're not costing us a lot a lot, and we don't even know the future of this project.

So why should we bother to consider the security? That could be a valid point for for us as leader for this project. But when you ramp up, when you approach to, say, like a million customers already on your platform using your service, then you realize, okay, some ransomware get into my system. What should I do?

Wait. The cost of revamping the whole system is much, much higher than you designed in the beginning. Like, we implemented certain frameworks, make sure our we have a security policy. Even it may not be perfect in the beginning, but when you gradually improve or scale up your system, you may hire new people, you may, you know, have dedicated team hereafter to take care of the security part.

That will be like your security is improved as you scale up. That can help you save some, you know, problems or pay a huge cost to certain incidents. Okay, it's certainly a valid point and besides, let's say, this cost issue or cost savings, there any other, let's say, reason why businesses should take active interest in cybersecurity when it comes to this shift in AI? Well, think cost is important, but I think the trust from your customers, your reputations, your brand is even more expensive when you put those things in at risk.

So while it's true, everything has a cost, but when you put your reputation you put your brand in such a risky position it's not only about how much you pay it's more about whether you access to a certain category of the customers. So that's definitely we encourage the leaders to consider. So ultimately cybersecurity becomes one of the protectors of the brand, right? Which is a strategic category, not anymore operational.

Yeah. Yeah. Yeah. As we discussed earlier, when you look at those model inversion, those, you know, data pollution, That's the problem that when the customer don't see your brand as a trusted brand, should be a disaster for any of the enterprises.

Okay. If we can go back just a little bit to your article, so you mentioned that also companies should treat their suppliers in a different way, also tools that they're using, AI tools that they're using. Can you explain a little bit further what you found in the article? Oh, sure.

I think there's a If I only want to, you know, make one advice or recommendation to to the leaders. I think that to be to avoid vendor lock in, I think, is the first priority. So we see many, many of the new tours. Some of them may not have the actual competitors.

When you when some companies we see our well, our customer, when they have a they they're porting a a service from another AI company. Then when they try to shift from their current platform to another platform, they just realized that new platform doesn't support that. So that means a huge cost. It's not not really.

You know, sometimes it's not about your vendor policy, how you balance different vendors in certain categories. It's more about when we look at this AI landscape, it's more complicated. Their dependencies, actually, we have a principle we call it like a seven layer seven layer infrastructure. In each layer of those infrastructures, we see at least 1,000 dependencies.

So with with so many so so many dependencies for your software, you are likely to be able to diagnose which part is more critical than others. So that's why we suggest that you better have an open source strategy. Make sure every component of your software, of your architecture, is kind of have a replacement or have options to avoid Vanderlumpine. That's what we advise.

Yeah, it's very important, yeah, to have this kind of a resilience, so to say, built into your supply chain, right, to be able to overcome any potential issues. Okay, you also mentioned in your article this AI risk management framework from NEIS. So why would you say that this framework is important for cybersecurity? I think that's the framework I find.

And maybe Well, I think a framework is quite important for any leaders to consider to starting an initiative. So this framework is I think now it's the one I feel like comprehensive and has many aspects that you trigger the leaders to consider. I know there are many movements around this and are joining this more community and try to improve. So I would like to say that no matter what kind of framework you want to adopt, at least you have a a starting point to to, you know, gradually add adding layers, adding components to a frameworks.

So that's that's part of the learning process for us as well. Good. So to wrap up the call, what would you say are the kind of tough things that security officers should do actually to start this shift of cybersecurity focus because of AI? Well, like, no, as we explained in the title, so the conventional cybersecurity does not protect AI.

Yeah, the word is having changed, and as we discussed earlier, that AI can find some vulnerabilities, so people never find them for twenty seven years. AI can actually conduct any sort of the packs in the speed that people never can contact. And also AI can help you to find out your to diagnose, to help you to see your system in a more transparent way. So for for this leaders, I think it's a good time for us to revisit this area.

Because agents, unlike when we say conventional, what is conventional? In conventional wisdom, we consider software as tours. So tours does not distort your opinion unless you make a mistake. That only amplifies your mistake.

So but in the AI area, in this in this age, AI is acting more like an agent, like someone represent you, someone, you know, can capture some of your skills. It's a lot more like managing a team of people. Right? That rather than extending your your tools to us to a large scale.

So for leaders, I think, you know, to have the management principle is more important to have a policy to control, manage those agents, to make sure someone you can dedicate your policy making process to some human, or maybe saying someday to an agent to control other agents. So this is a new practice. I think cybersecurity has to change a lot, not only from the infrastructure layer, from the old layer above, but also from the field, the overall picture, the framework, how you make your policy, how you make sure the policy controller actually follow your guideline.

Great, Thank you for these insights, Hugo. It's been a pleasure talking to you. Likewise. Thank you.

It's really enjoying to talking to you. Great. Thanks again, Hugo. And thanks everyone for listening or watching this podcast and see you again in two weeks time in our new episode of Secure and Simple Podcast.

Thanks for making it this far in today's episode of Secure and Simple Podcast. Here's some useful info for consultants and other professionals who do cybersecurity governance and compliance for a living, on Advisera's website you can check out various tools that can help your business. For example, Conformia software enables you to streamline and scale ISO 27,001 implementation and maintenance for your clients. White label documentation toolkits for NIS 2, DORA, ISO 27,001 and other ISO standards enable you to create all the required documents for your clients.

Accredited Lead Auditor and Lead Implementer courses for various standards and frameworks enable you to show your expertise to potential clients. And a learning management system called Company Training Academy with numerous videos for NIS2, DORA, ISO 27,001 and other frameworks enable you to organize training and awareness programs for your clients workforce. Check out the links in the description below for more information. If you like this podcast, please give it a thumbs up, it helps us with better ranking and I would also appreciate if you share it with your colleagues.

That's it for today, stay safe!