Hardware Hacking - Essential OT Attack Knowledge [the industrial security podcast]

Welcome listeners to the Industrial Security Podcast. My name is Nate Nelson. I'm here with Andrew Ginter, the vice president of Industrial Security at Waterfall Security Solutions, who's going to introduce the subject and guest of our show today. Andrew, how's it going.

I'm doing very well, Thank you, Nate. Our guest today is Marcel Rickson. He is the founder and lead instructor at fox Grid International, and our topic is hardware hacking. Picking apart the hardware, finding the vulnerabilities, you know, arguably essential attack knowledge.

We need to understand how we're going to be attacked if we're going to design effective defenses. So that's that's the topic for today. Then, without further Ado, here's your conversation with Marcel. Hello Marcel, and welcome to the podcast.

Before we get started, can I ask you to introduce yourself. Please tell our listeners a little bit about your background and about the good work that you're doing at Fox Grid. Yeah, thank you, Andrew. Hi everyone, my name is MARSA Riksen and if it would introduce me in one sentence, I am an automation engineer turned ot security nerd.

To my background, I have a master's in automation engineering. I have global experience in commissioning automation systems as well as programming planning industrial operations. Now, during my day job, I am an OT and IoT security consultant and a product owner of our in houset remote access solution. During my nighttime, I am an hacker, or if you want to put it more formal, I am an independent OT security researcher that looks at what makes and breaks OT devices.

Coming from that, I also founded Foxgrid, where I want to teach industrial cybersecurity and safety to newcomers. Thank you for that. And our topic is hardware hacking. Can we start with an example.

You've got a couple of reports out, Can you pick one? Can you tell us about, you know, a concrete example of what what is that? Yeah, let's talk about the hardware hacking that led to a CVE that I found last year where I found hard coded root credentials hidden deep in the device's filmware memory. Can you go a little deeper?

What was the device, how's it supposed to work? And you know how important is what you found? So the device is a remote access skateway that machine builders usually built into the electric cabinet so which connects the machine to the service provider. In case there's an unplanned interruption or any other operational bug coming up, the service provider can directly connect over the cloud portal to this edge device and start troubleshooting.

If I made. This is something that's used in manufacturing. When you say the manufacturer, you mean someone who's building a robot, someone who's building a stamping machine, someone who's building I don't know, conveyor. Is that the use case here?

This basically can be used in any operational so from your maybe water treatment planned to your manufacturing to your building automation. There are really no limits. This is really a network connection from the service engineer's laptop directly into the heart of the device or into the heart of the operation. Okay, So it's not just used for like a robot for a manufacturer of equipment.

It might also be used by a service provider, by the engineer who's responsible for you know, occasionally coming in and servicing parts of a water treatment system. It's used to access systems as well as devices, is what I'm hearing. Yes, correct, So this acts as the gateway to the machine or operational network. So you've found the default credentials.

Does that mean that any fool who wants to can connect to the cloud connect into this thing, or how would you use those default credentials. Luckily, the attack vector is really narrow. These default credentials. They grant root access to the device, and you only can get root access when you are physically connected to the device.

So luckily the cloud attack surface or the cloud is not exposed to disvulnerability. Andrew, I don't know if I just missed it, but what is the actual device that we're talking about here? It is an Ixon device I XO N I forget the exact name of it, but it's it's physically it's a little device about you know, six inches square in an inch thick. And.

In my understanding, it's you know, it's a remote access device. You can connect into it from the cloud. Who uses this? The sense I have is that it's used in manufacturing.

If you're building a laser cutter or a stamping machine, you might build one of these into the thing so that when the customer calls you up and says your machine isn't working, something is worn out, you can remote into it, do the diagnostics and say I think it's this part, replace this part, see if the problem is solved, because you know, moving parts where out friction is the is the enemy of moving parts. But you know, when I asked the gentleman Rick, he said, yeah, manufacturers of physical equipment use it so they can maintain the equipment or diagnose the equipment remotely.

But it's remote access. You know, a service provider, an engineer who's responsible for, you know, keeping the automation running at a dozen small water utilities in the geography might well buy, you know, a half dozen of these and drop one of them into each water system to access the HMI and the automation and whatnot. So it's remote access. The sense I have that it's used frequently by manufacturers of equipment that's used in manufacturing, but it could be used by service providers as well.

And I think it's the remote access thing that has me a little bit confused here. We're talking about hard coded credentials as vulnerability, something I'm rather used to in the IT space, right, Like a public repository or a server that's been incorrectly configured will leak credentials to the web that then actors could use to get in and we're talking about a remote access device, and yet I think he mentioned there that you can only actually exploit this vulnerability if you have local physical access to the machine.

So can you help me explain that gap. We go into this in sort of more detail later in the interview, but let me let me let people know kind of what's happening. This is you know, there are basically two user interfaces to the device. One is the remote access user interface with users configured and blah blah blah.

That's not where the vulnerability is. The other interface is if you touch the device and you connect to it. I don't know, I was a little weak on the details. If you connect through the USB port or if you connect you know pins, you know, you're too electrically to pins sitting there on the on the on the circuit board.

If you open the device up, you can get access to the operating system of the device. And it's the operating system credentials that were lead so in order to use those credentials, they don't work on the remote user interface. They work locally when you're when you're able to physically touch the device and plug stuff into it. The CVE was twenty twenty four, dash five seven, seven nine zero.

It was given a five or a five point nine or something like this, not a ten. This is not a remote code execution vulnerability. You can't do this remotely. You have to be local.

It's a local escalation of privilege vulnerability. That explanation makes a lot of sense to me, But why is it that, Like, how can you even leak credentials to somebody who's physically using a computer? Right, Like, any credentials on my computer that get leaked to me doesn't matter because I'm the user. So what I suppose I'm asking attack scenarios are we worried about with this vulnerability.

Well, actually we didn't go into that, but you know, as far as I know, the scenario is that you're there locally touching the device. Now normally, you know, you look at the device, it's got network ports, it's got you know, one of the ports is connected out to the world. You come in remotely, you know it does its thing. There is no other supported user interface.

But if you touch the device, you can get in there. You can tamper with the firmwaire, you can tamper with stuff. You can you know, you could presumably create credentials that you could use remotely. You'd need a little bit of skill to do that, but you know you could break the device, so it's again if you're standing there with a hammer, you could also break the device.

This is why it was given a lower priority. Yes, technically it's a vulnerability, it's not a really alarming one. What's interesting is how did you find it? Because the way that he found it, the technique is what he teaches at Foxgrid you can also use to find more interesting stuff.

What I wanted to ask you about is, you know, we've never had someone on the show who picks things apart like this, or maybe we did once about three years ago, but it's been a long time, you know. Can you talk about the process? How did you find this? How does one pick these things apart?

What is what does that mean? If I would just describe it to you maybe in a pub or over our coffee. This really is like a hardware and digital scavenger hunt because you have to look at so many things. You also go down a rabbit hole, see it's the wrong way, turn around, and then keep digging.

And to find this I just needed tools for about thirty euros, so a multimeter, screwdriver, prying tools, a U s BI logic analyzer, and a usbut interface was all I needed to find this ulnerability. Can you give me a little more detail? Those are the parts. I've never used a logic analyzer.

I mean, how technical is that? Do I need to be an engineer to use a logic analyzer? How did? How did you go about this?

What you know? Can you tell us a story? You know, what did you start with? What do you do next?

What does that mean? And you know a blind alley went down? How did it work? Yeah?

I can walk you through the all the six or seven steps that led to root access, from opening up to the device until I was greeted with the root banner. Okay, And before anyone gets started with hardware hacking and picking a dior, before anyone gets started with taking a device apart, here are four electrical safety rules that you should follow because your own life is more important than your curiosity. And never ever open wall plucked devices because if they're directly plucked into the socket, this means that hazardous voltage is inside the PCB, inside the device, and there's the risk that you can touch a live wire and it's urisk and electric shock.

Therefore, use devices that have external power adapters only so that the voltage conversion happens outside the area where you're working with. Also avoid mixing power sources. This is important for example if you really go into firmware extraction. And of course prevent in short circuits because this will fry your PCB and then you have a very expensive brick.

But if you stick to these rules, you can open up the device and first start with a hardware reconnaissance. We'll just take a look on what chips are on there. And many industrial embedded devices they'll run on a so called system on chip and they are somewhere close to the system on chip the flash memory filmware chip. So this is basically where the brains and oil information are still and once it is and once the system is powered on, the system on chip pulls the filmware information from the firmware memory chip.

If you identify these, then you take a look around the board. Are there any debug interfaces and and on this device I found the so called Ewert debugging interface. So with these we can move to the next steps. And first we do some electrical measurements just to prevent that our us be US BWARED and USB Signal analyzers get fried because they are very sensitive to voltage.

And first things first, we first confirm the common electrical ground on the debug interface we identified. Once we identify the common ground, we know where we well connect the ground wire of our USB logic analyzer. Then a viewered interface usually has two more pins RX and TX, which stands for receive and transmit. And then we turn on the power and then measure these pins against the electrical common ground and in most cases we will find a voltage range between three walls and five walls, which means the devices are communicating on the so called transistor transistor logic level.

When this is identified, we can move on with the logic analyzer, power off the device, connect the logic analyzer. We connect the logic analyzer's ground to the boards ground, and then the RX and TX WIA. Although the RX and TX pin are already labeled and we only could connect to the TX, it's always good to connect to all pins that we have a full picture of what's going on. Because Andrew at this board is what's this what's easy mode?

The pins are already labeled, but that's not always the case sometimes you just have a three four five pin sticking out and you don't know what I mean. Then you also do the same procedure. You measure for the electric common ground and then start measuring the voyage levels, and this gives you an idea if you can find logical signals going on. So thanks for that.

I mean, that's you know, giving giving us some insight into the mechanics of dealing with the device. I mean, I'm I'm a software guy. I never have to worry about electrocuting myself if if I bring up a compiler on my laptop. But let me ask you know, you've talked about two sort of devices here that seem to ring a bell with me.

There's the the you aret, which a quick question is that you are USB or is it RS two thirty two. This is an RS arras thirty two connection to USB. So this basically converts the signal, converts the logic serial signals to USB so that your machine, your computer can work with that. Nate real quick here, I had a very short sort of interaction with with Rick there asking about the difference between USB and RS two thirty two.

For anyone who didn't quite track that RS two thirty two is a very old hardware signaling protocol. I mean I remember using RS two thirty two back in the day to connect you know, this was thirty years ago to connect to three hundred bits per second modems. Okay, ancient ancient technology. Why would there be such an ancient interface on this modern device, is roughly what I asked him, and he said, there isn't.

What there is is is a USB port. It turns out that what he connected to, the t X and RX he connected to were signaling USB. And when he looked at the signals, he discovered that the messages coming across the USB were r S two thirty two over USB. So he looked around.

He said, well, you know, I have a dongle that can that you know, can take USB and gives me RS two thirty two, and he connected to it and there he can see the messages coming across. So that's that's what's going on there is it's a US thirty two thirty it's a USB connector on the device, but the signaling is r S two thirty two over USB. The other one that that struck me, and again I'm a software guy. You mentioned the flash chip.

You know. To me, if I get what's on the flash, I can start looking at instructions, I can start running my business. Embler, Is it possible to to sort of go under the nose of the device and just read the flash chip? Or do you have to go through the front door?

Do you have to go through the CPU in order to get access to the flash? No, you don't. You also can basically perform a chip off of the flash chip and then read out the contents with a programmer. This is also possible, but at the time of my research, I didn't have such equipment here, so I went through the front door.

Okay, that's fair, So please carry on. You're you're talking about the u R. I interrupted you. You know, finish finish the story here.

How how are we? Uh? How did you get in? Okay?

The logic analyser revealed that indeed, logical data is transmitted over the t xpin. So this means we can connect our us BURET interface and open a serial console to that. That's fair. I didn't realize it USB will I mean, I knew USB was cereal, that's what it means, universal real bus, but I guess I never put two and two together that you could just, you know, I don't know, connect an RS two thirty two to it.

Well, you always need this interface device that you pluck between that you plug into your USB socket and then on the other end of the device you can connect it to the target device. So once the usbu WED interface is connected, and I started the terminal on and I started a serial console on my Linux machine. Then I powered up the device again. I could see the bootlock flashing in front of the screen.

You know, it was a basic Linux lute bop. It was a very It was a basic Linux bootlock, and at the very end there was a lock in prompt to lock into this device. This is where it got interesting. And here my curiosity was really on fire because I really wanted to get into this d and I started to look at the bootlock itself.

First. Here I learned that the filmware memory is partitioned into several partitions. And if you look at the common IoT hardware hacker courses, then they always tell you to go for the root FS file system because that's where all the binaries are stored off this Linux device. But there was another petition that was interesting for me.

This was the so called factory partition. Scrawling further up in the bootlock, there was also a brief prompt to pass a There was also a brief prompt to press space bar to enter the bootloader, but andrew the timing for this was so narrow that it was almost impossible to hit the timing right to enter the bootloader. You know, you can imagine I was jamming the I was hammering the space bar like a lunatic. And then maybe at the fourth or fifth time, I succeeded to get the timing right, and then I was presented with the next option to choose the operation.

And here a very interesting option was presented to me. By pressing the number four, I would be able to enter the boot command line interface. And this was something what I was interested in and wanted to go. But with this narrow timing, I turned to chat GPT, asking it is this way.

Is there a way that I can automate the keypresses and can send a space bar press and a number four press at rapid speed? The AI gave me a five line shell script code which uses an on board tool of Kelly Linux to send space bar and number fours, and this immediately lended me into the bootshell. And the bootshell of this device is based on the U boot bootloader, and all the hardware hackers out there that are familiar with you boot would immediately see that this is already a very stripped down and secured restricted version of you Boot.

There was almost no way of manipulating the device, but they're left in the so called SPI command which enabled me to read the content of the factory partition. So that's what I did. I issued the command to read the factory petition and then and then it printed out the content of the factory petition in the hexadecimal format. And here's something really strange occurred to me that the data was not always represented in two hexadecimal digits.

You know that, You know hexadecimal data always needs to have two digits. If not, the data gets misaligned and then gets corrupted. So the problem I was facing here is that some digits where or some data was represented with single digits missing the second digits, so the data was not usable for me. Okay, Then I used another script to align the data and then convert the text has the text hexidescimal data back into binary hexadescimal data, and then I was able to view the binary data and the ask the interpretation of that.

And here's something really interesting stood out. There were basically three strings of data that at first made not really sense to me, but somehow felt familiar. And suddenly I realized this is the information which is also printed on the device's label on the site. I could see that in this factory partition of this device, the version number, the serial number, the device version, and the lock in password for the web management surface for the web management interface was stored.

But there was another string that also kept me guessing and puzzling for quite a while. But this unknown string had the same characteristics as the weblock in password. It had ten characters, capital and lower case letters and numbers, and I tell you this had to be another password. So I restarted the device again and at the very end of the boot process, I was prompted for the lock in once more.

I entered the user name route and entered this data I found inside the memory, and this gave me root access to the device. Andrew, it's not that anything that Marcell said at any point there wasn't clear. But we've now gone a while and he's expressed a lot of technical steps. Can you just give me the big picture summary what we're talking about here, what he achieved and why it's important.

Absolutely, he real quick managed to connect to the boot shell with the space for you know, script constantly blasting, and he got in and discovered there was almost nothing he could do there, but he could look at this one tiny partition and he managed to you know, get the data, decode the data, and he looked at he said, you know, this looks like a serial number. It looks it looks like a password. And so he said, well, let's try it, and he reboots the device again. He doesn't do the space for this time.

He lets it completely boot and it comes up and says, okay, I'm ready log in. Do you want to log in? And he said, yeah, let's log in as root and it says, well, what's the root password? And he says, well, here's the string that I saw in the partition.

He enters it and he's in and now he's in route. It's not like you looked at the file system and said, oh, here's files. You know, look, there's a file with the name password. It wasn't nearly that obvious.

Now it was very well hidden, but I think also on purpose, because there's nothing written in this memory area before or after this partition. You really just find the version number, the serial number, the web management password, and well the root password. So somewhere in production when the device gets so to speak, gets the breath of life and the data for the label, at this moment, the data must be flashed into this filmweb memory chip. Okay, so this is a very small partition.

Then we're not talking, you know, tons of megabytes, We're talking tons of kilobytes. Yeah, it was very small indeed. Okay, Okay, cool. So you found the vulnerability and then I assume you know there's something called a responsible disclosure process.

I assume you contacted the vendor, you contacted the government. What was the next step there? So the next step was to contact the security contact of this company, and luckily I was already in contact with him on LinkedIn. So on a Sunday morning, I sent him a screenshot of hey, mister x y Z, I got root access to your ot gateway and within two hours he replied and said, okay, this is very concerning.

Please send your findings and everything you have to our security email address, and we will look into this first thing Monday morning. Then I wrote a quick report attached the screenshots and the proof of concept video, and around Monday lunchtime they replied said, yes, this root password is uniquely generated per device and inserted here during production. But since everything is uniquely they kind of hinted at that they're accepting the risks so that the probability of this being exploited is rather low.

They also said, if machine builders, integrateors operators stick to their security requirements, they do not see really a risk of this being exploited. Okay, so the vendor said, you know it's a low priority because you know people are expected at physical security. No full off the street can come in, take one of these devices, walk away with it, pick it apart, bring it back. That's that's not a realistic threat.

Do you agree with that? Yes, totally. I agree with that from on my experience on the shop floor and in the field. You cannot just walk up to an electric cabinet, take out their device, screw it open, extract the root credentials and then put it back in with the back door you implanted right, This would hopefully catch some attention.

Okay, and you know, can you can you finish the the the thought? I mean, you wound up with a CVE for this, you know you've interacted with a vendor, then then what how do you how do you finish the process. Then I contacted MITRO to file a CVE, also reported the things I found and the implications for this, and after two months the CVE was. Assigned and at that point you're able to disclose publicly.

Is that right? Yes, all that being said, there is a tiny, tiny risk that you may receive the backdoor device. But then someone really must be targeting your operations. They need to know that you're operating such device, and if you expe acting a new shipment, they could intercept the shipment, open up the device, extract the root credentials, implant a backdoor, pack it back up, and ship it forward to your operations.

So for that, if you are operating something critical, or if you're operating or if you're having critical infrastructure and operations, you should definitely opt for temper detection and protection. You know, some devices they have these little sticker on their warranty void if removed. So fascinating stuff, at least at least to me. I've always wondered, you know, how some of this hardware hacking was done.

But you know, as far as I know, you don't get paid to do the hardware hacking, unless I don't know there's a bounty or something. You know, how does this relate to making a living for you? This is not my day job. And I also don't get paid to find these fulndabilities.

Let's just say this is a very expensive hobby. I've been in the I've been in the domain of automation systems for half of my life, and after my work, I'm still interested, especially in what makes and breaks these devices. And that's also how my trainings got born. I took all the experience I made from while breaking these devices and turned them into training.

Okay, and you know this is what you do at Fox Squid. Can you go a little deeper? I mean, if I sign up for one of these courses, what are you going to lead me through? If we stay with hardware hacking, you could sign up to Industrial Embedded Systems hardware penetration testing, where you will also go through these six or seven steps from investigating the PCB to hopefully getting route access.

But this course has a unique approach because if you look at the IoT hardware hacking courses, you usually hack some key camera or a home router, but it's almost impossible to hack an industrial device because there is an entry barrier problem. First of all, this hardware is really expensive. You usually pay five hundred dollars or more, and it's risky because you can brick it and then you wasted five hundred dollars. To get around this, I built a custom firmware for a cheap ESP eighty two sixty six micro controller that mimics the behavior of an industrial device and introduces the student to the same challenges I faced.

Okay, so that's the hardware hacking. Have you got other courses? Yes? I have my flagship course, Practical Offensive Industrial Security Essentials, which gives students from diverse backgrounds, whether they're automation engineers, IT professionals or total newcomers, a holistic introduction to industrial cybersecurity.

Of course, there are assrom gaps that needs to be filled, but anyone with anyone with enough curiosity will get through this course or will have success with the course and then get a holistic understanding of industrial cybersecurity. So if I can take you sort of on a side trip real quick here. You know, throughout this interview, I have been surprised by you personally. I mean I had always had a stereotype in mind for people who found vulnerabilities, who hacked stuff, hardware, software or whatever.

The stereotype that I had in mind was sort of somebody with a big ego, somebody with an ego, you know, saying to themselves, I'm smarter than you are. I can find these problems. You know, you the vendor have messed it up. I always thought you needed that kind of attitude to be able to go in and tackle you know, the vendor's defenses in you know, and incorporated in the product.

I always thought that you needed attitude. But you know, what's coming across from you is something different. Can you talk about who, you know, what do you need sort of in your brain, in your personality to be successful here? Well, to make sure, you just need curiosity and persistence.

I think people with a big ego they're more successful in finding more vulnerabilities. But like I said earlier, this is more an expensive hobby for me, so I do not really have the pressure to find vulnerability after vulnerability For me, it's more well being on this scavenger hunt to go away that or find a way to operate the device it was not intended to and then really find a way in. And to be honest, I also have a whole box of scrap OT devices where I did not find a vulnerability.

So this is where we come back to the expensive hobby. I think if someone is understanding a bit of the domain these devices are operated in, and have enough curiosity and then persistence to stick to it, they can definitely find some vulnerabilities, or if not well, I can at least learn a lot about the devices, how they operate, and how they interact with other devices in the OT domain. So, Andrew, we've been talking hardware vulnerabilities. It seems relatively serious, but bring it to a practical level for me, if I'm running an industrial site and I discover, you know, a hard coded issue in one of my gateways, am I running around red alarms, ringing to patch immediately?

Or am I more focused on these systems and data flows around it that enable sort of legacy technologies to occasionally have vulnerabilities like this. How would you interpret it in the grand scheme of things in. The grand scheme of things, there's there's sort of a couple of different questions. Let's let's let's pick it apart.

What we've been talking about primarily is how to find these vulnerabilities. Once you've found a vulnerability, now you've got to ask the question, you know, a can I patch the system? Because if it's you know, a vulnerability in your safety system, well I'm sorry. The testing cost of the new version is going to be prohibitive.

It's just really hard to patch some things. Other things are easier to patch. So can I patch it? Second question is do I need urgently to patch it?

And that's sort of a different skill set. It's one skill set to find the vulnerability, it's a different skill set to say, well, how would an attacker? So it's an imagination thing, imagine how would an attacker use this against me? And you know, we talked about two scenarios for this vulnerability.

One is physically walking up and stealing the device and taking it apart and putting it back, which seems not a very credible threat because you're going to be discovered. The second scenario was, you know, someone with much more resources discovers that you've just ordered fifty of these intercepts. The shipment bribes the driver to go take a long coffee break, breaks into you know, five or ten or fifteen of these devices, inserts malware, packages them all up again again, is that a credible threat.

It's a credible threat for some very high value targets. Is it a credible threat for a small bakery? You know, probably not. So you know, first step is fined it.

Second step is figure out can I even patch it? Third step is how would a bad guy exploit this? Are there credible threats? Is there a third scenario that we haven't imagined?

So it's a question of imagination and studying what people have done in the past, and then you know, the decision part part of it is you know, how easy is this to exploit? So we're talking about devices generally, we're also talking about cloud connected devices because a lot of the devices that Marcelle has focused on that he teaches you about his industrial Internet devices, they're connected out to the cloud. So that's more Internet connected, more Internet exposed. But really what he looked at here was an O T cloud remote access device.

It's arguably the most exposed piece of technology in the OT network. It's the technology that gives Internet based users access to the OT system. So normally you would set these things on automatic update. Why what if they blue screen, Well, nobody cares if their blue screen.

It's inconvenient if they blue screen. If the bad guys get in, you know, they can work whatever they want sabotage on your OT network. So normally people pay a lot of attention to defects in there. You know, there are too too vulnerabilities in their O TI remote access.

This one we just you know, we couldn't imagine a credible attack scenario pre mere mortals. You know, it might not be that that big to worry about, but generally speaking, you know, this is the kind of device you want people like Marcel picking apart the most thoroughly because this is the device that has to be the most thoroughly protected. Before we let you go, can I ask you to sum up for our listeners, what should we take away from this? What you know, what's important to know about this stuff, and how do we use it going forward?

Okay, looking at the vulnerability, I found this was a prime example that just one part of security was completely overlooked. When you look at the device from a network perspective, you see a very fortified device, but security doesn't stop at the network in our face and also the PC be the hardware level should be taken into consideration. And in general, I think OTI security needs more curious minds that are looking under the hood. For example, if you're an engineer, you already understand the industrial processes and here I just can recommend you to level up your cybersecurity skills.

And this is exactly what I'm doing with Foxgrit. This platform exists to teach industrial security in an affordable and practical way. The flagship cause Practical Offensive and Practical Offensive Industrial Security Essentials comes with an open source lab where you not only learn about penetration testing tools, but also how you can use them on simulated industrial controllers and that way you also can understand how your real devices would behave under such conditions. For the next steps.

If you're curious, check out Fox Sprit for resources and connect with me on LinkedIn, and of course keep pushing OT security forward. So that seems suggest about do it. For your interview, Andrew with Marcel rick Zen, do you have any final thoughts that you'd like to share before we leave today. I guess so.

I mean, I had always been curious, you know, how people do this stuff. What surprised me about the interview here was that I actually followed what he did. I kind of understood it. You know, I thought it'd be harder than that, and I suppose it could be if you have to, you know, if you don't have a small amount of information to look at, if you've got to look at the entire firmware and start, I don't know, disassembling megabytes of firmware looking for vulnerabilities, that would strike me as harder.

You know, this seemed really straightforward. You know, I don't know if I don't know, if I'm curious enough about, you know, how this stuff works that I would do the work myself. But I sure wouldn't mind another two or three guests like this to walk us through how they did the hard work, so that you know, we can satisfy our curiosity and beyond my curiosity. You know, I agree with Marcel.

We need people tracking down vulnerabilities. That's you know, it's because that's the good way to persuade vendors to invest more in security, to you know, make these devices more secure. To begin with is to point out afterwards they've got problems and you know, the next time around, hopefully they will be more careful. The bad way is to wait for the bad guys to find the vulnerabilities and exploit them and take advantage of us.

So, you know, we need more of the good guys. You know, we need more more technical, curious people out there fight in the fight. So you know, thank you to Marcel. Well, thanks to Marcel for satisfying our curiosity.

And Andrews always thank you for speaking to me. It's always a pleasure. Thank you. This has been the Industrial Security Podcast from Waterfall.

Thanks to everyone out there listening.