Lessons Learned From Incident Response [The Industrial Security Podcast]

If you didn't listen to a single thing I said, you can listen to these three things, Collaborate, plan, and practice. Welcome listeners to the Industrial Security Podcast. My name is Nate Nelson. I'm here with Andrew Ginter, the vice president of Industrial Security at Waterfall Security Solutions, who's going to introduce the subject and guest of our show today.

Andrew, how are you? I'm very well, Thank you, Nate. Our guest today is Chris Cistrunk. He is the technical lead of the Mandient ICs or ot security consulting team, whatever you wish to call it.

Google purchased Mandian in twenty twenty two, but they're still keeping the Mandian name, so he still identifies as technical lead of Industrial Security Consulting at Mandian. And our topic. You know, they as part of their consulting practice, they do a lot of incident response. He's going to talk about lessons learned from incident response in the industrial security space.

Then, without further ado, here's your interview. Hello Chris, and welcome to the podcast. Before we get started, can I ask you to say a few words for our listeners about your background and about the good work that you're doing at Mandian. Okay, thanks Andrew.

I'm again at Mandient on the ICSOT consulting team, been doing that for over eleven years now, focus on ICSOT security consulting around the world with every type of critical infrastructure, doing incident response, strategic and technical assessments, and doing training as well. Before that, I was electrical engineer. I still am, but for a large electric utility energy was there over eleven years as a senior electrical engineer Transmission dish should skate a substation automation and distribution design.

So that's a little bit about me and again just working for Mandian, part of Google Cloud. And our topic is incidents. It's lessons from incidents. But let's talk the big picture of incidents.

I mean, Waterfall puts out a threat report annually. I'm one of the contributors. You know, we go through thousands of public incident reports looking for the needles in the haystack, the incidents where there were physical consequences, where there were shutdowns, where you know, sometimes equipment was damaged, and we rely on the public record on public disclosure. And so I've always believed that we were under reporting because I'm guessing again, I don't have that many confidential disclosures that people tell me about, but I'm guessing that there's a lot more out there that never makes it into the public eye.

You folks work behind the scene, you know, without reaching any non disclosure agreements or anything. Can you talk about the big picture. Do you see incidents, especially incidents you know, in the industrial space with physical consequences, incidents that triggered shutdowns, incidents that are not public reported. How many are there, what do they look like?

Can you talk anything about sort of what I would not see by looking at the public record. Sure, thanks for the question. You know, I think we're talking about cybersecurity incidents here, and there's many incidents that happen every day, right but life goes on. Squirrels happen right in the grid.

But for cybersecurity incidents, I do believe we're seeing an increase. I can't go into how many. We actually have a report that m Trends Many has put out every year. It's going to come out later this month and for RSA, and this is a yearly report.

We report on the different themes, the different targeted victims, the different threat groups, the TTPs. But for cyber attacks that impact say production or cause of a company to shut their operations down. I don't have any hard fast numbers to talk about, but we have seen an increase and you can look in not just our report, but also the reports of others IBM X Force, Verizon, d b I, r Drago's others. There are increasing reports of these, and a lot of it has to do with things like ransomware UH and ransomware either directly impacting the control system environment, which we have responded to in a manufacturer and a few others, but we have seen in the public, you know, news where a company might have to shut down operations due to indirect impact.

Maybe their enterprise resource planning software or manufacturing execution software was impacted, which is an indirect impact to the OT critical data flowing that was halted, which means I can't produce my orders anymore, or track shipping or logistics things like that. So we're seeing a lot of those. There's others in the electric sector that they kind of have to be reported to OE four seventeen reports. If there's a material impact, obviously they'll be filed in the U or they're supposed to be filed in the eight K or ten K with SEC, and so I think if you take all of those source and look together and see, we see there's an increase of operational impact.

Mhm. But it's I think the engineers are doing a good job of UH and the folks that run these systems are minimizing the impact in these situations, and especially for electric and water and other critical infrastructure. Manufacturing is critical. But I'd say it is probably the highest targeted outside of you know, healthcare and other other areas.

So work with me on on the numbers, just you know, for one more minute. I'm on the record in the in the the Waterfall Threat Report speculating as to what's going on with public disclosures. It's my opinion, but you know, I have limited information to back it up. It's my opinion that the new disc closure rules in the SEC and other jurisdictions around the world are in fact reducing the amount of information in the public domain rather than increasing it.

And the reason I suggest this is because it seems to me that with the new rules, every incident response team on the planet roughly you know, I overgeneralize, has a new step two in their playbook. Step two is called the lawyers, and what the lawyers say, they say say nothing, because if you disclose improperly, if you fail to disclose widely enough, you can be accused of facilitating insider trading. If you disclose too much information, you might get sued. I mean people have been sued for disclosing incorrect information about security into the public.

People buy and trade shares, and then they know, they find out the information was incorrect and they get sued. And so to me, the mandate for the lawyers is say the minimum the law requires, because if you say too much, you risk make a mistake and getting sued, and you don't want to get sued. And if you you know, if you say too little, you're going to get sued. You know, the lawyers minimize.

And if you have, you know, a material incident, you must report it. If it turns out the incident is not material to the finances of the company, you don't have to report it. And again, to minimize the risk of getting sued by reporting incorrect information, you report nothing. So my sense is that we're seeing fewer reports because of these mandatory rules, not more of than what do you see?

You know, you see this from the other side does this make any sense? Do you have a different perspective. I can say that, you know, as an incident responder working with you know, victims in critical infrastructure, but also outside. I think this is a broader question you bring is.

I can definitely confirm that we work with external counsel that a victim may have hired to bring in to handle a lot of these reporting or not reporting requirements. I can't say or confirm that the lawyers themselves external counsel under reports. I can't say that I don't know. I'm not a lawyer, nor do I play one on Facebook.

So I will just stick to say yes, we have worked with external counsel, and usually we do not say anything in public for us as the incident responder, unless the victim company or our client asked us to, because sometimes sharing information is a helpful thing, especially if it's a big breach. Sharing that lessons learned about what has happened to them with others, just like we did back in the day when we had the solar winds breach. So you know, there's two ways of thinking of that, and maybe you can pull on that throw with some other experts, but not me.

I don't know about the External Council part. Andrew, you'd referenced Waterfalls Annual Threat Report, a report which I've covered in the past for dark reading. I'm not sure i've seen this year's iteration, So maybe you could tell listeners just a bit about what the report covers and what the numbers are showing lately. Sure, the report uses a public data set.

The entire data sets in the appendix. You can click through to it if you wish we cover we count in our statistics. We count deliberate attacks, cyber attacks with physical consequences, not stole some money, physical consequences in heavy industry and critical infrastructure, the industries we serve in the public record. Okay, no confidential disclosures.

The numbers last year were seventy two attacks, I believe with I don't know some you know, a one hundred, one hundred and fifty something like that. I forget the numbers sites affected. Many of the attacks affected multiple sites. This year, you know, we're up from seventy two.

We have seventy six attacks affecting a little over a thousand sites, So there was more sites affected, but the number of attacks did not increase sharply. And this is why, Again, I speculate, why have we sort of seen a plateau. We went up from you know zero essentially give or take in you know, let's say twenty nineteen to you know, seventy two, and then you know in twenty twenty four to seventy six. Why do we seem to have a bit of a plateau?

And I'm speculating it has to do with the SEC rules. People are now legally obliged not just in you know, the United States, the Security and Exchange Commission, They're legally not just in that jurisdiction, but in other jurisdictions around the world. There's similar rules around the world. If you have an incident that is material that any reasonable investor would use as grounds to buy or share or sell or value shares, you must disclose it.

But I have the sense that we are seeing fewer disclosures because by law you're required to disclose material incidents. And again because I speculate that because the lawyers are involved, we are seeing fewer disclosures. You know, they disclose the material incidents and they squash everything else. Is the sense I have.

But you know, you asked about the numbers seventy six last year. Nation state attacks are up from you know, there were two the year before, there were six last year? You know, is this a trend? It's still small numbers, who knows?

And industrial control system capable malware malware that understands industrial protocols and that is apparently designed to manipulate industrial systems is up sharply with there were three new different kinds of malware disclosed last year or you know, found in the wild last year that had that capability versus you know, seven in the preceding fifteen years. Again, small numbers. Is it a blip? Is it a trend?

Is AI helping these people write stuff? We don't know? So these are all sort of you look at the numbers and you scratch your head and you go, I wonder this. Doesn't you know what's going on here?

So that's that's the threat report in a nutshell. There's other statistics in there, but those are sort of the headlines that leads us into the topic of the show, which is lessons learned from incidents. You folks do incident response all the time. You know, can you talk to me?

What are you seeing out there? You know, is there an incident or three that sticks in your mind. As you know, Andrew, the most important thing I have to tell you is and you know or the most recent Where would you like to start? Okay?

Sure? We have been doing OT incint response since I've been here, and I can give you a few examples. Last year in twenty twenty four, we responded to a North American manufacturing company that had their OT network for looking at a Purdue model. It's the third layer or level three of the network was directly impacted by the Akira ransomware game.

And what had happened was an unknown Internet connection was made by this third party who was running the site. They had put in their own Cisco ASA firewall, and it just so happened to be that there was two critical vulnerabilities in that firewall at the time, and the Cure ransomware game was targeting targeting those exposed firewalls. So don't necessarily think this was a targeted manufacturing OT attack. It's just ransomware gangs doing what they do, trying to make money, and so they were able to log in and get in through these vulnerabilities and deployed the ransomware on directly on the OT network, which was flat and every system but about five or six or seven were completely encrypted, including their o T DCS vendors, And there was multiple not pick picking on any one in particular, but ge A, B.

B Rockwell several others that were there, and the backups server was impacted in the back up of the backup server was impacted. They were all on the same flat network. So this was a really tough situation since the company manufacturing did not have any backups that were offline. The OT vendors, like I mentioned for I had to come on site to completely rebuild the Windows systems, the Windows servers, the engineering workstations, the hmis, all the things that were Windows and or Linux that had to completely rebuild.

Client didn't pay the ransom in other words, and so the lessons learned here. Work with your OT vendors and OEMs and even your contractors to make sure that your Windows systems and Linux systems have antivirus, make sure that you have OT backups that are segmented from the main OT network, and keep offline backups and test them on a at least a year basis. Backups will get you out of a bad day, even if it's an honest mistake at five o'clock on Friday. So this is a basic win here having good backup strategy.

And then in the last case here we recommended they eliminate this external firewall and leverage the existing itot DMZ firewall that came in from the main owner of that site. And so they had a back door essentially that this third party contractor had installed in a new Internet connection. So give away from the shadow it go back to your normal itot dmz with jump box two, factory authentication and all those things. But if you do the basics and do them well, keep good segmentation, have backups, and patch your firewalls on a regular basis, I think that will go a long way, especially in this case.

You know, I feel like I've heard of variant of that advice that Chris just gave a million times, and I don't work in industrial security, so you folks must hear it all the time, or it must just be such basic knowledge that you don't even think about it. So are there really industrial sites out there that still need to hear that you shouldn't be making an Internet connection from your critical systems? Short answers, Yes, you know, people who do security assessments, you know, not just incident response that we're talking here, but security assessments come back and say they regularly find connections out to the IT network and occasionally straight out to the Internet.

The connections to the IT network tend to have been deployed by the the you know, the engineering team or the IT team to make their lives easier. You know, people with gray hair enough gray hair like me, they talk about, you know, how the systems used to be air gapped. This was a very long time ago. We're talking thirty forty years.

The systems used to be air gaped, and you know, people with gray hair like me might assume that's still the case. It's not. You know, everybody who does audits reports these connections. The really disturbing stuff, you know, yes, it's it's disturbing that there are connections to the IT network that are poorly secured, that, you know, But the really disturbing stuff is the vendors going in and if you do an audit on a site time and time again, I hear people saying, yeah, they discovered three different Internet connections.

The vendors stuck in there, and you're going, well, wouldn't you notice if there was a new Internet connection. I mean, no internet service provider gives you a connection for free. You got to run wires, you got to You got to pay for this thing. Every month.

It's showing up on your bill. No, it's not. You know, there's a lot of wires being run while stuff is being deployed. You don't notice a new wire.

And the vendors pay month after month for the Internet connection. Doesn't even show up on the bill of the owner and operator. Because the vendors are providing a remote management or vote maintenance service and they want to minimize their costs, they want to maximize their convenience in terms of getting into the site, so they deploy rogue DSL routers, they deploy rogue firewalls to the site's Internet connection. They deploy they might deploy rogue cellular access points where you know, there's not even wires to run.

It's just a box sitting there that has a label on it saying, you know important, do not remove. And of course God makes it invisible to everybody who's looking at it says, oh what, don't touch that one. Yes, it's very common. The advice I try to give people is when you do like a risk assessment or a walk through or an audit of your site look for these rogue connections.

Unfortunately, you're probably going to find one or two of these contractual penalties with the vendor help, but they're no guarantee. You said that the victim decided not to pay the ransom. You know, do you see victims ever paying the ransom to recover an OT network, to recover the HMI, to recover worse than that, the PLCs and the safety systems. You know, why would does anyone trust a criminal to take the tool the criminal provides and light it on their safety system and you know, restore it because they trust the criminal.

Does anyone trust the criminal that far as that does that happen? We have seen traditional I T systems where they pay the ransom and get access back to these systems and some that are OT adjacent such as colonial pipeline and hospitals. Right, we do know that those systems and both of those incidents or well those examples the are colonnal pipeline are name a hospital breach. In some cases there were OTS type data OT type critical information that was impacted and so they of course paid and due to the fact that there's if someone didn't trust these ransomware gangs to do what they say they were going to do, they would those ransomware games would be out of business.

So if you pay them, they don't, they decryptor doesn't work, then it's no good in their ransomware gain job is over with at least for this thing. But for OT in this instance I talked about, they did not pay. I don't have enough data to know if OT direct on the control systems themselves, the Windows, HMI s, engineering workstations, DCS servers, SKATAS servers, if they've paid in those instances. But I'd say it's it's plausible, And it really comes down to the business decision of the plant owner, the CEO of the cut based on what the engineers at the lower level, Hey, can we get do we have backups?

Can we get the vendors to come in? And so I really don't have enough information to say about do OT asset owners like a plant, like a site, like a you know that directly operates h O T, if they trust these ransomware games or not. It may just roll up higher than them about that. Uh.

Also, there's usually an advice from a ransomware negotiator that's a third party that specializes in negotiating with ransom so they may advise to pay or not to pay, or to get a reduced pain as well. So it's very, very complicated. I know I didn't answer your question directly, but in the instances we've seen, we have seen them not pay and we have seen them pay and what's OT or not. So so coming back to our theme here, you know, lessons from incidents.

You know, the lesson from this incident is, you know, get rid of that firewall, use the existing infrastructure, and you know, look at your backups. I mean, if the backups are encrypted, it's it's all over. That makes perfect sense. What else have you got?

What else you know have you been running into lately? That that's that's interesting and noteworthy. Yeah, I mean it's basically boths down to either ransomware or commodity malware. So I've got I've got another example about ransomware.

Electric utility was impacted by ransomware on the IT side, but they had a good incident response plan and they severed the IT and OT connections there and even down to the power plant type networks, and so that was really amazing, and so that's a good story, and we were able to actually verify the IT team were able to verify that the threat actor, the ransomware game Quantum, were scanning the OT DMZ, but they didn't get a chance to get be let through. We did do a full assessment of their DMZ and looked at the domain control of the firewalls and even the domain controller and the firewalls and others down inside the OT networks, and we found that that they were actually pretty lucky because they had some weakness in some of the firewalls.

So eventually do if they had enough time, if the ransomware actor had persisted long enough, they could have gotten through that firewall, made it to the DMZ And the active directory had some weaknesses as well, and they could have gotten domain access domain admin and pivoted to the OT network. But the great thing is highlight again that they had a good instant response plan. They were able to segment quickly and then they were able to have their OT vendor in this case it was Emerson Ovation were able to go on site and they were able to not only take the IOCs that we had from the ransomware, but they were able to sweep because it was their contract to do so, to look in the POC logs, the OT workstations, endpoint protection and all that stuff.

So we all worked in concert together in this incident. And then actually they hardened the firewalls, hardened the domain controllers, hardened the workstation configurations before doing anything else. They did all of that. When the ransomware was eradicated and hardened, then they said, okay, now we'll reconnect everything back the way it was.

So that was a really great lesson learned with another ransomware, and it wasn't a direct impact to OT, but this is a great opportunity to leverage that incident response plan that they had. So nate the concept of separating it from OT networks in an emergency. This is the concept that I see increasingly. I mean, I think we've reported on the show here a few times that this is what the TSAD demands of pipeline operators, petrochemical pipeline operators ever since colonial the ability to separate the networks in an emergency so that you can keep the pipeline running wild it's being cleaned up.

I haven't I'm told I haven't actually read the translated the Danish law, but apparently in Denmark there's a recent law in the last twelve months saying exactly the same thing. You know, the TSA applies to pipelines and rails, in Denmarket applies to critical infrastructure, and it says, in an emergency, you have to be able to separate. They call it islanding the industrial control network, and as Chris points out, it can be effective, but it relies on really rapid intrusion detection and rapid response because as Chris said, you know, the bad guys had been testing the OT firewall.

If they had had just a little bit longer, they could have gone through. So, you know it, even though it it's you know, imperfect, it is a measure that I'm seeing increasingly required of critical infrastructure operators and you know, recommended to non critical operators as a as a measure that helps, especially on the incident response side. Have you got another example for us? I mean three is the magic number you've given us, you know, sort of two sets of insights.

What what else have you got for us? And in terms of lessons. Learned, yeah, there's lessons learned I can just name a few other lessons learned from it, just about any attack, right, making sure that you have these at least Windows systems with antivise. In a lot of cases the OT network didn't have anti just basic antivirse, not necessarily an agent or EDR solutions.

If you have those, great. If you don't have any antivirse, that's you need to get at least corner version of Windows or operating system and with antivirus, having good backups, having good that good vendor support. Now, this last incident we responded to was using a living off the land attack, So we responded to electric utility in Ukraine in twenty twenty two and it was a distribution utility that the attacker came in through the IT network deployed their typical wiper malware.

This was the group APT forty four or Sandworm team, which has been targeting critical infrastructure around the world for quite a while, and they were able to pivot to the Skater system and used the feature of the Skater system to trip breakers using a tool that was built in the skate of system itself, So just giving it a list of breakers to trip and calling that executable in the system to trip those breakers on. Behalf of the attackers is the too long did read of that incident?

And so the lesson learned here is targeted attacks. They're going to not use malware. They're going to use the features or the inherent vulnerabilities in an OT network stealing valid credentials like an operator workstation or an engineering administrator account. And if you can even spearfish an engineer or administrator network AM on the IT network and you don't have good segmentation of roles from IT to O T, then that's that attacker is going to use every one of those tools to evade detection, to bypass your normal detections because they're they're coming in as a valid user.

So the lessons learned there is to limit the amount of administrative access. And this is you know, role based authentication, right and does the person that got promoted and now is in a different department, does he still need ADMIN rights? You know, does this person having enough control for just their area only or their responsibilities too wide? And now we say, okay, we need to reduce the amount of admin.

Do we require two factor authentication or even hardware two factor authentication to really reduce the attacker down to an insider threat, because remotely that's very hard to do to to to to bypass hardware token based two factor authentication. And so there's some there's some living off the land guides out there. The US government DEWE has put out a threat hunting guy for living off the land attacks after the volt typhoon announcements last year. But I would also go as step above and beyond that, learning good ways to detect anomalous logins, even from your own folks.

If it's out of a normal time, out of a normal location, you're really gonna have to have some tuning on some of these detections. And the only way to really test those is with a red team that's trying to be quiet and not trigger your detections. And that is some of the more advanced asset owners and end users they're using leveraging red teams, hiring red teams like what we do at Mandiant to come in and see if we can do living off the land attacks to bypass their detections. Since Chris mentioned it but moved on before we could actually define it, let me just for listeners, living off the land is the process by which an attacker, rather than using their own malicious tooling, would make use of legitimate software or functionality of the system they're attacking to perform malicious actions on it.

It's been a growing trend in recent years, I believe, because it's so effective in that it is so difficult to detect. You know, you could spot malware with certain kinds of tools, but can you spot somebody doing things with legitimate aspects of Windows or whatever you might be using. It sounds though like Chris is talking about detecting living off the land tactics, which seems difficult to Andrew. That's right, I mean, I you know, have been have been following Living off the land to a degree.

You know the what's the right where the short The short answer is you run an anti virus scan on a machine that's been compromised by a living off the land attack and it comes up squeaky clean, there's nothing nasty on the machine. And what I heard Chris say is that this is because the bad guys are using normal mechanisms, especially remote access, to log into these systems as if they were normal users and use the tools on the machine to attack the network or you know, to wait for a period of time until it's opportune, and then attack the network.

And you know what I heard him say is that because it's a lot of a lot of this is remote access, he says, you can detect this by focusing hard on your remote access system. A. You can prevent it by throwing in some hardware based two factor. You know, that will solve a lot of the problem, not necessarily all of it.

There's always vulnerabilities and zero days, but two factor helps enormously. It's way better than not having two factor. But that's preventive. On the detective side, he said, pay attention to your remote access If normal users are logging in at strange times, that should raise a red flag.

If normal users are logging in from strange places, the IP address coming in is from China, Well is Fred in China this week? No he's not. So you know what I heard was one way to to you know, help detect living off the land techniques is to pay close attention in your intrusion detection system to the intelligence that you're getting about remote users logging in. So one more question, you know, we talked to a lot of folks on the on the podcast, A lot of them are our vendors with technology that we talk about, and you know, sort of a consistent theme for most of these vendors most of these technologies is operational benefits.

Yes, the technology whatever it is, is helping with cybersecurity, but often this stuff helps with just general operations in sometimes surprising ways. We've been talking about incidents and lessons, and you know, a lot of what you do is incident response. Are there operational benefits that you run into that people say, you know, I did what you told me and everything is working smooth more smoothly than not just on the security side. Can you talk you do you have anything like that for us?

Oh? Absolutely. And one of the things that I always teut is looking at your network, looking at the packet captures in the network can aid in not just cybersecurity benefits, but these operational benefits. You can see things like switch failures happening, TCP retransmissions happening.

All this traffic may be went like your Windows hmis maybe trying to reach out to Windows Update, but it's blocked by the it OT firewall or anything else. It may not have a connection at all as trying to reach out all this unnecessary traffic or indications of proper configurations, misconfigurations and things like that. So just looking at your network with some of these tools that are out there, free tools, pay tools, ICs specific tools, or IT specific tools. It doesn't matter if you look at If you take any one of those, say just even wireshark and look in your OT network, you can get an idea on what traffic doesn't need to be there that you can eliminate it, make your improvements to the system, and now I have better visibility if there is an incident.

I can easier to detect if there's a cyber incident or if something's operationally wrong like a switch failure or something, and so there's a really great benefit there. Also helps improve reliability. We've done an assessment at a company that had a conveyor belt that they were having problems with. If the conveyor belt wasn't timed exactly right, if they had too much latency on the network, the conveyor belt would stop and all the things on the conveyor belt would just go everywhere, and it was a disaster.

So we just looked in the network, Oh, you got all these TCP re transmissions is and you look at the map in the in the software and say, oh, it's coming from these two IP addresses. Oh, we know what those equipment is. And we had the network person coming, Oh, I've been trying to figure this out for weeks and just looking looking, just using a duel like that. Uh, they were able to find and fix the problem and they fixed their latency issue because of that.

So going back to you know, incident response, having these having an incant response plan. A lot of ot already has this because of disasters, fire, floods, storms, it spills, air releases, safety issues, and that's all part of their normal disaster recovery or incident response plans. If you already have one of those, you've already done ninety percent of the work to have a cyber incident response plan. You just now have a cyber incident response added to that.

So that's the whole premise behind things like ICs or ICs incident Command system for industrial control systems and having that say, chief person in charge of cybersecurity for a site for a paper mill, for a power plant, for manufacturing facility, even though that's not your day to day job all the time. If you say the lead that and you say you have multiple plants, have multiple leads for those plants. That still the every decision will go through the plant manager, the general manager of the plant or site.

But at least you have someone that is in charge of cybersecurity, just like you have a designated firewatch person or anything else. So if you take safety culture that we've known about for over one hundred years and mold your cybersecurity culture to fit with that, things will make a lot more sense. We've already we've already invented this. We're not reinventing the wheel here now.

We're just including another paradigm of cyber security, network security, and endpoint security into these things that we have been doing. There's a fire, okay, let's put out incident response. So and if you have a plan, that's great. If you don't have a plan and you run around, that's not good.

So if you have a plan, you can at least prepare for it, and sometimes that's the win. Being prepared is better than not being prepared. Well, this has been tremendous. Thank you Chris for joining us.

Before we let you go, can you sum up for our listeners what you know? What are the key points we should take away here. Sure if you didn't listen to a single thing I said, and you can listen to these three things. Collaborate, plan and practice.

So collaborate. You get your IT teams, talking to your OT teams, talking to your manufacturers, and identify the right roles within each of those and make sure you get together and talk about these things. Have some donuts and coffee. So collaborating.

Knowing who is in charge of what is half the battle. Knowing who to call when plan, having an incent response plan, or including OT security in your incident response plan and or engineering procedures. That's gonna help when an incident impacts OT directly or indirectly. And then practice even can start with a simple question, Hey, what would we do in an incident or even going to having a tabletop exercise collecting logs from a PLC of security logs?

How long does that take? How many devices do we have? If the general manager says how long is this going to take to pull all the logs from all of our systems, you won't be able to say I don't know. You'll have to say no, this will take two hours and forty five minutes.

Because we've tested it so collaborate plan and practice. If you need help with OT security or IT security, we do that. At Mandian. We offer incidant response retainer that covers IT and OT.

There's no separate retainer. If you have an IT incident and don't need OT, not a problem. If you have OT only incident, not a problem. If it's IT cloud and OT all at the same time, we can help you around the world twenty four to seven.

And lastly, if you want to learn more about this, you can reach out to me Chris's drunk at Google dot com, my email LinkedIn social media blue Sky, and we check out some of our blogs on the Google Cloud or Mandian security blog. We have great content out there that is actual actionable not marketing fluff. It is actual actionable reports. The next M Trends Report is coming out next week r s a time prime end of April, so that's a free report.

It's a great report to look at and gain some insights on what we've been responding to over the last year. And with that, I appreciate it. Collaborate plan practice, Andrew. That just about concludes your interview with Chris Sistroke.

Do you have any final words to add on to his to take us out with today. I mean Chris, Chris summed up collaborate plan practice. You know, what I heard, especially earlier in the interview, was, you know, do the basics, guys. They some people call it basic hygien is.

Basically do on an OT network as much as you can of what you would do on an IT network. Put a little anti virus in on the systems that tolerated, you know, get some backups, get some offsite backups so that if the bad guys get in they can't encrypt the off site backups. There's somewhere else, you know, look for the vendors leaving behind internet connections, get rid of them, you know. And you know, on the in terms of living off the land, you know, he gave some very concrete advice that I've never heard before, saying, look, these people are coming in as users.

Get two factor. Two factor will do a lot to breaking up living off the land attacks. And in your intrusion detection systems, look hard at what your remote users are doing, and if it seems at all unusual, that's the clue that you're being a tact And you know, in terms of his collaborate, plan and practice. I really liked his his his uh the fire warden analogy, saying, look, you know, if you have an industrial site that is flammable, okay, your firewarden does not just sit on her hands until the place bursts into flames.

Okay. The fire warden is someone who's active in terms of actively h you know, looking at managing, raising the alarm when they see dangerous practices in this flammable plant. It's not you know, it's not just a reactive position. It's also a proactive position.

And we need that for cybersecurity because basically every site is, you know, in a sense, a flammable cybersecurity situation. So it's not just that they sit on their hands until there's an incident and then they're in charge. They are actively looking around, just like a fire warden wouldn't say, you know, we shouldn't be doing this. You know, my job is not just to put the fire out when it occurs, or coordinate putting the fire out.

My job is to help prevent these things. And so I love that analogy. That makes so much sense. Anyhow, That's what I took from the episode.

Sure well, thank you Chris for speaking with us and Andrews always thank you for speaking of it. It's always a great pleasure. Thank you. This has been the Industrial Security Podcast from Waterfall.

Thanks to everyone out there listen.