
MSP 1337 · 2026-06-30 · 43 min
Key moments - from our scoring
Substance score
43 / 100
Five dimensions, 20 points each
Measuring cybersecurity maturity requires moving beyond tool deployments to establish sustainable processes, documentation practices, and accountability frameworks. Jim Harriman and Chris Johnson break down practical KPIs for MSPs seeking to track progress: implementation status (yes/no/maybe), defined review frequencies aligned to industry standards (typically 12-month cycles), and concise policy documentation. The conversation highlights that most organizations struggle with change management, knowledge base utilization, and maintaining consistent policy reviews - often allowing critical security practices to become stale when priorities shift. Harriman shares his experience requiring third-party accountability through SOC 2 audits and peer groups to maintain discipline, while Johnson emphasizes that mature policies are typically 2-4 pages (not lengthy documents) and that the ability to demonstrate regular reviews is more important than perfect initial implementation. This episode targets MSP operators and small business owners trying to establish measurable, scalable security programs without getting trapped by underutilized tools or unsustainable review cadences.
Focus on three measurable KPIs: whether a control is implemented (yes/no/maybe), whether a review frequency has been defined (typically 12 months per industry standards), and evidence that reviews actually occur on schedule. This avoids subjective quality debates and tracks the ability to maintain consistent security practices over time.
Most organizations allow policies to become stale, which signals that security practices have become reactive and ad hoc rather than intentional. Showing evidence of consistent policy reviews - even if no changes are made - demonstrates that your organization is actively maintaining its security posture rather than setting it and forgetting it.
Mature policies are typically 2-4 pages long and establish principles and requirements, while immature policies are lengthy documents that embed process details. Longer policies require more frequent updates and become harder to maintain consistently, leading to stale practices.
Engage external accountability mechanisms like SOC 2 audits, third-party compliance reviews, or peer group accountability communities. Jim Harriman found that external parties holding him accountable - particularly through annual SOC 2 Type 2 audits - was worth the investment to maintain consistent discipline.
Knowledge bases, PSA systems, and ticketing documentation - most teams find solutions via Google but fail to record those solutions for future use, forcing the organization to rebuild solutions repeatedly instead of leveraging institutional knowledge.
Our reviewer’s read on each dimension, with quotes from the episode.
The episode contains a handful of practical operational observations - policy length norms, review cadence standards, and integrating a security program into EOS meeting rhythms - but most of the runtime is consumed by meandering anecdote, generic platitudes about process vs. tools, and conversational throat-clearing. The useful content is real but sparse relative to 43 minutes.
the best investment to me is not a product or a service or a tool. It's a process
mature policies are very distinct at, uh, two to four pages
The episode recycles well-worn MSP conventional wisdom - tools are over-relied upon, process beats product, leadership buy-in matters, accountability requires external parties. The one mildly interesting angle is weaving a security program into an EOS cadence, but it is mentioned briefly and without real depth. Nothing is contrarian or first-principles.
we've taken an EOS approach to, To a lot of our. To a lot of our, uh, evidence collection
nobody has to do this in our, in our industry. I mean they, they, they probably should if they want to survive another decade
Jim Harriman is a genuine small-MSP operator who has run a security program for nearly a decade and completed six SOC 2 Type 2 audits - actual practitioner credibility. However, he operates at a small, unnamed scale with no verifiable market impact, and the host functions as a peer rather than an interviewer, limiting the depth that a stronger guest interrogation might unlock.
actually just completed our, like, sixth Sock 2 audit
we've been doing this for almost a decade now
There are a handful of concrete specifics - six SOC 2 audits, 20 - 25 monthly review items, 12-month review frequency as a framework norm, one-to-two page policies with data governance at five pages - but no dollar figures, no named client examples, no measurable outcomes, and no external data citations. Claims like 'if you use more than 10% of it, you're in the elite category' are stated without any evidence.
if you use more than 10% of it, you're in the elite category
there's like 20 to 25 of those things that we're going through
The host adds structural value by proposing a maturity model (initial/developing/established/optimized) and drawing useful distinctions like policy length and review cadence, but he rarely challenges Jim's assertions and the conversation drifts frequently into loose analogies and filler. The episode closes with an off-topic exchange about fiction novels, underscoring the lack of editorial discipline.
How do I bluntly put this in a nice way?
if I was really dialing in a. An accountability tool, that the initial piece of the. Have I filled in my answers
Computed from the transcript - who did the talking, and the words that came up most.
The cybersecurity industry loves to sell tools. What it rarely talks about is the hard work required to make those tools effective. In this episode, Jim Harryman joins Chris to challenge the idea that technology alone creates security. They discuss why mature organizations focus on governance, accountability, documentation, policies, review cycles, and operational discipline long before they look for the next shiny solution. If you've ever mistaken a technology purchase for progress, this conversation may be the reality check your organization needs.
Transcribed and scored by The B2B Podcast Index.
Speaker A: Foreign.
Speaker B: Welcome to MSP 1337. I'm your host, Chris Johnson. A show dedicated to cybersecurity challenges, solutions, A journey together not alone. 37. I'm joined this week by Jim Harriman of Kinetic Technology Group. Jim, welcome to the show.
Speaker A: An hour with you, man, is always, uh, a pleasure. Anything.
Speaker B: Let's try to keep it less, uh. We've had these conversations over and over and over again not on the show, about measuring progress, measuring maturity with our cybersecurity posture. And you and I have had extensive conversations with others around. How do I know when I'm actually making progress? How do I know? How do I maintain accountability? What are the metrics that I should be using to determine good versus bad? Or even a better way to say it, and this is what you and I were talking about earlier, is how do I know when I'm starting to shift away from what is good, long before it becomes bad? Because we're all going to eventually drift in and out. By the very nature of the change in the threat landscape, the change in our technology, the change in our staff. There's so many things that impact, uh, as we, as we kind of navigate this, this path. And I was just curious, you know, to kind of maybe share it with the audience of like, what are things that they should be doing to help better understand not only where they're currently at, good, bad or otherwise, you know, what's the risk posture that they've accepted or have unknowingly accepted without looking, versus, like, how do I start working on shifting it to an alignment that I am comfortable with as an organ?
Speaker A: Sure. Um, I, I think the. Yeah, I think the biggest thing that, especially just in our industry largely is that we're way too dependent upon, uh, tools and services and things of that nature. Um, but, you know, to me, the, um, the best investment to me is not a product or a service or a tool. It's a process.
Speaker B: Right.
Speaker A: Okay. So the, you know, I mean, you're going to have help from these services and tools and things of that nature, but fully relying on them and whatever, you know, golden, uh, you know, award that they, or whatever it is that they're going to, the prize at the end of the rainbow pot of gold. Right. That they, that they all offer us, um, you know, they're going solve all our problems, really. They, they don't and they, they probably never will.
Speaker B: Sure.
Speaker A: And we have to just constantly maintain and developing the process is really the thing. And continually tweaking that process within our organization, our, our Our program, our security program, whatever that looks like for your company, that is going to be an ever evolving, maturing, uh, process. And so um, that's where I view it. I think for us just really getting started and realizing that we couldn't depend on the tools necessarily to make things happen for us, that there was going to be work for us to do in this regard and that it was going to be ongoing and that we were going to have to figure out how to incorporate that into our new operational mode. Right. I mean it was a conscious decision that was made knowing that we were going to have to do it and it was going to take resources and we were going to have to, you know, have some pains along the way. Yeah, but that, that's really, for me that's kind of what, what it's all about.
Speaker B: Well, I think, to what you said, if I think about all the pieces that go into good governance because really at the end of the day, your process and procedures, your policy, process, procedures, tools, all that equate to the more dialed than they are, the more you can see staff alignment with it because they obviously have had it explained well enough to either understand and accept or to maybe not fully understand and suggest change or provide pushback to help evolve it into something that they can adopt. Right. Because you know, just doing a uh, knee jerk reaction saying from here on out we're not going to do fill in the blank oftentimes is met with resistance because no one likes change. Um, but when we put those types of ultimatums out there, we run into a problem which is if the person I'm telling the ultimatum to doesn't understand the why, they are immediately going to be defensive. You're, you're hurting something in their process that was what you hired them to do, that you've now impeded their performance. Even though there may not be any impeding happening at all.
Speaker A: I mean that's, that's very true. I mean the um, uh, the burden of uh, the things that we have to do and maintain having a security program in place, um, it does change. It changes workloads, it changes schedules, it changes all kinds of meetings, I mean report, I mean documents and meeting minutes and everything else that you, you know, do and you put into place, I mean it is, it is all a, a drain on, on your resources. And so you know, we did, the thing is for us we've been doing this for almost a decade now. And so it's. Initially it was really, really tough. Uh, we had some blowback from the team members that we had at the time and some of them aren't with us anymore, some of them still are. Um, but I think that the people that got it and as our culture started to change internally, m, they realized that look, we're really, you know, we're focused on maintaining our ability to be in business, you know, for the long term. We're not short sighted. We're actually looking towards uh, the future. And, and I think the people that, that got that stuck around and said, you know, I understand that, that this is part of my responsibility now. This is, this is, you know, uh, an, an important part. And matter of fact, it might even outweigh some of the other things that might be more client facing, uh, that we do, um, it sometimes have to have, has to outweigh that. Right. So you just, it's a balancing act for sure that we have to play. But um, I, I, it's definitely achievable. Um, but you've got to, you've got to generate the right, um, culture and excitement about what you're doing. You know, it's not just we're doing it because we, we have to or because we don't. Nobody has to do this in our, in our industry. I mean they, they, they probably should if they want to survive another decade in this industry. But um, you know, it's not like we have to, but it's, it's, we, we do need to, you know. Right. And, and we should want to. I mean it's, it's really the, become the foundation of what we do. You know, it's changed. It has completely changed as you know, over the years. Been around a long time too. So you know, uh, it is, it is the new foundation of, of, of everything. It's no longer, you know, can you put in and support a firewall and do AV and all that other stuff we've talked about before. This is the foundation now.
Speaker B: Well, you mentioned the drain and the strain and I was thinking about that as you were describing sort of some of the maturity challenges. And what comes to mind for me is when the client asks you to solve a problem or challenge, in fact I shared one with you earlier. I'm like, I got a laptop that's not working properly and I spend 20 minutes trying to kind of do a precursory troubleshooting. They're like, oh, by the way, we had a power surge that happened when this suddenly occurred. You're like, huh, that would have been a really good piece of information to have shared with me. And I think that Goes hand in hand with what you're talking about with that strain and drain. Right? Change management. Well, what do you mean I have to schedule the patch or the take the server down. I can't just do it. Or, uh. What do you mean? No, uh, one is, why do I have to document what we did to that machine yesterday to make sure all the updates were current? It's like, well, if you don't document that, then when I come in to troubleshoot whatever's not working, I have no data to inform me of what might have changed since it was in a good working state. Like, well, what did it work yesterday? It did. Okay, what time yesterday did it stop working? Well, I don't know. Right. And if I have the documentation to show me what has occurred. And I think this is one of the first big steps that MSPs start to take from a maturity standpoint. Because when you have automated patching, that data is automatically logged versus I manually patch something and I now have to dig to see what actually happened because it's not going and telling some other system or reporting on it that it happened. It's just what I've done. And I think that when you start talking to MSPs from a maturity standpoint, you ask questions around that like, well, yeah, but we're a small shop. It's like, okay, well what happens if you're saying that internally change management isn't important? But if I describe the problem that you have with a client, you're like, yeah, I hate it when that happens. Why are you causing yourself grief internally, but, but getting upset about it when it happens with your clients?
Speaker A: Well, we can, we could spend uh, a whole, you, you could probably have a, uh, year's, uh, worth of podcasts about, you know, why, why clients are upset about whatever. But you know, I, the, the, the change management is a, you know, I mean, I think that that's an area that um, that we all, that we all struggle with. And what's interesting to me, you know, having come out of, you know, corporate IT and you know, back in those days, I mean we did follow like ITIL kind of guidelines and things like that. Where we were, you know, we, we had a, um, it, um, process that was kind of like a framework. It wasn't a cyber security framework. It was just like, right, this is, this is how you do it, right, this is how you, you know, do all that kind of stuff. And it's like change management was a big part of that. And I, I think that it's also One of the things that largely we struggle with, um, as, as MSPs.
Speaker B: Yeah.
Speaker A: You know, is, is that and, and tracking that and how we do it effectively, um, and how we communicate that stuff and people knowing where to go to make sure that all that, you know, it is, that is a, that is a struggle. And it's honestly still a struggle for us. I mean, we're constantly looking for ways to improve that aspect of our business because, um, we, we're finding, you know, the, the more mature that we get, um, the more holes we find. And, and we're looking for better ways to, to do it. But sometimes the better way takes a little longer.
Speaker B: Sure.
Speaker A: You know, and all of a sudden you see efficiencies start to, to, to drop and you're like, concerned, okay, we're not as efficient. Okay, well, it's temporary because the things that take longer, usually it's just because, okay, now I have to do this, this, this and this. Right. You go back and look at how long it took you to do something before it actually took you longer. Right. I mean, and that's, that's, it's. Usually it takes longer at first. It's, it's uncomfortable as any changes. But, you know, we, we just continue to try and improve that. But that is a huge area that I think we're, we, we lack in generally.
Speaker B: Well, you remember the first time, you know, you, you were troubleshooting a problem, you googled was pretty, you actually found something that was pretty clear and concise, is like, step one, do this. And it kind of stepped you through, uh, a process. Uh, and, and it, and it did in fact fix the problem. You're like, wow, that was really slick. When we first started doing that, most of us didn't then take that information and put it in a knowledge base to say for future use. You don't have to Google this anymore. Here is the answer to the problem. And still, uh, to this day, I don't know how many times someone says, well, I just went and googled it and I was able to fix problems. Like, yeah, and you put in the ticket fixed, but you literally did a whole bunch of work that if you were to have recorded that information into the answer and the ticket of saying what you did is saving somebody else in the future feasibly you again from having to, you know, start building the wheel all over again.
Speaker A: Sure. Yeah. I mean, it's like we've got all these again, going back to tools, you know, whether we have a, you know, we have PSAs and ticketing systems, We've got documentation systems. We've got all these things and all the, uh, all, all the stuff that we pay for.
Speaker B: Sure.
Speaker A: That is highly underutilized.
Speaker B: Right. And if you use more than 10% of it, you're in the elite category. I mean.
Speaker A: Exactly. Yeah, that, that is.
Speaker B: Which is scary.
Speaker A: I mean, I've, I've learned a lot on the, uh, on the documentation side. Our team has learned a lot over the last, you know, several years about, you know, we thought we were great at documenting something until we, you know, were working with a much larger organization, um, in our space and saw how they were doing things and we're like, oh, wow. Yeah, that's. This is mind blowing how.
Speaker B: Right.
Speaker A: You know, this, this is actually using this tool the way that it was intended to be. And we're just, we weren't even scratching the surface. And so it really changed our whole mindset about how we were, uh, relaying and documenting information about internal things, about our clients, about everything else to where it really is, um, becoming a, a key, A key piece of, of, of what we do. Yeah, it's, it's just there. But I mean, that doesn't have anything to do with the, the security aspect of it, but about just maturity in general. And I, I think that that is, we, we should always be looking for ways to improve. I mean, that, that is ultimately the, the, the key statement, I think. And as we put it into security, um, if you don't think you need, or if you don't think you can improve, or if you think you've got it all figured out, um, you're wrong.
Speaker B: Right. How do I, how do I bluntly put this in a nice way? Um, no, uh, you know, it's, it's funny because, I mean, we've talked about maturity, particularly when it comes to, excuse me, how we address doing things. Like. So it's really easy to say, okay, I address the standard or the control by doing X. But when you look at it through the lens of like, I adopted this best practice today, yes, it's feasibly a best practice, but tomorrow it might not be because there's something that's still better or can be done to reduce the potential compromise. But what's interesting collectively isn't about how well you've implemented it the first time. It's about the ability to show evidence of how often. Or a cadence that says, I reviewed it to see if there was an opportunity for us to improve upon it and if that improvement was beneficial to our company because in some cases, obviously, the financial burden or some other things might not make it so. But if I looked at anybody's overall, uh, say, the dashboard of what is the, you know, your source of truth, I would go in and go, oh, well, that's interesting. It looks like here you haven't reviewed these policies in three years. That's not optimized. That's not you adhering to a best practice. That's you just saying, I did something previously and it's done. I think that's the area that I think all of us can say we are either in trouble, have been in trouble, or potentially will get into trouble because we allowed that one key metric to become stagnant. It wasn't important enough. Something else took our priority away from. I didn't check the backup logs. I didn't fill in the blank. And that tends to be where the bad things end up showing their face.
Speaker A: Sure. No, I agree. Um, I mean, having gone through, I mean, what really pushed me down this road and knowing about myself, and I've said it, uh, a thousand times, if not 10,000 times, and at least twice on this podcast over time. It's like, I know that if left, left up to Jim Harriman, that if it was solely on me to do something that, you know, it probably wouldn't get where it needed to be because I, I would, I would prioritize other things. Yep. And as, uh, you probably should. Exactly. I mean, it's just, it's just the way it is. I mean, you, you just, you do it, you, you put out the fires and you just keep going and you're, you're doing whatever you do. Um, until I, Until I engaged with a third party to come in and basically hold me accountable, um, there was, There was really. There were things that were happening, but they weren't. It wasn't a purposeful.
Speaker B: We call that ad hoc.
Speaker A: Right.
Speaker B: And that's a nice way of saying reactive.
Speaker A: Yeah, it was, it was not a purposeful endeavor.
Speaker B: Yeah.
Speaker A: Um, and so, uh, once I did that, I actually brought a third party in. We, we decided to go, uh, Soc 2 at the time and actually just completed our, like, sixth Sock 2 audit. Right. So, I mean, it's, it's, it is, it is a way for us to get accountability and, um, and, and keep us on track, especially when we do the type 2, because they are going through not just a date in time, but they're going through a span of time, uh, making sure that we are consistently doing these things right. And so that, that Is to me what, what really, uh, got. It kept us, it has kept us in line and it's. And it's. For me, it's been worth every penny to go that route. Um, and I, I think that that is, um, you know, ultimately the way to go. Now. There are other ways to do it and there are other people way more disciplined than I am. Okay. And that is, that is a, uh, good, uh, thing, uh, because, I mean, you know, with the trust mark, I mean that I love the trust mark and have gone through that process and plan to continue to, to do that as well. Um, and so I think that that is a great place and it's a community and it's there for you to, you know, the help is there, the support is there, Everything is. You know, you got a question, they have answers, you know, and if Chris doesn't have the answer, he knows somebody that's got the answer, you know, so
Speaker B: it tends to be the case. I know a lot of people, so it's been very helpful.
Speaker A: Ultimately, I do feel like the, the community aspect of it, whether you're engaging with auditors, you know, for uh, another third party coming in, whether you're engaging with it with gtia, whether it's a peer group community, you know, some level. I mean, our peer group started basically a cyber security, uh, accountability group when we were all chasing cis, and it was a group of people that were just holding us accountable years ago. I. Exactly. That's so, I mean that, that. But that kind of stuff is. I, I really feel like. And we've seen some, Some great success stories come out of, of everything that I just discussed. And I think ultimately, um, that is, uh, that that's the way to go. That is the route to go. You have to get involved with other people outside of your, you know, outside of your circle of influence and grow that circle some and um, get some, get some sound advice from, from others.
Speaker B: So I thought it would be interesting to kind of take the, you know, how do you measure this a little bit further? We know, we talked about is it stale or not stale. And I think that that uh, in and of itself is pointing to a level of maturity that's in many cases for an msp, very futuristic, right? To have a review state that's active or a review frequency that's active would imply that best practice has been successfully implemented, uh, by its very nature. And so I was just curious your thoughts. So I went to, I'm going to say, great lengths to try to navigate what KPIs would be useful without getting carried away with say, quality of work. Because if I do, quality of work becomes very subjective. That's you and I debating whether or not SMS versus, you know, uh, a duo is the right way to do mfa. And while we could probably really agree on SMS not being good, there are definitely many different ways in which to do this that aren't necessarily one better than another. So I don't want to get into like how we measure quality of delivery outside of saying the more mature you are, the better defined your evidence likely is going to be to go along with your answers. So I broke it down and I said, okay, first and foremost, I don't want to try and establish the maturity level on your ability to implement a best practice. I just want to know, did you implement? Yes, no, maybe. So we're just keeping it super simple. This would be like your first time going through it. The second thing that I'd want to look at is when you implement, did you define a review frequency for it? So, um, at a minimum, just looking at most frameworks, most frameworks, most of the controls or best practices are set at 12 months. There's nothing that I have seen that goes out further than that. And while there are some that are more frequent than that, the average sits somewhere around a 12 month time frame. Um, and again, I'm sure someone listening is like, well, no, I think you should be reviewing your DHCP logs every day. Okay, well that may be true, but that's very different from sort of the status quo. And I think it would be. I'd be very careful about having frequency across best practices, having too many that happen too often, where your resource consumption of reviewing is not sustainable or not scalable. Um, so that would be my first two pieces that I would measure. Is there any others that you would want to add for someone that's doing this, say initially of what should they look at as far as looking at good momentum or good progress of a maturity cycle?
Speaker A: Um, I would say that the, you know, look everybody, it's, it's the big, the big P word, the policy P word. Right, okay. Um, because that's the area that I feel like, you know, most people mo and I'm not just talking about msps, I'm talking about small business owners in general. Paperwork, just general. Any small business, I mean, uh, outside of an employee handbook, if they even have that right, they don't have a lot of written policies. They're going off the, you know, seat of their pants on every decision that they make. Uh, everything that they do. And so I, I think that, I think that that's the area that, that you're going to spend a lot of time in, number one. Um, and, but it, but it's also the area that, that you can, you can get through it relatively quickly. But, you know, and it doesn't have to be as complicated as you think. I mean, you and I have talked about this. You know, keep the process part out of the policy, right? Keep the, you know, make, uh, it. Make it as, you know, as straightforward and, you know, concise as you can possibly make it. So you don't have to make a lot of changes to policies. You want to review them yearly. But, you know.
Speaker B: Yeah, let's talk about that for a minute because I think mature policies and immature policies have one very distinct difference. Most immature policies are pages and pages long. The mature policies are very distinct at, uh, two to four pages. And four pages is still probably a pretty long policy. And I would, I would counter that to say that if it is four pages, I'm hoping that I'm seeing versioning controls for the policy are baked into that document, which has made it grow beyond that two to three pages. But I think the reason why I call that out is when you think about review frequency and you think about what you're trying to address with a policy, the more things you put in it, the more often you have to touch it, and the more often you have to touch it, the less likely it ever gets to a point of being comfortable with what's in it.
Speaker A: Yep. No, absolutely. I mean, I think that if you start to get, uh, this big, huge policy, you need to look at. Okay, am I trying to cram too much into a single policy? Number one, like, I mean, our, our biggest policy is like data governance, data classification and all of that, data handling and classification. I mean, it's, it's probably five pages long and, and we've, we've tried to, uh, shrink that down considerably. I mean, it is definitely the biggest one. Most of them are one or two pages. I.
Speaker B: Right.
Speaker A: But that one, that one, that one's pretty big.
Speaker B: You know, you know, acceptable use policies tend to be big too. And I think in some respects there's a good reason for that because it's not a policy. Its policies, you know, byod, um, you know, MFA might show up in there. The things that you want employees to follow directly. That has a cadence, not necessarily of change, but a cadence of employee needs to re. Sign off on or review annually. And so it's a great opportunity to have that, you know, front of mind when yeah, you're going to initial this again. Here are the eight addendums that I want you to let me know that you've read. Because your employment may be directly tied to whether or not you did or didn't follow what's in there. Yeah, um, I like that. That's a good one. So that kind of goes into that documentation or evidence commitment, which I think starts to show that second level of maturity. So one of the things that I threw out there from an implementation standpoint. Yes, no, maybe is not a maturity metric in my opinion. It is just a way to start the process for measuring your frequency of review. Because if I say I've done it, then I should have a review frequency attached to it. But then I added maturity levels of like initial, um, developing established and optimized. And going back to the review frequency, your optimized would get kicked down to established if you miss your review dates. So the idea here is that you initially create a policy, you initially implement a safeguard. And I think that is, yes, it's implemented, but it might be initial. And I think when we use the term developing, it's you're improving upon what you've done the first time and that leads into established because you're probably not able, you probably don't need to make any more changes to it. Um, and I don't know what that frequency looks like to say I went from developing to establish established to optimized. But what I was trying to get away from is saying, you know, using the NIST model of 0 to 5 or 1 to 5 of like giving you like these numerical scores that are like, oh look, we're a four, we're a three and that's great. But our goal here isn't to look at your organization through the lens of yeah, you're doing a great job, you're perfect. It's to look at it through the lens of how do I ensure that I'm not drifting away from what good should look like knowing that I'm constantly going to have to develop something new. I may have to start something that I haven't done before. And I think that's really what this is about because we're going to have creep, right? New safeguards. Uh, AI is dominating everything that we talk about today. And uh, quite honestly I don't think it's all that far fetched for it to be talked about. It's really important to the future, uh, of what we can do, especially in the it space of the things that it can allow us to do more of without having more staff. So, you know, how do you, how do you stay on top of that? And I think if you have a good to your point process in place, then this shouldn't be whether or not it's a daunting task or how difficult it's going to be. It's going to be. Why haven't you started?
Speaker A: Sure. I mean, in a lot of ways what I can say is that, I mean, you know, not to just toss another framework out there, but we've taken a, uh, we've taken an EOS approach to, To a lot of cult model, to a lot of our. To a lot of our, uh, evidence collection and everything that we, that we're doing from a, you know, just how we're operating. Yeah, our meet, our meeting cadences, all the, you know, the things that we do, the things that are our agendas that we talk about in the meetings and everything that we're doing in that. We worked all of our cyber stuff, our security program into that. Right. And so it's become just a, it's, it's just a part of our. What already existed. Yeah, we've woven secure, our security program into that. And so I think that's what actually made it easier for us to adopt it over, over that time period. You know, it's just that, you know, we. Whatever your cadence is within your organization, if you have some kind of established cadence, meeting wise or whatever, you weave the security program into that cadence. And, and it does, you know, it does feel a little more natural than, you know, just completely changing everything. And, you know, I mean, we did add one meeting within our deal. We have a security meeting, ah, specifically to discuss, um, things, you know, relevant to that, you know, that week, client wise, whether it's threat intelligence, whether whatever is coming across there. But we also use that time to hold each other accountable. On reviewing security awareness training for the company, um, security awareness training for specific clients that we're working a program with. Um, you know, I mean, so it's, it's built into what we're doing. And, and that is. I, um, think that's what's made it somewhat successful for us. You know, I mean, I'm, I'm pretty hard on us when it comes to those categories. Whether we're, you know, developing, established, optimized, you know, somebody. Our auditors might come in and look at us as like, oh man, you guys are killing it. And I'm like, uh, oh, what Are you talking about
Speaker B: we're going in the wrong direction?
Speaker A: We can't seem to do anything right, and yet you're telling me what a good job I'm doing.
Speaker B: But I think that gets into the assessment side. I think to some extent, the bar is relatively low. And so when they see somebody that's above what they. And I don't mean like they set this bar, I mean that what they've seen, based on industry knowledge of doing these assessments, when they see one that's doing a lot to improve their posture, to them, it looks good because it is better, significantly better, than what they're regularly exposed to.
Speaker A: Right? Yeah.
Speaker B: Uh, so just a little bit real quick, and we'll wrap this up. You talked about sort of the EOS way and not wanting to say one should adopt any of those programs necessarily, but I kind of looked at it through the lens of this goes back to the metrics. What are the metrics we're using to keep us accountable? Which I think that's the piece that's missing for a lot is how having accountability outside of your organization. Because if you're the owner, well, you're the only one, really, that's holding you accountable. Because the staff can choose to say something, but the reality is they tend to not have the authority to. You know, outside of maybe shaming or making you feel bad that you didn't do it, your staff are expecting of you, which is totally different problem. But like your key metrics, your quarterly priorities are rocks. You know, what is the health of the things that you're trying to accomplish? What do they look like? I mean, goes back to, if it's stale, it's stale. That's not healthy. Um, and then the last one is like being able to show, I think you kind of said this more than once, is what does our meeting candidates look like? What does the staff involvement of our own maturity process truly look like? Because when someone tells me like, oh, yeah, we've designated this to so and so in our organization, they're responsible for our Trust mark program or CIS top 18 or fill in the blank, that is not leadership buy in. That is not organizational buy in. That is a. I am treating this like a checklist and hoping that I improve my posture. And ironically, implementation of best practices does help improve your posture, but it's not sustainable and it will not last.
Speaker A: I agree 100%. If I were very standoffish about it, and I just expected it to happen, sure. I don't know that I, I all flows from the top from the top down, you've, you've gotta, you've gotta establish somebody. It doesn't mean that you can't assign roles to people within the organization to, to do things right. But it is, I mean, I, I spend more time on the security stuff probably in our company than maybe a lot of owners do. Uh, just because it's intriguing to me and I'm fascinated by it and that is one of the burdens I carry. But it is what it is. Um, the metric side of it is important and it's not like you can treat it to me the same way that we treat operational metrics. Right. Because it's a completely different thing. Um, you know, there, it's not necessarily numbers. Right.
Speaker B: Well, there might be one parallel though. I think operational maturity. If you really have mature opera, if you really have operational maturity happening in your organization, then I would argue that your ability to implement security best practices and truly get staff buy in is going to go back to. Because of your operational maturity. Your timeline for adoption on things should not be, uh, you know, scary undertaking or feel like that. It's, you know, it's going to be forever before we can do this.
Speaker A: No, you're, you're absolutely right. I just meant that it was different from, you know, you're not looking at, you know, leverage and, you know, you're not looking at, at, you know, accounts and, you know, RIM and all that other kind of stuff that we look at for efficiencies in that, in that regard. And, you know, and us not being a, you know, we're not a security operations center, so we're not looking at, you know, the number of threats or whatever that we're, that we're keeping at bay or anything on a, you know, on a dashboard. Right. And we're more concerned with, um, maintaining the process. Right, right. So we're, we're, you know, we have, on our weekly deal we've got, you know, reviewing, um, devices. Right. So, you know, did we have any, I mean, on our network, you can't even get on our network without authorization. So it's like.
Speaker B: Which would be, makes it really easy to have that review and be like, oh, wait, whoa, what. How did this happen? How did this device show up? And it's like either confession is happening or a compromise has occurred and it means that it needs to be addressed either way.
Speaker A: Exactly. Right. So it does make that quick and easy. Right, sure. There's not a lot of big question marks there, but that's a weekly thing, so we're doing that we're reviewing, you know, vulnerabilities, threat intelligence and stuff like that on a. That's more daily, actually. But in our meeting, we. We kind of go through what has come in over the.
Speaker B: Like a recap.
Speaker A: Yeah. And do that kind of stuff. Um, you know, we go over what we consider our risk scorecard. Right. Where. Where are we kinetic at at this point? Um, and then we also review connectivity for our. Our SOC partner. Right. Making sure that our networks are all connected. Partner doesn't alert us when those connections don't work. So we have to go in and do all that. So, I mean, it's kind of a manual thing, and it sucks, but that is. That is the way that it is.
Speaker B: You can't offload everything to your right with our.
Speaker A: With our current deal. So, I mean. And that. So those are weekly for us. Those are things that happen weekly. Um, the monthly list is much longer. I mean, there's like 20 to 25 of those things that we're going through, from, you know, reviewing some security awareness trainings to, you know, um, going through, uh, all the. All the sock stuff and. And incidents. I mean, we have alerts coming in if there's a. Like a real incident, but then we go a little bit deeper on the things that they made maybe didn't raise up to an incident to us. And we just kind of, you know, go through those and review and make sure, hey, maybe we do need to check this out. Because, you know, even though they didn't think it was at that level, it might have raised a flag for us for some reason.
Speaker B: So it really makes me think that if I was really dialing in a. An accountability tool, that the initial piece of the. Have I filled in my answers of having some sort of posture that's really only step one, step two, kind of going to your point, I should be able to look at, you know, Kinetic Technology Group and in your dashboard, I would see things like meeting cadence and the types of things that we're looking at across the 18 domains and what our concerns are and where we have commitment or evidence issues and where we're asking for help from other peers because we've done the due diligence of implementation, which is where most end up getting stuck. Right. They stop at, I did these things, therefore I'm done. And in fact, have not established process, they've not established true cadence within that frequency model that we're talking about.
Speaker A: Right.
Speaker B: All right, well, hey, um, last question. Uh, are you reading a book by chance that the audience would find useful.
Speaker A: I don't know that they'd find it useful though. Oh, I mean, well, I'm a fiction guy, man. I like, I like thrillers and mysteries and stuff.
Speaker B: That's perfect. I mean last week I had the. What, uh, is it Something Carl? The, uh, it's the, it's kind of like a first person, uh, DND style book on uh, dungeon. Dungeon Something Carl I think is the name of the book. It's part of a series.
Speaker A: Gotcha. So I'm reading a Michael Connolly book right now.
Speaker B: Okay.
Speaker A: And so I don't know if you're familiar with like the Lincoln Lawyer. Yeah, yeah. So that, that's the series that I'm reading right now. So.
Speaker B: Awesome. All right, well, there you have it. For those of you listening, this has been an episode of MSP 1337. Thanks and have a great week.
Other episodes covering the same guests and topics, from across The B2B Podcast Index.