The profession of Penetration Testing with John Strand

Speaker A: Yahoo announced a security breach affecting upwards of 1 billion user accounts. Cyberattack leaves 145 million eBay users at risk. Target announced up to 110 million customers may have had their identity and financial information compromised. Cybersecurity breach at Equifax could affect 143 million American consumers. And now your host, Nexus IT Group.

Speaker B: Welcome back to another episode of Hacked. Today we are on the phone with the owner of the penetration testing and security architecture company, Black Hills Information Security, John Strand. How you doing, John?

Speaker A: Doing very well and yourself?

Speaker B: Doing fantastic. Appreciate you taking some time here. Excited to talk about some pen testing and how to become a pen tester. A couple things that we're going to cover. I'm going to cover the business of penetration testing, the five year plan to becoming a pen tester and of course we're going to talk about Black Hill Hills new information security card game, back doors and breaches. So really excited to talk about that, but, uh, let's get things kicked off here. With quick background on you, John. Uh, what's your path been in technology that's led you to owning Black Hills Information Security?

Speaker A: You know, whenever it comes to security, especially pen testing, I try to explain to people that there's no straight paths to get here and there's none when you arrive. Um, it's, it's really amazing to me how all the different diverse groups of uh, like, paths to actually become a pen tester. Mine actually started at Accenture Consulting. Um, I was part of the largest class action lawsuit, um, in history, Cobell vs Department of Interior, had um, to do with misappropriation of Native American funds. And, uh, worked that for quite a while. A lot of fun things happened there. And then I got picked up by Northrop Grumman, worked at Northrop Grumman doing um, offensive testing for classified programs and got out and started teaching with the Sands Institute. And then, um, uh, you know, startup Black Hills Information Security. And I would say that by and large, how I actually got into pen testing is there was a lot of auditing work, there was a lot of compliance work, especially from the fallout of the court case, uh, with Cobell. And it was funny because if you said that you should do the right thing in computer security back then in early 2000s, um, people always asked you to prove it. And I was one of the people that could actually prove and demonstrate attacks and demonstrate risks. And it just kind of became part of my resume over time.

Speaker B: Yeah, that's fantastic. So when did you feel that the general technology landscape was ready for penetration testing experts.

Speaker A: Um, it's funny, I actually didn't think that I would have believed it would have been after 2008. Um, I remember I was sitting in my living room, my house. I just moved up South Dakota. I was teaching for Sands, doing offensive training for the SANS Institute. And, um, right around then, it was early 2008, there was a whole bunch of, uh, articles that were talking about next generation security products and how they got four out of four stars or five out of five stars, and how they stopped all these different attacks. And I remember being genuinely afraid because I was thinking that, you know, this whole offensive thing wasn't going to take off. And then as 2008 progressed, you had 08067, you had Conficker, you had Dan Kaminsky. So I really believe probably late 2008, is whenever I finally kind of said, hey, I think I could make a living for a long time doing penetration testing.

Speaker B: Right? So how did you personally get into, uh, or gain skills needed to simulate acts and at the end of the day, be able to stop, uh, them from happening?

Speaker A: Um, I would say when I first started doing consulting, it was kind of by accident. I was doing a lot of training with Sans, and Stephen Northcutt pulled me aside and he said, look, if I ever find out that you're just teaching for the SANS Institute, I will fire you and you will never teach for us again. And I really felt a strong desire after that to feed my family. So I started doing consulting. And, uh, it was amazing to me, stepping out into the world, not just with the contracts I had worked on with Accenture or Northrop Grumman, but whenever I started working with other defense contractors as a consultant, Whenever I started working with, uh, various banking entities as part of an instructor, they hired me on to do pen testing and started looking under the hood of a lot of organizations and realizing, oh, my God, this is a nightmare. And then moving to another firm and finding the exact same types of things. Missing patches, weak passwords, users with very poor, uh, security awareness training. It didn't. I always thought that it would be hard to break into organizations, but it turned out it was really, really easy. And it just kept on snowballing. And that's really why we started the webcast. That's why we started doing as much as we could for community outreach, is we desperately want to make our lives difficult. I say the only job of penetration tester is to make your life hard. That is, your whole purpose in life is to make your organizations better at Security make your life more difficult for the next round or the next penetration tester that comes through. Um, so it really was something that kind of was a snowball effect once, um, I stepped out and started interacting with multiple companies starting in 2008, and then rolling through up until even now today, just realizing how bad security is on a weekly basis from our assessment. And really, the whole goal is to try to make people better.

Speaker B: Absolutely. So in today's landscape, how would you define penetration testing? What's its value? And, uh, generally speaking, are organizations using it effectively?

Speaker A: Well, that's one of the problems I see in penetration testing right now, because anybody that you're going to talk to about pen testing is going to say, well, here's the problem with it, but it's my turn. So here's the problem is everybody's trying to define things in specific terms, right? Like a red team is this, um, purple team is that a penetration test is these things, and it can't. They really try to define really narrow definitions for what these things are. And I think that that's a problem. One of the things I've learned about running a company is a red team. A penetration test, a vulnerability assessment, is always part of negotiation. Um, you have some companies that want a red team for an app. They want you to come in and emulate what it is an adversary would do to break into that app. You have some companies that want a penetration test, and, uh, they really want a red team. But I would boil down a red team. What are the core components of what a penetration test should be? And if you take any of those away, it's no longer that thing. And I would say you would have vulnerability assessments, right? You're going to run a scanner. As soon as you cross into the line of actually exploiting and confirming those vulnerabilities, you're no longer in a vulnerability assessment because exploitation is on the table. Penetration is on the table. You're in a penetration test. And we can further break down and define, like, what is a red team, a purple team? Sometimes we call other, uh, types of assessments black teams. But basically, a penetration test involves exploitation to actually demonstrate risk.

Speaker B: So organizations that are seeking the need for a firm like Black Hills to come in and conduct penetration testing, how often are they appropriately educated and know what they are actually buying versus needing to and get up to speed on what is best for their organization?

Speaker A: And this is actually a big question. And BHIS is fundamentally different than a lot of penetration testing firms, um, just from the fact that a lot of Our customers come from sans, uh, and a lot of our customers come from us doing webcasts constantly online. Our customers are actually a much better, more aware group of customers than I think most firms get. Um, I know Dave Kennedy with his podcast and the different webcasts and thing that he, things that he does with ions, he gets very, very, very sharp customers at Trusted sac. Same thing within Guardians. But whenever you go off of these firms that do a lot of community outreach and education, uh, Secure Ideas does amazing community outreach and education rendition does, there's a whole bunch of firms that are just great. If you go into the firms that actually dial for dollars, right? The ones that are actually, they have sales teams that are calling, well, we're going to try to crack the nut of medical institutions in Illinois. That's not the way that we do things. Once you get off of that, the general security awareness falls very, very, very quickly. I would say you would have probably 10% or less of the organizations are very aware as far as what a pen test is and why they need it beyond compliance. And the vast majority of firms out there that maybe don't have as much of a community outreach as some of the firms that I just listed, they deal with actually working with those organizations. And that makes for a much more difficult penetration test because the expectation and the awareness is so poor before you actually go into the engagement.

Speaker B: That makes a lot of sense. Now in what situation would it be advised to have an in house, uh, penetration testing team or a red team versus hiring a firm to come in and kind of ah, business to business, um, level.

Speaker A: I would say you should always, at least once a year have that outside audit. Right now how intensive you need that audit depends on the organization specifically what the goals and objectives are. But almost every single company out there should have that audit. Mind you, that's completely self serving, right? I do this for a living. It's like you know, a dentist saying you should always work with a dentist, you shouldn't practice dentistry at home. Um, that's our livelihood. And that's a biased answer. And I hope that your listeners understand that because I understand it because I think it's important. But I also understand that I'm perfectly biased in that. However, I would say that almost any larger organization, let's say around 5,000 people and more, they should have somebody that has these skills, um, it could be the security representative for that company, um, but they should have at least some basic understanding of how to approach security assessments and they can learn that online. There's Plenty of resources online that exist, from BHIS webcasts to uh, learn security online to getting training like the OSCP or GPEN from sans. There's plenty of opportunities for people to get educated on how to do cyber offensive operations. It just requires them to sit down and put the time into it. But I would say probably, let's say 5,000. You really should have at least one person that is trained and up to date on what cyber offensive operations look like.

Speaker B: Sure. Okay. Yeah, that's definitely interesting and I definitely appreciate. I know the listeners will as well, knowing, uh, it is a biased opinion, but still there's, there's truth there and it's all in goodwill, so. But, uh, I guess one thing I'm really curious about is the skills that you brought to the table and other folks that are in this space that have been very successful. What were some of those technical and or soft skills that led you to be successful as a penetration tester?

Speaker A: You know, this is. This also kind of leads into the question of how do firms like, like ours hire. And I, I honestly don't care what degrees you have. I know that that's harsh. I mean, I have a master's degree in information security, so it's weird for me to say that I honestly don't care what certifications you have. And I know there's groups like OSCP and the SANS Institute that may not agree with me on that. Um, I say that these things become kind of a body of knowledge. They don't define the candidate and they don't define your skill set. If you come to me and say, oh, I would like to get into pen testing, but I never took a class, that's garbage. Because the best pen testers in the world are the people that have the ability to actually dig on their own. They're the people that go to conferences and uh, not expensive conference, but even regular conferences and they dig in and they do capture the flag and they do online capture the flags. And they're constantly trying to learn as much as they can about their craft, regardless of whether or not there is a training class around it. Now, training classes and certifications tell me if there's a basic understanding of some core principles and that helps with some initial questions. But honestly the skill set that you need to have is just that ability to keep digging, to never give up. A lot of pen testers, you know, they won't say, well, there's no criticals or highs in this nessa scan. I guess this company secure, uh, that's not the way that they look at things, you know, they're looking at those scan results, their port scan results, their shodan results, what they found whenever they did recon, using tools like Recon NG to build out that picture of a target organization in order to try to attack them. They aren't relying on something telling them, yeah, here's the vulnerabilities that you can go after. So that ability to dig and keep digging is huge. That's why things like Net wars are so important. If somebody comes to me and they say, I am a Net wars champion at Sans Orlando, um, well, that's, that's just monstrous, right? That means that they came out on top of like 275 people. And that just shows tremendous aptitude to keep digging. The other thing is communication. Can you speak well? Right? If you're like, yo, yo, yo, we totally pwned you that I don't care how you are, if you cannot communicate effectively about what risk is, then you're going to miss the boat. Your goal of communication should always be about risk reduction to the customer. It should not be, look at me, I'm elite hacksaw. I have no time and interest for that. And then also write, um, a lot of the pen testers at BHIS write the equivalent of a novel a week. Um, you know, our reports, there's lots of screenshots in that. And that's not just Nessa's scan results. They're actually writing in their methodology what they did. And many of their reports will be 100 to 200 pages per week. And once again, that's not just regurgitating scan results. That's not copying and pasting. That's literally typing out the narrative of what they found for recon, how they got access initially, how they pivoted, because we need to communicate that effectively to our customers. That's what I look for, is the ability to dig, to not give up, and to solve problems on their own on the technical side. And then on the communication side, the ability to speak well and the ability to write even better.

Speaker B: Now, from a more technical perspective, you know, say you use me as an example. I've got general, uh, understanding of what needs to happen, but I don't possess any real technical skills outside of, you know, these capture the flag and just digging, kind of learning their way around networks and environments, um, by being there and trying things, um, are there core skills or core indicators that someone is going to have a good chance of being successful as a penetration tester?

Speaker A: From a technical perspective, yeah, we actually did this webcast. We talked about the five year plan, right? And this was geared towards college students. Um, so if somebody's out of college, you can accelerate it as much as you want, but you have to have firm understanding of operating systems. And I would include active directory with that. Right. You need to have solid, solid understanding of networking. Um, it was really horrifying to me. Um, Jeff McJunkin is a sans instructor with, uh, me, and he was up on stage and like one of the teams at the Hacker Jeopardy, this Hacker Jeopardy question came out and they said, what port is Telnet on? And I'm watching Jeff's face as people are trying to answer this question. They're like, 20, 22, uh, 21. And Jeff has this look of absolute horror and I have this look of absolute horror. It's like, my gosh, this is basic. This is absolutely primal. For everything that we know, you need to understand that basics of networking. Then the final thing is a coding language. Um, a lot of people are gravitating towards Go or Python or Ruby. Find whatever language speaks to you and start learning that language. So if you have the basic understanding of operating systems, a good understanding of networking, and then you have a coding background, then the world's basically your oyster. I see far too many people that are like, well, I'm going to get into security and I'm going to learn assembly for exploit dev. Yeah, don't do that. Because if you don't have the basics of understanding how to write an exploit in Python, big, um, Endian versus little endian and how memory address spacing actually works. What is a heap? What is a stack? If you don't have the framework to communicate effectively with a language like Python or Ruby or whatever, then none of that makes sense. Um, from an operating system perspective, you won't understand the service that you're exploiting if you don't understand what is RPC on a Windows computer system. Um, you need to have a basic understanding of those operating systems, network protocols and learn a language. I don't care what language. You show up and you know, Pearl, I might make a joke about how you're old, but then I'll reminisce about Perl. But, uh, you need to have at least those core fundamental understandings. Because here's the dirty secret about pen testing. Very rarely do you pen test something that you know inside out and backwards almost consistently. You're always encountering something new. I have heard of firms that say, well, we don't pen test X technology because we don't have that skill set in it. If every firm had that view, then nothing would get tested. So I really need those intrepid explorers that can actually go forward and they can research and they can dig deep. But you can't do that if you don't understand an operating system. You don't understand network stack and you can't code.

Speaker B: Right. Okay, now, what are some of the techniques that you use to identify people that have those skills and that are hirable?

Speaker A: One of the things that we say is we're always hiring and we're never hiring. Um, I mean, years ago we put out a general tweet saying, hey, we need a pen tester. And we hired B.B. king and a couple of other people and they're great. But the biggest thing for us is keeping, like, a dynamic in the firm that we get along. So with a lot of the testers that we bring on, we'll actually work with them as subcontractors or they're going to be at conferences. We're going to hang out for them, hang out with them rather. Um, we're going to see them perform at capture the flag events. We're going to see what they give back to the community. You know, what are they giving to the community as far as webcasts and blogs and capture the flags and things of that nature. And then we basically ask them to join our firm. Another weird thing is I don't like hiring unhappy people. Um, if somebody is talking to me and they're like, I hate my job and my life sucks, and I just, oh, I want to come work for you, that's probably going to nix it for me right there. I really like hiring people. Let's say that they're very happy where they're at, and it would take a lot to get them to move to a different place. I want to hire happy, happy people. Um, and if you hear a chicken in the background, that's because, like I said, I'm out in my barn on my, uh, farm right now. So I apologize. But, um, but no, we're always trying to get people with the good attitudes and we're really trying to find those people that are constantly giving back to the community. Community. And sometimes it's years before we actually bring them in.

Speaker B: Sure. Okay. Do you have any stories or experiences of finding someone in an unusual place?

Speaker A: Finding someone in an unusual place? Um, let me think. A lot of the people that we've hired, we've actually pulled, uh, from sans conferences. They were facilitators, they were TAs, they hung out all the time. And that's kind of boring, right? Um, I will say Egypt. When we hired, uh, James Lee, I had talked to him. He had been in a whole bunch of CCDC competitions. And if you ever had an opportunity to sit around Egypt and just watch him, right, he will sit down with these kids in college. I could say kids, right? But they're kids, right? They're, they're in college, they're getting their degree. And, uh, you would see him walk through the most basic mundane things with these, these kids, uh, these college students. And it was always with the utmost amount of respect, it was always with joy. Even though he was talking about quote, unquote, noob things. But he was always right there working with him. And I saw him do this at like 3 or 4 CCDCs. You know, this is a guy that's one of the big developers for Metasploit. Finally I just said, hey, man, if you're ever looking for a change and you want to try something different, oh, we would absolutely love to talk to you about joining bhis. And about six months later he called me up and he said, I'm ready to have that conversation now. Um, and that was just really, really, really cool to see that person who was so helpful and so nice to people at these conferences be willing to come over and basically join Black, uh, Hills Information Security.

Speaker B: Yeah, that's fantastic. You know, one thing that I've kind of taken from what you've described here when looking for talent is it really just comes down to passion. If they have the passion, they show the passion. If they give back to the community, if this is something that they, you know, live and breathe, they've got a space on your team is at least what I'm hearing.

Speaker A: Yeah, absolutely. Um, without question. As long as they have that passion, that is ultimately what we're looking for. And to be honest, it's weird, but coming up through the SANS Institute, whenever Ed Scotus and Stephen Northcott, myself and Rob Lee, we're looking at new talent and instructors. We're looking for the exact same thing. We're looking for that joy, that passion, that happiness, that willingness to share constantly. And that's ultimately what we're looking for. And it makes for an amazing work environment. The thing I can't stand is when you have people that are like, I am so much brighter than everybody else, I'm smarter than my peers, I'm a 10x engineer and I'm just so good. I've worked with a number of those people. And almost every single time they're miserable to work with and they're wrong and they make messes and they don't bring any humility to the table. So that joy and that humility is absolutely key.

Speaker B: Sure. Okay. So what do you think about a black ah hat transitioning to pen testing team?

Speaker A: I think that there's always an opportunity. Right. It makes it difficult for me to hire people that are self described black hats because that term means so many different things. Right. You'll talk to some people and they'll think that just writing an exploit is a black hat activity. Right. And those people, like, I've written exploits and I've released them publicly and I didn't go through responsible disclosure, whatever the hell that is. Um, like I'm a black cat and I don't, I don't. Absolutely, I'll hire you. Right. I'm talking to somebody who's a black hat and they're like, yeah, I stood up a wireless access point. I'm compromising all the people that go through it. And these people are all sheep. And uh, you know, nah, I'm not interested at that point. So I think it really depends on what their definition of a black hat is. Like I said, if it's exploit dev doing things and releasing things on full disclosure, I'm not that concerned. But if somebody has that attitude of I am better than everybody else, therefore I can hack everybody else at will, then we're done. I'm uh, just not interested in working with you.

Speaker B: Right. Well, I do want to learn a little bit more about this new incident response game that your team has created. Um, I believe it's a card game of sorts. Could you describe backdoors and breaches a little bit more?

Speaker A: Yep. So the idea is I wanted a card game that could train organizations for incident response and I could teach it to them in six minutes or less. Um, so the basic premise of the game is you have a set of cards that are designed for initial attack and comprise for ah, pivot and escalate for command and control and persistence. So you draw four random cards from each of these decks and there's about 10 cards for each one of those decks and that builds your incident dynamically. So if you're the incident master, kind of like the dungeon master, you just built that entire exploit. So you have spear phishing was the initial attack and compromise, pivot and escalate using link local multicast name resolution. And then persistence would be DLL injection on the uh, computer system. And then Command and control, the HTTPs. Uh, so you have XFILL over HTTPs. So that's your instance right? Now they keep those cards and there's random cards like I said, There's 10 for each of those sets. Um, for the initial attack and compromised pivot and escalate, C2 and exfil, um, and then persistence mechanisms. There's, there's about 10 for each of those. And then it's up to the incident response team to use their procedures to try to figure out what those cards are that the incident master is holding. And I think we have about 25 uh, procedure cards. I'm not sure. It might only might only be 20, I'm not sure. And on those procedure cards it basically has things like log analysis, SIM analysis, network flow analysis, host analysis, endpoint security analysis. You have all these incident response procedures that an organization should have and they get to draw four and they work through the incident by rolling 20 sided dice. For every action you take, you roll the dice. If it's over 10, that means 11 to 20, the action's successful. If it's under 11 or 10 or lower, then the action fails. If you roll a 20, you, you pull a uh, a randomized inject card. If you roll a one, you pull a randomized inject card. So it gives the game some flair and it makes it so it changes. So you basically create these stacks of cards and then you use them at appropriate times. But I can teach anybody. In fact we have a webcast coming up here in a little while where we're going to train people how to use this game. And it's a standard deck of cards, uh, size playing size, and you can sit down, you can hammer this out. For every single one of the procedures for offensive tactics and defensive tactics we have links and recommendations so people can use it as a training platform so they can learn about what are the different attacks for lateral movement and persistence and C2 and exfil, and what are the different incident response procedures that your organization should have. So my hope is this gets conversations going and they say, wow, we have these procedure cards that we should have in our organization, but we don't have these procedures and we really should develop this capability in our organization.

Speaker B: Yeah, very cool. So do you need to have a certain level of knowledge to engage in the game or can anybody pretty easily and learn?

Speaker A: Absolutely not. One uh, of the things uh, that I think is really helpful is it can be used as a learning tool. So ah, Jason can jump on. We are sending out decks of packs of 20 decks to educators all over the United States. And the goal is to get it in the hands of university students. And then these university students can read about these different tactics, these different procedures, these different technologies. So it can really help you build up and bolster your own awareness for these different types of attacks. But, uh, yeah, as long as you have a basic understanding of like, networking and operating systems in Active Directory, it's really a playable game. And honestly, even if you don't play it, like I said, if you just go through the cards and learn, then you're going to do great with it.

Speaker B: That's fantastic. That's a really innovative way to, uh, get people involved and it's always fun. So that sounds incredible. Excited to get our deck as well. Um, one more question here before we let you go. I'm curious what you're most excited about in the industry right now.

Speaker A: I think. Well, there's two different things. Um, the thing that has me the most excited is we finally are developing a common framework for understanding the attack tactics of adversaries and even penetration testers. If you look at the mitre, ATT and CK technique matrix from an example perspective, it is amazing because it's breaking down all the different types of techniques that attackers use. Now there's variations in those techniques, um, like, you know, different ways of obfuscating PowerShell or different ways of doing DLL attacks and things like that. But finally, you can sit down as a student and you can say, I'm going to start in this box and I'm going to start reading and understanding these different attack techniques and you'll have a pretty good understanding of what it is attackers do. That's number one. Number two, I think that there's this really cool awareness that's happening that we done screwed up like 15, 20 years ago. The whole concept of blacklisting failed us. You know, it failed us for antivirus, it fails us for IDS and IPS system like automated blacklisting just doesn't work. And people are waking up to that. People are also waking up to the fact that their SIEM is absolutely worthless in most regards. I mean, if you look at the, uh, Verizon, uh, data breach report, 1.5% of the incidents that Verizon worked were detected by people. Sim. So now we have people that are standing up and they're like, hey, this stuff doesn't work. What other opportunities and what other technologies are there? And they're starting to look at user behavioral analytics, they're starting to look at advanced EDR they're starting to look at network analytics and doing pattern matching with tools like rita, um, Real Intelligence, threat Analytics. And they're starting to look at their networks. They're starting to look at SIEM and looking at it from terms of Sysmon and using ELK stacks, actually get proper data out of our event logging stacks out of our Windows computer systems. This is beautiful because it's not about coming into this field right now and saying, oh, it's the same old, same old. Do these five, 10 things and you'll be secure. You won't be secure if you don't do them. It's dynamic, it's changing. And I think people are waking up and realizing there's a different approach for learning about attacks. There's a different approach for trying to defend their networks. And I think for the first time in a long time, I think some organizations have a real shot at actually securing their networks. Not being completely hack proof, mind you, but at least being a lot better than where we were instead of getting out of this horrible cycle we've been in for at least 20 years. Computer security.

Speaker B: Right. Thank you for everything that your firm's doing. Definitely taking an interest in the profession and helping those along the way that are interested in becoming successful in it as well. So thank you for what you guys do.

Speaker A: You bet. Thank you.

Speaker C: We want to thank everyone for listening to today's podcast brought to you by Nexus IT Group. If you're looking for a new career challenge, let's chat. If you're looking to hire new talent, reach out. Or if you just want to talk about cyber security, email us@infonexisitgroup.com until next time, stay safe and stay secure.