Why Contributing to Open Source Matters for the Mission

Speaker A: Welcome to the Defense Unicorns podcast. I'm your host Rebecca Lively and today I'm super excited to have Brant Keller on the podcast. Brant, I'd love it if you could take a minute and just introduce yourself to our listeners.

Speaker B: Absolutely. So I'm Brant Keller. I'm a software engineer at uh, Defense Unicorns. My background I think really stems from the defense sector, something I've always been passionate about and in particular I think kind of going all the way back. My, I'd say my, my grassroots interest and passion there starts with joining the Marine Corps out of high school. No, no interest of going to college. I know sometimes spicy, um, among those in the software domain, I think technology has always interested me, but maybe not to the same extent as others. I think what really caught my early attention and interest was problem solving. So in the Marine Corps I did communications for infantry. So I spent more time working with radios and RF comms and then hiking a pack than I did touching a computer. But it was uh, it taught me a lot of valuable things I think in reflection. Valuable, valuable experience to my life. But overall the problem solving aspect really started to like, I think bloom during that time to the point where when I started to get an interest in technology, there was just this innate like click, light bulb moment for me of there's so much room to solve problems in much better ways than we do today. There's so much room to use literal vehicles if you will. These, these platforms I worked in light armor and reconnaissance and we had these vehicles that were like, so had so much potential if we could find out how to get the technology on board. And so as we like really started to test new things or look at all the old stuff and be like why, why don't we use any of this? Or how could we replace this with better stuff? That's where I kind of started to dive into technology. And for me getting out of doing that in the infantry domain was going to be incredibly difficult. And so my pathway there was to leave the Marine Corps after my active duty enlistment of five years and go to college, spend a little time doing some entry level software development, uh, in the paratransit software domain for state and local agencies. Learned a lot of awesome skills. Kubernetes containerization still in its infancy around 2017ish. And then kind of to wrap up this, this background. It is really just the uh, I wanted to find my way back to defense at the end of the day. Really loved software at that point in time. Really loved still, still trying to grasp and learn. And from there it was which, which companies really had a, a defense domain that they provided or worked with. And I had no, no connections to anybody in the military who worked software. Software in the military for me was something I didn't even know really existed. So kind of like branching off, found a company called phantomworks, which is, which is a subsidiary of Boeing. Eventually kind of got pulled into the Boeing defense and space systems side of the house. And from there DevSecOps and all the things kind of led me, led me innately to where I probably am today. It probably put me on the path of, uh, trying to really solve some of these larger core organizational problems that kind of haunt the public sector.

Speaker A: I'm curious at what point in that journey. So it sounds like you went to college to solve a problem rather than, I think a lot of people go to college to. Because it's the next thing that they think they're supposed to do. So you went to college because you became obsessed with the software problem. Is that a fair statement?

Speaker B: I think it's fair. Uh, I think it's probably two levels. There's probably still half of it. That is, there is a unfortunate prerequisite among a lot of software employment for having a degree. I don't know that that's innately still as prevalent today as Maybe it was 10 years ago, but there was definitely a piece of that that is like, there's a core requirement out there. If you don't meet it, you're going to get automatically flagged and not accepted for interviews and things of that nature. I think in that same domain again, infantry, Marine Corps, I have no connections to software. How do I do this as a. What I want to do with my career and the GI Bill? Being able to support me in kind of taking that next step and furthering my education, very much a great part of that. But yes, I think to answer your question, there was this other part of the problem, space, which was like. I've lived with a number of technologies that I think we joke about at the end of the day, like they were acquisitions from the government who bought something at economies of scale much larger than infantry platoons, which are small groups of individuals on a specific mission, and so much room for better capabilities. And how do you start building a skill set that can at least start to discover, like, what does this require? I don't know that I even can really fully answer what does this require today, But I still have this, like, immense passion and hope that those vehicles that are still in use today and the future evolutions of those vehicles and the warfighter can have more optimal technology to actually focus on the mission instead of what we used at the time.

Speaker A: No, I think that's super fair and I definitely know that that's a really common refrain of people who've spent time in the military. Is this idea that maybe military grade technology doesn't mean what everyone thinks it means?

Speaker B: Yes, very much.

Speaker A: When it comes to software for. I know you're also a, uh, Cloud Native Computing foundation ambassador. Is that true and if so, what does that mean?

Speaker B: Yes, yes, great point. I don't know if people are going to be able to see on video, but I guess, um, a separate part of my background, I have a day job in software engineering. I think we'll probably discuss it at various points, but I work on Zarf and that's always a very fun topic to talk about. But then more broadly is probably two, I'd say two other areas of like space where I try to influence. One is cncf, more broadly, Cloud Native Computing Foundation. And uh, a lot of, I would say the core technologies that we use on a lot of these new and exciting mission sets or that we want to use kind of stem from the CNCF and some of the other foundations that support them. Kubernetes kind of being a flagship under the cncf. A lot of the other required infrastructure of things we put on top of Kubernetes to execute a mission is still a lot of public and open source projects. This is wonderful. And so kind of, uh, as I was working across horizontal and vertically through the foundations, something stood out to me, that there are ambassadors for the cncf. Each one offers kind of a unique perspective on what are we trying to do with the cncf. Who is included, like what does this inclusivity look like, as well as where are there areas of underrepresentation? And this was a really big one for me. Uh, in my application I really felt like there's an area in the CNCF to make some changes that would bring things to light. There are others who I could mention by name even that we're looking at the public sector space and saying, okay, well how do we make representation of the public sector more known and inclusive so that we can get other practitioners of cloud native technologies in the public sector to kind of come and talk about their use cases, their gripes, their constraints, their requirements. And so for me that was really the front line of my application that I believe was why I was accepted to be a CMCF ambassador. And why I still am, um, today is because I want to kind of take my core ambassadorship responsibilities and try to find ways to have those discussions with projects. I work with a lot of projects and something else I'll probably talk about in a minute, but also from the domain of how, how are you aware that these use cases exist? And a lot of times I'd argue projects are uh, completely unaware. Oh, you want to run this in the air gap? What is the air gap? Tell me more about it. And you start telling them like, oh yeah, we just, we can't make a call here. So instead of we replicate that service internally or we pull data from somewhere else and they're like, I never knew that that existed or that somebody would want to use the project I created to go and do that. And I've had that conversation more than a handful of times. And so I don't think that there is any lack of projects wanting to support those use cases, but more so unaware of uh, those use cases maybe even existing.

Speaker A: So I'm curious how you became involved in the open source community and what called you into it.

Speaker B: Yes, I want to answer this question very carefully because I think there's a way that I have answered it in the past that comes across as maybe a bit more blunt. Some of the employers that I've worked for in the past had a, uh, I would, I'd say still had some maturity in their open source utilization journey to understand exactly how to contribute to open source without believing that they were losing ip. And so in that regard there was a lot of legality and legal processes wrapped up in me, ah, as a software engineer, wanting to do something in my free time to an open source project because I believed it helped everyone who consumed that project and had really no, no proprietary information involved in that work. It could have been updating a unit test, it could have been doing something very meaningful to documentation. But still there was so much ambiguity and confusion wrapped up into that that it led to a lot of surfacing of like people maybe not wanting to do so because they were just worried. We don't want to, we don't want to lose our job, we don't want to create a mess, we don't want to violate some agreement that we may have agreed to in taking this employment. So I say all that to say that after I had come to Defense Unicorns at the time, 2022 January time frame, years ago now, and it was even smaller startup than it is today and there were essentially there was no one telling me I couldn't do anything. I laugh about this with people. I say that I just didn't ask for permission. No one, I wasn't doing anything nefarious to like need permission to go and do something. But it was just this is my chance and like uh, there's so much room for opportunity here to, to figure out what is my pathway, but also what is this open source stuff. So that is a uh, long way of saying that I've only been in open Source now as a contributor to Open Source for maybe something like three plus years. But once I had that opportunity it was very much just like dive in, try to figure out who I could be a mentee of and kind of learn from them. What does this look like? How do you do these things? How do you work with people? Where do you go? And I think the biggest learning, the, the biggest learning curve. But the thing that had the most impact wasn't actually contributing to projects, it was showing up. The, the biggest part of like being involved in a collaborative activity was just that like people that might need a rubber duck and figure out like I'm trying to solve something here, I don't know what I want to do or simply like just building that, that network of people that you've worked with, building that, hey, uh, what do you think of these things? Let me work with you on trying to solve this. And at the end of the day like the more that you continue to show up and support people, the more that like they just kind of recognize you as somebody who wants to be a part of trying to do something in this, in a community, in a project. And so worked with a couple different layers of open source. Open Source isn't just code and I think sometimes that that is something that is a barrier to entry. But Open Source I see as so much more and it's a, uh, it's a really a space for us to educate people on where, where to make improvements that can make their enterprise better, something more horizontally scalable for like hey, here's how you can do vulnerability management better and you can impact again in whole enterprise. You can impact all of the projects across the CNCF just by working on a piece of thought leadership that is, is more than just how do I put this in CI on one project. Here's, here's the fundamentals, here's how we can teach people to fish and make a lot of impact.

Speaker A: One of the things I'm curious about is actually like Open Source and the government I know I uh. You've uh, spent time in government and now in government adjacent roles like do you see there being, do you see the government embracing open source? Do you see the government using open source being afraid? All of the above. I'm curious what your experience has been like there.

Speaker B: This is a really good one because it's a really tough one. Right. I think you're going to see a quite, quite a spectrum across the government. Right. And like you'll see I think isolated pockets of people uh, who do it extremely well. You'll see others who I think groups have kind of stamped things in the past. I think we've made a lot of improvements ah and helped people understand. But some might still see open source as the scary thing. They've seen the instances of how it can go incredibly wrong. And um, those things are completely fair. We need to learn from those. But in some regard they see any transparency in their supply chain about the thing that they're using, the thing that they're actively deploying to production and trying to measure the, the criticality of this thing running in production is incredibly important. It uh, runs the water utilities for a state or something like super critical where if something failed there would be huge implications. And you uh, start thinking about okay, well what's the risk involved with somebody knowing that or somebody knowing the internals of a dependency that we use in that system. And it's a hard one to really try to teach into these pockets of the very like open source adverse security uh, front and foremost mindset to say no. There is a lot of reason for why open source can help you build confidence about your security posture, how using these dependencies and where they are built, where they come from, who contributed to them. Again kind of a spicy area of the open source and the government conversation that really is quite critical and why you would not want to try to compete with these technologies. I don't mean compete so much as like you're going to try and offer an alternative service but compete for just sheer. You're going to build it yourself even though this thing is built out here and it's even more optimal, inefficient at doing the job.

Speaker A: So you're talking more about the like the government trade off between the government building it themselves and the government using an open source tool. Or are you talking about maybe the government using a closed source tool versus an open source tool. Then again one example you used it's that like security through obscurity with if we use an open source tool then how it works is transparent. But I'd venture if they're using a closed source tool, it's even more frightening if they didn't build it themselves. And maybe in some cases even if they did, because they still don't know how it works, or they definitely don't know how it works. And sure, neither does anyone else. But that, I don't know, I feel like that introduces its own risk. Do you, do you agree with that?

Speaker B: Yes, I'd say very much so. We've elevated the domain of supply chain security so far. There's so much more room to do things and I want to say like the job is done, but over the last decade and then the Open Source Security Foundation, OpenSSF becoming very prevalent in this conversation I think is where it's important to highlight that uh, like the floor of supply chain security and open source has been lifted so high that even with some of the fundamental flaws of something being compromised via uh, new and unique ways, like we've improved this baseline so much and we found ways and practices and guidelines and things like that to really help us build confidence about a thing, even if we're not innately aware of all of its wiring versus that of a closed source piece of software operating as a black box. Since it's going to do something, you have probably an agreement with that provider that it will do that thing. But there's going to be this innate evolution of technology that just continues and continues and compounds and compounds and open source really allows us for this tight feedback loop of uh, not only like the risks involved, I think that's an important part of this conversation, but also just like the sheer efficiencies and compatibilities with how technology evolves. You don't really have a lot of guarantees. I would argue. I'm not a closed source aficionado if you will, but you don't have a lot of guarantees that the domain evolution of something like OCI artifacts and how we use them today versus how we use them a year ago, which is even a pretty big difference, is going to be something that your closed source software could, could potentially do and use and how and where, versus an open source project where it's like, hey, we've seen this, here's the patterns. You're using this pattern. I can see that, I can see how you're using it because I can look at it, I can debug it and I can see that if we switch this thing here that we're going to be more efficient, we're going to optimize we're going to be able to evolve with a specific being updated from one version to the next very uh, expediently and not run this course, which I would say we have ran time and time again in the government acquisition domain of being sold something. And now that thing is kind of put on like it's not evolving but it's being put on life support. We want to keep it running because it provides a critical function, but there's no path to get us off that thing or to get us to the new thing, which is insanely important.

Speaker A: So shifting gears just a little bit, one of the open source tools I know the government uses a lot is Kubernetes. And you posted something on LinkedIn recently that I thought was really interesting. It was the list of top 20 Kubernetes contributors, period. Like which companies were contributing. I'm going to pause at that. I'm going to ask you what you found surprising about that list from the get go.

Speaker B: It's easy to talk about things that are not surprising. We have a lot of the big companies that offer Kubernetes distributions at the top. There's a lot of scale involved in that. That is sheer people and time that you can't uh, I would say at some point like it's very hard to compete with without that being your sole objective. And so that those are some of the things you get out of the top. It's, it's kind of, it's blatant but then you start getting down into it more and you either a get a mix of I've never heard of this company before, what are they doing? Where do they do it, how do they, how do they do it? And then on top of that you get to companies such as Defense Unicorns which is still a quite small startup being at in the top 20 which is um, frankly is like exciting from a unicorns perspective but at the same time like almost. I don't want to do any injustice to the people at Defense Unicorns or in some of these other companies and how they've position themselves because there's a lot of other great companies in that list tetrate to name, to name one. But the amount of scale that required to get to number 20 wasn't something that should have been a uh, hard practice for all the companies that tend to evangelize doing open source and being involved in open source. And I think that's where we get into a lot of ambiguity of what does it mean if you market yourselves as a company that does Open source or is open source? Are you producing things and that's your kind of contribution? Open source. That's great. I don't think there's anything innately wrong with that. But I think more so what comes as a surprise is how many companies with much, much larger scale that do things with kubernetes are not on that list. One, but two like purport to be security and risk adverse and don't have anybody monitoring those projects to such a degree where they're being involved in the discussion. And I think that's really where I think it's important.

Speaker A: I think maybe wasn't surprising but was disappointing to me is that none of the major prime contractors, and I'm not going to name them but, but I could because none of them are on there in the top 50. I don't know that I saw any of them when I scrolled through it. But other than like maybe like Microsoft, who has lots of different reasons for contributing that aren't about the federal government, do you think that that's, do you think that's a security risk? That we don't have contributions back by the major defense primes?

Speaker B: Yes, that's not a spicy take in any way. I don't think anybody is really going to argue that statement. I think what's really important about it is that we have all of these companies that are consumers of technologies and that is where they stop, they consume, they integrate. And I get, and I don't mean to say any of this is to like point fingers at people, but I think it should be more of like a reflective activity of hey, what technologies and dependencies do we consume? All right, wonderful. Are they properly supported? Like let's get down into them. Like hey, this thing, it doesn't even have to be kubernetes. It could be a tool that you use in production every single day. Like who, who maintains that? And if you look at it and it's one guy doing it in his free time, like there, there should be things you start thinking about of like okay, well how do we support them? Is having their, their knowledge of that thing that we use in production. Could that, could that be a game changer for how we use that thing in production or the way that we use it in production in the future, like key hiring conversations that begin to be had I think are super interesting. But then when we get back to security, like as soon as you become a consumer of technology, I think you have an innate responsibility to be monitoring one, how you use it. Two like uh, I would say almost to the degree of what's changing. When something changes versions, I think people fall into the spiral of I've got some automation. It tells me when there's an update and I update it and make sure it still works. The renovate and CI workflows of the world, which are wonderful. Again, that's talking about a broad piece of guidance that was like, hey, you need to keep your dependencies up to date constantly. Yes, you do also need to know what's changing in those updates and not updating when you see something that should be flagged. Right. And that's where this, this cycle, I think becomes really interesting for risk when we start saying, okay, the people who are doing it correctly are saying there's an update on a technology I consume. Do I have somebody who is, let's say, on staff, who is a contributor or has worked with that source code very deeply and knows it. Can they tell me what's changed between these two things and evaluate is something in here malicious? Right. How many. I think we can point to a number of times malicious code has been merged into source code in an open source project because either a, that was the goal of whoever this nefarious person was where they got angry at their consumer base and said, whatever, I'm just, I'm going to put this in here on purpose. Which has also happened. And I think that's where it becomes really key to say, like I'm a consumer, I see that there's updates coming, I need to be involved in how those updates kind of came to be in some way. There's a scale problem there that's not always solvable. But then reviewing the changes, I think if you can't look at the change log for a project that says, hey, Zarf went from version 0.53 to O5.4 and say here are the changes of code and look at them and say, I get what's happening here, that you aren't the right person to be making that PR review on your use of that dependency unless you can spend that time and energy to go and look at it. And I would say that's probably one of the bigger areas of risk with regards to open source and dependencies.

Speaker A: Can you solve that in some ways by relying on more mature open source tools? I think there's a big difference between Dan in his mom's basement maintains this tool and is literally the number one and only contributor and something like Kubernetes, where you have thousands of companies backing it. Do you see that as something you maybe can rely on more without necessarily needing to go through every line of code as it updates.

Speaker B: Yes, I think that's a fair point. The scale involved with say Kubernetes changes from one version to the last. There's a change log involved there that frankly like, would take up a lot of my time and energy to go and review all those things. And maybe uh, maybe I could do it in a month. Like there's so much involved with that that would become untenable that I think at some point, yes, you do need to have some kind of innate maybe like trust and it's something that needs to be built. I don't think that you should ever just trust something because it exists and you use it, but you should have some varying levels of ability to say, here's what I'm consuming, here's where it comes from and it is the Kubernetes project. Here's how they control changes, here's how they review changes, here's how they release, here's how uh, they get new features, here's how they deprecate old things. Lots of things involved there that have a lot of quality gates and process gates involved. And so I do think, yes, you get at a point of like, yeah, no one is probably looking line by line and saying what's changed here? But the maturity of that project definitely helps. That said, I would say that there maybe is still room for Dan's project. And he, he has this thing and he's like, hey, like, I want people to adopt this thing. And so I'm going to be forward leaning into ways that make the adoption pathway more optimal or efficient. And I could point to uh, probably a couple different initiatives, one being the open ssf, like baseline project. And it's just like uh, it's kind of a way for projects to start, like codifying, like why should you trust this? Or why is the security of this project that I'm building something that can be trusted? And so again, it's a spectrum. It's really hard to pin down and say here's the only way to do something, because that's impossible. But here are ways for us to build confidence appropriately.

Speaker A: And maybe for some places a reasonable first step is actually just knowing what software is even going into what you're building. Because I think there's some places where there's not even a basic awareness of that in the government.

Speaker B: Yeah, I think we've tried to iterate in uh, a lot of positive directions like the software bill of materials and SBoMS, conversation is something that gets some people really excited and makes a lot of other people really angry. And it's one of those, that's where the conversation isn't finished. Right. Like, as with all of this, there's still things that we need to do in order to get to a meaningful point. Just because we produce SBoMS doesn't mean that the trust or the security of that project, uh, is any worse or better. Cool. You have all your dependencies here. Did anybody look at this sbom? Did they know it says right here you're using a vulnerable thing that's been exploited and compromised. Like, oh, well, no, we just kind of accepted that you gave us all the, checked all the boxes. Right. And I think that checkbox phenomenon is something that we all have started to frown upon. But at the same time there's ways to where the pathway of identifying that a gap exists in its present and where can we potentially take this in the future? Becomes uh, a leading part of the conversation, such as with S bombs there, the story isn't finished, it's still being written. How can we ensure that that makes its way to something, uh, and I really like to name drop open source projects, but something like guac, which takes the SBOM enumeration, ah, problem and graph analysis to a layer of being much more provenant in the discussion of what does this mean.

Speaker A: Yeah. So I have one more Kubernetes question and then I want to pivot and talk about ZARF and air gap and why air gap matters. But the last Kubernetes question comes from, uh, something I saw on LinkedIn after we posted the top 20 contributors and defense unicorns was on there, but so was ByteDance. And someone said the US government can't rely on China for critical tech. And I think their sort of implication of what they were saying was that we shouldn't use kubernetes. I'm not sure that that's an actual option at this point, given how embedded it is in lots of different government systems. But I wonder what do you have to say to people who are like, holy cow, bytedance is a huge contributor to kubernetes and that feels like a risk.

Speaker B: Yeah, this is a really, it's a really hard one to answer, I think. Like what we need to, we need to get to is what, what questions are we asking when we're evaluating the technologies that we're using? And uh, it's going to be so diverse, it's going to change, it's going to be situationally dependent upon how this thing is being used and where and a lot of things. And also going back to our conversation of like confidence and trust about where those contributions are going in such a way that you don't have to again examine every line of code in order to have confidence with the thing you need to. I think there's going to be some level of like, variance in there of uh, you should do these things and you should do this to some degree. But at the same time, yes, we have a lot of tensions with regards to foreign contributions to projects, with regards to the US Government and its consumption of software. And it's going to be, it's going to be a consistent problem. And I believe we need to move away from um, maybe the area of origin or nationality around contributions and try to focus more in on like what is the core problem that we're trying to solve here. Hey, there are risks involved with this code coming from a foreign country. And is that the reason that we don't use a thing? Or do we hire technical domain knowledge to help us solve this problem? Do we adopt other technologies or change a new LLM that enables us to kind of have it take a different perspective, build another better testing tool or a static analysis tool? Something to kind of get at more of the problem of like, well, what were you trying to solve here? Like there probably, probably there are a lot of people operating with positive intent no matter where they contribute from. And uh, uh, I think the, the problem of stating like, yes, there are tensions here and so we're not going to use this thing one of the other. You have that choice and you can make that choice wonderful. Like maybe that is how you select technology at some point in time. The inclusivity of getting contributions and innate domain knowledge from across the globe is something you can't ignore. And we need to find pathways of saying like, okay, if this is, this is a risk, how are you evaluating that risk? And it shouldn't just stop at or area of contribution. It should really start with okay, how, how do we address these risks accordingly?

Speaker A: Yeah, and I'd say as the, the resident pessimist in this conversation, you said you, you can't. There's positive intent no matter where, uh, or there could be positive intent no matter where it's coming from. I'd say the opposite of that, which is there could be negative intent no matter where it's coming from. And so you, you need to be able to vet the security of your software without relying on something as like, right, wrote as what country did it come from? Because There are definite cases where US contributors have, have contributed malicious code. And if you just accept it because it's from the U.S. well, that would become a foreign adversary tactic pretty quick, I suspect, which is, hey, let's just contribute from the U.S. so I, I do want to talk about Zarf and Air Gap, and I think you, you wrote a blog recently, or maybe a couple of blogs on why people who are building cloud native tools should care about the air gap, which I think is a really interesting question, because one of my questions has been that exact thing historically is like, as the world is increasingly connected, why would we care about disconnected environments and optimizing for them in any way?

Speaker B: Yeah, uh, this is a really fun one. Again, mentions of all of the CNCF ambassador things. These are areas that I'm, I'm really passionate about, and I think they all stem from, like, this. There's a, as with everything, kind of a variance of, you know, what, what kind of connectivity do we have and what's allowed? And I've worked in classified environments and no connectivity in or out. And you're like, okay, well, I need to design a platform, uh, DevSecOps platform, if you will, that can be used. And if you're just looking at that problem set, okay, DevSecOps, it's these, integrating these tools together. And then we ultimately provide a pathway of like, build and deploy, CICD, etc. And that's great. And you build the thing. And I am as guilty of this as anybody else, where it's built and it's ready and it's deployable on aws. And you're like, wonderful, like, we can use this thing. And then they're like, oh, yeah, well, can we also use the same thing, but in this classified environment? And you turn around. Well, like, the answer is no, because we assumed all of these things. We assumed you had somewhere to pull the stuff from. We assumed that there was going to be places to put the data. Here's all these assumptions we made. And so this is kind of like my way of telling the story of building a platform first and foremost with connectivity in mind, not air gap friendly. And when you try to take something that is not Air gap friendly and make it air gap friendly, you quickly find out that you made a lot of assumptions about how this thing would be used and where and kind of the, uh, underlying infrastructure and when you tried to work back for them, that it's, it's difficult, it's not something you can't overcome. It's not insurmountable but it is difficult. But you also find out that there's just a lot of areas for resiliency that you didn't also plan for that apply to connected environments. And so this is where I've kind of been diving into this more and more lately, to trying to describe and build some knowledge around why this is important for kind of building any application today. It may be a little uh, niche to go to the extreme of air gap, but I believe like there's still some of these underlying cloud native fundamentals. That is like if you start with the ability for knowing how your architecture adapts to varying levels of connectivity, then you're probably building a stronger, more resilient system overall. Even if the base case is deploying it to AWS in a public cloud and there are no connectivity restrictions, something's going to happen. We've seen cloud like DNS fail, we've seen cloud other things fail. And it has brought down production systems in uh, a lot of places simply because an application could not handle losing connectivity for some point in point in state. And so I think that builds better systems overall, like full stop. And then we just start abstracting from there into these other pockets that are super important. If you go to the full extreme of air gap, we can point to a number of systems that they are only going to be air gapped at this point in time. And I say at this point in time because I think technology and connectivity is just going to continue to evolve and we'll start to see more and more connectivity. Um, but it doesn't go from air gapped to high speed Internet in one day. It's hey, every, every day maybe I get a satellite link that's super slow, but it gets me, it gets me the pieces of the things that I need that are super critical. And then we get faster and faster and that'll be important. And I've talked to a number of practitioners now kind of across the world about air gapped environments and there's a piece of it where they want to say they want to use this pattern because it'll help them with building production systems in ways that they can focus on the connectivity requirements being slimmed down to the most critical path and saying this data is the most critical thing. If I can air gap the rest of this and have confidence that it is going to be able to run in isolation and by itself, but there is connectivity allowed over this very weak link to perform something, then I want to use that connectivity for the most critical pieces and not for doing Something that doesn't support the mission set. And so very broad, very, very broad topic. It's going to be hard to really dissect all the key things, but I think there's a, there is a full spectrum here for let's try to consolidate around which patterns are air gap friendly. Where would you use air gap and why? And I think, uh, putting that into your design requirements or going to a project and saying here are the constraints of using it in this environment. What can we do about these?

Speaker A: I think one of the things I know a lot of people I've talked to who don't have as much defense sector experience, one thing that they're surprised to learn is just how many air gapped environments there are and just how many things are air gapped for various reasons, some of them maybe even unnecessarily, but they definitely are. And others definitely very deliberately and by design. So I think realizing that there's that entire market and use case out there in and of itself is a good first step.

Speaker B: I would agree. Every we talk about edge and I think the edge conversation is really important because it almost alludes to connectivity, but the connectivity is never a defined, consistent requirement. Uh, and so kind of, as I mentioned, could be on at some point during the day, it could be off. We've seen IoT and industrial settings, IoT and I would say agricultural settings where again, like there's a lot of cloud native utilization across a lot of these broad and different environments. And I would say that probably a lot of people operating the air gap are doing so because there's a requirement that they do so. I would say just as likely is there are people operating with a subset of capabilities that they could have because the constraints of that connectivity don't allow them to deploy more effective software. And there's still going to be a lot of room for trying to solve that piece of the puzzle and use something like Zarf and let's air gap the deployment of the workload and get it deployed on that thing. But then the connectivity of that thing, just needing to reach out and grab data, which can be very small in size, is something that is still allowable, can advance the mission.

Speaker A: Uh, that makes a ton of sense. What have I not asked you yet that you, that you wish I would ask you or that you really want to talk about before we wrap up?

Speaker B: Uh, let's see. I think the, the, the last thing I really wanted to hit on was trying to not leave this conversation with people thinking, okay, yes, I need to be more involved in open source, but I don't know how.

Speaker A: It's a great question. So how can folks get more involved in open source if they think that that's something they should be doing?

Speaker B: Yes, I think that this is probably like a uh, cake diagram at some point at the end of the day, but I think there's a couple of different pieces of that. The picture, something that I didn't mention as part of my background is within the cncf. I'm also a technical lead for the Security Technical Advisory Group. And so if you just think really quickly about the cncf, the Technical Oversight Committee kind of oversees a lot of the day to day project governance and evolution underneath them. Like you, yes, you have the full CNCF landscape which is 200 some different projects now that all have one to n contributors and maintainers. It's a very large group of people. Then you have end users and consumers again an even larger group of people in that you need to probably delegate some amount of authority from the oversight committee to groups of people who are domain experts if you will. And so for the Security Technical Advisory Group I serve with some um, other volunteers from across the industry, fellow tech leads or co chairs in that group to really try to find ways to work with projects and help them with their problems and then to try to evaluate the security of projects from maybe more of a assessment perspective and provide guidance. Right, provide guidance for people who consume, provide guidance for projects who build and everything in between. And I say all of that to say that if people want to get involved again my journey in open source has been very short to date and I think as I said earlier, like showing up has been the biggest answer to the question. 10 times out of 10 people will like sometimes just get paralyzed by looking at a project, looking at an issue, their issues, looking at things that the project needs help with which to be honest a lot of projects don't do very well just because they don't know that they want to ask for help on something or don't know that they can or need to. And a lot of these people are doing this as their free time projects. But I kind of say that as to like show up and find the place to show up first and foremost. So for like for me security is something that I believed is innately like a part of every day as a software engineer in Open Source and in the delivery of open source to end users. And I wanted to be involved in that conversation. So I'm um, googling right, CNCF security. Oh There's a group where all they do is talk about like security practices and some people that might be really boring, but to me it's like, hey, CrowdStrike is having a very poor evening because of an outage that lasted a significant amount of time and impacted a lot of people. What did we learn from that? And are people having this conversation of hey, here's an experience we had as somebody using a project or building a project and this, this really sucked. Like how do we make this suck less? And to use Brian Fincer's words. But the, I think the kind of important piece in there is like once you get over the show up piece, you've already like, you've already at least done something. And for me, every single time that I have been like huh, uh, should it, should I do this thing? I don't know, I'll just start showing up and see what they're talking about. And every single time you just consistently start showing up and people are like, what do you think about this? Does this make sense? Oh hey, uh, I know you work in air gapped environments, right? What about this changes or isn't a necessity or straight up can't be used? I think those pathways are important. I think there's a piece of this that I mentioned earlier which is are people staffing for open source, open source security and some of these other like positions that innately are going to be very hard to make an argument for. It's hard to say I want to hire somebody who doesn't build a product for me that makes me money. I want to, I want to hire somebody because they have innate knowledge about the technologies I use. And there are companies that do that, they are, that are well above the curve here, uh, of doing it appropriately. But then there are other companies that are like hey, you've used this, you use kubernetes before. Like you, you can solve these problems and continue to solve these problems as part of a consultancy, a prime contract on some, some government initiative. And I think that we need to try to find ways little by little to like build some breadth in what people's day to day responsibilities look like. If you're not involved in cloud native technologies, maybe you don't have this same requirement. It's probably far and few between in software these days. But if you do have cloud native technology utilization, then it's like okay, well what things are we monitoring? Are we monitoring vulnerability feeds first and foremost? Like start to categorize where you can and how you can and from there I think you need to continue to ask yourself one from a organizational perspective of why does this matter? And if you can't come up with the reasons why it matters, please find people who can help you because, uh, largely the answer isn't going to be it doesn't matter. It's going to be you weren't aware of all these areas that are significant gaps in kind of your risk reduction framework. And I think that's super key for me. At the end of the day, again, nobody was telling me to go do any open source things strictly. It's just more of a company culture and things like that that are well supportive of it. And for me I wanted to like, find areas of kind of like, I don't know, differentiating my skill sets a little bit more. We all work on these things. Who has more, uh, of an active mindset for security, like where do you go and get information about these things to help inform you that there is a better way to work with containerized images or to work with helm charts? We can work in a silo and get a job done. I believe that is fundamentally possible with engineers. But are we doing it in a way that positions us well for security, positions us well for technology adoption and evolution so that we're not building products that people will consume and have different levels of criticality that ultimately are unsupportable at the end of the day and have no pathways forward?

Speaker A: I think that's a really great point and I'm actually, I'm glad you shared it. I have just a couple of quick, hopefully fun questions that I like to ask. So the first one, um, I like to read, I like to hear what other people are reading or what they've read that kind of shaped their views or vision, anything from like technical to leadership to fiction. What book do you wish that you could make other people read?

Speaker B: M. Yeah, I think I could point to, I could point to probably like different books. I, I, I always enjoy myself a little bit of like productivity and professional self help books, if you will. And anything by Adam Grant I think is going to probably be on my list at the end of the day. I don't know that I have a real like standout for something that really continues to shape the way that I view things. I think what I enjoy most of all is trying to find those pieces of literature, of people, maybe challenging my assumptions a little bit. And so as much as I wish it were a book, I think more so my data source for some of these items is probably blogs and either blogs or other pieces of like short form piece like literary authority, authoritative sources. And so I, I don't have an answer so much as it uh, continues to try to pick at people who I think fall into this software domain of like what, what things can we try to be building skillsets for? It could be open source, it could be open source evangelization and I really love hearing different points and counterpoints for that. Security is always a big one. And then on top of that I think uh, at some point we, we start bridging into less technical topics and more into what, what kind of things empower people to be involved in conversations and those sort sort of like all sort of melt together. And so I have a non answer for you is what I'm trying to say.

Speaker A: I love it. I accept your non answer and I think the gist of it is if I had to summarize it's short form. Look for things that challenge your own assumptions so that you can be more right at the end of the day.

Speaker B: I think more right may be an interesting way to put it. Ah, for me it's like challenge my assumptions in such a way that show me how I might be wrong and then make me more right by being able to be an advocate for that point of view.

Speaker A: Yeah, I think that's how I meant it. But I'll, I'll, I'll revise it too so that you can be less wrong every day.

Speaker B: I like it.

Speaker A: What is your favorite open source tool?

Speaker B: Hmm. I'm m like laughing at myself because I don't want to say Zarf because now I work on Zarf, but like I've told people this story time and time again. Like I built, I with other engineers built a delivery mechanism that was a lot like Zarf, but it was not using all of the things that Zarf uses today. And so it was a lot less optimal. And so that's what really like attracted me to this problem set of defense unicorns and Zarf when it was really in its infancy. And so uh, probably Zarf's really high on my list. But uh, it sounds like a very biased answer. Outside of that I think kind of starting to. Which one would I pull on? I would probably start to bridge more into kind of like the, the observability domain, if you will. I think there's a lot of like untapped potential there and so there no one tool, uh, on the surface I think, I think I'm more so I'm like a specifications person at the end of the day. And so, like, when oscal was a really big focus area for me, I wasn't the expert, I wasn't the, the, the expertise behind that specification, but I wanted to learn it and jump into it. And so things like S bombs still, like, really get me riled up because there's so much potential. Things like codified compliance still, like, really gets me moving in the right direction. I feel like the open SSF in general, if I could point at. And point at it, is a really great organization because it advocates so much for trying to get a, uh, people. Trying to get people to collaborate on something. Not because it should be like a project in its own right, but because people are like, I have this problem. Oh, I have that same problem. What can we do to fix that? And so if you look across the OpenSSF domain, like, I think there's a lot of really great instances of things like bomctl and Guac and other tools that are taking the specs and really trying to make them usable.

Speaker A: Our, uh, last question, and this one is super important, so prepare yourself. Is cereal soup? Why or why not?

Speaker B: I don't know the definition of soup. Now I'm questioning everything. I thought I knew I would say yes. I can't find a good delineation between the two, where it seems like they have some property of a base liquid, whether that's broth or cereal or milk, and then they have, uh, some sort of makeup that has all of the consistency. They seem pretty similar. I'm going to have to say, yes, they are the same.

Speaker A: I accept it. Awesome. Well, thank you. Thanks for being on the podcast. I really appreciate it. Thanks for your time and insights. I don't know what the answer is for soup and cereal, so I was just looking for some advice. So thank you for that.

Speaker B: That too, yes. Thank you. Thanks for having me.

Speaker A: Thank you for listening to this episode of the Defense Unicorns podcast. If you enjoyed the show, please share it with your network and rate the podcast five stars. Wherever you listen, tune in next time for the latest in DevSecOps.