Shipping Software Faster and Safer with Pepr

Speaker A: Pepper will always be open source. And the reason why it's open source is because frankly, open source software, when your software is open source, you expose the application or the software or the platform, whatever it is, to exponentially more eyes and more eyes over time and then more people start adopting it and using it.

Speaker B: Welcome back to the Defense Unicorns podcast. I'm Rebecca Lively and today I'm talking with Case Wiley about how we're making Kubernetes security simpler, faster and a whole lot more developer friendly with Pepper. Case, welcome to the Defense Unicorns podcast. I'm so excited to have you on.

Speaker A: I love it. Thank you so much. I'm happy to be here.

Speaker B: I'd love if you, if you don't mind just starting with just a brief intro of yourself and how you got into the defense industry.

Speaker A: Absolutely. So I'm Case, I'm the lead on Pepper, which is an emission controller and an operator framework that was Created by Jeff McCoy at uh, Defense Unicorns. And so we've been maintaining it and building uh, features onto is a huge fundamental part of uds as the UDS operators built in Pepper and I got into Defense because I was actually at a previous company, I met Jeff and Rob and uh, some people where we were doing kind of some work in the public sector area and we were building container hardening pipelines and building applications to ensure trust and ensure artifacts could be trusted and they have software buzz of materials and are security scanned. And I realized that there is a lot of opportunity for innovation, like for people from the private sector to come help out in the public sector as well. And I just really like the mission. It's something that's critically important. And every day I go to work and it's not like I'm working because I want to get paid, um, I'm working because we're enhancing national security, we're enhancing our security posture and it's just very meaningful work.

Speaker B: So you weren't like prior military or prior civil service, you just sort of found the defense mission and fell in love with it?

Speaker A: That's correct, Yeah. I actually came from Red Hat and after Red Hat I worked at Solo IO on service Mesh and API gateways. And then I went back to Red Hat. But it was actually during my first couple years at Red Hat I ended up working a little bit in the public sector. It was supposed to be a one off. I was supposed to build a cloud native application kind of this one off time but, but I ended up getting like extended and extended and extended on this contract. And I ended up being out in Colorado Springs for like six, eight months right before COVID And that's where I met all these great folks at uh, Defense Unicorns. And I love their innovation. They're very much product led organization which is the best organizations. So yeah, I have a completely different background, no military experience, but I'm still learning the acronyms. So I've learned a lot since I've been here because I've almost been here for two years now.

Speaker B: Coming up, have you created any of your own acronyms yet?

Speaker A: I haven't, I haven't, no.

Speaker B: Okay, so that's, that's goal, goal for you. You, you mentioned pepper and UDs and we actually don't talk about the products as much on the podcast as I think some people might expect. So can you just give like a quick, I'll start with a quick overview of like what is UDS and why does it matter? But then really like let's into dive, dive deep into what is Pepper and why are you building it and why is it open source and all of that fun stuff.

Speaker A: Absolutely. UDS is a way to deploy very complex networking and infrastructure in air gapped environments so that you can meet a very, very high level of controls. And the beauty of UDS is the automation. So you know, let's say you have a control requirement that all network, uh, traffic must be encrypted, which is a very common control that must be satisfied. We deploy things like istio, we deployed things like keycloak for authn off Z. We have monitoring stacks that we deploy. So from a high level what UDS does is it abstracts the whole network and the whole infrastructure away from the developer. So the developer is only focused on their app because, because they are the expert in their app, right? And they have all the domain knowledge. So what we do is we provide the infrastructure. So now all of a sudden they have to deploy what's called a UDS package. It's a CR in kubernetes. They deploy a UDS package and as soon as they deploy that package they automatically have a global security posture across the whole cluster. They have uh, authentication and authorization so they can actually access it through keycloak and make sure the correct roles are required. They automatically can have service monitors and network policy which effectively the service monitors are scraping the application that they deploy. So it's collecting metrics really common thing for Prometheus. The most common Prometheus rule is called up. It's like is my application up? And if it's down. You can send a page, it's, it's called a. Yeah, you can send a page to yourself that says like, hey, my application just went down. And so imagine that you have 50 or 100 applications. You don't want to be doing all that by hand each time, right? And because it's going to take velocity, it's going to, it's going to introduce the risk of human error. So you want everything to be the exact same every time. Like if I deploy my application, I need network policy, I need istio, I need for this to be configurable so I can actually hit it from an actual like DNS entry. And I need authorization, I need authentication, I need my metrics. And I want a great security posture with istio. I don't want to have to think about that every time. So it's taking away the risk of human error and it's really increasing your velocity. And of course it works in the air gap. So like that's one thing that UDS is known for. It's a best in class solution for air gap and it completely takes care of the network and all the underlying infrastructure and then you just focus on your application. So therefore we work together and then you can actually get more done.

Speaker B: Why you talk about the air gap and you talk about not coming from dod. I guess I'm kind of curious. Are you surprised that the air gap matters as much as it does within the dod?

Speaker A: Um, I'm not. I'm, um, not. It's, it's also, it's also really important in banking, really important in data centers. It just makes sense to do things air gap, right. Like you, it's, it's the best way to kind of ensure that you have integrity. Right. So your application has integrity, that is, and your infrastructure has integrity. You, you never know. Like, you try everything, you try to, you try to apply all the security controls that you can, but there is like people could be sniffing on the network, right? They could, you could accidentally expose something that could lead to you getting hacked. And in the dod it's like the risk is so high, right? The uh, the like, it's just, it, you don't even want to take that chance. So you do want to do things in the air gap and you want to be as safe as possible. So safety is insecurity are principal concerns for the DoD. And so that's why we want to make it very easy for them. So they can enter, they can innovate in their space and then we can help them by Saying like, look, we want, we. You are the expert in your app and you want to develop features for your app. So therefore you can react to things quickly. Right. And m. Maybe you, you decide you need another app and then you can deploy another app. But that deployment of the software and the day two operations with UDS is going to be a very simple thing. So the UDS is going to take care of that for you.

Speaker B: No, that makes a ton of sense. Because if you spend a whole bunch of time, every time you write an app having to figure out all of those basic security requirements, uh, then 90% of the time is spent on that and it's something that's repeatable versus where I suspect a lot of DoD developers want to be, is building the thing that actually matters for their mission. I mean it all matters for the mission, but they don't want to think about security at every single layer, every single time, with every single app.

Speaker A: Absolutely. And imagine environments that are ephemeral environments by design. I need to have this environment up right now and then I need to have it down, up, down. Right. Maybe it's up for two days. It's not a perfect world. Right. Like sometimes you can have your apps running for years and that's like very much a lot easier to maintain. But if you have to come up with an environment very, very quickly, deploy the network policy, have your global security posture, make sure all these controls are met each time with velocity. That's another reason why UDS shines. Like, obviously it can do the long term stuff just as well, but because of all the configuration, because of all the automation that it contains, it really shines in ephemeral environments just as well.

Speaker B: That makes a ton of sense. So I'd love to pivot just a little because I know Pepper is part of uds. But what does that mean? What part? And, and why do we like, I guess, see it as a separate tool at the same time?

Speaker A: Yeah. So Pepper is completely separate from uds, but Pepper as an admission controller and an operator framework. You create your own operators in Pepper. And so Pepper and UDS is actually the UDS operator. So the UDS operator is domain specific for uds. It is designed to one, of course, create a global security posture across your whole cluster. So you don't want pods running as root. If your pod is running as root, you know what can happen? You could reverse shell out of the pod and get onto the node and now you have access to everything potentially. Right. Like this. These are things that potentially happen. So it's going to automate the global security posture. It's got mutating and validating webhooks. So it's going to say like, hey, this application's coming into the cluster. I see that it's not using any security context. How about I'm going to assign something that's sane, a uh, sane default that follows best practices. And so it can mutate that. But if there's already a security context there, then it's going to say, okay, I'm not going to change your security context. You know what you're doing. But then it's going to do a validation phase. And that validation phase is going to say if you are running as root, this is a huge security vulnerability. It can block it from coming into the cluster because you don't actually, you don't want to have trust. You want to have basically like zero trust. So you can't say, oh, the application developer needs to run as root. It would have to be explicit trust in terms of a UDS exemption that says like this application has permission to do this. So that's the validating webhook. The validating webhook is going to say you can't come in unless you meet said requirements. And that can be security posture. In terms of security contexts, running as root FS user invalid mount paths, like mounting on host paths of the node. Like that can be really dangerous too. Uh, but it, it doesn't have to be like only security, right? It can be resource requests and limits of pods, right? Like you can't say or you probably shouldn't say. It depends on who you are or what your, what your target app is doing. But you might not want to request 10 gigs of memory for every app because there's only finite resources in a cluster. So it can do anything. So the mutate can change things and it can add things. It can add same defaults. It can assign resources. And then the validate says, okay, this is the final stage. This is the object that's coming into the cluster. Will I let it into the cluster? It. If it meets the requirements, I will. If it doesn't meet the requirements, it's going to be rejected from the cluster. And I think we can go into a nice metaphor too about admission control. That might make it really easy to understand admission control for people that might not be super familiar with Kubernetes Admission control.

Speaker B: Yeah, that would be awesome. Because occasionally I know, not even just occasionally, I listened to this podcast long before I was a Defense Unicorns employee. And sometimes I'd Be like, wow, it would be really great to understand that. So if you have like an explain it like I'm 5 version, that would be amazing.

Speaker A: Absolutely. Kubernetes Admission control consists of two phases. A, ah, validating phase and a mutating phase. And the first phase is actually the mutating. So there can be as many mutating webhooks as you want, as you need as are deployed. You can deploy Pepper, you can deploy Hiverno, you can deploy opa. There can be as many mutations as need to happen. And then after all the mutations are finished, there's the validating phase and there can be as many validating webhooks and checks as need be. And so if I'm explaining it to a five year old or actually like a five year old that knows about the club, right, because this is a really easy way to describe it.

Speaker B: A hip five year old.

Speaker A: Yeah, a hip five year old. Maybe let's watch some Disney movies. And like there's a dance club that they want to go into, right? So at this club, you're outside of the club, right? There's a line to get in, there's the line of people to get in. And there's two types of bouncers. The first bouncers are the mutating bouncers. And so what they do, which they represent the mutating webhooks. So they're going to look at your outfit, right? So in this, in this club, right, they also like, they serve food, they have cake, they have drinks, they have ginger ale, they have everything. So the first bouncer, what this bouncer is going to do is he's going to check everyone's drink ticket, right? It's like, hey, do you have a drink ticket? If they don't have a drink ticket, he's going to give them a drink ticket. And this says, hey you, you can drink no more than five drinks, right? For one person. The reason that he's doing that is because he doesn't want the club to run out of drinks. And then everyone leaves, right? They're watching a show and everyone's thirsty and now there's no more water. They want to stay open. They want to make as much money as possible. So they're trying to make sure that everyone has enough. So he's going to give you a drink ticket. The next one is maybe this club has like a 3D screen, right? And the next bouncer is going to give you the 3D sunglasses. So now you have your drink ticket. Now you have your 3D sunglasses. And now the next mutating bouncer is going to check your headwear. Like, if you're wearing a hat or something that might be blocking people behind you, they're going to say, hey, there's no hats in the club. And they can just. They can say, hey, I'll save it for you, but I'm going to take your hat off. So now you have your drink ticket. You don't have a hat on you. Don't you have your 3D glasses. You seem like you might be ready to go to the cluster. So they can, they can give you things or they can take things away from you that you were planning on using. Now the next. Okay, so now you're done with the mutating clusters you have. What do you think you need? So now the next bouncers are the validating bouncers. And so first of all, they're going to say, hey, before I let you in, do you have the. You have the drink tickets, right? So you, you have your drink tickets, right? So now you can go in and drink. Do you have a hat on? No. So you're not going to be blocking people. Okay, perfect. Then they're going to say, like, do you have your 3D glasses? Perfect. And maybe they do one more check, right. That the mutating mouser didn't say. Maybe they say, hey, there's, uh, a. There's a requirement for, like, comfy pajamas at this club. Do you have on your comfy pajamas? If you do, go ahead. I grant you entry to go into the club. If not, you have to go back through admission. You have to go back through the line when you feel like you've satisfied the dress code and the requirements for this club. And that's what Kubernetes admission is. So when a POD is going through the admission webhooks to the mutate and the validating phase, it actually, or the POD or whatever Kubernetes resource, it does not exist in the Kubernetes cluster yet. It's not in etc. Only once it goes through admission does it actually exist in etcd.

Speaker B: One of the tools you mentioned a second ago is Kyverno. And I hear a lot like, why don't you just use Kyberno to do what Pepper does? So I'm curious, how do you distinguish. And are you competing with Kyberno? Are you complimenting it?

Speaker A: Yeah, Kyverno is definitely a great tool, and a lot of people have a lot of specific knowledge with Kyberno. And Kyverno is an admission controller. Right? It can do the validations, it can do the mutations. Pepper though, is not competing, although there is overlap there. But the projects have very distinct goals. Pepper is an operator framework, so if we had to compare it to something, it's a lot more similar to Operator SDK or Kube Builder. By the way, both of those do have Admission controllers as well built in that you can opt into. But Pepper, I would say Admission is a first class feature. It's a first class thought. But just as much of a first class thought is Admission is the Kubernetes watch. And so that's how you react to things that are existing in the cluster after admission. So you're basically what Pepper does is it allows you the ability to react to any change that occurs in your Kubernetes cluster. So what you're doing is you are effectively automating mundane tasks, right? So if we, if we are, we think your car is broken and every time your car breaks you kick the tire and then it like starts up again. You just this one thing you have to do, Epper is 100% equipped to do that. So it's almost like a consolidation of tooling at, at one point, right? Because the difference is epper, you have a whole programming language, a whole strongly typed programming language with a very obvious and simple interface. So really in Pepper, I can say, when a pod is created or updated, I want to mutate that pod. I'm going to say set label, Pepper has checked this. I'm going to set the label on it and then after the mutation, right, because it's a mission control, I can have a validate and I'm going to validate that the pod is not dangerous. It doesn't expose things, it doesn't expose security context. That could create a liability for me, for my cluster later. So it's the API that's different. And another thing, like that's standard. But another thing we could do in Pepper is say when a pod is created, I want to replicate this pod in AWS US East 1, just to replicate it for redundancy. And then I also want to replicate this pod in gcp. So I want to go across clouds. So, so pepper is 100% cloud agnostic and it can really, it really doesn't have an opinion on what you do with it, which cloud it runs on. It's just, it's a Kubernetes operator framework and it's an emission controller. And really short summary, the way it's an Admission control or the Kubernetes operator is because it has A Kubernetes informer built in a custom Kubernetes informer that can handle massive massive scale. So basically what a Kubernetes operator does is it calls up the Kube API server. It says, hey, look, I need to be informed anytime this deployment changes because I need to react to it. So anytime this deployment changes or anytime I see this package because we have an extraction. UDS operator uses an extraction for applications that are called packages. So hey, Kube API server, every time someone deploys a package, let me know. I need to know the lifecycle. Are they creating the package? Are they updating the package, Are they deleting the package? The reason I need to know is because now I need to deploy network policy. I need to do an API call out to Keycloak to configure that with the client id. So I need to make sure my application has access to Keycloak. Um, I need to go ahead and generate service monitors so I can collect metrics for my application. And conversely, another example is around network policy. If we are, we have very strict network policies because if you're, If a, if a service is talking to a service, we have to be intentional about allowing that. If not, there is plenty of things that could go wrong, including like trying to do a denial of service attack. Many things, right? You can try to pound it and take it down with like a 503. So it's. That's why we have very strict network policies to say if A is talking to B, then they have to explicitly be allowed to talk to each other. And that's configured on the UDS package spec. And that's. And so if IP changes, right, if there are IP changes, if a node is rotated, we're actually watching things like endpoint slices in Kubernetes. So we know like which IP ranges are allowed to talk to who. So that's why like that secondary non admission aspect is just as important as the admission. We have to be informed on changes. So it's really like you are building out your operational logic so that it can be. You can deploy 100, a fleet of 100 apps and you can do it across many clusters just as easily as you can do one. And it's going to be the same every time. There is going to be dynamic policy updates based on environmental change. And so we are 100% adding automation to mundane tasks with centralized guardrails from a singular point.

Speaker B: You mentioned doing that in Typescript. I know. One of the questions I get a lot about Pepper is Why do you guys use TypeScript instead of go?

Speaker A: Yeah, that's a great question. So the reason that Pepper was created in TypeScript is because Pepper's goal is to be accessible for everyone. So I actually worked in OperatorsDK a lot when I was working at Red Hat. Uh, I built a lot of operators. But with Operator SDK, you need to know the Go Kubernetes clients with Pepper's interface, where we call it the Fluent API, it's designed so that anyone can pick up Pepper and create an operator. So it's designed for Velocity. But it's funny you mentioned Go because our company does something called Dash Days, and I've written Pepper's Informer and Go three times and compare them side by side. The last time I used nats, which is, uh, a cloud native messaging system. And so I had a NAT service and then I had my Informer. I was using a Kubernetes dynamic shared informer beside a Pepper in a sidecar. And the dynamic shared informer was sending events to nats. And then NATS was. And Pepper was subscribed to the NATS server and then the NATS server was sending the events back to Pepper. So it was like a little triangle. So they'd be informed on the subscriptions. I was able. They used effectively, very, very similar resources in terms of memory and CPU. They were both handling 6,000 requests per second without drops over a good amount of time. And so at one point, I kept making the stress tests harder and harder and harder and harder. And then when it was like, ridiculous, when I was like, whoa, whoa, like if this happens in an actual Kubernetes cluster, the control plane is just going to keel over and die, right? Like, it's, uh. We. There's. There's a. There's like a finite amount of resources in a cluster. This is never going to happen. But I saw the performance the same. So in terms of performance, it's extremely performant. And another thing, right? So why do it in TypeScript? Well, we talked about the Informer side, right? It's just a call to the Kubernetes API server. So this means there is very minimal latency, right? Like it's not going to be a lot more efficient than doing it in Go when you're going like one hop, right? You're just talking to Kube API server through admission. Kube API Server is just sending you Admission review objects and then you're sending them back. It's like the latency is minimal. Pepper literally only talks to the Kube API server. So different strokes for different folks. But the, the real, the real powers with the fluent API so that anyone can write with it. And, and that's what we think. We think if you pick up Pepper today and you need to, I don't know, do your mundane, your mundane task, you need to make sure pods aren't running as root and if they are running as root, you want to reject them. You can say when a POD is created or updated, then validate and you look at the pod and you say like if the pod spec securitycontext run as root is or run as root is true, when run is roozer run as non root would be false. Or if run as user is equal to zero, like integer zero, that's like running as root. So you can say, okay, I'm rejecting this because uh, it's violating security context. It's creating a liability in my cluster and I want to enforce a global security posture across my whole cluster. So my devs and work on their development environments and they don't have to have like their YAML files with like 300 lines, right? It's going to avoid configuration drip too, right? It's like, hey, delegate this. And this is cloud native software. What does ISTIO do? It delegates the network. This is delegating the security context, uh, the security posture. So it allows you to focus on what's really, really important. And it still gives you the benefit of 100% security posture. And then you can layer on your additional rul rules automate. But you know, another thing you can kind of like do is you can have your standard rules that don't even have to do with kubernetes, right? Like maybe, uh, when I see if I have like some CR for a database, I want to actually like replicate the database. I want to like make this database available in another zone. And then because it's, it's typescript, right? You have the whole power of that just like trigger a database migration or when a POD is created. I want to do a dad joke. I want to fetch a, uh, dad joke API and write it into a config map and then people will laugh at it later.

Speaker B: I don't recommend that, but I, but I love the possibility. So is it fair to say that part of the typescript choice comes down to wanting to make sure that you're not limiting your Pepper users to people who are already familiar with like go and platform engineering?

Speaker A: Yeah, yeah, we want someone that has a task to not have to go and study kubernetes client Go, which is like the underlying library. In order to know how to do this, we want to make it so obvious. So if you have a task, you know how to do it, you can easily automate it in Pepper, you can automate it easily, you can test it easily. And that's like, why we built the Fluent API with like the human in mind first. And it also has intellisense. So when you're writing something in Pepper, it's going to suggest what you can write. And that's very intentional. So you say when a pod, it's going to say is created or is created or updated or is deleted, and then it's going to make suggestions. So it's actually a very easy framework to do to get off the ground and running.

Speaker B: So I want to pivot a little bit to one of the other questions I get or one of the criticisms I hear of Pepper. And I'd just be curious your thoughts on it. And I guess I'll start with like, can you give just a brief, maybe explain it like I'm 5, maybe like I'm 10 of like, what is GitOps? And then the question I get all the time is like, how does Pepper work in a GitOps workflow?

Speaker A: Yeah, GitOps is when there's. GitOps is when you have parity in your infrastructure in terms of your source control management system, so like your GitHub, and then in an actual physical running cluster. So what a GitOps server does is the first thing you do is you say, hey, GitOps server, look at this repository in the folder that says kubernetes. And I have like a deployment, a, uh, service, a POD or whatever. I have some things that I want you to deploy. And what the controller does is it reconciles constantly to make sure the desired state, which is the state of the repository, meets the actual state, which is the state of the cluster. So if I delete the service, the GitOps server should be configured to say, hey, I just noticed you're missing the service. So it's going to redeploy a service that actually has kind of like almost nothing to do with Pepper. It's going to be like. But we'll call it the UDS operator, right? The UDS operator deploys what's called UDS packages, and the packages are, uh, what represents the applications. So if we were going to use Argo CD or some GitOps tool or Flux, we're going to point the GitOps server still in a repository, but we're going to point it to the packages and then so what's going to Happen is the GitOps server is going to be reconciling the desired state versus the cluster state. And it's going to say like, hey, your actual state in the cluster does not meet your desired state because this folder in the repository has uh, three packages and it looks like it's missing the blue package. So it's just going to deploy it. So they play like this is how operators are when you work with GitOps, right? You have to tell them what to do. But uh, the kind of benefit, I think the thing that a lot of people would be interested in hearing about this thing, operators versus GitOps, there's GitOps is really good, right? And it's a good way to get up and running really, really quickly. But this is what, what an Admission controller does is it allows you consistent configuration management so you're not having to. You're going to have a global security posture, right? You can have very simple, uh, Kubernetes deployments if you want. And then you are basically delegating the security context to the Admission controller. So like I want to deploy hot info, a very simple app. I want to deploy an app that says hello world. I don't want to have to go in and say like, okay, uh, let me look at my security context. Let me make sure I'm not running as root. Let me make sure I haven't enabled some like something on the network. Like it's called like drop capabilities, things like that. Make sure I'm not like adding capabilities or dropping capabilities. I want my YAML to be expressive, I want it to be simple. And if I have created an operator, I know the operator is going to take care of the security posture and the network resources. So now I'm pointing at my simple app, my deployment, my service, my service account. And it's making sure. And the GitOps controller is ensuring that those objects exist in the actual cluster all the time. And then once they do go into the cluster through admission, it's going to assign the same defaults that I've decided so that now I do have my global security posture.

Speaker B: No, that makes sense. So I think what you're saying is they're completely complementary, but potentially using Pepper simplifies a lot of the actual deployment of an application. Is that a fair summary?

Speaker A: Yeah. In terms of your configuration management, which is really closely tied to GitOps, like, we want to enhance our like golden path in code. Like we don't want these snowflake deployments in YAML and consistencies like, hey, this team over here and the green namespace has a lot more experience in Kubernetes, whereas the blue team is like, they're great at application development but they don't know about Kubernetes. So now like there is very inconsistent configuration management in terms of what those actual objects look like, right? This team's thought of everything. This team's thought about basically nothing other than their application. But now they can both go deploying and their security posture will look the same. And this kind of gets in a little bit to, to multi tenant safety. Like if you, if you want to, if. Since we're talking about two different namespaces, we call them Blue and green, but let's say like Team Billing and team, Team Sales. Maybe Team Billing has a namespace, they have their application there. Sales has uh, a, uh, namespace and they have their application there. But what if the requirements for sales have much less security restrictions, right? Like, what if there's like really nothing sensitive whereas billing has like customer identifiable data. So this is like a great thing about an admission controller and an operator builder. You, you have multi tenant safety and that uh, you choose your own rules, right? Like you probably want a global security posture that says like you have to meet at least this threshold in order to gain entrance to Kubernetes through admission. But after you're in it, right? Or like after that, after the global layer, it's like which namespace is this thing going to. Because maybe we can have a slightly more relaxed security posture for this namespace, whereas this namespace no, like we must have a crazy rigorous security posture. And that's all things that you can program into your operator.

Speaker B: No, that makes a ton of sense. I'd love to talk a little bit about why. Maybe not like the, if you want to get into the philosophy, uh, of it, that's great too. But like why Pepper is open source and like what that means and whether you think that's always going to be true or if that's something that might be temporary.

Speaker A: The easiest thing is Pepper will always be open source. And the reason why it's open source is because frankly, open source software, when your software is open source, you expose the application or the software or the platform, whatever it is, to exponentially more eyes and more eyes over time. And then more people start adopting it and using it and saying like, hey, you know what, I do have this simple thing I always have to do in my cluster. Maybe I try Pepper for that, right? And then they do it with a simple task and then they Say, hey, you know what? It would be great if Pepper could do this thing and they put in a feature request and then we develop that feature request, or they develop it and they submit a PR to Pepper. And now Pepper as a whole is better because now you're using it, I'm using it, they're using it. Everyone's. The more people that use it, the better it's better in terms of security because there's more eyes on it, there's more people kicking the tires. There's more people, um, in software, everyone has different expertise. So it really gives the exposure to a lot more people to say, to find things that might be broken too, right? Like, hey, we use it for UDS Core. But UDS Core never thought about doing this thing that this other company, abc, is doing, is using it for, and they just found something that maybe is broken or maybe could be better. So it's a way of working together. And also we're proud of the products, like, we're proud of the work that we've done. And we want it to be free and open source and we want people to build with it and give us feedback and, uh, create feature requests and everything.

Speaker B: So how can people contribute to it now?

Speaker A: Yeah, it's really simple. Our, our repository is GitHub.com defenseunicorns pepper, you can, There's a contributing guide there. So you just basically read the contributing guide and that's. It's almost too boring to even say on a podcast, but you basically, you look for an issue or you create a new issue. You say, like, I want Pepper to do this, and you put in a feature request and someone reviews it and we say, you know what, that's a great idea. Do you want to do it or do you want someone else to do it? Do you want someone on the team internally to look at it? And if the person wants to develop it themselves, they'll say, I want to do it. So we'll say, all right, that sounds good. And we will assign the issue to them. And then they'll basically, they'll clone the repository, they'll create their own branch, and they'll go working on it and working on it, making sure all the tests pass. And then when they're getting ready, they're going to push the, uh, feature through a pull request to GitHub, and we're going to be reviewing it and we're going to say like, okay, this looks good. Maybe this could change. We're gonna, like, there might be a little bit back and forth, but We. We genuinely, like, love contributors, and we try to make it easy for the contributor process. So we just want to make sure it's gonna pass tests, it's not gonna break anything. And then if there's a little cleanup, we'll probably just accept the PR and say this looks good, and then, like, immediately open up a new issue. And, like, we don't like to, like, nitpick people for all these little things, so we just open up a new issue after and fix anything that we would need to.

Speaker B: So one of our other tools at Defense Unicorns is Zarf, which we donated to open SSF last year. We haven't donated Pepper yet. Is that part. Is that. Is that in your plans? And if so, like, what. What do you still need in order to be able to donate it?

Speaker A: Yeah, donation is absolutely. It's a goal of ours. And what we need right now is we need to be. There needs to have more than one company contributing to the product. The reason is these foundations, they are trying to protect the applications that have gone to foundation so that if one company is. If company ABC has this great product that everyone is using, and then they say, like, hey, we're not going to fund this product anymore. And now these 100 companies that were relying on it are now without a product. So you have to have two companies regularly, regularly contributing to the product. And that's actually, like, the main thing. Right? So we want to look to get it donated maybe around Q3 or Q4, but we're just. We're in the process now. We're making sure our docs are great, so it's very easy to contribute. We're doing. We're in the kubernetes slack at Pepper. So we are very, like, social in the slack. Like, we write Pepper tips weekly. We write our release notes. We're there to help people, like, if they have questions, if they want to do things or if they want to contribute. So we're just trying to grow our community right now, make sure our docs are great, make sure our process for people to contribute is a good process and something that people want to do. Like, we don't want to make it a pain at all.

Speaker B: Uh, that makes a ton of sense. What have I not asked you yet that you wish I would have?

Speaker A: One thing. I think it's. We did a metaphor about admission. I think a metaphor about what an operator is would be a really nice addition to. We are in. Let's say we work for the Department of Transportation in Texas, right? So there's we have a lot of highway to cover. So we have this new incentive, right, that we have to change all the road signs that are. Well, we have to change all the road signs to be in italics instead of just regular, right? It's got to be in italics now. So what's the process? We're going to have to one print, maybe like tens of thousands, if not hundreds of thousands of signs everywhere. We are going to have to go out to every physical sign, human labor, right? Unscrew the sign, install the new sign, go to the next sign. That would be very, very manually intensive. Work by hand with like maybe you accidentally install a sign like crooked or something, or you install uh, a sign backwards right after you've installed 300 signs, like you might be pretty tired. It would be easy to accidentally make a mistake. So the, the operator side of that is now imagine that these signs were digitized. So all of a sudden they can be, they can be any color that you want. They can be italics or non italics. They can be in English, they can be in Spanish, they can be in French, they can be in whatever language. All we have to do to now change thousands and thousands of miles of signs is simply press a button and now we're done. So an effort that is manually intensive in that it's prone to human error now can be completely automated away. And so now, right, imagine that we had done all that by hand. And look, this is the real world. Now they're like, oh, I like the italics, I like the italics, but I think it should have actually been bold. Now we've got to go back out. If we uh, have already have our operator in place, these day two style operations are now extremely simple and we can ship them out immediately. So we can do hundreds of thousands of things. Network policies though, in the cluster, network policies, configure, keycloak, configure Prometheus. We can do anything very, very simply now because it's going to be the same for everything.

Speaker B: No, that makes a lot of sense. And I think we should digitize the road signs because that might help.

Speaker A: Yeah, absolutely, we should.

Speaker B: And also italics on road signs is a terrible idea, but I totally get. Is there anything else you wish I would have asked or anything, Anything you want to make sure that we cover?

Speaker A: No, I think, I think we've really covered it.

Speaker B: So I just have a couple fun questions, or at least I think they're fun. And, and then we can, we can wrap up. So I'll start with. And this one might be way too Easy for you. What is your favorite open source tool?

Speaker A: Kubernetes.

Speaker B: Okay, if you said Pepper, I was gonna. I was gonna let you, but I would, I would. I think Kubernetes is a better choice.

Speaker A: Well, Pepper doesn't exist without Kubernetes.

Speaker B: That's a. That's a fair point. Is a hot dog a sandwich?

Speaker A: I don't think so.

Speaker B: Why not?

Speaker A: Because you don't have to have, like, a bun or bread.

Speaker B: Okay, interesting.

Speaker A: It's just like the meat.

Speaker B: So a hot dog with the bread, Is that a sandwich?

Speaker A: It can be considered a sandwich. I let it pass for a sandwich, yeah.

Speaker B: Okay. And then my favorite is just like, what is a book that you recommend that either everyone read or every developer read or you can couch it however you want, but what book recommendation do you have for this podcast?

Speaker A: I think Atomic Habits is one of my favorite books because it's. It's like really lessons that you can apply. And I felt myself becoming, um, kind of more efficient as I was reading it and creating these positive habits and then sustaining the habits and kind of adding on more and more and more. So I love that book because it gives you something that you can apply as soon as you start reading it. So by the end, you have all these great ideas and great habits that you can apply and then hopefully that you've been applying while you have been reading the book.

Speaker B: That's fantastic. I appreciate it. Thank you so much for your time today. I. I really appreciate having you on and. Yeah, uh, that's. That's all I've got.

Speaker A: Appreciate it. Rebecca. Yeah, it's been a pleasure to be here. Thank you so much.

Speaker B: Thank you for listening to this episode of the Defense Unicorns podcast. If you enjoyed the show, please please share it with your network and rate the podcast five stars. Wherever you listen, tune in next time for the latest in DevSecOps.