Journey to Tech, Iac, Terraform, Cybersecurity, and More with Ned Bellavance

Speaker A: Hello and welcome to Ops and Hops, the podcast for those who like beer but have to work in tech to afford it. Today I've got Ned Belavance, founder and curious human at Ned in the Cloud. Welcome, Ned.

Speaker B: Hey, Derek. How you doing?

Speaker A: Doing pretty well. Thanks for joining. I know we've talked at a lot of conferences and, you know, we've had a lot to discuss about Hashicorp and all of that, but it's great to finally have you in an official capacity. So what did you bring to drink today?

Speaker B: I, uh, really like Stone Brewery, and so I have their fear movie Lions Hazy Double ipa. I went with a smaller can because I don't want this to get, uh, too ridiculous.

Speaker A: I mean, that's the point, right? Yeah, no, I've had that beer before when I used to live in la. I got it a lot, and I gotta say, it's a. It's a really good beer. I've got the. Once again, I. I've kind of cheated lately, honestly. My fridge is so full of beer, I've got to get rid of them. Um, I've got the Working for the weekend Double IPA from Spiteful Brewing. It is a local brewery here in Chicago. It's also a little bit of a spicy beer, so it's going to keep. Keep things interesting, I imagine, as I, uh, as I get through about half of this. But we'll see where we are in the conversation whenever it starts to hit.

Speaker B: Yeah, it's interesting because Stone Breweries from California, I believe. Um, and I born and raised in Pennsylvania, so why would I even know about Stone Brewery?

Speaker A: Shouldn't you only be drinking Yingling?

Speaker B: Oh, God, don't insult me. That's not nice. Um, but this actually goes way back to the early days of listening to podcasts in, like, the 2005, 2006 era. So, I mean, we're talking, like, beginning of podcasting. There was a show called Slice of Sci Fi, and the hosts were all from California, and they made a point of talking about beer a lot on the Slice of Sci Fi podcast. And I don't quite know why, but they would talk about Stone Brewery endlessly, and I was desperate to try it. And it wasn't until many years later that it actually made its way across the entire country for me to drink. And the wait was worth it, I guess, is what I'm saying.

Speaker A: Well, that's kind of funny because Yingling, um. Although, again, not a fantastic beer, but better than your Bud Lights and your Budweisers and things like that.

Speaker B: Sure.

Speaker A: And down In Georgia, when I was, where I was living, uh, where I grew up, we really wanted Yingling. And one of the states near us would get it, but Pennsylvania just would not distribute it anywhere.

Speaker B: Hmm.

Speaker A: And so it got to the point there was actually a liquor store down the street called Bootleggers. Fancy enough. And they would smuggle in Yingling across the state line and they, they have it in the coolers. That was the only place in the state you could buy it. So I started drinking it a good bit. And then finally as I got into IPAs and things like that, I started to realize that maybe it wasn't that fantastic of a beer, but it's not bad for a hot day around the grill, to say the least. So.

Speaker B: Uh, yeah, yeah, absolutely.

Speaker A: Yep. Well, that's great. So anyway, Ned, I know you do a ton of DevOps stuff. You've got, you know, your own shows, you talk about it a whole lot. But obviously there was some sort of story that got you to here. You probably didn't just fall out of the clouds and start talking DevOps. So share a little bit about that journey that got you to where you are right now.

Speaker B: Oh, geez. Yeah. I mean, we can go back to pre me working in it. I've always loved computers. That's been a thing my entire life. Uh, my parents bought a Franklin 1200 when I was a lad, which was an Apple Iie clone. That was back when you were actually allowed to create Apple clones for that weird period of time. And so that was my first computer. And ever since then fell in love with technology and computers. That was always a thing in the back of my head. Uh, I tried going to school for engineering and that didn't work out because engineering, apologies to engineers out there. Engineering is kind of boring. You don't get to talk to people much and actually like talking to people. So pretty quickly realized that engineering wasn't for me. Ended up just getting, uh, uh, an associate's degree. And at the same time, I had been working retail part time for years, just as, you know, some extra pocket change support myself. And I had a chance to be promoted to a, uh, store manager position at Hot Topic of all places.

Speaker A: Yes.

Speaker B: And so I decided to take the position of a full time manager at Hot Topic. And my district manager was a big fan of me. Like, I worked hard, I did the thing, and she thought there's a potential for you to move up and become an area manager and then a district manager. And I thought that's where my career was headed. That was My trajectory. And, um, it wasn't until about a year before I quit that one of my sales associates looked at me and he was like, dude, what are you doing here? I know. Not that you can't be smart and work retail, but he saw that I had potential to do more and that I had interests that were far afield of, you know, just working in a retail store. He's like, why? Why are you here? Why are you doing this? And it really hit me because he was like an 18 year old about to graduate high school and go actually join the Marine Corps. And he had ambition and he saw that I should have ambition. And that was the kick in the pants that I needed to start looking elsewhere. And about a year later, I landed my first help desk job in it. And that was the beginning. Um, I got to work with computers and I got to solve problems. And I really like solving problems. And that is what help desk is designed for. Every phone call you get is going to be someone with a problem and it's up to you to either try to solve the problem or work with them and somebody higher up to figure out what the problem or the solution is. And, um, that was my entree into working in tech. And so I kind of fell into an ops role pretty naturally after that, because Help desk is all about solving problems, but at a smaller scale. And then ops is about solving problems at a bigger scale, probably with servers or something. And yeah, then I worked ops for a number of years. I was, uh, sysadmin, network admin for a small organization. And then I moved to a university where I was doing that for a while. And then I had the opportunity to go do consulting and got into consulting and did that for about seven years. And seven years in consulting is like 20 years in a regular job. I mean, you know, it's. You just learn so much because you have no choice. And so after pumping my brain full of knowledge for seven years, I decided to take a break, do something different. And that brings me to my current iteration where I create educational materials for a technical audience for a living.

Speaker A: Yeah, uh, that's. God, that's almost identical to my journey, actually. Yeah. Went to school, got bored. It was actually a trig class that finally broke me. I think. I was like, how does this have anything to do with tech? Obviously I had no idea that AI was going to be this big now. But, uh, you know, so there was that, um, had an Apple Iie actually when I was growing up. Growing up, um, yeah, school didn't work out Worked for Best Buy. Manager promised me, you know, here's a path all the way to the top.

Speaker B: You should just.

Speaker A: He literally told me, he's like, you should drop out of school because I make plenty of money and you'll, you'll run one of these stores one day, blah, blah, blah. Little did I know the $8 an hour they were paying me probably wasn't as much as I probably could have been paid at the time. But you know, young college kid, who knows?

Speaker B: Yeah, you don't know.

Speaker A: And uh, yeah, help desk, then consulting, then kind of moved through a few other things until I started creating educational content. So cheers to that. Um, so, yeah, so, you know, you saw school wasn't working out and all of that. You saw the retail, you know, you got the, the same, just like kick in the pants. It's like, hey, I should do something else. But I imagine that retail experience probably affected your abilities in the help desk immensely, tremendously.

Speaker B: Uh, in part because my first help desk job was for a retail organization. So now I was on the other side of the phone, essentially answering calls from people at the register or in the back office who were running into problems. So I had a tremendous amount of empathy for them. Like, I've been in that situation literally where the register doesn't work, I've got a line going out the door and I just need things to be fixed. So I felt that pressure. And also, like, back office software was terrible in the early aughts. I'm sure it still is.

Speaker A: It's still a nightmare.

Speaker B: Hasn't changed. And so, yeah, the nightmare of dealing with back office software and trying to fix weird payroll things or inventory problems, I, uh, understood all of that. And so when someone would call in, I wasn't dismissive or I didn't know where they were coming from. I was able to empathize and go, okay, like, I gotcha, uh, we're going to get through this together. And the other thing that I learned is how little most people understand technology. And that's not a knock on them, that's not their focus. If you're working at a Hallmark store selling cards, you don't care what a USB versus a PS2 port is. But I need to know that and I need to express that to you. So understanding what the level that people were coming in with and how to educate them enough or uh, prepare them to a certain degree to help me with the troubleshooting, that was something I really enjoyed. And I think that led into the fact that I really like teaching 101 and getting started stuff, because people are coming not knowing anything. Then I remember what that's like, and I understand sort of the fundamentals that I need to start with and the concepts that I need to work with to build up that foundation. And then at a certain point, they don't need you anymore because they have that foundation. They can start building on their own.

Speaker A: So one thing I found when I was kind of coming up in the consulting world and even the help desk world and everything else was. And this may just be my overbearing anxiety over the course of my life or whatever, but I always got compliments on my sense of urgency. Even if it was an easy problem to solve, I. I owned the problem. And it sounded like, you know, when somebody would call, they're like, oh, wow, you seem like you really care about solving this problem for me. Did you find that your experience in retail may have given you some of that sense of urgency as well? Or are you more of a calm, cool, and collected troubleshooter?

Speaker B: I'm a pretty calm person in general. That's been remarked upon by many people. It's like, you seem very chill, and I might be, like, boiling over, um, on the inside, but I tend not to show it. Um, so if you got me, uh, in the help desk call, you would have just a calm, patient person on the other side who's willing to walk you through the process. Not that I'm going to slow you down, but I'm not going to heighten your level of anxiety that you're already experiencing that caused you to call help desk up in the first place. So I think that calm demeanor helped a lot to just put people at ease when they called in, and then we get the, uh, issue resolved, and then they, you know, were generally pretty happy about that.

Speaker A: Yeah, that's great. Yeah, it's always fun to hear kind of how people handle things. And I know overall, people in help desk a lot of times mirror the client, which is sometimes a good thing and sometimes a bad thing. You know, you probably don't want to start shouting back at them. But sometimes showing, you know, that you are involved helps somebody who's stressed out, but also someone who's calm. So that's. That's interesting. I like hearing the ways people solve these problems because even in, you know, managing these big consultant projects or even teaching people, learning from, like, help desk and things like that seem to help a lot. Help kind of get you in the right frame of mind for that. So.

Speaker B: Yeah, I think that's true of Any service job. And that's why I think it's really important, as you're growing up or whatever, to work a service job for a certain amount of time. Uh, because you get the perspective of the people who are serving because you yourself are now in service, but you also get the perspective of people who are kind to the service folks and people who are not. And so in my experience, uh, that meant that I will never be one of those people who is immensely rude to the folks working the service jobs because I know what a drag it is and how like, you've got enough on your plate without me being difficult. It doesn't mean like, be a wonking mat, but it does mean, like, start with empathy and that they're doing their best and they're stressed out and go from there.

Speaker A: Yeah, that makes perfect sense. So in the consulting or help desk or, you know, any of that type of service customer facing type stuff, did you ever have a time where you were just absolutely wrong? And I mean, I know you did. Uh, we all do. We all do. Whether you admit it or not, we all do. But, you know, can you think of any of those times? It kind of. Do you have any advice to someone who is coming into, you know, again, their first help desk job and they're just terrified, you know, they're sitting there every time that phone rings. Their blood pressure's through the roof. You know, is there anything you can, kind of any advice you can give to those people to help help them if they are ever just wrong, if they just make a mistake?

Speaker B: Well, I'd say the biggest piece of advice is if you're wrong, acknowledge you're wrong as quickly as possible. And like, don't. I mean, we've all watched sitcoms where somebody lies and then they lie to cover up the lie and cover up the lie. Like real, real life is kind of that way. If you make a mistake, covering it up just compounds the error. So if you were wrong and you said, hey, I need you to unplug this, and you accidentally shut down the server for the entire store. Random example. I know. Uh, and you'd be like, oh crap, I think I just shut down your server. I'm so sorry, let's see what we can do to get that fixed and you back up and running and then we'll deal with the next thing. So like, owning up to it as soon as possible is good. And also not being afraid to ask for help. Now, I know some people feel more comfortable asking for help than, than others. And some people are worried that asking for help is a sign of weakness or showing that you're unable to perform your job duties. But in my experience, if you ask for help and it's rejected or you're looked down upon, that's a terrible job or environment. You need to get the hell out of there. Um, you should be able to ask for help when you genuinely need it and receive it without comment. Like, yep, I'll jump in, I'll help you out. And that was as I moved up from help desk to like second level or third level help desk. That's what I was generally called upon was this person who was very new, only knew what was in the solution manual. Basically, they hit a wall and then they would escalate and I would jump right in. And I'd never be like, oh, you're an idiot. I'd always be like, okay, did you do this, this and this? Okay, perfect. I will help you. We'll get this solved. And that's, that'd be my biggest piece of my two biggest pieces would be own up to the, the fault right away. And also, if you're in over your head, ask for help. Don't be afraid to ask for help.

Speaker A: So we've taken a lot of, um, liberties whenever you, you just explained that one of the big words that I heard was, when you genuinely need it, can you add a little more color on what genuinely is, like, how much should be expected of someone before they actually reach for help? You know, where do you set that threshold and how do you know that you've done enough to go and ask for help without encountering one of the graybeards that might actually give you flack for asking for help?

Speaker B: Uh, you know, I did work in an environment for a while where there were a few graybeards, and if you came to them for help and you hadn't done what they thought was the requisite amount of research and self help, then they would shoo you away and just point you in a general direction. So if you haven't read the manual, basically they don't want to hear from you. That's not always a helpful attitude, but I kind of get where they're coming from. They want you to be able to help yourself. I think if you genuinely need help, it's because you've already exhausted the resources that are in front of you and you've realized there's a time crunch on this. There's a limited amount of time I can flail on my own before I need to bring somebody else in. To get an issue resolved. I can't just let the house burn while I try to figure out how to turn on the faucet, right? Like, no, grab firemen, get that faucet open, let's put this blaze out. Uh, so if it's that kind of situation, you probably want to ask for help a little sooner. If it's not a time sensitive issue, then personally I really enjoy learning and researching things on my own. So I'll bang my head against a wall for a while before I'll put my hand up and ask for help. Because for me, that's part of the learning process. I have to get a bunch of things wrong before I get it right.

Speaker A: Fantastic. We've got our quotes for the show. I appreciate that. That was, that was really good. That was, that was fantastic advice. I think that's going to help a bunch of people. So now you are really diving into the infrastructure as code world. Was that a natural progression, just the easiest route for you, something you worked on a lot in consulting, or is it just something you just genuinely enjoy?

Speaker B: Well, I genuinely enjoy it now, but it was certainly born out of consulting and it's not the first time I've had to code anything. I, uh, did eventually go back to school and get my bachelor's degree in computer science. And so I had to do a, a fair amount of programming, especially for the Capstone project that was c and net 1.1. To give you an idea of the era we're talking about. I'm so sorry everyone who had to live through that with me. But anyhow, um, coding was not totally foreign to me, but I hadn't done a ton of it. Uh, I really fell in love with Powershell when that came out in exchange 2007 era because I really batch scripting. I lived in a very Windows centric world. Batch scripting was terrible. PowerShell was like, Here's a way to make it better. So I really embraced PowerShell for both exchange and just general administrative tasks. So I got very comfortable with scripting, writing pretty complex PowerShell scripts, even ones that had modules and invoked functions and all that kind of stuff. I started writing pester tests at a certain point. Um, yeah, I know, I got a little extreme.

Speaker A: Wow. So were you exchange 2007? Remind me, was that, uh, pre or post, like verb noun in PowerShell for exchange? Because I know after. At first it was pretty much like a very low level access to the Exchange API. And then after that they abstracted it a little bit more and it's like get mailbox instead of like before. Like it was really nasty to get a mailbox. You had to add a bunch of extra stuff.

Speaker B: It was still a little nasty when I started with it. Exchange 2007 was the first version of Exchange to support PowerShell. And it was when they sort of deprecated the administrative console that worked with exchange. Uh, 2000 and 2003. Some tasks just couldn't be done in the UI. You had to use PowerShell for them.

Speaker A: I do remember that. I think I started with either 2007 or was it 2010 maybe? I remember there were a lot of things that we had to do and I ended up. It was my first programming, uh, experience also actually I programmed a text user interface for PowerShell with Exchange because I was so tired of creating away messages and doing all this stuff through the slowest UI on the planet. It's like 50 clicks just to go and change one admin's mailbox or one VC guy or whoever we're dealing with that day, their mailbox. And it got to the point where I just created this tui and it's just like, all right, press two for away message, press three for the user, type in the away message and hit Enter. Like that was it. And it was really fun to build. I do say that I probably stopped a little early because I quit that job. But it was really eye opening, you know, kind of seeing all the structures in programming and trying to figure out how to make the code more dry and all of that type of stuff. And I actually developed a love for text user interfaces after that. And I go after them anywhere I can find them now.

Speaker B: Yeah, I hear you. The web ui for exchange 2007 was pretty bad. Uh, 2010 was better. 2013, they got most of it ironed out. But that was because Exchange Online had been progressing rapidly at that point. And so everything that happened in Exchange Online made its way down to the on premises product. Uh, there was a middle ground though,

Speaker A: that was a disaster. The docs were always hard to find the right things. It's like, oh, are we talking about365? Are we talking about, um, on premises? And then it got to the point where it was so split brained that you just wanted to quit, which I did.

Speaker B: Yeah, no, I hear you. Uh, so a lot of my early consulting was Microsoft based projects where I was doing exchange 2007 upgrades to 2010 and then 2013. So I got to know the ins and outs of, uh, Exchange intimately. And there was a point in my life where I Probably could have sat there was like an experts exam for exchange. I was probably close to that level in terms of my in depth knowledge to the behind the scenes of exchange. A lot of that has fallen out of my brain since then because I don't work on it anymore and I'm happy about that.

Speaker A: Yeah.

Speaker B: But at the time, yeah, I was really deep into database availability groups and do you go with a JBOD or you hook up to a san and then there's all the stuff with the ese utils like it's just. Let's not worry about that.

Speaker A: Yeah, you're bringing back some really bad memories now of seizing of a uh, of an 80 box that I found in a closet one day. It was running uh, it was running the time server also. So we're wondering like why all of a sudden the entire network is having all these massive problems and it turns out there was just this server sitting a closet somewhere that somebody did not fully deprov and ended up having to my first production Fizmo season which was quite exciting as you probably know.

Speaker B: Look at you.

Speaker A: I know growing up it was the worst.

Speaker B: Oh my goodness. Yeah, yeah. I have, I won't say fond memories but I do have memories of doing all those kinds of things with, with Microsoft products which actually don't touch Windows very often anymore aside from being like my daily driver for my desktop. Um, but you'd asked about infrastructure as code, so in the world of consulting I did a lot of PowerShell because there's a lot of automation to happen. And then around 2015 or so started being asked to work on AWS and Azure based projects. And so my first encounter with infrastructure as code was cloudformation. Um, another consultant had built out a prototype environment for a healthcare company using cloudformation and he wanted me to production ify it if you will. And so I spent uh, about four or six weeks just working through JSON because this was pre YAML support. So I just thousands of lines of CloudFormation JSON and if you know anything about cloudformation, you know it doesn't have a whole lot of functions. So lambda function for everything coded right in there.

Speaker A: Yep.

Speaker B: Um, it was a nightmare but it was an eye opening experience because I could provision an entire environment including domain controllers and SQL servers and web servers and everything by just typing a command and letting the template do its thing. And yeah, it took like an hour and a half but at the end of the hour and a half I had a fully built environment that I could test against and the healthcare company was very happy because they needed those kinds of environments to spin up and tear down. So that kind of put it in my head that, wow, infrastructure as code is really powerful. And it also introduced me to the idea of a declarative approach to deploying infrastructure versus the imperative approach that I'd been using with PowerShell. Then I started working on Azure projects and started using ARM templates, which at the time were like a, uh, kind of a step up because they had more function support, but kind of a step down because ARM templates didn't have the managed portion like cloudformation did, where cloudformation keeps state of the environment and can detect drift and you can post updates to it and, you know, see what the changes are going to be. ARM templates were more like, no, you just deploy it and then the stuff's deployed and then if you need to update it, you go in the portal or you run commands. Uh, then it was around that time that I first discovered Terraform. And Terraform was a beautiful union of all the things I liked in infrastructure as code. And a lot of things that I didn't like were gone. So now I could work with hcl, which was much less verbose than JSON and more readable. And, um, it had a whole bunch of functions to do some common things. It had the idea of reusability in modules and it maintained state so you could actually use it to manage your infrastructure ongoing as opposed to just like running at once, creating everything and then throwing the state file away. Uh, so then I started sort of evangelizing Terraform around the consulting office and a lot of us ended up using it for the majority of our projects going forward.

Speaker A: So do you remember any of the big upgrades of Terraform? You know, uh, I think it was 0.08 or 0.8, I believe. And then there were a couple more after that. I can't remember all the really big ones. I think it was like 0.12 or 0.11, something like that. How did that go? Did you encounter any massive, just horrible experiences?

Speaker B: One big upgrade I think was around 0.9 to 0.10, which is when they split pro providers out of the binary into their own plugins, which was just a net good for everyone. It really freed up development of the providers to work at a different pace than the development of the core binary itself. So I don't remember any big headaches over that transformation, but I do remember some issues, uh, with the state file around that time and just like some Changes to the way the state file was structured. So, so you had to be very careful around, uh, which version of Terraform you were using in a given project. He didn't want to upgrade accidentally. Uh, then the really big change was between 0.12 and 0.11. So in 0.11 was HCL version 1, which didn't have strong data types. And then 0.12 introduced actual data types to the language, which meant it could actually understand an object versus a list versus just plain text. And that freed up, uh, a lot of the weird kludges people had put in to do string manipulation, basically, because when you passed things between different modules or different configurations, everything was just a string. And so you had to do your own interpolation. And now it natively understood these objects and it was like, no, I don't have to do that anymore. I can remove all these things I've done. But it was, uh, for some people it was a little bit of a painful upgrade. And then they had some trouble figuring out exactly what they wanted to do around provider versioning for a while. So there were like three different iterations of how to version the provider until they finally settled on what is the current version 1.0. It was probably the best version of the different ways they went about it. Uh, but you know, I see some people who really enjoyed some of the interim ideas. So maybe they stayed on 0.13 or 14 because they wanted to do it that way.

Speaker A: Yeah, I remember when the AWS provider decided to make that big change on, uh, S3 buckets.

Speaker B: You mean version four of the provider? No, I don't know what you're talking about, Derek.

Speaker A: Exactly. Yep. Good times, good times. I was like, yeah, why didn't they just do it like security groups and roll it out in phases and give you both options until, uh, until they made the change. But instead they broke a lot of things.

Speaker B: I think because it was a major version upgrade, the thought process was expect breaking changes.

Speaker A: That's exactly what. But a lot of people, uh, decided they didn't feel the same way.

Speaker B: Well, that's when a lot of admins learned that you should pin your versions.

Speaker A: Right.

Speaker B: You learn the hard way and then you don't make that mistake again. Hopefully.

Speaker A: Yep. I've definitely eaten my shirt a few times on, uh, oops. Version changing. I didn't. Oh, Postgres actually was a big one. I think it was Postgres 11 to 12 took down prod for me. I had everything else pinned. I did such a Great job. And there was just that one container that I just, it just slipped my mind. Yep. Took down Prod on that day, everything. I think there was a power search and uh, our Kubernetes cluster, you know, went down, came back up. Everything came back up fine. But it pulled the latest container of Postgres and bam. Flames everywhere, babies crying, you know, just cats and dogs loving each other. It's just ridiculous. So, um, yeah, really fun.

Speaker B: Never use latest and always pin your module versions.

Speaker A: Absolutely. Yeah. So do you ever find yourself going back to cloudformation? Do you find times where it's just like, you know what, you're an idiot if you just don't keep everything stored in aws and you know, obviously there are some good reasons to do it, but do you find that the benefits of Terraform almost always outweigh cloudformation or do you find that cloudformation is pretty useful?

Speaker B: I haven't touched cloudformation in. Well, I've used it once or twice, uh, for very specific purposes. Uh, one of those very specific purposes was I was developing a AWS networking course and I didn't want one of the prerequisites to be, you have to know Terraform. So instead I chose to use cloudformation for the initial setup of things. But generally speaking, Terraform's my go to, and that's because I tend to work across multiple clouds and things outside of the three major clouds. So knowing that I have a common language to fall back on, it's sort of like the powershell of infrastructure as code. Right. Like I, I know the language so well that I can pretty easily apply it to whatever the target environment is. And it doesn't abstract any of the complexity that's behind the different cloud environments, which I really appreciate. I want to know how GCP actually works under the covers and Terraform doesn't hide that from me. Whereas some other infrastructure as code languages try to create these higher level abstractions like a VM that just deploys across all the clouds and it's like. But yeah, it's more complex than that. And I don't want you to hide complexity from me when it's going to be important to the way that I deploy things.

Speaker A: Yeah. You know how you use, uh, Internet Explorer to download Chrome? That old joke, obviously Edge now it's the only purpose I use cloudformation to deploy the S3 bucket and other IAM resources that I need to work with Terraform.

Speaker B: That is a common question that I actually get is if I'm going to Store my terraform state in an S3 bucket. What do I use to provision the S3 bucket?

Speaker A: Right, it states all the way down. Yeah, you just don't want to do that. Yeah, I always use cloudformation to deploy my dependencies like that. Keep it, you know, of course in separate admin account or whatever and throw that key away pretty much. And then keep everything running from Terraform after that.

Speaker B: Yeah.

Speaker A: So uh, with arm, I'm just curious, have you played with BICEP at all?

Speaker B: I have, I had no choice because I was creating a course, uh, for an Azure certification and one of the tasks, or I forget what they call objectives, that was in the certification requirements was being able to deploy the environment with bicep. And I was like, well, I guess I'm learning BICEP today. And it turned out like it is so close to the way that Terraform works that it only took me about an hour to understand, okay, this is that and this is that. All right, I got it all. I'm just gonna use it. And it has the same concepts of like modules and the way you do things declaratively. So I found it very intuitive. Would I use it over Terraform? Probably not. I don't see a reason to do that. But I appreciate that Microsoft wants to offer as many options as possible and they also recognize that writing ARM templates manually is a nightmare. And rather than supporting YAML, which they still don't for some reason, they decided we'll create this other language that essentially just transpiles into ARM templates.

Speaker A: You know, in the 90s if uh, Microsoft started copying something, we'd all be up in arms. But now for whatever reason, we're actually kind of cool with it.

Speaker B: I'm fine with it. Whatever makes admins life easier I support. So if you're an all Microsoft shop and you want to use bicep, I mean it's better than ARM templates. Go for it.

Speaker A: Right? Right. Yeah. So you know, whenever you're experimenting with all these new things, you know, imagine you hit a lot of walls, I imagine, you know, a lot happens. It sounds like overall you're pretty much focused on Terraform, unless you need to teach something. But obviously if you're teaching something, it's worth it to someone. You know, these, these things are still being used out there. Mhm. And it sounds like you're able to move between them pretty easily. Do you have a strategy behind that? Is it pretty much do you start with the TerraF, like Terraform docs and kind of move over to the other, or is it usually pretty easy to dive into the docs and figure it out after that?

Speaker B: I think at a certain point, once you've done enough programming and working with infrastructure as code, you have these core concepts to fall back on. While all these languages have different syntax and weird foibles, they all have the same core concepts. So once you understand what those core concepts are, you've built a framework in your brain. And so when it's time to learn a new language, it's just like, okay, I have this outline, I'm just going to plug in all the words of this new language and oh, okay, these two things are a little funky. So that goes in an addendum somewhere. But I think just having that baseline understanding of the fundamentals allows you to pretty quickly learn another technology or language. And I think that goes across not just infrastructure as code, but technology in general. You and I have been in tech, uh, for, I don't want it to say, but I've been in tech for over 20 years now.

Speaker A: Same.

Speaker B: And you know, while the melody changes, the tune is the same. It's like, yeah, I cut my teeth racking servers and you know, cutting my knuckles on cage nuts. But like the fundamentals of compute storage and networking haven't changed that much. It's just implementation details. So not that those implementation details aren't important and not that they aren't challenging in their own right, but at least I come to every new technology with that baseline foundation and that helps me rapidly get up to speed on that tech. Uh, it's one of the reasons that I've shied away from like database administration and now more recently AI and other data heavy driven things is I never really built a solid foundation in data and analytics and like complex data structures. And so I always feel like I'm at a disadvantage learning any of those technologies because I feel like I'm missing a whole gap and to fill in that gap could be like a couple years of work. So I'm just going to stick to the stuff I know right now and maybe I can coast on that until retirement.

Speaker A: Man, that is such an age related thing because I'm kind of going through the same thing as well. I'm like, okay, I know all of this stuff, this is great. But uh, this is, this is hard. I don't like it anymore. You know, 10 years ago, oh my God, I'd be up for like 12 hours just staring at a screen, you know, really hammering into this. Now I'm kind of just getting Old and grumpy, I guess. I don't know. But, um, it's funny.

Speaker B: It's a matter of priorities, right? Yeah, it's important. Like, people talk about a work life balance, but what that balance looks like changes throughout your life. And so for a certain amount of time, that balance might favor work very heavily. And then at a certain point that might tip more to not work enjoying your life. And I feel like that's what's happening to me right now. For a certain period of time when I was consulting, I was also building up my catalog on pluralsight so that I could. Well, I didn't know it at the time, but eventually it could just work for myself. Right. And so I was hustling, you know, I was working my day job consulting, and then I would come home at night and after the kids went to bed, I would spend two hours or three hours working on my pluralsight courses. And I did that for like three years. Just hustled both of them at the same time. And that was, that was rough. And then once I started working for myself, I was petrified, Derek. I was like, I am now responsible for all of the money.

Speaker A: Yep.

Speaker B: And so every opportunity that came in the door, I was like, yes, yes, yes, I will take one of each of you. And so I was still working like these 12 hour days and working at nights. And my wife was like, didn't you quit your job so you could do less of this? But we have to eat. Um, and I have now hit a certain point where I won't say things take care of themselves because I still have to do work, but I don't have to grind the way I used to. I built up a base of revenue and now it's tilting in the other direction. I want to have fun with my kids. Like, I want to hang out with my wife. I want to play video games. Like, I have other priorities that are not work at this point. And I did the grind, so now that I don't have to. And it was difficult for a year or two in that transition to give up the grind because I always felt like I should be doing something. And then I've slowly let go of that and go, yes, you could do things, but why don't you?

Speaker A: Not so real talk behind that. I think there's a really good message there that you kind of illustrated. And obviously things are moving faster than they used to, of course, thanks to AI and everything else. But overall, you know, what do you think the life cycle is of any skill that you can learn Today, you know, this is a question I get a lot is if I learn Jenkins, am I out forever? You know, in a year, is this stuff going to disappear? Well, they'll always be these jobs. I mean. Two questions. One, do you think there's anything that you should definitely not learn right now? Just period, like just stay away from and to, you know, what do you think the life cycle looks like for most everything else that you can learn nowadays?

Speaker B: Man, don't, don't learn about exchange 2010 that much. Seriously though, you know, that's kind of the point though. Like my exchange skills are totally irrelevant at this point.

Speaker A: Right.

Speaker B: Almost everyone has moved to Office 365, Microsoft 365, whatever it's called. And those who haven't, there's very few of them. So it's kind of like COBOL programmers, right? Like those who have stuck with COBOL can probably pull down some decent salaries, but it's a very small market and I think exchange admins are kind of falling into that same thing.

Speaker A: Believe the COBOL actually is used by the New Jersey, like Department of Transportation or something. There was a, or something. There was a really big story about it. I remember one of the, uh.

Speaker B: It was their unemployment system.

Speaker A: Yeah, that's it. The unemployment system. Yep. But that's.

Speaker B: It was during the pandemic because a lot of unemployment systems crashed and burned because the uh, 10x load and when they brought in developers to try to fix that, it turned out that it was all running COBOL on mainframes and suddenly those people were very important.

Speaker A: Right, Right.

Speaker B: I think my larger point, it gets back to the thing I was saying earlier about learning fundamentals and building a foundation. If you can learn the fundamentals of a programming language, and I don't mean like go learn JavaScript, but I mean like the fundamentals of this is a function slash, a method. These are the different data types that you're going to be interacting with. This is how to build looping constructs. Like that's the sort of stuff that is just going to be applicable across all of the programming languages. So even if you specialize in one, it's going to be easy to pick up the other. And infrastructure is the same way. Get a fundamental understanding of how storage works, of how networking works, of how compute is provisioned, and then whatever, like new whiz bang thing comes out. Like webassembly is the new hotness. Like webassembly is just a process running on a Linux kernel. That's it. Like there's Nothing magic about it. Can it do some cool things? Absolutely. But if you can bring it back down to fundamentals, if you know Linux, then webassembly is not going to be a mystery to you. So I'd say start with those fundamentals. If you're trying to learn something today, then don't be afraid to specialize. I think a lot of benefit comes out of specialize and sometimes you'll end up specializing in the wrong thing. I spent a couple of years really focused on Azure Stack. Ask me how that's panned out. It wasn't my sole focus. And even though that didn't pan out, what did I learn? I learned a lot about what it means to run a private cloud. I learned a lot about, about disconnected scenarios and why you might want to run your own cloud versus just using the public cloud. So there were broader concepts that I learned in the process of specializing in a thing and I think that's probably another big point is keep an eye on the bigger picture while you're learning the minutia because the bigger pictures, uh, pictures ultimately the thing that's actually going to help you learn the next thing.

Speaker A: Yep. So speaking of Azure Stack, um, thanks to especially the Broadcom stuff and everything else, Broadcom purchasing VMware, you know, we're seeing this huge resurgence in like Proxmox and as people like dhh, sorry, uh, I never remember his name. David. Uh, was it Heinemann, I believe?

Speaker B: Sure.

Speaker A: Um, the creator of Ruby on Rails.

Speaker B: Yeah, yeah. VHH is what I go by.

Speaker A: Yeah, same. Yeah, he made huge, huge waves talking about leaving the cloud and all this and of course he has a pretty good case now. I'm not going to say it's a rock solid case and it's a very, you know, it depends on what your team looks like. There's so many things and I've, I've discussed this ad nauseam. We don't really have to get into the merits of in the cloud or out, but he's invigorating this on premises idea.

Speaker B: Mhm.

Speaker A: Are you picking up any waves of that? Are you starting to see that with anybody maybe you've consulted with or within your circles or you're starting to think that maybe it's a good idea to pick up and understand how some of this stuff works on prem?

Speaker B: Yeah, I think that would be an excellent idea. Uh, I spent many years working deep in VMware as well and you know, when I wasn't doing exchange migrations, I was also doing VMware migrations from four to five was one big purple screen.

Speaker A: Just wanted to see if you'd look visibly pissed.

Speaker B: So, yeah, I did a lot of VMware work and I think learning on premises stuff makes a lot of sense still. Do I care about VMware specifically? Not really. Um, I actually have been building out a Proxmox lab locally because I want to know more about the technology. There's lots of other solutions that are going to come up in the private cloud space. I think through the acquisition of VMware, Broadcom, with their licensing changes and their partner changes, has created sort of a hole, Ah, a vacuum where other players can rush in to fill, especially in the SMB space. And they will. I mean, you've seen concerted marketing efforts from all different folks across the spectrum. I mean, I've certainly seen it from nutanix. I've seen Red Hat getting in on it. I even got something from humanitech and I was like, you have nothing to do with this.

Speaker A: But, okay, I've seen that push as well. They are really getting big on that and I'm not seeing it. It's a. It's a stretch.

Speaker B: It was. It was a wild missive that I got from them that was like, you could build a private cloud on us. I'm like, no, you can't.

Speaker A: But I started reading it and I was like, oh, they're releasing a Proxmox clone or something like that. And I'm like, nope, just completely. Just their product just kind of hobbled together, bolted onto a thing. Got it.

Speaker B: Yeah. So I think that this is a great opportunity to. If you didn't have the VMware background, it's going to be a little bit of a slog because again, those concepts just, like looking at Proxmox, I'm like, oh, yeah. Just my mind immediately makes the connections to, okay, it's called this and Proxmox instead. Um, but. But I would look at that. That's a really good solution. Uh, Zen Server is still around if you ever played with that. It's called, I think, xcpng for some reason, because they hate vowels.

Speaker A: Um, and then Next Generation's in there. So you're good.

Speaker B: Sure. I remember when all firewalls were next generation.

Speaker A: Yep.

Speaker B: And, uh, another interesting solution, and one that's being championed by Red Hat, is Cube Vertical. So they've basically pushed the overt project to the side and said, we're going to dedicate our resources to making Kubevirt a thing for, like, OpenStack and OpenShift and so if any of those tickle your fancy, I would definitely go down the rabbit hole on any of them. Uh, I'm planning to in the second quarter of this year, maybe do a series of videos on my Proxmox adventures, uh, once I have a little bit of time to dig into it more and actually get some different scenarios up and running.

Speaker A: Yeah, I'd love to see that. Uh, I know that in our IoT deployment or industrial IoT deployment for a startup, I did Covid kind of killed it. Um, long story. Basically we missed the wave that would have carried us into being very rich by just not quite being ready whenever it happened. Um, but it was a really cool project. Basically we deployed, we deployed kubernetes and uh, daemonsets to all of our nodes out on the floor. We managed all the application upgrades. Everything through kubernetes worked super, super well. It was almost bulletproof aside from the occasional key rotation or, you know, whatever. Power outages can't do much about that. But it was pretty cool. And Proxmox was one way we started to investigate how to use to give us a little more resiliency and Proxmox with CEPH and all of that, of course there's a lot to consider. You got to dive into networking. You've got to figure out a lot of that type of stuff. And I've actually seen someone now use USB4 as a network interface for Seth, I'll have to send that to you. Um, it was really interesting. They were able to, you know, within Linux assign a network interface to the USB 4 which gives uh, like a 10 gig link or even more. I can't remember how much, how much it is now, but basically a very, very cheap 10 gig link that will support CEPH for multiple things. Because that was one of the big things is whenever you're distributing that type of data back and forth across a cluster that gets pretty heavy as you add more and more kubernetes workloads. So m being able to utilize USB like that, I mean not only taught me a ton about manipulating the kernel and stuff like that, figuring out how things work within Linux, but also a lot cheaper than a bunch of 10 gig links. So that was really interesting. I'll send that over to you as well and hopefully that comes out too because that'd be really interesting. I think the other thing people would love to see is CEPH and managing Quorum on the cheap.

Speaker B: I mean the challenge is always the witness server, right?

Speaker A: Yep. And I know a Lot of people have been setting that up on Raspberry PIs.

Speaker B: That's the one. Well, I happen to have a rack of Raspberry PIs, so that won't be a problem.

Speaker A: Yeah, no, I'm, I'm genuinely excited. I will definitely be watching this course as you knock it out. I don't have to manage the stack anymore. Luckily we've passed the cluster on, but it's something that I'm going to keep in my back pocket and I strongly suggest everybody else do as well.

Speaker B: Yeah.

Speaker A: So how are people going to watch this stuff? You know, tell us a little bit about what you got going on today and I'll start grilling you and adding extra stuff on my own side and we'll, uh, let's see how people can actually learn from Ned in the Cloud.

Speaker B: Oh, sure. Um, if you're interested in Terraform related material or vault related material. So on the Hashicorp side of things, uh, I have a number of courses, many courses on pluralsight. So if you have a pluralsight subscription through your employer or you want to get one, you can do that. You can all sign up for a free trial. I've got, I think 30 some odd courses on there. It's a lot.

Speaker A: Wow.

Speaker B: Yeah, yeah. Uh, beyond that, uh, I have a YouTube channel, Ned in the Cloud, and that is pretty focused on Terraform. But like I said, I want to do a Proxmox deep dive and that's not going to be a course necessarily. My initial intention is to just publish a series of YouTube videos about. Here's a basic home lab setup. Here's how you set up live migration between two Proxmox hosts, like little things like that. Basically taking my VMware knowledge and how do I apply that to Proxmox? Because I think you're going to see a lot of VMware admins, especially in the SMB space, going, how do I do that on proxmox again? I'm like, okay, I got you. Here's how you do it. Here's the storage vmotion. There you go. You're welcome. Um, so, yeah, my YouTube channel is a good place to find me. And then I have two podcasts. Day two Cloud. If you're interested in ops type conversations like this, Day two Cloud is definitely the place to look for that. Uh, if you're looking for more just silly conversations with a, uh, historical lens, then Chaos Lever is my other podcast I do with my buddy Chris Hainer. And in that we just take a current technology trend or concept that's going on and add some historical ideas behind it. We, uh, just did. Cybersecurity insurance will be our most recent episode. And you think it's not interesting, right, because it's insurance, but it's actually pretty intriguing. And Chris pulled uh, out all the stops to do some research around the history of insurance and how it impacts current cybersecurity insurance qualification.

Speaker A: So real quick question on that because based on something I just read, uh, somebody was going off on CVEs and how easy it is to submit a CVE, you know, and say like, oh, here's a thing that's incredibly niche, probably will never affect you ever, but we're going to rate it as ultra critical and if you don't take it out, you know, you're screwed. Uh, that sounds like one of those crazy conspiracy theories where these insurance companies could be submitting CVEs and stuff to inflate their usage. But I might be insane.

Speaker B: Yeah, I mean, the main force behind cybersecurity insurance right now, or the main things that they're dealing with, is an escalation in the number of cybersecurity incidents across the board. Uh, it's like doubled in the last two years and the fact that they were a little lax in their policy writing in the past. So they've been losing money on these policies as they get cashed in. And so now they're being much more stringent around who qualifies for a policy and what the premium looks like. It's going to get way more expensive, basically.

Speaker A: Yeah. You know, I mean, I know I'm sorry, the last thing I said sounded like we were ending everything, but you got me on a, got, uh, me on a corner here. Um, I've actually filled out one of the, the items for cybersecurity insurance. And as I'm reading it, basically if you are telling the truth, which of course we all are telling the truth all the time, if you're telling the truth on these reports, why in the actual hell do I need your cybersecurity insurance? Like, basically, if you follow it, you are probably the most secure organization in the entire world. Like everything on the cybersecurity page is basically, you know, perfect. Like, if you get hacked with following all of those steps, you are probably the most sought after. And uh, you know, you're like basically the US Government or something like that. You know, like it's so unlikely that I don't really know that buying cybersecurity insurance makes sense if you actually follow everything. They require.

Speaker B: Yeah, but I mean, there's always zero day attacks, right? Yeah, you can have everything patched up to the nines. But like we had the Barracuda email security gateway incident that was, I think, earlier this year or last year, and the advice was, we can't even patch this. You just have to pull it out of production immediately. And you could have been doing all the things right, you know, and you have this email security gateway in place to stop phishing attacks and all that. And now that is an, uh, exploit that is a point of attack you now have, because even though they issued patches, it wasn't good enough and you actually had to rip the thing out. So I think cybersecurity insurance makes sense even if you are following best practices. And there's a really good chance if you're a publicly traded company, your board is going to demand that you have it as a base requirement even if you are following all these security practices. So, yeah, like I said, it's a very interesting conversation that Chris and I had. So if that piques your interest, that's going to come out on the 15th of February. So keep an eye out for that. Go to chaos lever.com.

Speaker A: awesome. No, that's fantastic. All right, well, do you have anything else you'd like to share with the, uh, with the audience before we end it?

Speaker B: You know, we talked before the show about how important experimentation and failure is.

Speaker A: Oh, my God.

Speaker B: Learning in your career.

Speaker A: Four times here, and we kept missing it. So that's fine. Let's get after it.

Speaker B: You know, you and I were experimenting. We were trying out different things and going down different topics, and that's how you explore and learn. And so I guess my last thing that I want to say is I encourage everyone in your career to explore and experiment and be okay with failure, because failure is a learning opportunity. Every time I failed in my career, I've learned more about myself, what I want to do, and more importantly, what I don't want to do. And so that has made me a happier person in the long run and a more successful person in my career. So don't be afraid to experiment and don't be afraid to fail.

Speaker A: Awesome. That's perfect. All right, everybody. Well, that has been Ned Bellavance, and this has been a wonderful conversation. So if you could just give everybody a little cheers. Oh, nice. Hey, remember I had a 16 ounce. Cheers, everybody. Thanks for listening to Ops and Hops and feel free to reach out to either of us if you have any questions.