Engineers Are Becoming AI Orchestrators

On this episode of the show, I have with me Kevin Haggard. He is the Vice President of Engineering at Barracuda, and we're going to be talking about AI and engineering culture. We're going to dive in and ask him a little bit about how things are going there at Barracuda, talk about AI Dev Days, talk a little bit about adoption, different levels of it happening, and maybe a couple other things we'll throw in there as well. But Kevin, thanks for taking the time.

Hi, thank you. Appreciate being here. Absolutely. All right, before we start, Barracuda, what do you guys do?

Yeah, so we are a cybersecurity company that focuses on cyber resilience. Essentially, we are protecting customers across their email, data, network, and applications. We try to pair AI with people to be able to make, and the people are making judgments and adding context while AI is correlating data. But essentially, we are trying to make cybersecurity easy and help organizations be resilient.

All right. Well, I mean, cybersecurity and AI, they go well together. I guess in this case, we're going to be talking about AI and engineering culture. You guys mentioned cybersecurity.

In terms of where AI sits within the context of the product, obviously everybody ends using it. But I guess for you guys, what does that look like? Yeah. I mean, so we've got lots of data.

And so for us to be able to correlate that data, be able to identify patterns across that data is really important. but having people use that because of the amount of data that's coming through, allowing people to be able to make judgment calls, understand the context, understand what's a safe thing to do. So for an example of attacks happening, we want to shut that down. We want people to shut it down.

And, and then, but based off of data and signals that they're seeing, and AI is great at scale at being able to correlate all of that and be able to enable someone to take, take action quickly. absolutely when you look at the product obviously um within cyber security you still have to include a person i mean a lot of times that's very important where where does where does ai sit in terms of the solution itself where does it start where did where does uh where did where do people start yeah so it's it's embedded basically throughout the whole pipeline so when we think about like data ingestion data coming through we're monitoring what's happening through that, whether it be through emails or through data that for customers who want to back up their data, we're monitoring through that to see are there things that are happening.

Then when on the email side, if someone needs to take an action, we see that phishing alert happening or a detection takes place. Someone can then take that judgment call and say, yeah, this really is an attack. Let's stop it. Let's put a halt to it.

On the data side, We think about people who want to restore from a backup. That's a perfect time to check for malware. Make sure that if they're backing up because they had a malware attack, well, let's don't put that attack back in place again. Let's actually protect them.

So we check for it there and then allow for the person to decide, do you want to continue with this backup or not? So it's got to mix all the way through our products. It has to be embedded into the whole workflow to enable us to make good decisions. Absolutely.

When it comes to engineering culture and where AI sits, I guess everyone is all over the map in terms of where they're at. And even within teams, people are at different stages. I guess for Barracuda, where are you guys situated on that maturity curve? I think we are, it varies.

I think we're on, um, we've got some teams that are orchestrators that are, um, managing agents and orchestrating those agents from requirements all the way to deployment to where some teams are still trying to figure out, or some people are still trying to figure out how does it help them improve in their job? So I think we're, we're, we're kind of all over the spectrum, which is not surprising in my opinion for an organization of this size, But it's really important that we help those who are struggling or helping trying to figure out how to do that, that we set them up in a way that allows them to identify that and learn some of the lessons of some of our more advanced teams right now.

Absolutely. I guess when it comes to individual teams and how people are taking advantage of AI, obviously within engineering, there's a big growing trend of using one of the agentic coding solutions. As an organization, I think I hear a couple of things out there. One, anyone can use anything they want.

Two, well, we're still testing it out. And three, hey, these are the product stack. we've incorporated it's moving moving through the solution i guess from your perspective are you guys at a stage where you're still testing individual solutions you guys are a big enough company where maybe individual teams will find more synergy with a particular product is there has a strategy kind of started formulating there yeah so we've we do have a strategy around this for us as a cyber security company it's really important that we have guardrails in place it's very easy to leak data or to leak information that you don't want to have going out.

It's also very easy to have vulnerabilities installed. So for us, we do have basically like an AI gateway or proxy that allows us to only access certain LLMs through our various partners. We have to have that protection in place. We have to have the control plane in place so that agents just can't take actions without understanding what's going on.

So, um, it's what we, but we do it in a way that doesn't prevent people from exploration or experiments. Like when a new model is released, those are pretty, those are available pretty quickly for people to experiment with. Um, but we, we do have to have those controls in place. Absolutely.

When you when you looking at and you mentioned adoption is at different stages How do you support adoption I guess in a bigger engineering work What are some things that you've seen work? What are some things that you guys are trying to help promote it? So we started out with working with some of our partners. So like Microsoft, AWS, Anthropic, Atlassian, all of them.

We've had them come in, help us do training sessions broadly for the whole organization. and with engineering, that includes product and design. Those have gone well. But then we've also had hands-on sessions to where we'll bring in a partner to help us, people who have gone through this multiple times with multiple customers, and help them go through like a workshop.

We've done a couple of two-day or three-day workshops where it has a facilitator come in. We try to get people on site to work together. And those tend to be a little better because they get a little more hands-on activity. But one of the things we did recently, in fact, just last week across the whole organization, is what we call AI Dev Days.

And that was taking three days, fully protected, meaning no meetings unless you want to meet with your team, but it's protected time for people to spend time with the tooling. And as they're working through their particular task in their projects or the SDLC experiment. It wasn't about the output. It was really around, can we change default behaviors and focus on working in a new model?

Fantastic. I guess when people run these, I mean, I guess, I think AI depth is really interesting to kind of go. So when you see people who haven't really had a chance to work with AI and actually, you know, maybe some of the resistance out there. Is there a light bulb moment for somebody?

Because, you know, it's not easy sometimes to go, the craft that I have spent years building can simply be adjusted. And I don't think replaced. I'm a big believer that those that are trying to cut headcount by AI will have bigger problems than those that are embracing it to bring efficiency to teams and actually hire more to do more. I think there's two different camps there.

But what does that light bulb moment look like when somebody's like, oh, okay, I see why using this will help me, even though it's going to change my job? Yeah. For some of our more, I guess, resistant people, when we, with AI dead days, we made it mandatory. Everyone had to participate.

And so to allow for those light bulb moments, did everybody have a light bulb moment? No. uh but a lot of people did they're like wow a task that normally take me three days only took me a couple of hours i'll give you an example of like we have some of our product managers take word documents and that turned into jira tickets within hour with all uh with well actually within a few minutes with a lot of fidelity and it just made it they you know it's the first time they were doing this so it gave them that light bulb moment of wow i don't have to spend as much time copying and pasting or rewriting things.

So it really helped. On the engineering side, we had some people work very closely with our design team, be able to create prototypes. And we saw some on the front end engineering side of this, going from that prototype to actually having working front end code tied to the APIs connected up, just took hours instead of days and weeks. So we see those light bulb moments.

And we still have people who are like, well, it didn't write the code as good as I thought it could, or it wasn't, I thought it was a lot of slop and it didn't find, you know, it didn't do things I expected it to. Those are going to happen. We know that. So why we think people have to continue to be in the loop and review things, but we also realized that some of the people didn't necessarily have some of the setups that they needed to, or some of the skills in place and some of the tooling that you can use for like spectrum and development or other things to, to enable there to be a more structured feedback loop.

So if people were just using, for an example, claw just out of the box, yeah, maybe they didn't get the same results as someone else. But that also helped us understand we have to do a better job of sharing learnings across the teams, sharing those skills, sharing the things that are working so that everyone can benefit from it. Because people do struggle for it. And we also, I think the teams that were most successful from our AI dev days were the ones who didn't take themselves out of the loop.

They were constantly giving crisp instructions back to the agents of what to do. They were constantly reviewing the changes that were happening. And the feedback that we were getting from them was that they didn't find them as a replacement. They found this as an accelerator.

That's what's really important, I think, for people to get is that in the industry, we all have to adopt and learn the tools. There are things about it that aren't going to work well. There are a lot of things that are going to work very well. So I agree with you.

Companies who are cutting a bunch of staff because they think AI is going to replace everyone, I think that's a flawed strategy. I think it's really about how are you enabling more productivity and better quality to get out to your customers and add value a lot faster. So that part excites me. And I think that's what we're seeing at Barracuda.

And we're trying to, you know, we really want to see that adoption throughout our organization about how do we accelerate? How do we do things faster because customers want things faster and better than what we can do today? Absolutely. You know, it's interesting because I feel like the bigger the organization, the processes are definitely more defined processes.

So a startup, I could see a startup can grab a solution. It's part of being a startup. Agile nimble they can implement it But once you start looking at bigger companies with bigger product deliveries at an enterprise level those processes are I not gonna say rigid but they entrenched. I mean, it gets hard to shift the actual SDLC.

If we're going to actually go, hey, this is, there is a process, there's a life cycle. When you start thinking about that shipping life cycle, that SDLC, as you start seeing incorporation, what are you seeing the effects on the actual frameworks of software development? Because obviously, Scrum has been around for a long time. I'm already hearing some people say, why a two-week sprint?

What's the point of it? We're delivering within days, like we're no longer in it. So, are you seeing that? Are you starting to sense that there's other bigger structural changes around this that's coming?

Yeah. I've had this conversation a lot internally. From what I think is changing or what is required is that it doesn't matter about Scrum, Kanban, whatever methodology people want to follow. You just want to be able to ship faster.

And if you don't solve for your bottlenecks early on, you'll still have a problem in delivery. So let's just say we added 100 engineers instead of the AI agents. If I have a bottleneck on QA and testing at the end of that lifecycle, that hasn't changed. So I actually have to work on improving from how do I ship code faster and work ourselves backwards.

And so when I think about continuous deployments, that was something we all strive for, want to be able to do. Whether we actually deploy continuously or not, it's different. But that creates mechanisms for you to be able to have the right test. you've had the right reviews, you've really worked out your bottlenecks.

That applies just as much today, if not more than what it did before. So it's having those principles in place that you have a good way to release code. You have really safe mechanisms to roll back if you need to. But that's the same for AI because what's going to happen in the coding part, we're going to create a lot of velocity, but it's going to break things.

We're going to see where our bottlenecks exist today. They're just going to get highlighted even more. So if we don't start solving for them now, we're going to all of a sudden have all this, our cycle times are going to reduce. We're going to have a whole bunch of code just waiting to ship.

And that's going to be a bad outcome because what we want to be able to do is ship code quickly, but we're going to run into these bottlenecks will exist. So we have to take from the process, work backwards. And how do we optimize all the way through that? And I can help with that.

It can help with testing. It can help with deployments. It can help with code reviews. It can help even in the earlier parts of the process.

I find some of our bottlenecks are problem definition. How do we actually articulate well what we want to do? What's the customer need? So addressing on either end of that process is really important.

So I think the Scrum or Kanban, those methodologies, when they don't matter as much, it's really how do we articulate what needs to be done? How do you move efficiently and get that to the customer as quickly as possible. Yeah, I love that. And I think it's interesting when you literally think about the bottleneck and you think about a bottle, you've got a wide side, you've got a narrow side.

I don't think I've talked about this, but I actually feel like with what we're seeing with AI and engineering, we've taken the bottleneck, which was before the actual development time was the bottle. And then once we got it passed, there was, I mean, there's other stuff. but it can flow downstream. There's only so much coding you can do and so many people you can hire.

There's constraints. If we're going to flip the bottle and go, well, here's all this code. Now, as you mentioned, we're going to have other issues because, well, there's other security factors that potentially we need to look at. We have to make sure that it's a code base that's still supportable.

I mean, there's a litany of stuff. I mean, we could just endlessly talk about that stuff. When you think about that bottleneck and we talk about the human in the loop and the role of the engineer, is the role of the engineer then shifting? Because the bottleneck was how, I mean, there's a whole, you know, Frederick Brooks' mythical man month, right?

I mean, there's a whole book around adding people late to project doesn't solve the problem. We've always had to hire more engineers to solve that actual task. if we are going to go and look at code as easy to produce, are we shifting where we are hiring from that side to potentially a little bit downstream? Are we shifting people in a different part of this bottleneck, maybe long-winded way of asking you?

Yeah. I don't know if we're going to shift it that much. I still think we're going to hire engineers. We're still going to hire people who have domain expertise or a specific skill, but their role is going to elevate more.

Meaning like maybe their craft isn't writing the code that they did before, writing the automated test or writing the requirements. It's really around how do you articulate and how do you review the work that's being done, which gets back to the orchestrator. How do you become an orchestrator of this versus an operator? And I think that is what's going to fundamentally shift for people.

It's going to be a hard adjustment because if you are the engineer who's used to writing a lot of code. You've built your whole career around that. That domain expertise is really, really valuable, but you may not, writing code isn't necessarily the thing that's going to be most important. Your system design expertise is going to be far more important.

And that's what I think we're going to start hiring for. Who can see the big picture and who can articulate across that to instruct agents or even other people. When you think about hiring like principal level engineers, we aren't hiring just for their coding expertise. We're actually hiring for their expertise in, around system design.

How can they enable others to go faster and how can they coach and mentor So I think those people are still going to be extremely important They still going to be doing different things They just may instead of instructing humans probably going to be instructing agents to do some of that work which also scares me because then if we're hiring the principal level engineers, how are we hiring or how are we maturing the people that need to grow into that role? So we create a skills gap over time if we're not careful.

So I do think, you know, in the industry, we have to think about that skills gap. How are we training people to get back to that, what I would call the principal level, who should be the people who are excellent at orchestrating across all of this? I've heard this conversation a lot. I've had a chance to think about this for months because it's the thing that people talk about the most sometimes when we do these topics.

and I was thinking about when you're a child and you're sitting there with a parent and you're like you're seeing them do something at least in my day not now it's like hey I want your kid to try something fail it but in my day it was like you sit and watch why well you might mess it up okay so you sat there and watched and once the parent figures it out then they can kind of piecemeal and show it to you. I almost feel like we're at that stage of AI engineering where it's like somebody who has deep understanding of what's happening underneath that has to figure it out.

Somebody's got to take that step and figure it out. But once we do talking about some constraints and bottlenecks, you're going to want other people who can execute smaller pieces that are more manageable to come in and look at them because one, just a series of principal senior engineers will be overwhelmed with the amount of code that's flowing through a system. I think that might be what will happen. I could be wrong.

But I think that's what's going to happen. And I think it kind of fits even with engineering projects. If a company is launching a new project, you typically want a principal, a senior to be the first ones on, figure out what to do, figure out what needs to be built. Then we'll dole out smaller pieces of work to people who have less skill because they need to.

But I also think with AI, we might level set skill. Maybe as we see these tools get better, that system-level design might be brought down lower. And I actually think there's a world in two years where you just need to hire much more junior entry-level people because the tools will allow them to create design. Obviously, they won't know if the design is right or wrong, which is where then we need a principal engineer.

The circular loop. Yeah. Well, that may be true. Yeah.

Yeah. I mean, the advancements of the LLMs, like the models have been phenomenal over the last two years. It's amazing how far they've come along. What I, for at least for us right now, what we haven't been able to see is things that take into account scale very well.

So we have to be like for us, for a company, a barracuda, like barracuda size, we have to be really cautious with some of the changes that we make small change could have a really big impact on our costs. And so that, we have to continue to account for. We also, security vulnerabilities. We've got, there's a lot of things that are, a lot of tooling now that allows for us to detect vulnerabilities faster, make changes.

But some of that is we still need people to evaluate what are the right things to go solve right now. But I agree. I do think, you know, some of the systems design, some of that abstraction that's going to happen will make it easier for other people to do that. But I still think we've got performance and cost issues that have to be taken into account that we have uh that we're going to need those very senior people with the skill sets to be able to do absolutely that's that's i i think i think it's a it's a people call it the messy middle and i feel like that's where we're at because we're still figuring it out in real time and we still don't know and we still are not sure and i think it's the the part that's really complicated And we'll see.

We know one really, a year ago when people talked about these tools, people, it was very defensive. I don't want anybody to use it. That was what I heard. I don't want anyone to use it.

And now it's like, feel free to use it when you do your coding assessment. If we want to hire you, we'll set you up with the AI. We want to see what you do with it. And it's flipped within 12 to 18 months.

It's nuts how quickly that's gone. Yeah, it really has. But I personally find the messy middle fun. because we don't know what the outcomes are going to be.

We're actually very fortunate this time in our careers to get to live through this, to be able to be a part of it, help direct what does the future look like. I think about when mobile apps came out and people started building for that. We moved from just building web apps to building mobile apps. That was a fun evolution.

And we've seen that with cloud computing. We've seen it with moving from desktops to web. But I've been fortunate enough in our careers to get to see this. This is prime time.

This is fun time to be in. It's scary to a certain extent, but it's also really fun. And what people do today is really going to lay the foundation for what the future looks like. And we get to have a say in it.

100%. Very interesting times. Well, Kevin, I appreciate you coming on, sharing with us. If somebody does have a follow-up question, especially if they're maybe at a bigger org and they're kind of curious to follow up on something you mentioned, what's a good way of connecting with you?

Yeah, LinkedIn is the best way. Okay, fantastic. We'll make sure people are aware of that. Thank you for taking the time with us.

I really do appreciate it. Thank you. I really appreciate it too. Absolutely.

All right, that's it for this episode. Be back again. Different guests, different topic. Until then, two things.

One, we talked to Kevin about what's happening at Barracuda, how AI and engineering culture is mixing. We talked about AI dev days. We talked about bringing everyone along for that ride and what he is seeing as the company is incorporating AI within the engineering team. So please share this episode with somebody else who also might benefit from hearing it.

Also, like, subscribe, comment. Let me know how the show is going for you. Until next time, thank you and goodbye.