Inside Real-World AI Systems (AUDIO)

Speaker A: Welcome to Software Without Borders, uh, where we explore how technology, leadership and execution come together to shape modern software organizations. I'm Tomas Hilliard, Chief Strategy Officer and I'm joined by my co host Olivier Poulard, CTO and Managing Director of Software Strategies. Today's guest is Ben Kaufman, SVP of Engineering and Product at Stack. Ben has led engineering organizations across financial services, construction, tech and AI driven platforms, scaling teams from early stage environments to large complex systems.

Speaker B: And what really stands out about Ben is that he's not just talking about AI, uh, in theory, he has actually implemented it inside real systems like computer, uh, vision pipelines, 3D modeling workflows and large scale data infrastructure. All this while improving delivery speed and team performance. Ben, great to have you on the show.

Speaker C: Great to be on the show. Thank you guys.

Speaker A: Awesome. Well Ben, thanks again for joining us and I've been lucky enough to get to know you over the course of a few conversations over the past month or two. Um, but for our audience, could you please, uh, to start, give us a short description of your professional arc and maybe something that's influenced who you are as a human.

Speaker C: Ooh, that's a really broad uh, question to start up with. Uh, I would say uh, for professional arc, I would definitely say, you know, computers in general, just starting as a young child and then using it as kind of the platform to kind of build and shape my life around. And uh, it's, it's, it's been a wonderful journey and you know, main thing is just being susceptible to change that always, always happens.

Speaker A: So as far as influence as a human, you've just always been interested in computers, technology and maybe uh, what software development can um, produce as far as results go.

Speaker C: Yeah, we were uh, uh, my father was just in town. Spring uh, break, uh, we, we did a little camping on the beach and we were kind of reminiscing about uh, the books that we read as a kid. He was going to be handing down one of his books to me that uh, you know, it was like a little family heirloom. And he's like, what was your book? And I was like, oh man, it's so nerdy. And it was, it was Bill Gates, it was in uh, uh, like early high school, maybe middle school. It's called the Road Ahead. And he's talking about what the, what the Internet holds. And I just remember laying about and reading that at night. It was in thinking, man, I'm a nerd. But also I am so excited about what's ahead for the Internet. And here we are having a podcast over the Internet.

Speaker A: Absolutely. And, uh, you know, it's no surprise to anyone here that software has touched every industry. But I'm curious as to how you found yourself in construction tech and what led you there.

Speaker C: Yeah, that's a great story, too. Uh, I worked for some more traditional organizations to begin with. I kind of took the traditional route. Um, Capital One had a huge influence on me. I got to see a lot of people leave Capital One and do startups, and I realized that that's something that really catered towards my personality. It's, it's, you know, it's very much a meritocracy. You. You get in there and you show your worth real quickly, but there's a lot of, uh, upside and a lot of swings up and down, and that's just something that I fundamentally enjoyed. So at one point, I. I made this huge decision that I want to go work for a startup. Uh, but to everybody else, I was just this guy that worked for these, you know, big, stodgy old companies, and it. It took a lot of convincing. And, uh, just out of dumb luck, I got Procore, and then Procore ended up IPOing, and that was it. I was hooked.

Speaker A: Very interesting. And so now we're at the chapter where you're at Stack, your SVP of engineering and product. Can you tell me a little bit more about Stack, uh, something about the organization? And you're in a unique position, uh, leading engineering and product. Can you tell me what interesting nuances that might bring?

Speaker C: Yeah, I love it, to be honest. You know, the. The one area I always found a bit challenging with engineering, and I'm, um, I'm sure it was vice versa with product is There's. It was really hard to get this cohesive vision that incorporated engineering efforts and product efforts together. And so I was really pushing for some sort of role where I could kind of drive both of those in dual streams and make sure that each is well represented, but also reflects the outcome that you're looking to drive the most value to the company. And so Stack had this wonderful, uh, offer to be able to do both product and engineering, but they also really wanted to dive into AI and they wanted to dive in from two prongs, not only from the engineering standpoint, which has been spectacularly exciting, but. But also from the product standpoint. And that just really, uh, resonated with me and hit in a wheelhouse that I knew would be a lot of work, but it'd be something that I'd be very, uh, I think I'd be very successful at.

Speaker A: So you brought up the topic of the day, the topic of probably the last two or three years. Obviously, AI. We've, We've mentioned to start off the podcast, that you've led multiple teams through, you know, cloud, mobile, microservices, and now AI is at the forefront. So tell me, when did you realize that this wave was not just a fad, that it was actually going to stay and change the way that engineering teams operate?

Speaker C: You know, that's, uh, that all started at Pro Core. Um, I've always been an early adopter of technologies. It started, you know, of course, with the web and then mobile and people. You'll know you're in the right area. People are like, it's worthless. It's not the right one. It'll never, It'll never. They used to, you know, say, AI, ML, uh, what's the difference between that and statistics? And you're like, well, just give it time. You'll. You'll get there. Uh, Procore was the first place I started at, and it was great. We built. It was like a little ragtag team, uh, like five of us, and we built something pretty special, uh, while we were there. I was very proud of it. Uh, basically, in short, it was. We used the Inception model from Google. Um, and it was. So it's a computer vision model. And we used that to recognize various aspects of photos that were being taken, uh, on site and construction and be able to easily categorize them with the full intent to start analyzing them wholly and comparing them against schedules and costs and saying, hey, it cost you $10 to lay this amount of cement on one project. It should cost you about 10 on the next. Well, that got, uh, the CTO of the time, Sam, he, He really liked the idea, and that was. That was also very invigorating. And so he kind of pulled me in and we. I got to speak to the board a little bit about it and really, really pitched the idea. Um, at the time, they were really getting close to an ipo, so it was hard to take pretty big moonshots. And, uh, it didn't quite get as much traction as I would like. But later on, it led on to the work at Scanify, which is a 3D, uh, software where we do drone, ah, 3D modeling, uh, and utilizing a drone and using AI to recognize objects and deciduous trees, and then which led to Stack.

Speaker A: Hmm. And so I'd be curious as to when does it begin to show up, uh, more so in your engineering teams? And in your day to day coding?

Speaker C: Well yeah, that is the rule. So um, scan and fly was the first place it showed up in. And uh, I had a little insight, I had a hack, I had a good mentor of mine that worked at Meta and he's like oh my gosh, uh, coding. AI coding is taking off like it's going crazy here. You need to get on board. If you're not on board you're going to get left behind. Uh, there's a double edged sword there. First it was kind of selling the CEO who was a little bit more traditional in nature and then engineers at first glance, um, the ones that didn't fully buy off on it kind of felt threatened by it. That would be my analysis. It could have been many things. Um, and then as time moved on it and I, I came over to stack, the crew there would just, you know it was based outta Ohio so a lot of uh, Midwestern guys, I'm, I'm originally from Kansas, I'm Midwestern. But they were into it, they wanted the chain, they, they dove right in. So this is kind of the first area that we really saw mass adoption that I really saw mass adoption of, of AI code and that also goes in line with, with the maturity of, of like platforms like cloud and chatgpt. Codex.

Speaker B: I've got a few questions. Uh, you know there's been a lot of noise of course about AI in general. Co pilots and agents, automation, whatever. Uh, from your perspective, what do you think? Where have you seen AI actually delivering real value? Especially with engineering teams?

Speaker C: Yeah, with engineering teams for sure. That's easy when that's the low hanging fruit, our crews, you know, uh, buying into it, um, hook, line and sinker. There's several caveats that come around with it of course. One of the big ones is code reviews because you are producing a lot more code a lot more frequently. But one of the most surprising areas to me was product, um, a byproduct for product, uh, was uh, that engineering now was chewing through their work a lot faster and in order, instead of hiring more product people in order for them to keep up, they needed to be able to research and provide the body of work for the engineers to build along with the design. And so we started using AI to not only help out with the research and help out with the uh, ticket generation but also started using AI to help build out some quick designs for the devs to work off of to move to keep up the speed that they were working with uh, using, using AI. So to me that has been one of the nuances of the journey that I wasn't prepared for, but really, really enjoyed.

Speaker B: So you're talking about more or less pre prototyping? More or less, um, before the work starts.

Speaker C: Yeah. And you know, in the traditional uh, set it would, you know, uh, um, a product owner would reach out to a customer and talk to them quite a bit and then uh, they would reach out to design and they would kind of work together and iterate through designs with. And then they'd break it down into these chunks of work. And uh, with these chunks of work they would move that over to the engineers to start building. Now we're utilizing AI to not only kind of scavenge all of our sales and customer service calls to try to preemptively get an idea of what we're looking for, but we've created these uh, architectural design patterns that they can import into Claude and, and create a uh, I wouldn't even say like a prototype. Create a real time mockup for them to use, uh, for the developers to use as they're building out their, their, their, their code.

Speaker B: Okay, good. And then uh, you know, conversely is, are there any areas where um, you haven't seen the needle move? Really?

Speaker C: Where we haven't seen the needle move?

Speaker B: Yeah.

Speaker C: You know, this is a tricky one and this is a balance and I don't know the right answer to this one but you know, we're also looking broader in the organization on how we can utilize AI with customer service and sales. Now there's a few startups out there, I think Kixie's one of them. I spoke to their, their CEO a little bit. They have some great kind of preemptive sales. And the scary part is that you can't hardly tell if you're talking to a human or a. But there is a very humans general, ah, artificial general intelligence hasn't quite made it yet. So humans can still kind of sniff it out. And so there's a very fine line of, you know, do we want to risk the potential of prolonging the sale or not having the satisfactory customer support response when you're dealing with voice, not, not necessarily chat. Um, and so that part, I don't think we've seen it move the needle as much. But um, companies like Kixie Sierra with Brett Taylor, I think they're doing some really, really magical stuff.

Speaker B: But from an engineering standpoint obviously you've seen great progress and uh, uh, great help from AI obviously.

Speaker C: Wild progress. I mean and it was um, a little hot take on this that uh, you know, traditional engineering has been. Not always, not always. This is a hot take, uh, a little bit more of a young man's game. It's a, it's a, it's an athletic sport, if you will. Um, you, you tend to see the, the uh, people more mature in their career, moving towards different roles to, to accentuate the wisdom that they have. Um, with AI, there's this really unique paradigm that's happening that the wisdom honestly becomes the most valuable aspect. And that real sharp, you know, you know, twenties brain that it, that could, could solve these complex problems very quickly is not really a differentiator anymore. So we're seeing some of the best code and some of the fastest code come out of the engineers that have just been in the space for the longest amount of time. That, that was a wild curveball that I'm, uh, you know, kind of enjoying to see.

Speaker B: Excellent.

Speaker A: So Ben, just to stick on that point, I guess with this moment, uh, of euphoria, or not euphoria of Eureka rather, um, how does that change how you are organizing your engineering teams or how does that change how you're designing your operating model?

Speaker C: It, you know, just holistically, uh, AI has changed everything from how we set up our organization in the engineering and uh, product department, and then also on how we leverage the skillset and the talents of the individuals. From an organizational standpoint, um, it's very much now around speed and very little process. So we've started this, uh, this methodology called High Impact Pods. And I would love to take credit for it, but uh, a mentor of mine kind of slid a, uh, uh, uh, word doc over to me and said hey, take a look at this. What do you think? And oh, I love it. Right. And I'm gonna, I'm gonna run with this. And so, um, what the high, uh, impact pods entails is uh, you operate with a team of two engineers and a shared product owner and a theoretical shared designer. Now the designer is operating a little bit more abstract. Um, the idea being is that the uh, kind of in the similar transition that we went through from Waterfall to Agile and Scrum, this is even more lightweight and so that we still more or less use jira, but people aren't really logging into it anymore. They're kind of mcp' ing through Claude into it to grab their tickets and tickets to be created. It's more just like a data repository. And so the product people are running about three pods and it's in each pod has a couple engineers. And so we found that this organization way helps the developers not work, uh, work more independently but, but still are able to communicate effectively as a, as a team. Now there have been pieces that have been challenging in that arena because they're moving so fast in regards to people touching the same part of the code base at the same time and making pretty significant changes, but we're, we're working through those.

Speaker A: I'd be interested to hear your response to it and uh, knowing that you are working with AI native teams and bringing clients to some of our great software development partners around the world, I'd be curious to hear your perspective on Ben's POD setup.

Speaker B: No, that sounds right. I mean to me, uh, the smaller the pod, the better it enables better communication, less risk, et cetera, less misinterpretation of the requirements. Uh, and uh, typically those small teams are more nimble. That's why Agile worked well because now we went from a whole team of 100 to 10 teams of 10 or 12 teams of eight or whatever that is, uh, at the time. Right. But now we're talking about even having uh, half the size of what typical pod would be, so from 8 to 4 to 3, etc. So, but still, uh, some project will still require maybe a large number of pods. I would say, I would, I would

Speaker C: assume is all right, yeah, we, we're, we're right around 12, 13 pods and uh, we had to create a few new rules. Right. So what, what we're seeing is um, like so we use Claude, we use Claude code and people will be touching some similar code bases at the same time. Now Claude's getting really good at holding memory for the individual and their past and what they've done and kind of being able to have a little bit of a predictive nature of going forward. But what it doesn't share is that context across all the developers. Uh, and that would, I'm certain that that will be in the future. But what happens is if somebody changes a method, which happened traditionally in the past, and you would do code reviews and you would merge it, but if you're having you know, a thousand code reviews a day, I'm exaggerating, but maybe 150 or 100 a day. If somebody foundationally changes a method and 10,000 lines get written and another 10,000 get written over here, and this method is instrumental and so this works and this doesn't, then you, you get in a pretty hairy situation of trying to figure out what, what, what went wrong there. So this orchestrated role that we've, we've built, um, and we have one kind of. There's a person that's representational for product, but the main one is for engineering is as the work kind of comes in, he. Very much in a real time, it starts directing like, okay, this is a part of the code base that nobody's really touching right now. You guys go, go conquer. And this is a part of the code base that nobody else is touching. You go and conquer. The problem we're navigating with that now is the frequency of that communication. It's, it's frequent and it's, um, it's a very taxing. It's like an air traffic controller. That's actually exactly what it's like.

Speaker B: Yeah. And I've, uh, uh, I can, can vouch for this because, you know, I've, I've managed very, very large teams as well. And one, uh, of the problems was dependency management. AI or no AI. Okay, that was the, that was the problem because now you have, you know, I, um, had at the time 800 people on that team and uh, in five different locations offshore. And when you don't know who else is doing what, then, uh, you know, it can become very, very quickly a problem of dependency problems. Right. So it's all about dependencies in the end. So I'd be curious to know how you use AI from that comp. From that context, from, From a dependency management standpoint.

Speaker C: You know, I would love to tell you we have solved that problem. We're solving that problem. We're in the midst of solving that problem. Um, you know, there's the easy routes that I could tie and everybody be like, you know, yeah, sure, of course, that's, that's, that's it. But, um, it's, it's just a really interesting thing because dependency management in the past was just a little bit slower by nature.

Speaker B: Exactly.

Speaker C: And the changes were slower by nature. But now the changes are happening so rapidly that we're just kind of figuring it out as we go. Um, I would say the number one thing kind of, uh, going back to what you're saying, Oliver, is the communication. It's like, don't be afraid to use that huddle button on Slack. Don't, uh, be afraid to, uh, communicate with everybody. Um, but also there's a big part of the code reviews too. And so, uh, one way to help alleviate that is to use AI for code reviews. Um, and Claude's offered a great service. Um, there's also Code Rabbit and a few others. But Clauds is rather expensive. I don't know if anybody's had a chance to play around with it. But it's, uh, about 15 to 20 bucks. A code review.

Speaker A: Oh, gosh, yeah, Ben, that's where I was going to go next. You know, you've been talking about, uh, hey, we're more nimble, we're more flexible, we can produce more, uh, engineering is utilizing it, products utilizing it. We're seeing where we can implement AI across the organization. I'm thinking to myself, what are the financial impacts on engineering a product with the amount that you can produce? And how are you looking to, um, provide enough governance and oversight to make sure that that doesn't get out of control?

Speaker C: Yeah, so this is, you know, uh, the Patrick Donnelly. He's, he's the one, uh, that, that kind of gave me the idea of the pod. And so him and I meet weekly and we bounce each other ideas off each other, uh, chatting all. He's in Scotland, so we chat. I send him two in the morning and he gets message from me. From two in the morning. We have this theory that eventually, uh, it wouldn't be surprising to me if Claude starts charging the equivalent amount of a cheap engineer. So they start charging 60,000 to $80,000 a year for a seat. Um, for us directly right now, that's kind of forecasting to the future. For us directly right now, uh, our token usage is going through the roof. It's very high. And, um, I've, I kind of have to do have weekly meetings, uh, with our financials being like, all right, here's what we can expect, you know, and so far everybody's been pretty, um, appreciating of it. You know, it, we're, we're going up about a thousand to two thousand a week sometimes in terms of cost. But, uh, the one thing that the CEO keeps, you know, um, of Stack keeps hitting me up about is like, great, you're shipping a lot. Where's the value? What's the value of giving back to the customer? And so that's been another big thing that we've had to work really hard on from the product standpoint, is to really start quantifying value. Great. You know, you're shipping a lot of features, a lot of great features, but where are we, where are we getting the most value out of these features? And that's also been a part of the journey for us.

Speaker A: And I'd say that's probably your hottest take of the podcast is that, um, Claude, down the line is going to start charging, you know, very junior, uh, levels of, you know, engineering Salaries in order to have a seat. Olivier, what are your thoughts?

Speaker B: Yeah, I mean, I wouldn't be surprised, but I don't think the technology is there yet to completely replace people. So, um, they won't be able to justify that as of yet. I mean, probably in a couple of years, maybe. I don't know yet, but we'll see. Um, that being said, uh, I still think that the, uh, future entails both humans and machines. Uh, I see this as a, you know, code, uh, as a service, if you will. The same way we moved away from, for the most part, moved away from on premise, uh, compute to on demand compute. So it's the same thing here. We've got to be careful about how we use that on demand, essentially, uh, coding services. Right.

Speaker C: I would build upon that. Exactly. And I might have misrepresented. You know, I still believe, uh, there will be human engineers wholeheartedly. There's just too many pieces that have to be stitched together from a people and architecture and customer standpoint. I, um, just believe that, uh, instead of having maybe, uh, two engineers, you'll have one with an equivalent seat sitting next to it. Like a cloud seat, uh, sitting next to it that they're charging just as much for.

Speaker B: Yeah. At the same time, sorry, the, uh, you know, uh, I've used cloud, cloud as well. And ah, sometimes, you know, I. So this morning I made a, you know, I'm prototyping certain things. I made a very, very simple change in, uh, in a database. And I wanted to test Claude, you know, how quickly it could be done. I can tell you, that change, I would have done it in five seconds. It took cloud, uh, 15 minutes to do it. It burnt. I don't know how many tokens.

Speaker C: 20,000 tokens.

Speaker A: Yeah.

Speaker B: Or something in those lines. Not sure if it's purposeful or what. But in the end, okay, There are certain things that AI can be very, very good at and other things that are not. So the things that, to me, that AI would be very good at is to find the needle in a haystack.

Speaker C: No, that's, that's going back to your earlier point. I, I often question how, uh, ChatGPT and Claude, you know, anthropic kind of come up with these, uh, token usage. Seems like a pretty good hustle, right? It's not super clear. And they can kind of dial it up and dial it down and. Yeah, if you're doing a simple insert into a, into a table, that's exactly it.

Speaker B: Yeah, yeah, it's simple insert, update. In fact, it was an update in the table. You're removing one item and adding three or four whatever so that the dropdown would be a little bit different. But the uh, on the ui, but in the end it took. And it's not just the number of tokens, it's my time as well. I was sitting there watching it, you know, just to see you know, what, what would happen, et cetera. And that was interesting to uh, to observe.

Speaker C: One of the, one of the more uh, kind of fun aspects of uh, does at times feel like a little bit of, a little bit of a hustle. Um, I can't quite remember what the original question was that we, we, we steered out of that.

Speaker A: Well, well maybe to, to bring us back. Um, you know I, I think we've done a really good job of touching on where AI is affecting the organization as a whole then where it's affecting us in an engineering sense and in your sense specifically in your pods. Uh, but from an individual level, I'm curious Ben, what do you see habits, uh, that are changing the most because of an AI augmented world?

Speaker C: Yeah. So uh, oh, uh, going back to I think the original question Oliver said we are seeing AI, uh helping with training devs like so they're able to comb the code base and learn it very quickly now it's so the code base is almost self teaching when you have Claude and it's able to step through it. That has been tremendous especially for new hires. Uh, for me personally I just feel like it's up to my iq. I come across overwriting way, way more intelligent than I think. Uh, maybe I am right in that regard. Um, it's also just a great spot check for me to, with everything that I do to just kind of cross reference. So you know, outside of the coding, which is the most obvious stuff the day to day in terms of you know, writing emails or uh, presentations or coming up with budgets is you know, stuff that people don't even necessarily think to use it on. It's, it's actually really great as a partner in crime. And so it's become my kind of buddy if you will and say what am I missing? And we, we kind of have a joke around uh, stack is ask better questions. And we found that that kind of mantra not only helps everyone in terms of what they're doing uh, for their organization, but specifically an engineering product who's fully embraced AI helps them get the most out of AI.

Speaker A: Yeah, that's interesting. And you mentioned uh, you envision a future where AI and engineers are uh, collaborating together. And I would love to hear your description or your prediction of what that human who orchestrates with an AI engineer would look like and what characteristic and traits they need to have in order to be successful in leading an AI tool.

Speaker C: You know, just watch Star Trek. You get, get all your answers there. I, I, that's all that comes to my mind. You know, um, the best, the, the core, the core idea is that is the one thing that, that I have seen, especially for people kind of swearing in it outside of engineering, is learning how to wrap their mind around of just asking questions and giving it context. You know, it has to be like you're just talking to another human. You have to shape the context. You have to tell, you know, AI what's going on, and then you have to know how to ask it the right questions and know how to ask it to ask you the right questions. And, uh, that last piece is the piece that I almost see everybody kind of drop. And it's such a little hack of just saying, hey, you know, I feel like I'm missing something. Can you kind of give me. This area feels weak. Can you ask me some questions that you think would make this meteor or might represent something that I'm missing? And then it opens up, usually a nice little rabbit hole for you to go down.

Speaker A: Love it. Love it. And so, last question, on the individual level, um, new tools. We're teaching people what habits they need to pick up, what they need to learn. On the flip side of it, what habits are you discouraging your engineers or your teams from using, uh, now or from you from doing now that they have AI and it can be automated? Uh, uh, the efficiency can be increased.

Speaker C: I don't, the only thing I'm necessarily discouraging is not using AI, but that's usually the first question is, have you used AI on it? And if you haven't, please, please give it a shot. Uh, um, but everything else in regards to AI, it's so new. There's, everybody's figuring stuff out as we go, and I mean, it's just wonderful. You know, there's this, uh, Ralph Wiggum paradigm that's coming up that we've played around with in terms of software development with AI, uh, it's, it feels, you know what, it feels even better than the mobile era. It feels like the web where everybody's just figuring stuff out. And the iteration cycle on it is absolutely wild. It's gone from, you know, what used to be maybe months, you know, the quarters to now, like on a Weekly basis, we're, we're seeing change. So this really, the, the discouragement might be the discouraging of not using AI. Like, use AI, make it part of everything you do. Worst case scenario, you burn five minutes and you get nothing out of it.

Speaker A: Uh, before I throw it over to Olivia, you know what, Ben? We have all different types of guests on this podcast, and I really like that you put yourself in the category of I'm in the experimentation lane. I am in the. We need to try things out and continue to see what works and doesn't work. Um, sometimes we have the AI naysayers, sometimes we have the people who have absolutely no guardrails with AI. And I think it's really fresh to hear your perspective as far as the only thing I'm discouraging is, uh, not using it at all.

Speaker C: Yeah, thank you.

Speaker B: And, uh, still talking about people here. And then, uh, I'll follow up on that. On that, uh, one question. Do you see any differences between the more senior and the junior engineers into adopting AI or how they use it? Adopt. Adoption is one thing, of course, but then once it's adopted, you know, how they use it, why they use it. I don't know.

Speaker C: Yeah, um, I'm not actually. I think the, you know, if you look at what makes, uh, outside of, you know, intelligence, what makes a great engineer out of the core is curiosity and the willingness to accept change. And, uh, I can unabashly say it's stacked. That's what we have through and through. So in terms of, you know, years of experience or juniorness, uh, the, the idea of adopting AI has just been pretty consistent throughout the whole organization. The one thing that I have seen is that more juniors are pretty quick on the draw with the gun and which is to be expected. Right. Um, in, uh, in the past, if that's like 50, uh, lines or a hundred lines of code, that's, you know, cool. We've got you. Right? We'll, we'll help correct you, we'll talk you through it. Now they can do a lot more than that really quickly. And so that part, um, has really forced a lot of our senior guys to kind of, you know, create some standard markdown files that they kind of plop in their, their source code directory to make sure that when the best that they can, that they're, they're following some basic architectural guidelines and, and work, you know, as much as possible that, that, that these AI, uh, code development engines offer within guardrails, within a closed box environment.

Speaker B: Okay. And, uh, so beyond the uh, fact that in your teams of course you had different levels of seniority. You have also different roles, right, like uh, products more product oriented, more uh, architecture oriented. Dev programmers, I guess, testers, QA, DevOps. How do you see them use AI differently.

Speaker C: So we've separated our team out and now we, we have everybody still works in pods, but we do have an organizational structure where some of our most talented engineers are uh, operating, providing guidelines or structural constraints for the architecture. Um, they're loose, I will admit that, but they're there. And so in that regard that's one kind of slight deviation. It used to be architecture and anybody that's been in engineering for a long time, they kind of get a bad taste with architecture. They sit in their ivory tower and they kind of say something. This is not that this is very hands on devs, uh, that are just basically providing guardrails so that we don't get ourselves too, too deep into, into trouble. The other aspect I would say is product. You are, I am seeing product soiree into engineering. That's hot. That was a hot take at her at our dinner the other night. Um, I'm enjoying it. Um, now you have to take that with a grain of salt and you have to understand that for maybe copy changes and color changes, that's not too big of a deal. And that trust is kind of earned along the way with the product and the engineers as they go. But they are soireeing into the engineering more than they've ever done before. Um, the main component there we're pushing on is they have to learn GitHub, they have to learn some sort of source code repository. They have to have a basic idea of what they're doing and not just talk to Claude, hit enter and shoot it off and call it a day. Uh, but most of them have been pretty into that. So in terms of that kind of segmentation we are seeing you know, product kind of soaring engineering. And then kind of back to what you were saying originally is the main aspect is the organization is just flattening. So there's everybody, everybody is hands on these days um, to some form or fashion. And so there's less focus on kind of line level engineering managers and just more focus on everybody working in their pod and getting their, getting as much functionality and as much value as they can out to the customer.

Speaker A: Ben, I'm curious, does this change the type of engineer that you guys are going after? For example a full cycle, uh, or full development, uh, type of engineer might be more attractive in the future alongside an AI tooling rather than a more specific type of engineer in QA or DevOps, for example.

Speaker C: Yeah, this is, this is really interesting aspect too. Oh, QA is another one. We're fully automating that. That's, you know, and that has its own problems too. Automating QA is definitely a process, especially when you're looking at, you know, some of those functional based tests. Um, and, and uh, going back to what you're saying a little bit too, Oliver is uh, with the sre. Uh, um, they're still primarily infrastructure now they are also soaring into engineering quite a bit more now. Um, but you'll find some of the best SREs usually came from coding and so it's not too much of a stretch for them. Uh, jumping back to, you know, what you were saying, Thomas, is in terms of hiring, the main thing we're looking for is a willingness to accept AI. That's it. Uh, we still have our core for dot net, right. So, you know, having a good understanding of that is still very relevant at this point. Uh, but the, the interview process starts off with here's a problem solved with AI. That's, that's, you know, that's our kind of weed out. And um, there hasn't been as much industry adoption as you would think. People aren't ready for that or they're not ready for that in an interview. Maybe that might be a fair, a fair, a more fair, uh, analysis. But uh, and then the secondary part of the interview is just kind of making sure that there's some core architectural and fundamental understandings of, you know, Net and programming in general.

Speaker A: So maybe just a personal, uh, curiosity, tangent. I'm really surprised to hear that, Ben, that people are not adopting it or using it, or maybe they're just surprised to see it on a first round interview. What's, what's been your experience? People are, are still hesitant in using it or hesitant in showcasing that they're using it.

Speaker C: I, I can only make assumptions, right? I, I don't know for sure. Now. We did start this back in kind of November, December last year with our interview process in an AI world that might as well have been five years ago. So a lot has changed since then. I, I think there was. Well, I can tell you what some, some have told me. There were a, a few people that were hesitant about, uh, you know, adopting AI because the interview process was not utilizing AI yet. And so you start strengthening skill sets that are, uh, almost, you know, opposite of the skill set of, of typing out Code, and that's scary if you're looking for that next role. I suspect that probably by the end of this year, definitely next year, almost every engineering team is going to be doing, when they're hiring, going to be following a very similar process where they start with AI, they give a problem, they say, we want to see you walk through it with AI and then they'll have some sort of foundational core language that they kind of want to make sure that you have the basics on.

Speaker A: Very nice. Interesting. Okay, uh, moving on. Um, so you've mentioned that the POD system, and it seems like that's really great for scaling. Um, can you tell me if you anticipate, if you were to scale your team to double its size, would your POD system still, uh, be relevant and well, working in that environment and what would you have to change in order to make sure that scale is able, um, to be done and able to be done in a correct and healthy way?

Speaker C: Well, I'm going to give a loaded answer to that, but I don't actually know. Uh, yes I do. Uh, there will be problems. Um, this context thing of so many engineers writing code so fast. Um, I also have another great, ah, friend that is actually over at ChatGPT. They say they're having multiple code commits. I don't know if I'm allowed to say that, but like a minute, a minute, every minute they're having multiple co. It's, it's unfathomable to me. Um, and I was just asking him similar questions on the scale and, and kind of watching how, you know, they've broken down the problem and, and at least made some pretty strong attempts to do that at scale was compelling to me. I won't dive too much into what they're doing, but for us, uh, I think the appropriate way to scale is you start developing kind of a silo aspect in the regards that you'll have a grouping I like to think of like Kubernetes almost. I don't know if you have. In Kubernetes you have your pod groupings and, and nodes and whatnot. So you'll start having um, various nodes, various pods and in those groups together. And those will all kind of filter up to maybe one orchestration or maybe multiple orchestrations and a primary orchestration on top of that. I don't know what that looks like. But I'm really excited to get there. I'm really excited to see the problems that arise and how we as humans, how we solve them. But then also the technology that gets invented along the way, that helps make that problem more solvable.

Speaker A: Yeah. And it's really interesting because I've asked you about if you and your company are successful and you scale, um, how will you account for more engineers? Well, I guess on the flip side of that is do you need to scale or is the tooling going to get better? Is your process going to get better? Maybe scaling to you means, uh, in the current headcount, just becoming more efficient. So, um, you know, it's a way that I need to continue to think about. The questions that I ask for the future is what does success look like? And I don't think that's necessarily a scaling organization, but an organization who's producing more in a more efficient manner and driving revenue through that.

Speaker C: You know, I can pile on that one. That's a heart to heart that I have with, uh, our CEO vas a lot and for him is all about value, right? That's, that's the end of the day. Like we're, we're, we're an expense and what value are we driving to multiply that expense back to the company? And so for him it's really just that it's not um, about do we add more people or can we do more with less? It's uh, if we can do more with less and we want to add more people, are we compounding our value? Is there a multiplier in value there? And so that's kind of the back and forth that we're having with, with that, that conversation. So, um, I think it just all depends on the stage of your company, what you're trying to accomplish, you know, who your competitors are. Um, but I do suspect that as everybody starts to adopt this, it'll be the same racing game as before. It'll just the amount of functionality and the speed will just rapidly increase. But I, I think in the short term there may be a decrease in engineers, but in the long term, uh, there, there will definitely be that ramp because everybody wants to win.

Speaker A: Thank God you said it. As a software podcast, we love to hear when people are optimistic about the future of engineers and um, you know, continuing to, to bring in humans in the loop. Uh, but Olivier, my apologies. Please go ahead.

Speaker C: No, no, no.

Speaker B: Uh, well, talking about scaling and just finish on that on that topic, one of the, you know, first of all, it's very, very difficult to scale. And I'm talking, we're talking about people here to scale locally, right? So typically to scale you need to allow people to be scattered around the country or Even outside the country often. How do you see AI helping you with this, this, this, this fact that more and more teams are globally distributed, if you will.

Speaker C: Yeah, we're, we're definitely global. Globally distributed. Um, how has AI played a role in that?

Speaker B: Not yet.

Speaker A: Maybe.

Speaker C: Yeah, I, I would say, I'm trying to think of it. I would, I would say the, the main aspect that, um, I would say that AI has played a role in is through, uh, helping people understand what code was written and, you know, learning the code base and what, uh, they need to do. So in the past, if you were doing this appropriately, you might make some fairly significant changes and your time zone, your it's bedtime or dinner or you gotta go pick up the kids and they're waking up in the morning and they're getting their coffee. The, you know, what theoretically should have been the correct way to do that is you give them a big synopsis, a big lowdown. Hey, this is what I changed. This is where I'm at. Here you go. If I have a feeling you're gonna be using this. And so you, you give this big report, ride out. Uh, now we're not seeing that as much. We're just saying, you seeing guys go, hey, it's in there. Use Claude. Tell Claude to go take a peek and give us, give you a summary of what's changed and what you need to be aware of. And so I would say in that regard, that's the primary area that we're seeing. But again, I think we're just at the tip of the sword here.

Speaker A: Well, uh, Ben, uh, I think this has been a really invigorating conversation. I like that you've provided a lot of optimism for the, um, efficiencies and what AI can bring in the future. But you've also, uh, brought optimism to the fact that humans will hopefully be alongside of AI producing great software for years and decades and centuries to come. Um, so, uh, to me, it's really rethinking and reshaping how teams operate, how work gets done, and how organizations scale.

Speaker C: Yeah, Yep. I can't help but, uh, the nerd in me think about Star Trek. And, uh, it's just, it will be a part of all of our lives, but it'll just help us go to places, uh, we've never been before.

Speaker A: Yeah, absolutely.

Speaker B: I think that AI will help with the collective.

Speaker C: It will. I agree.

Speaker A: Well said. Well, thank you again, Ben. Really appreciate you joining us. And, uh, we look forward to continuing the conversation. And please reach out to Ben, find him on LinkedIn and, uh, continue to pick his brain because he's certainly a great resource.