96. AI Native Ops, How Brex Rebuilt Operations

Hello and welcome to another episode of The Operations Room, a podcast for COOs. I am Brandon Mensinga joined by my lovely co-host Bethany Ayers. How are things going, Bethany? How do you like the new glasses?

The new glasses are fabulous. I'm mostly convinced. I feel like there may be a bit too low. There's a bit too much eyebrow.

This is one where we're gonna have to do a bit of a video or have like a picture for everybody to see what I'm talking about. But on the whole I like them and they are, I think my optometrist was calling them multifocal rather than varifocal, so maybe that's the new name. He also calls them spectacles so he might just be inventing his own. Basically, there's the top is looking far away.

The middle is for me plus one, and then reading is plus one and a half. Okay, so I think I have two then. I have like one for my classic I Can't See Far Away, and then I've just got the pure reading one. So I was expecting because of the way everybody's talked about it that I was going to put these on and my whole world was going to be horrible and I was going to fall over for days, but it hasn't been bad at all.

Like I don't notice the, I think maybe because it's only plus one and one and a half, I don't notice big changes sitting down, everything was fine when I tried them on, then she had me stand up and I was like, Whoa, somehow I'm walking on a bouncy castle. But very quickly I got used to the bouncy castle and now like the world is starting to be a bit flatter. I mean, you get used to it super fast. I mean I was told the same thing.

I mean disoriented for a while because it's weird to like your eyes have to focus in the right spot. I think within, I don't know, two hours I was probably fine and I haven't looked back since, so it's not that bad. Yeah, and it's so nice to be able to pick up my phone and read and be able to look out and read, like it's just so nice. All right, so any special events this week?

I went to a Local Globe event last night. I think it's called Open Court. I always find it difficult when people like have their main brand and then they have other brands and then you have to remember it, all the different rules. But I think I was called Open court.

I think its happening once a month. It happened a bit last year and this was the first one for this year. And it's just bringing the community together and having different speakers. And I forget how good Local Globe are.

I'm not just saying that because they're an investor. You know, their offices are really, we've been to them, like for a COO event. They're really open and generous about using their offices. They do a lot of community work.

This event, the age range was probably 18 to 70. And everybody who was just interested in AI, the speakers had the business leader, I can't remember, he has a weird title, but the and then the head of AI and ML for Spotify. It was interesting, but also I was like, wow, despite being in this space, but being on the business side, so little. He was such an impressive person.

I looked him up afterwards just to see his name's Martin Gould. It's one of those people where he can make the complex easy to understand and compelling, and feels very much like a good professor, And it's also just. Clearly unbelievably smart when it comes to maths, but also just smart and well-rounded and personable and an amazing communicator, like just one of these people that God has touched more than the rest of us. Oh wow, sounds like a very impressive individual.

So impressive. And so he did his, I think, undergrad and master's at Cambridge and then did his PhD at Oxford in maths and then also has a great love of music and so combines those two passions to create amazing personalization to make sure we all access the music we want to access. Sounds like an amazing bio. Yeah.

And it was just a really, it was so nice to have all these different people come together. The questions were from what's your business strategy to the granola, like what's next, how are you going to keep growing to questions I can't even paraphrase on like the level of maths that people in the audience were asking the Spotify guy. It's great to see how vibrant the London startup community is and how diverse it is. It just got a real buzz from it.

Yeah, for sure. Local Glob is one of the, if not the premier seed company, has to be close to the top three, I would say, in the UK. And it's been around for such a long time and they've invested in so many great companies as well. That space that we've been to before, it's a great space.

And to your point, it's like relatively speaking open to the community, open to special events. If you're one of these seed-based companies, you can work out of there as much as you want type of thing. So it's fabulous environment. Interesting bits or a connection here is that I went to the 60 minute mentor live podcast with James Mitra and he had two guests talking about talent acquisition and one of those individuals His name was Charles and he's now I think like the chief talent officer of lovable and He was plucked directly out of local globe for that role so apparently this Charles fellow similar crazy background like he was in one of Elon Musk's, the Neuralink company as the talent lead for Neuralync working directly with Elon Musk.

And then he worked for some other company in a similar kind of a crazy awesome capacity. And now he's had lovable and apparently lovable kind of crisscrossed like all the VCs in particular for some reason looking for their talent lead and plucked out this as fellow to lead their talent. So the probation period, they tell candidates out of the gates, there is like a more than non-zero chance that you will not be here at the end of the three months. So the probation is deadly serious.

And if you're an incoming hire, you need to recognize that you might be out the door in three months effectively. Interestingly as a side note. The regulations in the UK are changing next year, whereby this kind of two-year cutoff point that classically has been there where if you're an employee less than two years, you don't have a lot of rights and the company can terminate you, the company can terminates you very easily. Once your past two years it is much more difficult for companies to terminate.

You have to go through performance improvement plans and special cycles to exit somebody from a company. So that two-year marker. Next april is becoming six months which is almost like probation away so the six month marker i think all companies in the u.k.

I have to change their approach in terms of how they do that for six months very specifically to ensure that there's much more rigor around somebody's performance in a way that previously i don't think we really had to. So this is more than i think the seo is that out there this is coming and that your marker is not gonna be six months come next april. And I think it's the orientation of how you think about performance in that first six months to ensure that the person is high value needs to stay in the business.

You're much more clear on that. What do you make of that? We've already done it. Oh, you've done it, you're always ahead of me.

Well, I mean, we've done it internally. Our finance and ops and people person has built in the structure, set up the alerts, put in the performance reviews and documentation. So we now have a much more rigorous zero to six month process than we did previously. So when it's rolled out, we're ready.

Oh man, you're going to have to give me some crib notes here on what to do. Yeah, and we're also, so we've done it all vibe coded for the moment, but what we're considering doing is we moved from High Bob to Rippling, but have not had a great experience with Rippaling, but I don't necessarily think that's a reflection of Rippiling, and it might be more of a reflection of how we rolled it out and what we are paying for. But rather than looking for new HRIS, we have decided that we're just going to Vibecode our own or code our own.

And so we have different elements of it right now. And then the next step is to turn it into the app that we all use. So we have onboarding and offboarding has been done and their probation period and performance management frameworks in general have been done. And now the last piece is to put it together and build the holiday and sick leave tracking.

And then we have our own HRIS. Yeah, that's awesome. So I'm going from rippling to creating your own situation. That sounds fabulous.

But just because we're not very complicated, we're 20 people, and we don't need to pay whatever it is for, we use half of it. And it doesn't work well, and it's a bit ugly, and it' annoying, so we might as well just do it ourselves. Yeah, that HRAS space is so weird because you're right, at the core center of it, there's really not that much that needs to actually be done for core HRAS, which is the reason why the high bobs and the ripplings have like 12,000 add-on modules that you can buy because they realize the core is so miniature in size.

So to your point, if there's any kind of like SaaS software that's pretty doable to get rid of versus like a HubSpot which is much more complicated, the core HRAs like front and center, I would say to kind of AI agent your way out of it. Yeah, and also like we've done it bit by bit because we've just automated different processes and it's like, oh, we've automated this, we automated that, we automate, okay, like, why don't we just link it all together and save ourselves some money.

We've done the same for a partner portal because we were paying for a partner portal that wasn't expensive, it was like three grand a year, but it had a lot of features and we were using it as basically a form and a database. I was just like, no, there is a much better solution to this. That's such a SaaS software thing, isn't it? Every SaaS software, you kind of use the core essence or the functionality, but all the additional stuff, which there's always a ton of it, you know, another 90% of features you never touch.

And then you're like, okay, do I really, you know again, same thing, like can I AI agent my way out of this just for the core, essence of what we're trying to do or because I don't need this other 90%. Yeah, I mean, I think when you're a bigger company and you need it all, fair enough, but at our size. And it's all just knowing to pay for somebody to like, I mean, the partner portal was like super basic. Why are we paying for this?

All right, so we've got a great topic today, which is AI Native Ops, how Brex Rebuild Operations. And we have a great guest for this, which was Camilla Matias. She is the COO at Brex. So the first thing I wanted to ask you was, Camilla had talked about, if you're going to follow a procedure, you don't need a human anymore.

Whatever is kind of like an L1 type role just doesn't exist going forward. What's your take on Camilla's? Well, I mean, I think she was talking about her business in particular, but it ties in a lot to what we've all been talking about of what's the role of an individual contributor. There are no more individual contributors, everybody's running agents.

I think the bigger question is how do people gain experience if there aren't any L1 jobs? And so I don't think that the entire company is going to age out and die or retire if we don't hire younger people. So I think we need to switch and start to think about. We need young people and increasingly the young people who we hire are going to be AI native in the way that the last generation were digital natives.

And so what are the skills that they need and how do you pair them with the experts in the business? Because what they're gonna have to learn is they'll come with the ability to do a lot more with AI than we have, but we have the business experience. And so how do teach them the business experienced? With their AIs, maybe a buddy system might work, but I think we need to hire in more young people and if we don't, we're going to be in trouble.

Yeah, yeah, so I think you're exactly right. It is absolutely fascinating because we talked to True Search to do a tire for a VP of product. What True had told us was that look, if you're looking for a VPA product right now, you're gonna get one of two people. One is gonna be the kind of like older, broad-based experience that's done your run before going from 20 million ARR to 100 million ARRR, but they're not gonna be AI native first, which is what you're look for.

And if you want to find that person that's a VP product, they're going to be much younger, much less experienced, much more of an ambitious step up role to become like a senior leader in your business. But they're gonna come with that skill set that you're looking for out of the box in terms of being AI native first. And this is not an L1 role that I'm talking about, obviously, but but I think your comment applies to this as well, which is, I think we're going to need AI native, first young people coming in that are like thinking purely through as opposed to us where we're trying to like backport our thinking to figure this stuff out basically and I think some combination of the two somehow this is what the outcome is going to look like.

Yeah, I think for Camilla's business it's easier because her L1 or customer service, I'm thinking I'd love to listen to how law firms are going to address this because presumably we're still going to need lawyers in the future. How do you train them? So when she was transforming Brex, and she did a fabulous job of this, and you'll hear this a little bit later on in the interview that we do with her, she was saying that the real unlock was intensity and operating like a seed-based company.

So she was say Brex is quite a large organization, but she had to think from like a C point of view in the way of like saying to herself and her team, you know, I'm the CEO of the company, I have responsibility for this entire organization, but you know what? I'm not gonna do that. What I'm gonna do is like cancel all my meetings. Pull together a small team of individuals, including myself, and we're going to go forward with this mantra of AI, the AI native and build it out from the ground floor.

So effectively in a weird way, she was almost like seconding herself as a CEO to lead the charge at the outset. And the outcome that she's delivered that you'll hear on the interview has been pretty transformational for Brex and I think all of us thinking through CEO's and CEO's in this case having the courage to do something like that sounds pretty radical and she had the backing of a CEO so that makes a difference i think but in any event what do you make of what she did. And her board.

I just think that her interview is one of the most important and most interesting and valuable that we've ever done. It's almost like, should we talk about it or should we just tell everybody, listen to it? If you want to understand how to cut your cost to serve by 50%, if you want understand how truly get efficiency from AI, spend the I don't know how long it's going to be, 45 minutes? Listening to Camilla.

There you go. So I think we've set Camilla up. Let's park it here and let's get on to the conversation. I remember like the beginning of like last year, Pedro comes to me, and we were having like our, Pedro is our CEO and founder, we were having like our feedback session, he was asking, Amila, what would you do if you're starting Brex today from scratch with like all the technology that we have in place?

And I remember, like the theme was like, oh, we're not adopting AI enough, or quick enough. And I look at him, he's like, Oh, I would build everything. Differently from, I mean, if we had to build now, if like everything that is available, the concepts would be different. It's just hard to change.

And he was like, okay, so let's rebuild from scratch. And that's the thoughts that I had. Like, okay, what if I would actually rejoin from scratch? What if I was hire my first operator, my first function in ops with all the technology?

I remember in the past, everything was about, Okay, how do I document things? How do I make sure that I have the right metrics in place, the right SLAs? How do make sure I have the SOPs that are codified that many people can do that, and they're not just making the decision based on whatever they have in their heads. When I started thinking about this, I thought, what if I can codify all of that with AI, and then AI is just part of the team, because it doesn't need to be a technology itself, but it should be part of a team.

And the managers, the leaders, they are not like leading humans and training your pupil, but they are also training the technology in their favor. So it's not that the business outcome of the role has changed. The leaders, everyone in the team is still have like the same business goals. It's just how you achieve the roles becomes different.

Could you explain what you mean by codifying everything from scratch? So did you just sit and talk to AI and explain to it everything you wanted to do? And have a big database of it? Or what do you actually mean by that.

I think if you fully start from scratch, maybe that's what you do, because you start like just talking with AI like about about the problem that you have to solve. In our case, we had a bunch of like all the SOPs, all the process all the decisions that we have made in place. So it's about okay, now that we like all this knowledge, how we codify it, I think one of the first things that we did, it's not only about codified acknowledgement of what was created, but it was like truly rethinking about the roles that we needed in place.

So if you think about very entry levels of operations, it doesn't matter if it's a support function or like an ops or like a risk function, but you normally are expected to follow in detail a procedure, like, okay, you have to like follow those rules. I think for me, the first thing is if you're going to follow a procedure, you don't need a human anymore. That should be fully done by AI. So whatever is like this L1 type of like role level, this just doesn't exist.

And as long as we are all moving towards like this new environment, this is not as necessary, that's going to be the beginning. The second is when you grow in your career, normally you're training more people, normally you're becoming a leader or subject matter expert. And you're passing their content, like, what do you do? What is all this institutional knowledge to the team?

You can still do that. But instead of passing to the Team, you're not only going to be joining at other different groups of people, you're actually going to join that with AI. So the expectation of this other type of role is, OK, you need to be able to put your SOP in a prompt. You need to able to.

Not only look at the metrics, SLAs of the teammates of a group, you actually need to be able to do an evolve and see if the agents are performing. So you still have the same type of fundamentals of an operator, but you're applying the fundamentals to the agents because the agent should be part of the team. And then you can still have, okay, but who's thinking about strategy? Who's thinking the design, like the small level tree as you grow in the ladder, right?

Those are still there, but instead of like, okay, how do you create like the organization environments? How do you think about August structure? You should actually be thinking about systems that's compound. You should be thinking like about process that I mean, you have the proper feedback loop that are only getting better.

You shouldn't be thinking about how you break down a problem into a hundred micro steps. Because if you break it down in many, many tiny steps, you can actually have like the agents building those, which in the past was just not possible. So I think for me, I'm also trying at Matomic to transition us into being an AI native company. And on the engineering side, that's a lot easier, partially because engineers understand more, but also because the technology has been built more for them and cloud code and context and the software around the models exists already.

I'm finding it more difficult on the commercial side Because. Somewhat because of people, but more because of technology. And some of the software just doesn't do what I want it to do yet. And I don't think it does what the teams want them to do.

Yet. So what are you recommending? What software do you think are things that exist are actually good so that I should think. Because it's not only about the software itself.

Cloud code is great. I think there is a fundamental change that you need to do around the talent and say, hey, now you're expected. You shouldn't be expected to do it yourself. You should be expected to leverage in AI and how you do that.

A simple example, and let's use support because it has a very customer-facing role. Everyone talks about, okay, everyone thinks about using the software, which is the chat bot. To really reduce contact rate. I think beyond that is now when we train the new support class, we train them without an instructor in the room.

We train them truly with AI. In the past, you had to read a bunch of things, watch a bunch videos, be trained to follow someone else that had done the job. Now, you're actually trained by an AI agent because that's what you should be doing. These AI agents can put you to any knowledge management.

If you are able to ask the questions very quickly, you are to instruct and gather the information. So if the training starts already different, you get a teammate that is much more proficient with AI to resolve a customer challenge. And then when you go deep, okay, Camila, but what's the software I'm trying to do that is not as simple because then people start using. I think that's where I think Brex had made the right investment.

We invest in ourselves building a lot of it, building a little like the infrastructure. So there was a lot of like engineering investments to getting the whole company ready and operations ready too. So I don't think it's just ops doing it by itself but it's actually a very close partnership into getting this done because you need to create like the tools, the systems and the first layer that allow people to build on top of that. Now, maybe what I think if I was going into detail here is, I remember that I had the same question as you, Beth, and I was gonna propose this whole change to our board.

And I was nervous because I thought, okay, the board will say, Camila, why are we gonna invest on reinventing operations if we can invest in having AI within the product? But when I presented, the feedback was actually very positive. Everyone was like, that's amazing, because if you do invest internally, and you have the right tools, you build the taste that you need to build the products that the customers want. It was actually everyone agreed, let's just double down and invest in revamping operations with AI, also from an engineering and R&D perspective.

What did that look like? Like? I mean, without revealing any trade secrets, additional engineering hired, what's the infrastructure like? Share as much detail as you can.

Then I got everyone on board. I sent them the memo and an email to the whole company. That's what I expected the road to be. And this became part of our roadmap.

But the reality is I didn't feel things were moving fast enough. And then this was maybe what, May last year. And I was like, oh, all those new companies, they are able to move quick. So what's the difference between what they're doing and what are you doing?

And then I think it was more like a joke in the beginning, you said, okay, what if we put people together? What if we say we have to operate as a seed company, and we have a group of people that will do that? And that leads it. So I remember that I was like, okay let's do it.

I was sitting here, let's let's recruit your engineers within Brex to do this project. And then, I think in two or three days, I basically said, okay, this is our budget from a- very underfunded seed company. We're going to work from this office. We're gonna cancel all the meetings.

We're not going to join like regular breaks routines. And including myself, I cancel all my external meetings. And I said, I will spend four to six weeks in a group with like everyone joining the office, going to the office together, spending a whole day with those problems. What we did, we chose great engineers.

That have been dealing with those problems that have MIMO-like AI proficient. And we got like the subject matter experts with them. And then it was like ops and ends together to truly try to rebuild all those solutions. To think, okay, this is like what SOP says and then engineer, okay.

But what do you do when you get that use case? Okay, so that's what you have to codify to the agent. Okay, and then like building like those multi-agent systems that allow us to like make better decisions when you're talking about fraud. Which was like high risk for us.

So having this room to like put people together and say, you don't leave until you find a new way of like this to operate was actually very helpful. In four weeks, actually in six weeks, we launched like our first new reinvented onboarding flow, which was for a very specific segment of customers, fully automated, which was also the foundation for a lot of like investments that we made after that. And then when you circle back to that board pitch, so this is kind of, I guess, earlier in the cycle slightly, in the board pitch like what was your pitch?

If you can describe it a little bit like, because always in these board meetings, there's this question of AI, you know, are you using AI? How effectively are you are using it? Why are you not doing more of it? Why are we adding all this headcount, et cetera, et cetera.

So for an operator, for a CO to come into a board meeting to kind of pitch them around. Kind of the concept of like reinvented workflows and job roles and whatnot. Like what was your, what was a bit of the detail around the presentation itself and how you kind of pitched it. I'll break it down in three parts.

The first one is I needed a business outcome. I need to convince them that if I was getting investments, that it would be real ROI. So here we measure cost to serve. So we have this whole operations, how much cost to serve the business.

If you're doing the same project with more commercial functions, you could think about CAC. But whatever it is, for me, this is the cost to solve of breaks today. I think I can bring this down by half, which makes Brexit by half. That's what I mean.

Well, no wonder the board was like, here, have some money. I think we're like already. So I said half in 24 months. We are thinking like halfway through that in less than 12 months.

So, I actually think we are going to be able to deliver like shorter than I actually promised. But anyways, I had the number. I had that's I want to do this is what we're gonna this is why And the reason behind that wasn't because I would cut roles or anything. No, it was because I would be able to grow as fast as we wanted without investing into all these ops machines, because that's where I would see all the AI benefits.

So then, okay, first, go business outcome. Second was, okay. How are we going to change the team? Which was what I explained to you.

That was my first thing. This is how the roles would change. This is the type of investments that we do in like L1 roles and that's why it scales so quickly with revenue. If we don't need to do this anymore, this is one of the reasons that we're gonna get true to the outcome that I mentioned.

So when I explained what the roles would look like and how we'd get to the business outcome, I was like, okay, but what is the level of investments we need? And then was there needs to be a priority for engineering too because we need to build those systems, the platform that allow Ops should be a contributor that allow ops to do the evolve themselves, that allow ops to codify with the prompt in a safe way. And that was the third piece to close the loop. So if I would go back, the business would come, how we would change the team, and what were like the investments from an engineering capacity.

And then how did you figure out what technology to use? Did you left that to engineering or did you already have some ideas? I had an idea of how the system would look like, how that we should have to break down the problems. But in the end, this was in partnership with engineering, right?

When we did this first, we call internally Hacker House, when we did the first Hacker house, we already had some development, okay, how we were investing to automate the whole KYC. But the reality is we didn't know how we could underwrite a customer without any human touch. So it was us all going together, OK, how do we make this even possible? But I wouldn't say there was a lot of like new software or new things.

It was a little like, OK. Using cloud, using like all the APIs with like, I mean, we tested Gemini. We tested Shared GPT. We tested cloud and we saw all the performance of the models.

And we still do. And using tools that we already have, we still use like Retool, which is a tool that we use. A break today, but the building honestly was just with the models that are available. We didn't need to retrain the model, there was no RL or anything extra advanced here.

There were instructions around how you build the agents, how they connect, which the engineer obviously had a ton of input there, but it was not any specific software. Well, I guess it's more, or if you've built things in-house, because the areas where we've ended up needing to build is our data structure so that we can access data in the right ways, some amount of sharing skills or sharing information between spaces. MCP servers aren't always great. The actions aren't always great, like there's a lot of engineering needed around the edges.

So it's not the model so much anymore that I think matters as. How do you get the best out of the model? And that's the part that we're focusing on building at the moment. So we use Clay.

It was one of the softwares that I think it was great that we had to integrate that gave us a lot of access to LinkedIn data. When I'm thinking, we use like some starting companies, we use Accent to spread financials, which was actually very helpful into how we collecting. We use a lot like data aggregators that had been adopting AI, but that have been in the market even before the last three years. So...

We use a bunch of them, not I don't want to pretend here that we built it all in-house, but it was a mix, but the infrastructure of how we would build the agent, we just mute our own. I think it's just because I'm so deep in it right now and I was like what's the silver bullet how can I do this in a way that's easier than what's happening because some of it is culture change but a lot of it I'm finding right now is people are ready to go and the technology isn't always there without needing to build.

Yes, one thing that we definitely saw as an example, I wanted to automate the full dispute process, right? Dispute, in my words, here is a customer use the cards, they don't recognize the transaction, they come to brex and they want to dispute. So there are mood steps here. We have like teams and teammates like walking behind the scenes.

So the first step to do that honestly started fooling operations without any engineering resource. We basically, as a teammate to say hey. What can you do because you're not going to be able to prioritize from an engineering perspective like this flow here? Then they created the gems, which at the time was like the agents that you could build with Gemini.

Just by then creating the agents, being able to record, putting all the knowledge management there and the hundreds and hundreds of pages from all the master cards, books of how you file the disputes, we were able to automate 50% of the flow. 50%, I always say that that's when people stop and get to the main. Mistake, because if you automate 50% of something, it will still, you may get 10% of efficiency. Because unless you fully automate the whole flow, and maybe this is the silver bullet for me, unless you're fully remove the whole floor, it's not enough.

So, okay, this feels great. So then we lay in the engineering resource to build the other layers that need to be done, to actually integrate this in fully. Finish the automation of the process. Now this is going live.

This has been tested and going live and with this, this is like one of the main things that is helping us to deliver our cost to serve goals because before I had more than 30 people working today with like a lot of our teams working overnight and now I don't need any of the other our teams anymore. So we've just raised a series B. We have a ton of cash. We're now spending it to hire a bunch of people.

You know, we're obviously on the software development side, things are, we're very cognizant of like reworking a lot of our flows and kind of the software developers themselves were be hiring probably half of what we would have classically done, expecting the software development team to be much more productive. And we can see we're on a good trajectory for that part of it. I guess what I'm wondering, is on the back office side of things. We need to create space for the organization to be able to have space to think about their jobs, think about what they're doing, think about our flows, start to think through how best to rework them.

We have a program or initiative in the company where we're pairing engineers with other people in the companies, like back office as an example, to work on particular problems of SOPs or whatever, similar to what you're describing where... There's a particular issue or workflow that we have that's highly repeatable, work with the engineer and that individual, reimagine it and automate it using AI effectively. So we're starting on that journey a little bit. But what I'm wondering about is the acceleration part of it.

And right now we all have day jobs, including myself. So, you know, in my back office team, whether or not it's worthwhile or sensible even to hire somebody that has 100% capacity. To work wholesale on some of the operational issues that we have with the software development team as opposed to piecemealing right now where we have bits and pieces of myself and some of my team, some of team working with the engineers on spots of a problems on a limited time basis. Should I hire somebody with my team to help?

Should I actually work with the engineering lead to hire more engineers perhaps to have more space on their side to work with our side internally to do more of this? What do you make of that? Excellent question, and I think that's spot on. Ideally, I would recommend you to get someone that is doing the back office job, that is leading it, that has a lot of expertise in the subject, whatever you want to automate to be part of the process.

For two reasons, they know best, they know the institutional knowledge. If you bring someone fully external, they would have to learn all of that and try to codify. And normally that's when people make a ton of mistakes. So if you're able to bring this someone to lead the initiative and pair them up with like the engineers, I think that is very important.

I would say that maybe hire a few engineers that know and have built in an environment like more like AI native does helps. So maybe I would invest there, but I leverage operations subject matter experts. And the reason why I would push for that or like I would advise for that is Because you're not going to need just one flow. You're going to needs multiple subject matter experts.

So you want the talent to be like, OK, you kind of create one. You prove like points with this. And you want this to be replicable, right? The engineering team can go and move on to resolve all the problems.

But then, whoever was able to create this new flow is who you'll be working towards how you actually QA, how you'll actually keep improving the agent itself. Because it's not a once and done. The models keep getting better. You need to always keep testing the response that are done.

And if you are able to be part of the build, then you create the new L2 of my crazy matrix that in my head does make sense and has been working. But then you created this new L two because this person can actually train in this new workforce model. Does that make any sense? Yeah, it does.

What would you suggest in terms of the time allocation? Right now, similar to Bethany, we had a company offsite. We spent a day doing this kind of activity. We now have this initiative that we've set forth, but we haven't really constructed the company in some form as to, yeah, you should be spending every second Friday doing this very specifically with your paired engineer as an example.

So, if you had to recommend kind of space-time allocation for... Of in-house experts, whether it's the operations team or it's GTM team members to work with their paired person to work on workflow issues like this. What would you suggest in terms of allocation of time? I wouldn't do like once a week.

I think this needs to be continued. So I would highly recommend at least a whole week, fully focused on that. Ideally, you're gonna need more. Across the company to be nervous.

No, I don't think any. No, no, no. I cross the company. No.

Because then you can't stop to only do that. I would try to choose, okay, let's prove that this model can work. Choose like a group of engineers that someone should lead, that will push back on everything. And then choose some, a few subject matter experts and say, hey, this is your day job.

Your group for the next one to four weeks will be fully focused on this. And you got to deliver. And Don't you create them. To choose the most complex workflow, but choose something that you can do end to end.

It did help that I was in the room because you normally have like leadership trade-offs. And even though the team was absolutely amazing and they did all the work, I think being there in the world allowed them to move quick, to not having to go to other teams to get feedback on how to do that because I was just breaking down like the decision right there for them. Oh, but coming up we do A or B. So instead, they're like, OK, so now I have to wait this Friday meeting to get everyone on board.

No, we made the decision quicker. And even if you can't allocate your full time, today's all like you don't have a leader that could allocate four weeks to this, allocate all the mornings. Allocate a block of like three hours a day. That's then all the decisions that this group needs should be done.

Someone is unblocking right away, and everyone in the company will respect that. I think so then the leadership, subject matter experts. And the technical background is the perfect combination. Everything you're sharing is so valuable that I feel like I just want to ask you, what do you, it's a bit of a reflective question, but like what are the five biggest tips you have or the five things you learned that everybody should focus on?

I think we definitely have leadership needs to be involved. You need intensity of work and do something end to end. Do a process end to and rather than trying to just optimize a process. What are your other pearls of wisdom.

Expert with the technical team that really helps. And then we already talked about the end-to-end. It's slightly different but very similar. I would say have a very clear goal.

What do you want to achieve? Not only like automate something but what actually you expect? Is it a full automate process end- to-end or is it 80% less of something or you want gets a much higher conversion rates, but having a very clear goal that helps with the trade-offs helped us here too. So I want to be able to address the markets that we are not able to to address.

I thought this was easier because for this first experiment that we had, I didn't want the whole company to stop. I wanted to do something that was incremental because if I chose like something to prove a concept that was part of like the day-to-day, I would be running into a lot of roadblocks. So I chose a segment that in the past we had failed to serve, so then it would be crazy if we fixed. So I choose the segment and said, okay, we're going to find a way to serve the segment.

But then it's kind of like a proof point. Okay, if you're able to solve this, now let's test with something else that scores the business. And the second one was like, okay. Actually, move from this little tangential thing to the largest flow that we have, to the largest volume, and we said, now we need to fix onboarding.

So before onboarding was take five business days, I want to do this. In five minutes. So this five to five, it was what led us to believe in what it means. So did you actually look through the business and identify your biggest wins and the most painful areas and then pick them off?

The way that I prioritize what we need to invest was different. So the first time we always had the prioritization and then I went then, hey, I need to deliver this business goal. I need you to drop cost to serve by half in 24 months. So that was very clear to me.

And then I had to prioritize, okay, what's driving cost to server and like, what are the things that I'm gonna tackle? So that's how everything started. For the first operating model experience, which was like how we created the Hacker House, I tried to create something that was actually. Not a business goal for that fiscal year, because if I was able to do that, I could move quick and deliver upside that then everyone would say, okay, we have to invest on this new operating model.

For the second one, since I had a very proven and clear model, it was more like, okay this is the way that make things happen and quicker. So let's choose like a real problem that will deliver the business's results for this fiscal year and tackle the cost to serve goals. And again, we did it! Now, I don't need to be in other rooms anymore because people have learned how to operate.

This has been spread out in multi-blocks of multi-areas of the business. What were your biggest surprises? The remaining. For the first one, I was truly surprised that it worked so well and so fast, in a sense.

I was like, okay, that's impressive. That's amazing. I was really surprised to see how Everyone was adopting to AI in different ways. It was so funny for me to be in the room and some engineers be talking with AI to code, also seeing operations being able to create evolved data sets that they didn't even know what it was, I myself connecting to an API to be able to do.

So I think I was surprised that when you put it there in the challenge and you remove distractions, everyone. Got excited to learn something new, and even if you didn't, that was a good surprise. On the not as great surprise, on the second one that we did, we chose a much larger problem that was already core for how the business ran. The first one, we could wait and release the code release, go live all together.

For the second, we released in batches. As an operator, I'm always pushing back on engineers when they release things and causing stands if it's not fully ready. So for the second one, actually this, this group that I was leading created a ton of incidents, which is a operator nightmare. So like, I was like, almost like when we would go to the stability meetings, I like, Oh my gosh, this was like a, there's this process that was leading, but I mean, you don't build until you break things.

Right. So it was interesting to see when you're really like speeding up. You need to be careful. So sometimes doing something fully on the side is much easier.

That's why companies that are studying is normally much easier when you have to do things on the core, you're gonna need to trade off and you're going to need to understand that the mistakes will happen. You need be there to fix quickly. And the outcome is real. And so you were just brave about making mistakes and accepting they're going to happen as long as you're there to fix them as fast as you can.

At the moment, I got some critiques on that and I criticize myself, but after seeing this, after the outcome was so great, yes. So there's a lot of bravery there. How many sleepless nights did you have? Oh, many.

It's funny. I was like leaving the office midnight, like those days when we were doing the Hacker Houses. We are having lunches and dinners in the office, but it was fun. I'm not joking.

Like, after that, it's like, yes, you're looking for things to go back to normal. But everyone involved on that was like, oh, this was like one of like the best moments of my career. I learned so much into this. So it's a trade off, but I think it was fun.

I think that's continuously you can do this every day, but that's why I think like intensity for like a short period and that, you know, beginning and. Is really helpful. So here's a question for you. So now that you're no longer in the room and there is something you inferred this slightly but throughout the organization you've kind of it's now self-perpetuating to some extent where folks are just doing this.

Organizationally as a CEO have you done anything to support the company to ensure that it was going to be embedded? So any kind of like you know showcasing of demos type situation or what was your way of vetting it. Two different things. One is from an engineering perspective and keeping the ops partnership with engineering because I think it's key.

I want to make sure that it wasn't a road map and I'm still very close to overall cost of the road map, how we're going to prioritize and why. So we continue to deliver the goals. On the other side, I want make sure that I have more and more SMEs to get ready for that. So we have monthly ops on hand where we always celebrate the AI spotlight.

During the whole month, we incentivize all the new, everyone in the team to showcase and make sure that they are creating their prototypes or things that are helping them to be much more productive so we can roll out across the organization. And we define change recording. The case studies for joining operations are completely different now. You are expected to use AI to resolve the case study and I actually asked you to see the prompts.

Of what everyone uses as part of the case study, and I joined the final round. So we have this entry level that everyone helps join this entry-level, and we have final rounds of interviews, and that's when we receive the prompts from everyone. So we talked a bit about some of the restructure in terms of removing the L1s or not hiring more L1. But if you found changes to roles, and are you rewriting job descriptions?

Are you just letting it kind of organically change? Yes, so it's definitely like rewriting job descriptions, like the job to be done change. In the past, you expect to make a decision, follow a procedure. Now what I'm asking you is to create an agent to do what was the procedure that you wanted and it needs to perform.

So the business outcome is still the same but sometimes the skills change. So as much as I want everyone to get there, to get the skills, it's also a different profile. So that's why I need to change the recording to get like the right profile in house. So it goes in a mix.

You make sure that you give opportunities for folks to get there, and you keep hiring more like this talent that is very native using all those tools. And like in this mix, you get both, like people that know the business, people that don't know the tools, one teach each other, and you move forward. And have you found, so speaking to somebody else who's measuring their AI adoption by some of the normal metrics, plus how many handovers have been eliminated? Yes, this is awesome.

We do have some metrics as handovers elimination. My only concern, and I'll go back to this, is sometimes when you eliminate some handovers, you still have a fee where that's whoever's looking to this. We maybe need to look at the whole history. So I always say that if you remove 70% of the handovers you never get 70% off efficiency.

You may get 10% to 20% of efficiency with the whole flow. So I really think like the more that you can, not only think about the handovers, but to think about this whole process, you get a real. Effective outcomes. Sorry, so that's what I mean, is the handovers are eliminated because the process has changed and is much more streamlined.

So that's the way I'm talking about it. Oh, okay. Yeah. So like then, basically what we mapped is like all the flows that exist that are like operational heavy, and we're going tackle one by one.

It's like an underwriting decision, a fraud review, a reward check, a the deal desk flow, whatever it's like, or how you talk with like the customer, how escalation happens, whatever, it is like those are the things that we're mapping to go tackle one more. You're mapping it and you're restructuring it rather than just automating what. Exists today? We are automating or like getting the team to get in the automations themselves.

But yes, we're basically re-automating, reinventing all those flaws in a way that they become I need it. And are you finding that the flows are actually changing or the flow is the same, but now AI is doing it? It's a mix. Sometimes for AI to do it, you need to slightly change the flow.

But in many situations, if it was a very prescriptive flow, AI can do it. But if it's not that prescripted, sometimes you have to redesign how the flow would look like. So it actually needs to be more prescript if not less for AI. Yes.

So when that's why it's the beauty, you have to break down to very, very, very small steps because if it is tiny, it's very prescrptive and that you can fully do. We are running out of time, a fascinating conversation, but everybody has to answer the final question, which is if our listeners can only take one thing away from the conversation. It's possible, intensive matters. Make sure that you have operators working with engineers because this partnership is very powerful.

Lovely. Thank you, Camilla, for joining us on the operations room. If you like what you hear, please leave us a comment or subscribe, and we will see you next week.