Autonomous Software Development at Enterprise Scale: Inside a 1,000-Developer Pilot (with Blitzy) | CXOTalk #918

Speaker A: The percentage of all the completed work done autonomously by blitzi it ranged from 80 and 95% depending on the use case. We accelerated development velocity of between 5 and 10x. The human is not writing the code. The human is directing a platform on how to write the code. That's a huge change in paradigm.

Speaker B: That's Enrique Ibarra, uh, CIO and head of Business Transformation at gnp, Mexico's largest insurance company. Our conversation covers AI adoption, change management and autonomous software development.

Speaker A: Our main operational system is a mainframe based map with IBM components. System has been running for a little bit over 20 years. The initial rationale for modernizing this application was basically cost. That was one of the concerns. But it was not the only one. In a few years from now, uh, it's going to be much harder to get COBOL resources. I mean if you go to the universities, students don't learn COBOL in the universities. There's no real interest. That's going to be an issue I guess in the future too in terms of how do we give longevity to this asset. We started a few years ago incorporating the typical um coding, copilot coding. We incorporated several informally. We started to provide all of our developers with uh, copilots without formally measuring their increasing productivity. We just basically use it, find it useful, take advantage of it. And that's being expanded among um, all of our developer base and we have roughly about a thousand developers. By looking at different industry solutions we find we basically learn about Blitzy. Uh, the value proposition sounded too good to be true. Uh, it was a little bit like magic. It's like give me the specifications, give me the code and we will autonomously generate everything. It was very attractive. So we went and visited Blitzy at their offices in Boston to learn what was their vision, what was their product. Just meet with founders on the management and we liked the approach. It uh, was very aligned with what we're trying to achieve here at gmp. So we decided to uh, execute a pilot that will sufficiently evaluate the abilities of Blizzy in our own environment and with our own code.

Speaker B: Tell us about that pilot and what were your goals for the pilot?

Speaker A: The goals of the pilot were to evaluate the capabilities of Blitzi uh, as a uh, software development platform. So um, what we did is we selected one, an existing system that we have, real system that actually had a number of requirements that had to be executed anyway. And what we decided is to test different types of use cases using this existing system. So one of them was a backend migration. I mean the backend was written in an old version of Java Java 8 and we needed to modernize that backend to Java 21. That was very clear. Same thing with the front end. Front end was developed in a very old version of Angular. I think it was angular 11, I might be wrong. And we needed to upgrade it to one of the latest versions. We also wanted to test a specifically use case to build a new feature autonomously that is providing the right prompt to the platform of describing the new functional feature in business terms that we wanted to build and having the system build the new feature for us. And then a fourth use case where we wanted to do the remediation of a number of security vulnerabilities that the system had. So we thought that with that breadth of use cases it would be a nice spectrum to basically test the capabilities of the system in a real life environment, which was our environment connected to our GitLab repository where all of our code resides and connecting it to our CI CD pipeline. Basically that's what we wanted to test.

Speaker B: Why didn't you just give these small teams the opportunity to vibe code their way into this modernization?

Speaker A: That was perfectly possible. Most of our teams, they're using copilots to basically work faster, um, and have the copilots partially develop what they're trying to develop and test what they're doing. But this is a different type of platform. I mean there's no IDE initially here, it's just a platform. You provide a very detailed prompt of what you want to achieve and then the system using its own internal agentic um, architecture basically autonomously creates all the different software changes or creates the new software that needs to be incorporated into your project. That's what we wanted to test, yes. I mean we could have given this to a team, um, given different, not only one but even several types of uh, co pilots and we're working that way already. But we have never worked with an autonomous platform. And the idea was to explicitly test a platform that has a different uh, paradigm of usage than the rest of the bytecoding uh tools.

Speaker B: So you were really looking at shifting the strategic value of technology development, right?

Speaker A: Exactly.

Speaker B: What does autonomous development mean in practical terms at gnp? And how does blitzi fit along with the other tools in your AI code generation stack?

Speaker A: We are interested in increasing agility as much as uh, it is feasible. That has been our objective for many years. Adapting methodologies, incorporating tools, streamlining our CICD pipeline. Uh, in order to try to achieve agility we keep improving, trying to make changes to improve agility, we want to change the role of humans in the software development process. Um, we want to do it to achieve agility by coding. Um, all the tools that assist workgroups are great, they do provide value. But we wanted to test a new generation or a new type of tools for generating software in an autonomous way. That's the value proposition of Blitzy. It sounded very attractive to us and that's what we wanted to test. What's the value of that is, um, you know, you have to learn a new skill which is being very good at uh, prompt engineering. You have to be, you have to create very precise and complete prompts for the platform. But if you do, and that's a skill that, you know, I've seen that our engineers have learned fast. Once you get the right prompt, then you get the right results in a speed that was very impressive.

Speaker B: What about enterprise requirements such as security, governance, architectural standards? How does this approach support the corporate technology requirements that are needed?

Speaker A: You definitely need to be very specific regarding your own corporate guidelines. We do have our own guidelines, technical guidelines, we have our own security guidelines too regarding characteristics that code has to meet, um, the type of test we want to execute on the components that we develop. But what we found is that those specifications, those guardrails, those are part of the input that you provide to a platform like Blitzy. So you not only provide the functional specs and the intention of your project, but with all the functional specifications, you also provide all the technical and security guidelines too.

Speaker B: What results have you seen, uh, across development velocity, quality of code, cost hours saved.

Speaker A: We were very pleased with the results that we got. We roughly, uh, were able to accelerate a development velocity of between 5 and 10x in terms of engineering velocity and um, the percentage of all the completed work done autonomously by Blitzy. It range from between 80 and 95% depending on the use case. For example use cases that you basically want to upgrade from an old version of Java to a new version of Java. And you do have to provide the guidelines of how to do the upgrade. It's not as simple as just mentioned and just upgraded, just upgrade from 8 to 21 and I'm done requesting. You do have to provide a lot of information on how to do it, but those type of projects, we basically uh, saw that the percentage of autonomous completion was close to 100% front end. Uh, modernization of front ends is different, it's a little bit more tricky. But we got around 80% but. And 80%. You know what, basically the team told me is 80% is just wonderful. So I mean we still have to do 20%. Um, and um, for the reminder 20% they do by code it bringing their ides, they bring in their copilots and they finish the reminder 20% and overall that accelerates the process a lot.

Speaker B: So they're using blitzi to do the heavy lifting and then using lighter weight tools to do the fine tune polishing.

Speaker A: Exactly. To basically complete the project.

Speaker B: Enrique, how did the developers jobs, roles and daily work change as a result of this shift to autonomous development?

Speaker A: The role of humans changes. You still need developers. If you're going to develop a system you need to provide technical guidelines. You need to provide guidelines related to the platforms where the software is going to run and an end user cannot do that. I mean in our case we deploy our systems mainly in Google cloud so you need to know about the technical features and then um, um, on the technical settings that you need to enable in the cloud platforms in order for the system to run. An end user is not going to do that. So you still need a programmer but the role changes completely. I mean the human is not writing the code. I mean the human is directing a platform on how to write the code. So that's, I mean that's a huge change in paradigm.

Speaker B: How do the developers react?

Speaker A: You have to be careful with the change management and the human resistance to change is something that you cannot overlook. In general. Some of them were skeptic at the beginning. Uh, it was like this sounds too good to be true. As we started work with the platform and getting results, their attitude changed very rapidly and they started to get interested and they started to see value. They were intellectually challenged by this new type of work. So they not only accepted it but they were very motivated in testing the platform, in testing different prompt uh techniques to try to achieve better results from the platform. And I think that overall the group that so far has worked with uh, blitzi at gmp, they're excited about it and they're very uh, enthusiastic about basically uh, expanding the use of blitzi to the rest of our software development.

Speaker B: What advice would you give to other CIOs on transforming their engineering organization to be AI native? In this same way you don't just

Speaker A: flip a switch to full autonomy. That doesn't work that way. Uh, you have to build trust through a phased human in the loop approach. You need to target the friction, uh, you know, we need to deploy agents to first to handle high effort, low risk friction points first such as, you know this modernization projects of basically upgrading the programming languages or um, writing system documentation for example, or uh, generating test suites. This type of tasks that generally are low friction. Then you need to shift the engineering mindset. We have to train our engineers to transition from being creators to editors and orchestrators. It's a different role. And um, the human leader's job now becomes defining the prompt, reviewing the architecture and validating the AI's execution.

Speaker B: You are a technologist, you're also a businessperson. Can you describe the strategic benefits that this approach and the faster development speed unlocks for gnp?

Speaker A: Development speed isn't just about writing code faster. I think it unlocks different strategic advantages. And one is like first to market innovation. I mean, you know, we can design, deploy and iterate on like new insurance products in weeks rather than in months, or allowing us to capture market demand before our uh, competitors even react. That's one of the goals that we have. The other benefit and the other objective that we have as a company is improving customer experience. So this speed and this agility empowers us to continuously ship digital improvements so like instant claim processing or seamless onboarding and basically try to meet the high expectations of the modern consumer who is uh, basically they're more demanding all the time. It shifts our technology organization from simply maintaining the business to actively uh, dictating the pace of the Mexican insurance market. That's our goal.

Speaker B: As you plan to roll this type of agentic product development out across the company, what are your thoughts? What are your concerns? Uh, what's the approach that you're taking?

Speaker A: We are now a process of expanding the use of uh, the blitzy platform. We have incorporated seven additional teams now that they're going to be trained and they're going to be tutored. They're going to be some handholding of working with them between blitzy engineers and our engineers that have already gone through the different projects the last four months. So we're going to gradually start expanding the use and what we are going to measure and make sure that it gradually happens and becomes a reality is that our speed of execution basically improves at a, uh, very noticeable and exponential rate. And gradually we will continue to expand. I mean initially we have seven groups. Once these groups are sufficiently mature, they will continue working in this fashion and they will continue incorporating jobs. We think that in a two year time frame we will be able to change the paradigm in the company.

Speaker B: And how many developers do you have

Speaker A: Currently we use in between our own developers and uh, external developers from software factories, we have around a thousand Ideally, we should not rely on external developers in the short to medium term, and we will only keep internal employees basically working with platforms.

Speaker B: So speed was the driver, but ultimately you will also be reducing cost.

Speaker A: Yes, definitely.

Speaker B: Any final advice on how engineering leaders can get started or should get started?

Speaker A: With autonomous development, you can easily pick within your organization a use case. Either a, uh, very old system that need to be modernized or a system that is giving you a lot of problems, you know, during regular operation, because this has a lot of bugs or it has a lot of security vulnerabilities. And, uh, try these type of tools. I mean, you know, pick a use case, pick an existing system, and, uh, just try it and sandbox it. Do it carefully, you know, select the right initial set of engineers to work in this project to make sure that they will be able to, uh, do the transition and to understand this new paradigm. Test it, see how it goes. Um, you basically elaborate from there.

Speaker B: You just presented a very practical and wise textbook on AI adoption.

Speaker A: I think it's just common sense, but thank you.

Speaker B: Enrique Ibarra, thank you so much for taking time to speak with us today.

Speaker A: No thanks to you, Michael. It was an honor. Thank you.