Building the Agentic Software Factory - Thomas Dohmke and Matan Grinberg

Welcome back. I just got the surprise that it's actually three people who are gonna be on stage now. It's not gonna be a presentation, but it's gonna be a panel where they're gonna discuss about the topic that I explained to you earlier. So it's about building the Argentic software factory.

So I want to introduce you to the three folks that are actually on there. Uh first of all, there's Thomas Domke, uh CEO and uh founder of Intire. If you haven't seen it yet, really, really cool intro video, really cool stage outside. So please, um, I don't know if he's ready or if they're still chatting.

Thomas Domke? So from left to right, we got um Frederick Ladinois, senior editor at AI at NewStack. We got Thomas Domke. And we got Martan from Factory.

And they're all gonna tell us about factories right now. That's have fun. That's the plan, thank you. Yeah, software factories.

That's the topic today. And I was thinking before we do that, factory AI, I'm not sure everybody knows it, despite your five billion dollar valuation, and tire is still pretty new. So why don't we kind of start with an introduction first? And I was thinking for factory AI, kind of explain to us what you do, and I think for a lot of the developers here that are familiar with coding tools like Cloud Code and Codex, what is it that's different about factory AI?

Yeah, absolutely. So I guess to start, our mission is to bring autonomy to software engineering. Um and maybe a more kind of concrete way of putting that is we want to make your software improve itself. Now, to make your software improve itself, it's not just a matter of, you know, do you have one agent running or ten agents running or a hundred agents running?

It's really you need to build a system which today we're calling software factories, that has the ability to ingest the signals from the market, from your users, from your incident tracking tools that will take them in and then determine, hey, what are the things we should be planning or implementing, testing out, things like that, but basically just ingesting into your software development lifecycle or your software factory, um, and then taking it all the way through into deployment, into production to then continue that measurement and kind of close that loop.

Um and so the means to create software that improves itself software factories, that is what we give to our customers, and we do it in a manner that is model agnostic, you know, so you can uh go across all the open AI, anthropic, you know, open models, um dynamically route between them, uh, just to make sure that you can optimize that software factor. Okay. Thomas, most people probably still know you as the CEO of GitHub, and Thomas said he cannot fix the outages, so you don't need to ask him.

But explain to us what Entire does and how does that fit in with what Natana is doing. Yeah, one day you don't have to say anymore that I was the CEO of GitHub, I hope. No, um uh if factory is here, we are at one layer uh below factory, we're building uh the semantic layer um that stores all your agent sessions, um, uh connects them to your commits, we call that checkpoints. Um uh basically the memory that you use across all your agents, including uh factory AI, which is one of our partners.

And then one layer even further down, we're building a new Git backend that can scale uh with the agents. Um Git, um, as we had it for the last um 15, uh, 20 years now, um, was designed for human-to-human interaction or collaboration. Uh, but now we have human to agent, and soon enough the factory is agent-to-agent collaboration. So the frequency of the pushes is much higher.

You know, these agents work 24-7. I think most of us um still uh want to sleep and have a nice weekend. And so we are building a new platform, a new GitHub, if you will, uh that focuses on agent to human collaboration. I got it, I got it.

Now let's go back to the software factory, because it's still a pretty new term. Can you go like just a little bit? How do you understand that, Thomas? What's a software factory to you?

The difference between uh like a real factory and a software factory is in a real factory you have your materials coming in on one side, you know, all the parts or whatever, then you have a um, you know, a conveyor belt or like a moving assembly line, and then on the other side the products are coming out of the factory, and you know exactly how long it takes, you know, from input to output to produce a single uh product, and then depending on how many shifts you have, you know, you know how much uh parts and and uh you can products you can make every day.

That's not how software ever worked. And you know, if if you have um uh worked in in a process like Scrum or Agile or or any waterfall and whatnot, you know that all of that is to some degree made up because we just don't know how long uh it takes to build a certain feature, and it's wildly different between a new codebase, a green field as we call it, or an existing codebase, uh brown field. Uh and so these these new factories they need to account for that. They need to uh account that even if sometimes you know my cloud code says this will take 15 days to build, and then three minutes later it has built the whole thing, right?

Because it it's it's using it's using wrong data to estimate how long to take. But these new factories need to be designed around that uh as software developers and as businesses, um we we we still want to be profitable. We still want to, you know, uh have an OI on the token investment, the energy that we ultimately put into these factories, and I think that's where you know a lot of the innovation will happen over the ne over over the next few months and years. Yeah, yeah.

What's that going to look like from your perspective as you're building this? Yeah, I mean, I think the the uh visual that we usually use here when we talk to developers or you know uh engineering leaders about this shift from just agents to software factories is uh we kind of reference, I don't know if you have seen videos of like Tesla's factories. But if you look, it's like you know all these robotic arms going and working, and there you don't visually see that many humans in there, but it's still a very complicated engineering problem to assemble that factory.

And it's just kind of the analog of what we're gonna be doing, but with software, where the human engineers, instead of spending their time writing the code, screwing in the little thing and doing that, it's instead the human engineers are building the factory that builds the software. Which is an even more interesting problem, because it's now you're going from let me go and manually do the thing, to how do I abstract away the process that we've been doing for the last 20 years?

How do I do it in a way such that uh we're not codifying the things that we have done but we shouldn't be doing? And how do we instead emphasize the things that we should be doing more of or that you know are lower leverage for humans, and so we want to go and automate that so that then the humans can spend their times time on the things that are way higher leverage. And mapping out this software factory and the decision-making criteria that you've built up over time, this is a really, really interesting and frontier engineering problem.

And most of the people who start you know going towards this future of software factories get to now work on that instead of the kind of boring implementation problems. We'll get back to that with the role of the engineer is there. But how close are we to having like an autonomous software factory? I mean, we have it today.

It's just how good is it, or how good is the software that's ability. It's kind of like software factory. How good is factory? Sorry?

How good is factory? Yeah. Well, I mean, I think with like with factory, you can do this right now. And I think software factories are kind of like schedules where everyone has a software factory.

Everyone has a schedule. It just might be very disorganized or very messy or very kind of not formalized. So whether you know it or not, even if you were literally writing code on a piece of paper, that is technically a software factory, just a very inefficient one. Now, um going and setting up like a you know decades-old bank to be a fully autonomous software factory, that's gonna take a lot of time.

But if you're building, you know, uh an application from scratch, you can build it in a way such that it is native to this autonomous development. You can do that today. Um the problem is just as you scale, it's a that much harder because there are different people involved who have their different preferences and different styles. How do you decide which ways to standardize?

Um especially like it, and it's again, it's kind of like GitHub where when you have you know different branches, except instead of branches of code, branches of your software factory, where one person maybe wants a certain process to look one way and the other wants it the other way. How do you reconcile that? Um so I think there's still some nuances and uh uh interaction patterns, both uh human to human, human to AI, and AI to AI, that we haven't yet nailed. Um but we're starting to see it emerge at the frontier, which is pretty exciting.

Thomas, how close are you to implementing a software factory at entire? I mean building on what Martana said, I think the the fundamental issue is you can you can today run uh agents 24-7 and they can open uh hundreds, if not thousands, of pull requests, both on your internal repos and and your open source, um uh your own open source repos, or even more so on other open source repos. And what happens is that those people all will uh uh go crazy about it because you're just increasing the the load that somebody has to review uh uh 10 or 100fold, and um and you're doing that by burning tokens.

So you could say, you know, in a way you're investing into open source by running tokens, um, agents, harnesses against an open source project, except the maintainer doesn't necessarily want that, uh let alone uh has that as part of their roadmap. Uh and now you're forcing them to either spend human capital or token capital to review all this. So, one, we haven't actually solved the code review problem. Right.

Um, we're not at a stage, I think, in most organizations, um, uh whether it's the enterprise or uh startups or open source, to just automatically merge or YOLO merge at all. Um and um and that's fundamentally a trust problem, right? We haven't we have seen enough of these incidents where something got merged and then it broke something in the system of an outage or worse, um uh a data leak. Um, and so we're not um and and the more we go into like uh regulated industries and and companies that have uh 30 years of code base, which are more and more companies, right?

Because now we're heading into the uh uh late 2020s, 30 years ago was the late 1990s, so a lot of the dot-com bubble uh uh companies now have 30 years of code base. You can't just run an agent on that, auto-merge it, and and then hope uh uh for the best. Um the other one, and Matan alluded to this, is which like um it doesn't make sense to pay five thousand dollars in tokens if I can do it in five seconds myself, right? Or it doesn't make sense to have an agent read my emails to then summarize these emails to me and I'm spending time reading the summary instead of just responding uh responding to those emails.

And there's I think lots of scenarios today where it's kind of cool and magical to use an agent, uh, but at the same time it's it's much more efficient if you if you actually do that yourself. Right. And finding that balance and figuring out how we can run these factories autonomously over the whole weekend and not waste a lot of tokens and energy and ultimately money, uh, it will be crucial for every business because otherwise I can't run my business if on Monday I come back and I'm proud that it ran all weekend long, but I'm frustrated.

And you know, I've had moments where I put in my prompt, what the hell did you just do? Uh it's oh, I apologize. I'm like, I want a refund, and it's like, oh, here's how you open a support request, and all of a sudden the agent can no longer be an agent, and it just sends me to human support at uh at one of the model providers. So I think we are there from a technology perspective, but we're not uh actually there from a business perspective.

And bringing those two uh into alignment, you know, I think that will be the challenge for many of us over the um over the time to come. For sure. The era of token maxing lasted, what, two months maybe? So we're already past that.

But you brought up code reviews, and you've talked about that in the past too as one of the bottlenecks, right? And I think it still is. Um, Matan, how do you think about that and how do you handle that? Yeah, I mean, I think it's there's kind of this same systematic approach to every part of software development, which is figuring out what are the time sinks that are low leverage and kind of a waste of our engineers' time, and okay, how can we standardize that and automate it as much as possible so they can focus on you know the higher leverage, you know, minutes and hours of their day.

Now, code review is obviously very high leverage because if you do something wrong, you know, you might uh it might lead to an outage to your customers, you know, very disastrous for the business. Um but there's a lot of code review, especially as it was done like five years ago, that is not particularly time efficient. Like, you know, oftentimes it takes like hours just to even understand what's actually happening. Or there are just certain things where to be methodical and like check every line, like humans aren't best at it, agents are.

Um and so we uh at like one of our goals is trying to get to a point where as many of our PRs as possible are able to be kind of fully reviewed and auto-merged just with factory on its own, but then also detecting, okay, which are the parts of the code base where we're gonna need a human to uh kind of be in there no matter what. Um because I I think it's important to know we don't want to go from just like completely human to just none at all. We just want to have better resource allocation.

So if we're gonna spend an hour on code reviews in a day, let's just make sure that hour is spent very well and very kind of uh methodical. Um but examples, I mean, to to the question earlier about software factories, like the agent that we build is model agnostic. These days there's like 10 models that come out every week. If we had a human go and make sure you know it works in an optimized fashion for every model that comes out, we would need to hire like a team of a hundred people to just constantly be doing that and then reviewing all of that to make sure it works well, and doing all the evals and then you know putting it into production.

That is something that we've built a fully autonomous software factory loop for. Where we used to have engineers working on that, now we don't, and it can even auto-merge. So a new model comes out, it'll go through our the software factory, it'll go through the evals, it'll iterate based on it. If it sees certain ones aren't performing well, it'll fix those, merge it into production.

Then there's certain things of like, you know, core part if there are changes to the core agent loop, we're not gonna auto-merge those. We're wanna we're gonna wanna get whichever engineer is responsible for that, kind of as much important context as possible and as much focus as possible, so they don't need to worry about the thousand other lines of code that don't really matter to the crux of the change that's happening. Um yeah, we we we really think about it as a resource allocation problem.

If we're gonna spend a certain number of minutes on this review, what where should that human spend their time and how do we get them up to speed as quickly as possible? Is that what you do at Entire as well? How much have you automated there in the review process? It's not automated uh in the way that an agent can automatically merge with no human supervision.

Yeah. Uh but it is automated in the sense that almost nobody reads every single line of code anymore. To give you a real example from this morning, um uh in the last few weeks, a friend of ours took our company uh robot, uh, we call him Marvin and made like a real 3D printed robot with electronics and all that. And the all the code is written uh uh with codecs, and um uh uh you know you built a GitHub repo uh and we made that.

I wanted to make this public this morning so when later he demos it on the stage uh on the booth, uh he can say just go to the repo. And when I was clicking the change visibility button on GitHub, uh I was thinking, hmm, maybe I should check that there are no open AI API tokens in the code base, right? And I was literally about to leave my hotel to walk over for the session. And sure, you know, I I could have postponed it and read every single line of code, or I could have tried to figure out how to install a secret scanner on my computer, or and that's what I did on my phone with codex remote control.

I just said, you know, scan this repo for any uh uh secret leaks and similar things, and it uh ran through, I think, seven secret scanners and it scanned the code base and also figured out there's some minor issues that have nothing to do with the secrets leak, but it gave me by the time I arrived here the the green light, there are no secrets in the code base. Uh and then I clicked the button, and so uh Stefano can can demo it later. And that's automation, right? Like I did uh I offloaded a process that honestly I don't want to do, you know, so I have more time for the things you know that I do want to do, like like this session.

But how many tokens did you burn just to do that? I mean, I it was you know within the credit limits I have. That's uh because the agents are now smart enough to uh not try to read every single line of code um uh and understand is it a secret or not, they're just installing installing a secret scanner as well. In fact, the whole firmware uh for the robot was all written uh by Codex.

And and I installed the code base on Monday on my computer so I can flash it myself and install all the tools to flash the microprocessor and all these things. But the point I want to make is that is there still a need for developers to understand code? Absolutely, because you need to find the moments when the agent does stuff that that is bad for your business or bad for the customer. Uh but are we uh better off when AI reviews every single line of code uh in a sh ever-increasing code base over us reading every single line of code?

Absolutely. Like there's no way, you know, on a Friday night uh before you want to head in the weekend that you're finding exactly all the issues that the agent can find. In fact, your human nature will mean you're finding three or four or five, and then you feel good about it. You had enough dopamine, and you can kind of say, look, you know, I found something, so clearly I did it.

Um, and uh and now I can hand off the work to or back to my my coworker who who authored the pull request. Yeah, yeah. Now, one thing, Mitan, you kind of hinted at it is what how does the role of the developer change in all of this, right? Because this morning I was talking to a bunch of people here, and I it was pretty easy to send some anxiety among developers, right?

What's next? How does this influence their role going forward, or does their role even exist going forward? So you kind of hinted at how you're thinking about that, but maybe go a little bit deeper there. Yeah, I mean, I think first of all, what got most people into engineering or software development in the first place, and what makes the best engineers and software developers is people that like solving problems, that like systems thinking, thinking around multidimensional constraints where the solution is really not obvious, and you have a lot of different trade-offs and a lot of kind of different constraints to be considering.

Those are like the interesting problems that people are excited about in the first place, or owning end-to-end outcomes, like building some new kind of entirely new product, or delivering some new experience to users. That's been that's what initially got people into engineering. But then the reality of engineering for the last 20 years, especially in large companies, is you own like this little thing. We can only ship three features in this quarter, and you must own this little thing in this part, like you know, you're you're a cog in the machine or you do this little thing and that's it.

Um and then it turns out that in order to go and solve these constraints or go and you know implement these solutions to these problems, you need to know the secret incantations to whisper to the computer in order for the computer to do the thing that you want it to do. Um now, in this new world, uh it turns out that you don't need to whisper the secret incantations anymore. Now, a lot of us feel very sentimental about the incantations that we've spent decades learning and understanding and you know, growing fond of, but we kind of need to remember what initially got us into this discipline in the first place?

Was it we really like the sound of the incantations, or we like solving the problems and delivering the outcomes to the users or to our friends or whoever we're building for? Um, and so I think there is a little, you know, it is always jarring to like be used to something for such a long period of time and now it's changing. But I think the biggest thing is just like remembering, like, and I actually ask this of every single engineer I interview, and even users, is I always like to hear like, why did you get into engineering in the first place?

And it's always something like, oh, well, when I used you know the Atari back in you know the 90s, or the I was playing these video games, and I love it's always things that are way bigger, not like, man, I really loved the syntax of C. And that just like like no one ever says that. Um, and so I I think we you know we kind of need to refocus on like why did we do this in the first place? And now these tools will allow us to go and do that in a way higher leverage manner, which I think is really exciting.

Like, we will solve so many more problems. I think people who say, like, oh, you know, engineers are going away, inherently or implicit in that belief is that basically there is a finite number of problems and we're solving all of them. So now there's no more need for engineers. I would argue there are a lot of problems in the world and we're solving a very small amount of those.

And I think the good news is people like engineers who are the best problem solvers in the world will now go and solve larger, more ambitious problems and get them done faster, and I think that's a great world to be living in. Just to push back a little bit there, I think it a lot of people just love the craft of building code, right? Yeah. They might not want to become architects or something else now.

What do you tell them? I tell them to keep loving the craft and evolving the craft. And just because you know I have now an Agent that can write code for me doesn't mean I cannot find time during my day to evolve my craft. In fact, I've been talking on these kind of stages a lot about that.

We need to teach all children coding early on. Like we can argue is it a first grade or third grade thing, but like in the same way, like why would I not learn coding, but I learn about Maxwell equations and Newton's laws and how a cell is constructed. In many ways, you know, coding is now an essential part of our life because we're all consumers of software on our devices that we carry with us, you know, uh at 24-7. When if you have a watch that you know tracks your sleep.

So coding is something that we should all learn and evolve our crafts. And there's people that are really, really good at this in the same way that people, there's people out there, and you can watch them on YouTube that are really good at manual woodworking, uh, and obviously manual making music, acoustic music, and and all that. And um, and so that I think that's number one. I don't think the craft goes away just because we have a machine.

Uh, and the same for what it's worth with many other things on the planet, right? You uh like to cook yourself even though you can get a pre-made meal uh in the supermarket. Um I think the second part uh is that um look, you know, the uh we as developers we've always evolved and we've always found ways to build more. Because let's be honest, my backlog is longer than ever.

And I don't know any company who's coming to me and saying, Oh, we have no more ideas. You know, we have we ran out of items in our backlog. We have no more bugs, no security vulnerabilities, no feature requests, no, no. The only people that have that is those that have no product.

And everybody else has an ever-growing backlog. And I don't think you know that we have found the factory that can you know get through the backlog faster than we can uh generate ideas. And and so every software developer will be a developer. That means they take ideas and convert them into software.

And they will be able to code and they will be able to understand code at a certain level, you know, who who still you know in the room uh understands COBOL or assembly language and those kind of things. But um, they will every day um you know take more and more ideas uh uh and and make real products out of them. And and for me at least that was always the dream ever since I saw a computer in the in the late 1980s. Now you've talked about the idea of the middle class engineer, right?

Talk about that a little bit more and what happens to those. Every engineer you know that has been in this industry uh uh has uh con has seen that there is um you're you're already you're all you're only preserving your job by constantly learning. Um we have gone through these uh different phases, you know, I already mentioned assembly language and came basic and I learned Pascal, uh, but it was before the internet, so you did it all learned by yourself and my first company. We shipped floppy disks uh in in envelopes uh to people uh in Germany.

And then came the internet, and all of a sudden, not only could you uh chat with others and ask them questions in the middle of the night, uh you could also download open source, right? That's that's really the the big shift that we went through in the late 1990s and then with GitHub again in the in the 2010s. Then obviously came the cloud and mobile devices, and so we have always this before and after. And and today, when you when you're running a startup, you don't have to buy servers, you don't have to think about data center space, network cables, you're not you're not hiring people to manage all of that.

Uh you can just spin up you know uh a Kubernetes cluster on any of the cloud providers um and and get up and running. And I think those transformations we have gone through many, many times, and this is just one of them. It's a big one because it touches the thing that is dear and near to many of our hearts, uh code. Uh but I think it also uh has brought so much fun and so much excitement to the to the scene.

And if you look at uh companies like Lovable, it has also brought you know hundreds of well not millions, hundreds of thousands um uh new people into the craft that will, because they have seen how easy it is and how fun it is to build software, naturally also start learning a little bit more about the code that Lovable generates. He has seen them in the computer museum and how many junior engine engineers have you hired in the last year? I think the it's funny, the the differentiated value of someone who has decades of experience is actually less now than it was before.

Um because you know, right now I think it is more accessible than ever for someone who is just very high agency to go and solve problems. I think traditionally, and people talk a lot about the roles of product and engineering and merging and whatever, I think generally the best product people and the best engineers are good at solving problems that have multi-dimensional constraints. The only difference is the engineers know the secret incantations and the product people don't.

That's basically the only difference. Otherwise, they have a lot of common traits. And I think similarly, uh the best builders these days, whatever label you want to put, you know, engineer, software, developer, whatever, the things that matter most are just high agency and just like IQ, raw, firepower, whatever you want to describe it, just like you know, they're smart and they have high agency. Um you can be 30 years into your career and satisfy those constraints.

You can be two months into your career and satisfy those constraints. I think that the biggest thing is how do you identify the one who is earlier in career and high agency, and how do you identify the one who's you know 30 years into their career and low agency or high agency? Because sometimes when you have a lot of experience, you can mistake that experience for agency. Um and I think that's the biggest thing.

But uh years of experience matters so much less now, generally, than unless you're doing some hyper-specific use cases than ever before. Right. Final question, like for the audience members here, what should they do to prepare for this new world of software factories? We already uh checked the first box, which is attend the conference and uh learn about all the uh uh uh new developments in technology and all the companies uh on the show floor.

But I think it's um just use AI and try to build something uh with an agent um uh every single day. And I think a lot of um uh uh Martan mentioned agency. I think autonomy uh is the other piece that plays into that. You need to learn how you actually prompt it and plan uh the agent loop so it can run through the session uh uh or through the through your day or hopefully through your night.

It's not an achievement if we wake up every 30 seconds or every every every every three hours uh in the night or during a meeting to give it another prompt. That's not how we want to uh design these loops. You want to be able to design of them, and I think um uh the ones that are leading uh the industry right now are the ones that have practiced this a lot over the last year or so. And uh we ourselves need to also ramp up and and and and and try it out.

And that's that's very similar to how we learn coding. You become a good senior or principal developer by learning and practicing the craft of coding. Now we have to learn and practice the craft of managing uh fleets of agents. Yeah, and I think just seeing you know, friends, you know, people who have joined factory, you know, colleagues, all of this, uh the people who end up doing the best and kind of scaling in this new era are the people that embrace, genuinely embrace the uncertainty and the high variance of everything that's happening right now.

And instead of being uncomfortable by it, just diving into it. It's like I have no, like it's okay to embrace and say, I don't know what exact skill is gonna be most important. I'm just gonna try doing it this way, try doing it that way, see what I like. Instead of like every generation ever, every single generation for the last thousand years is always like, oh, this next generation, they're messing it up, they don't know what they're doing, they're losing the important things that only we understand now.

Um and just remember that, and even despite that, you might find that you're right, but just go and experiment and see what is this kind of new era. Is it real or is it BS? Um, and by doing that, you're gonna stay at the frontier and learn and then you know be able to teach others. And I think just embracing that discomfort and uncertainty is probably the the most important thing.

All right. We shall go ahead and embrace the uncertainty. Thank you both. Back to the agents.

Thank you.