Agent Harness, State of Play, Risk and AI Company Culture

Speaker A: Welcome to a new conversation about software engineering. Today it's me and Heinrich talking with Birgitta Berkeler from ThoughtWorks. Hi Birgitta, how are you doing?

Speaker B: Hi Sven and Heinrich. I'm good. Thanks for the invitation.

Speaker A: Yeah. Who's Birgitta? She's actually a second time guest. The first, second time guest at uh, Case. Uh, check out our last episode with her. I think it was 2018 or 2019 about cultivating architecture principles.

Speaker B: I think it was actually the very first podcast recording I ever did. So it's still in my mind.

Speaker A: Yeah, it was fun and actually I still pass it around when people. It's still a topic. Principles are still a topic. Um, but who are you? You are, uh, distinguished engineer at ThoughtWorks, uh, focusing on AI assisted, uh, software delivery. Uh, you are a technology radar, ah, member for I don't know how many years. Um, you also have a, I would say a go to blog post, uh, hosted@martin fowler.com. so whenever to our audience, whenever you want to have a very good and trusted source, go to her blog post. I mean also my colleagues, they also refer to the blog post. You know,

Speaker B: it's like a series, yeah, we call it like a series of memos because Martin suggested this to kind of say with Genai, everything's moving so fast that, you know, it's almost like memos. You know, it doesn't have to be polished. It's like continuous thoughts. Right?

Speaker A: Yeah, yeah, yeah. But I think it's uh, well balanced. You know, it's not. Everything is awesome. Oh my God. Hysteria. It's more. Here is a new idea and we tried it out and here you can use it and there you can't use it. So I think uh, it's a very trusted source, I must say.

Speaker B: Thank you.

Speaker A: All right, Heinrich, how are you doing?

Speaker C: Uh, I'm doing fine. Excited to be here and have, ah, such a prolific guest and such a hot topic. Very good to be here.

Speaker B: Thank you.

Speaker A: Yeah, exactly. It's super hot, actually.

Speaker C: It was.

Speaker A: I have the feeling it's getting hotter all the time, but um, maybe we can cool it down a, uh, little bit. Uh, we want to discuss, uh, three major plugs. Uh, the first one is the current state of play, um, when it comes to AI assisted software development. Um, two weeks ago there was this ThoughtWorks retreat on the future of software engineering focusing on AI in Utah. And Birgitta was of course there. And we uh, want to get some thoughts out of this event. And also a, uh, last point is, uh, what can actually leaders and platform teams do to support developers, uh, when it comes to AI assistance? So those are the major blocks. I hope we can also talk a little bit about the AI vampire. But let's see. All right, um, the industry is moving super fast. Uh, what is the state? Big question. What is the state of practice on February 17, 2:00pm UTC, 1 2026. So that's our current recording time. And uh, it's like politics. You always have to give the timestamp because maybe tomorrow a new tool, uh, arrives. It's a big question. I don't know how to structure it. Uh, but, um, yeah, in terms of tools, culture, uh, workflow, um, it's up to you. It's up to you how you structure it.

Speaker B: Yeah, yeah. It's also like we were talking before the recording of this, like, what are we going to focus on? Because there are of course so many different aspects of this. Right. But, um, I mean, to talk about the current state, I think it always helps to look a little bit at how we got here or like what the developments were on the way. Right. And that's often what I give conference talks, uh, on. And usually I just focus on coding assistance. But there's of course more than that. Right. There's like all kinds of other things you can do with AI on delivery teams. But if we focus on coding assistance, which is definitely like the um, biggest area, I would say, both in terms of potential but also in terms of activity, like what's actually going on then? Uh, when I look back on maybe April last year, so almost a year ago I gave a conference talk about this, like, where are we now? And at that time I was talking about the agentic modes encoding assistance, which were like just a year ago were quite new. Right. So this is like the kind of like, almost like advanced chat experience where the, um, coding assistant, uh, actually, uh, can execute bash, uh, commands and kind of like constantly goes back and forth. And I was talking about MCP model Context protocol at the time as well, which was still relatively new. Or like it was just having its first kind of big surge of um, hype, which is almost a little bit over now. Again, I can talk about that as well. But, uh, there are some alternatives now. Um, and also at the time I was talking about these rules files, custom instruction files. Right. So that was like the first simplest form of uh, um, how do you configure the agent a little bit. And I was talking a little bit as well about um, emerging ways of working, uh, kind of like workflows developers started to all report, okay, first I plan and break down the tasks, then I design with AI, then I implement. So they were kind of first workflows and of course models were evolving all the time in parallel to that. But I almost don't focus very much in my day to day research and experimentation on models, to be honest. Yeah, they're going to evolve. Yes. We recently had a good improvement again with OPUS and the GPT Codex models. But the much more interesting area for me is actually the tools and the integration and everything around that. And I think that will ultimately uh, make the most difference. And so that was kind of like April last year. And then I was recently trying to look back at the last 10 to 12 months, right. What were actually the big maybe let's say three things that happened, right? Because every day when you're in it, it feels like so much stuff is happening. Right. And I would say like the last year was especially kind of the rise of like background agents. There's still not like a, uh, uh, more autonomous agents, right? So basically agents that you send off in the background and that in an unsupervised way without me sitting in front of it and like steering and looking exactly what they're doing, just go off and do a thing. So last year In May, um, OpenAI released Codex and then Google immediately came out with Google Jewels and Cursor introduced background agents. And so that was the first time that we could experiment with that in an easier way. And then almost at the same time, Claude, uh, code was released in May as well, which kind of started this whole era of command line based coding assistance. Right. So all of the other providers, uh, all the other product companies have their own clis now as well. But Claude Code is probably by far the most popular uh, coding assistant right now. And at first sight it seems like the big breakthrough here is the, the terminal experience. Lots of developers seem to really like it, right? But I actually think that cloud code is so successful because they're just really good under the hood. I think a lot of that terminal experience you can also reproduce in cursor or like IDE based um, coding assistance. And actually cursor almost has a lot of the features that cloud code has. But cloud code just really works well under the hood. But the big thing that came with the CLIS is also that they can be run headless of course, which is almost like an evolution of this idea of background agents because now we have these headless uh, uh, assistants that are almost like a building block that we can plug into our existing ecosystem. Right. So we can more easily plug it into our pipelines, uh, and so on. So it's almost like an evolution from that and again makes it more accessible to us as development teams to experiment with these things and maybe try things like a code review agent in our pipeline or try things like cleaning up feature flags, little routine tasks and stuff like that. Just making it more accessible. Um, and then the third thing maybe uh, in the last 12 months that kind of like exploded is context, uh, engineering. So this is a term that started floating around in June or so last year. And um, it basically means something like the process of curating what the model can see with the goal of getting better outcomes. Right. And in coding agents in particular, this has really recently exploded and a lot of people are very confused. Right. Slash commands and rules and uh, skills and MCP servers and all of that type of stuff. So I can talk a little bit more about context engineering, um, as well if you like. But those are kind of like for me the big things over the last 12 months. So like uh, background agents. So like a trend towards less supervision and making that more accessible and more pluggable into our ecosystem. And all of these like this, uh, in Germany we say storm undrang, right? This storming and norming of like context engineering and figuring out how we can best give uh, context, uh, to the agent.

Speaker C: Uh, what do you think about uh, this open claw thing that came out like two weeks ago or something?

Speaker B: Yeah, I mean that is uh, uh, kind of not so coding related I would say. Right. Though, uh, as far as I know, the creator uh, of it did actually vibe code at all. Like vibe code in the sense of like didn't look at the code and so on. But it's like I think more this trend towards like personal assistant experience, plugging it into, into everything. Right. So I wouldn't say it's that uh, related to coding in particular. M. That's probably true.

Speaker C: Yeah.

Speaker A: I um, I must say I'm mostly interested in uh, in. In context engineering because uh, I'm a bit thin on agents. We can talk about it in a second. But to me context engineering, um, it looks very promising. So I must say I'm usually half a year behind because the customer, I mean on my local machine I have codecs, but at my customer we use, um, let's say we are always half a year behind with everything. Um, so uh, we only introduced agents lately. But context engineering already a bit longer and um, use step nine by the way. Uh, So I think those skills and uh, slash commands, although I haven't fully tried it because for reasons but uh, to me it looks promising because uh, I can tell it exactly, I can tell exactly what are the constraints, what to look at. For example, um, there is this tool context 7. Um, if you use that, correct me if I'm wrong, but it tells your agent exactly. I'm using Spring monolith modulate and context 7 exactly tells it how to write proper ah, spring modulate code for example.

Speaker B: Yeah, yeah.

Speaker A: Another thing uh, which is again I haven't tried it but it looks promising is um, we have some cross cutting uh, concerns across the, not only across the code base but also across the team and like some security guidelines for example. And I can also write that, right? So I have my Agents MD file and I say and I can write rules or maybe coding guidelines. And the problem is always you have those guidelines and it's very hard to um, you know, to spread the word that everyone follows the guidelines, but now you, you can just codify them and that's it. I mean that's my dream. But how does it work?

Speaker B: It's not quite it, but yeah, I mean um, yeah, it's just like basically everything with coding agents is like kind of about increasing the probability that they give you the thing that you need, right? So it will never be 100%, you can never rely 100% that you follow all of the rules that all the coding conventions that you give them. Right. But it is of course a really powerful lever if we can like figure this out in an intelligent, no pun intended way, um, to give it the information that it needs just in time. And so it's basically a combination of first of all, uh, usually lots of markdown files at this point in time. In the repository you were mentioning Agents md, which is this kind of standard, uh, or evolving standard to have a file of that name in the root of your code base. Other coding assistants also look for Claude MD or like, like all of these different formats, but it's basically a markdown file. Um, the problem is that this file will always be put into the agent's context, always. So the longer it gets, the more space you take up in the context window and you want to deal with the context window in an economic way because even though technically they're getting a lot bigger, the fuller the context window gets, the more hit and miss the results are going to be. So there's some um, observations of the fuller it is suddenly uh, the agent or the large language model Forgets the thing that you just told it two turns ago. So you have to be economic about it. It's also often a good idea to start a new session when you can see that the window is too full. So you want to be uh, both technically but also you don't want to overwhelm the large language model with information that is irrelevant because it might actually get distracted. So that's the whole thing of like how put just the right information in there. So when you have these files you can also um, split them up into multiple files and maybe say only load this when you're editing a python file. So that already makes it a little bit more specific to the file type. Um, or you can have these. And this is like um, the new thing that Anthropic introduced, the thing that's called skills. So you provide a description of what your skill does. For example, um, documentation of how we write react components. And then the model will decide when it notices, oh, I'm writing a react component and there's more documentation there. Let me load that in now. So it's almost like lazy loading the context when you need it. So that's like one part of it, like conventions and maybe also prompts that give instructions. So for example, uh, yeah, I want to do code review. I have my code review conventions in that file, so I kind of like load that file. And then the second part of it is um, what I would call context interfaces. So that's basically all of the different tools that the model uh, can trigger. Right? So it's the built in tools in the coding assistance, like reading files, editing files, searching, uh, your code base and so on. Then it's MCP servers as well. So the context seven that you mentioned is an MCP server as far as I know. And um, it's the skills, right? So basically every time the agent asks the large language model, something part of the context window is already full just from describing all of those available tools. So all of the MCP servers that are there, all of the skills that are there, all of the tools that are there, ah, so that um, the large language model can use it. So that's also something that we need to optimize. A lot of that is happening under the hood in the coding agents. But I can also think about like do I make 50 MCP servers available every time, or is there a way for me to only make the ones available that are really important for my situation. But so that's then, so it's all of the markdown files, it's like, how do we intelligently load them just in time? And the third thing is those context interfaces and what's being made available. And then as a human, my job is to kind of like have some awareness of what's in the context, uh, what I made available, um, and maybe also have an eye on how full the context window is in these supervised sessions when I'm actually sitting there and driving it. Right. And maybe one last thing about the features that, um, help manage the context is. And that's like a feature that really unlocked a lot of stuff that's going on in the hype right now, which is what they call sub agents. Right. So that means that. And that's a feature that is more rare at the moment still. But Claude code definitely has it. I saw the cursor has it now as well. I think Copilot is probably also doing something. But this basically means you're in your main conversation, let's say with your main agent, and it decides every now and then to spin off a sub agent with its own context window. So Claude code, for example, does this a lot. Uh, in the beginning of a session it spins, uh, off these explore agents, that kind of research in the code base what code is there. And that usually takes up a lot of tokens because during the exploration the agent reads many files and it's all in the window. But then the sub agent just reports back to the main agent what the results were of the research. So the main session is not full of all of those research tokens. So that's just like one example of really good use of sub, uh, agent. And now in the hype there's all of this sending of swarms of these agents that can even talk to each other and so on. Right. But, uh, even in a simple way, these sub agents can be really effective at helping you with the context, uh, management.

Speaker A: Yeah, I mean, the swarm, I think, uh, is quite interesting because last year I attended a training from Henrik, uh, Nieberg of, uh, Spotify fame, and now totally turned into the AI world. And actually I attended the training because I wanted to know about, you know, agent swarms and stuff like that. And, uh, surprisingly he said, yeah, you know, we're not using it. I was like, okay, um, so it's obviously not so, uh, I mean, if he's not using it, it's probably not so super hot and I don't die if I don't use it. Um, but the question is, how does it actually, how does it actually work? You know, this, this Swarm. Because with one agent I have my understanding. With two, I have kind of an understanding. But since I never tried it and, you know, I'm on, then the question is, you know, how does it work? When should I. When should I use it?

Speaker B: Yeah, I think, yeah, that. First of all, there's also a bit of definition of what do we mean by swarm, I guess, Right. So, uh, what definitely lots of people have been eager to try is just having multiple agents that maybe have these Personas, right? So there's this idea of setting up a team with the typical roles that we have as humans as well. There's a typical sub agent use case is also a code review agent that maybe uses a different model than the model that created the code is something that I hear people do a lot. Right. So kind of like just these different Personas, us. And, um, Claude Code, for example, just released a feature that they call agent teams, I think, where if you tell it in the prompt, I want you to set up a team of the following types of agents. And then there's some orchestration going on. So that's kind of like a lightweight version. That's maybe what you were just talking about. It's a bit like you have maybe three or four, and you're thinking a little bit about what are those agents? Um, but what I would call swarms is a lot more, say, extreme.

Speaker A: Okay. Okay.

Speaker B: So the idea of swarms is really, you maybe don't even define what the types of agents are. You let the AI figure out what types of agents it needs, and in the extreme case, it even sends off like dozens, hundreds of them. And I guess the idea is a little bit like this, like, throw lots of agents at the wall and see what sticks. Right. It's kind of like a brute force approach. And that's maybe also, if you ask Henrik about swarms in particular, you also said not using it, because I would say it's very experimental right now. And maybe the hope is almost like we threw a lot of data into these large language models. I mean, that's very simple way of putting it.

Speaker C: Right.

Speaker B: And then we had, uh, capabilities emerge that even the creators didn't expect. Right. So I almost see it like, okay, uh, people saying, okay, what if we throw lots of agents out there? Are we going to have emerging capabilities again that we didn't expect? Right. As opposed to, you know, there's this concept in AI that I always botch a little bit in my understanding and definition called the bitter lesson. So again, I might slightly misrepresent It. But in my head I always think of it as like we had AIs in the future. Let's say that we exactly told them, um, these are the rules of chess. And that works to a certain extent, but it always hits a wall. Right? And that is the bitter lesson. So ultimately AI has shown in the history of AI to work better if you have this kind of learning kind of capabilities. And I guess that's what the swarms always remind me a bit of in terms of like what people are trying to achieve with them. And so, um, there's a framework called Claude Flow that has been doing this for like over half a year I think. And then recently, famously Steve Yegi created Gastown, which created a lot of attention and I would also call it the swarm thing. And even Steve Yegi says this is like one iteration of this. He uh, assumes that a lot of what he built in this or what he vibe coated in this, it's going to be obsolete soon or it's going to change. But so there's a lot of experimentation happening in public right now. Right. Um, so Gastown is one iteration, then there's going to be the next one. And um, these swarms also of course burn a lot of tokens because the idea of course is that yeah, AI will make mistakes, but then all of these agents will kind of cross correct each other. Right? But that takes a lot of tokens because there's all of these autonomous loops that are happening while we're sleeping. Right. So it's also a question, is that the future? Is it maybe the future for some use cases, but not for all. Um, and how exactly they work? To be honest, I haven't looked into the exact patterns. There's a lot of orchestration happening. There's like uh, how do we store state in case something goes down and needs to go up again. There's communication um, between the agents. So in Gastown, Steve implemented something with like he has this mayor Persona that all the other agents report to and that kind of. So you have like a scheduling component that you need, right? Like how do you actually schedule all of these tasks uh, to the agent? So there's like implementation patterns for this emerging right now, I think. But yeah, it's definitely not ready for users in your enterprise to build a new big feature in production when you are on call for it, Right?

Speaker A: Yeah, exactly, exactly. But um, in my financial services, um, uh, application or multiple applications actually I hope next week. Famous last words. But uh, we have now the agent Mode with uh, Tab 9 and um, it also has something like skills and slash commands. Um, some of them are not working in our environment. But, um, reasons. But to me, um, it sounds like a good idea, uh, to have those Personas. You know, I have a, I have a development Persona, I have a test Persona, I have a code review, code quality Persona. I maybe have a Persona for coding security coding guidelines or, um, you know, whatever. And that sounds like you're not overloading the task.

Speaker B: Yeah. So, yeah, when you say Persona, they've probably done a sub agent or something where you actually want to do it in a different context. Right. But, um, yeah, and also with these skills, commands, rules, of course, there's now a lot of, uh, noise about how do we share this with each other both in the industry, but also in an organization or just in the team or then with other teams. Which things actually are applicable across multiple teams and which things are more specific to one team and which things are maybe more specific to me. Right. Maybe I want to, uh, put something in like, oh, I'm an experienced developer in xyz. When we talk about those things, don't go into details. Right. Or like, I don't know, just as an example, there might also be individual rules. Right. So this sharing is interesting because it seems like a lot of people try to kind of like do it up front. Uh, like, oh, can I copy this from somebody else? And then it will just magically all be better. Right. But I usually recommend building it up. Yeah, I usually recommend building it up step by step. Right. Because then you don't overload your context with stuff that you don't even know if it's relevant for it. Right. I was talking about the importance of not distracting and stuff like that. And so it's usually like a process of like, oh, it's constantly doing this thing wrong, like, let me put it into a rule. Or like I constantly find myself in my development workflow doing the same step, like review and prepare a commit message or something. Oh, let me create prompts with my conventions for those. So I have to slash commands or skills. Right. And then share those with the team. Right. I guess some things you can, it's useful to copy and paste from others. But I just like, I think especially when people are starting with coding agents, it's good to first experience them in their unconfigured form, let's say. So that you get a feeling for like what the models can actually do. Um, because otherwise you might be confusing what's the tool, what's the model, what's the rules file. Right. So maybe start plain and then kind of like become more advanced as you go.

Speaker A: Okay, yeah, that, um, that brings me to, to the, to the topic, uh, where I thought we discuss it later. What can platform teams do? Because my, you know, I usually, uh, people have a gazillion things, uh, to do and now not everyone should think about everything necessary also like sandboxes and uh, all those things you need. So what can or what should a platform team provide? What's a good way of. Obviously it's not that we can bring a full platform. I say platform.

Speaker C: Right.

Speaker A: So a full experience maybe is a better word. Like here are the tools, here's already pre configured, um, uh, skills, for example, or let's say a pre configured workflow. Sensible default workflow. I'm just uh, inventing things here, um, and just go and see. But I hear that shouldn't be the way to go.

Speaker B: Yeah, I mean there's multiple challenges. Right. Like another challenge is also how fast everything is moving, which is always hard for a platform, uh, approach. Right. Because platforms usually harvest things that have shown to be useful multiple times. And then you're like, okay, this is ready to be shared with others. Right. So you constantly kind of have to assess, okay, is this now going to be the thing? Is it skills now or are we going to have another thing, uh, next week? Right. Um, so I mean there's always going to be churn and you have to expect some rework and stuff. But how do you balance it so that you don't build something out over three months and then it's totally irrelevant. Right. And so platform wise, I guess, I mean, starting from the basics, there are some enterprise, uh, environments where um, it's all about also access to the models. Right. Which models do you give your employees even access to? Do you need guardrails in front of them? How do you do cost? Uh, management? So it also has transparency for people. Um, sometimes it's of course procuring licenses for these agents. Technically you can actually switch between coding assistants all the time. You don't have a lock in, in that technical sense. Right. That um, you have to migrate code or something. Right. But of course procurement departments might not be so happy about that. Right. But technically you can actually switch. So that gives us some flexibility. Um, yeah. And then in terms of like the context engineering. Yeah, it's um, I think as with everything else as well, it makes sense to have like one or two pilot teams that maybe experiment more and that then when they have experience with things that Work really well, then feed them back into maybe a central system where people can, um, there's like a plugin system for cloud code, for example, at this point. And then it's in the end it's markdown files, right? So it's not like so magical to distribute them. I know that some companies build uh, kind of useful MCP servers that uh, people can use, right? Like for example, one that can access the observability data or stuff like that. Right. Like so common things, um, I said before, by the way, that MCP servers are being replaced. Some of them are being replaced by other stuff at the moment. And like a big trend right now is like, why do we need MCP servers when we all have clis running on our machine? Right. Instead of having an observability data MCP server, you could just tell the agent in a skill how to use the AWS CLI that is installed on your machine. Right. So that's what I meant by the ah, storming, norming. Kind of like we're figuring out things as we go and realizing, oh, wait, why do I need this special thing running over there when I already have a program on my machine that can access these things? Right. So, but yeah, that was just an example of like typical things, like again, context engineering, typical things that you need and that you have utilities provided maybe by a central team. Right. If, um, maybe you want a new CLI in your organization that can access certain data that before you didn't need a CLI for, right. But now because it's super useful for agents, you might want it and it's also easier to build that cli because AI can help you build the cli. Right?

Speaker A: Exactly.

Speaker C: Yeah, yeah. I mean, from my perspective, um, I, uh, think the context engineering part is actually the largest lever that platforms have. So, um, I think there's like a lot of experimentations that you should enable the teams to do so. This procurement availability question also, like which models do you give your engineers access to? Right. Some of them are very expensive and that kind of explodes any infrastructure budget easily. At the same time, can you really afford not to give them the best models? Because then they are like way less productive than they could be. So this kind of, uh, this is a platform question, but kind of enables experimentation. And then there is a question about standardization. And for me this is like as you described, there are some files that are personal, there are some files that are team local. And then I think there are some things that are company local, like security policies, some kind of things that architects do also context about the business. And my personal hypothesis um, is that this is the highest leverage thing that a platform team can do. So think about which are really company specific context and think about how to effectively make that accessible. And thankfully a lot of the thing can be oh, there's a, ah, Google Drive MCP which I can now use to just to read documents which is surprisingly uh, can be surprisingly helpful. But I think uh, if you are doing an engineering investment like writing an MCP server, uh, or distributing some form of standards as markdown files, it's probably a good uh, way.

Speaker B: And then there's kind of like almost like hot off the press. It's like this new hype topic right now. There's this term coming up, um, harness engineering. So we have the next engineering term. Everything is engineering of course, but the idea here is that again kind of like future facing, okay, if we really want more autonomous agents, there's all of this like oh, what if we don't touch the code anymore, what if we don't write anything manually anymore? Right. Um, then what is the harness the safety net around that and does it need to be more than what we have for humans right now? Right. So um, there was like a really interesting article by a team from OpenAI just a few days ago where they describe how they built up a harness over five or six months building an application like a real product from scratch. And they said they tried to never touch the code but always when there was a problem they would build more tools around it. So this is some like, I um, guess I would imagine harness as some like pretty uh, standard stuff like static code analysis and stuff like that. But also the OpenAI team also mentions custom linters which I've also heard more noise about recently. Right, Because AI helps you write linters quicker so you can write your custom rules that also have error messages that are basically prompts that help the agent know how to deal with the linting error. They also talk about uh, how it affected kind of their decision making, in which cases they pull in a library, in which cases they build something themselves because sometimes when the functionality is right in the code, it's more available and easier to use by the agent than when it's like something external. Of course they have a big knowledge base in the code base that they constantly enhance. They also talk about something that they call garbage collection where they have agents running all the time to fix inconsistencies in the documentation and to like, you know, find because they still notice of course, like a Drift and like entropy. Um, and they also talk about uh, architectural or like code based design, things like how do we keep the data structures stable and only in certain places. Or like modularization. That part is a bit vague, so I at the moment cannot practically imagine how they did it in the harness. The only thing they mention is like um, structural tests. Right. I imagine stuff like arc unit, maybe not literally arc unit, but like stuff like that. But the general idea is really like, yeah, how do you pull the automated, uh, deterministically in parts, automated monitoring of what you define as quality up to as high as it gets and maybe even choose different patterns in your code base to make that easier. And now to get back to your question about platform teams or maybe even architects in the organization that try to have influence on multiple teams then even though some of this might maybe be more future, but I think it's worth reflecting today. If we had a harness, what would it look like? And are we maybe starting a new code base where we can try some of the things like change the design so that it becomes easier to have this automated harness? Uh, yeah, yeah.

Speaker A: Actually I have a question on the rigor, um, which was something, um, maybe we can take a little detour here. So what you were talking about with the harness was also something which has been discussed on this future of software engineering retreat in Utah organized by ThoughtWorks. Maybe you can say a few words about this retreat and then we go to the rigor.

Speaker B: Yeah, um, the starting point was basically that it was just actually the 25th birthday of the Agile Manifesto, uh, which like um, of course Martin Fowler who still works at thoughtworks and also one or two uh, ex thoughtworkers were part of that group. And that meeting where they came up with the manifesto happened in a place called Snowbird in Utah. And so it just had its anniversary. So we kind of took that as an opportunity to organize an event also in that general area in Utah and invite, you know, there was a bunch of people from ThoughtWorks, a, uh, few of our clients, but then like a bunch of, uh, yeah, let's call them technical, uh, thought leaders for, for lack of a better word. So I, and we had an open space, right? So we uh, you know, where you create like an unconference where people create the agenda in the morning. And then we had like breakout sessions and discussions and there was one about rigor, um, I was not in that discussion, but um, I know, I think this whole discussion about rigor and using that word was probably started by uh, this blog post by Chad uh, Fowler, some people might know him, he's um, very active in the Ruby community back in the day

Speaker A: CTO Wunderlist. So he is certainly a guy to follow I think.

Speaker C: Yeah.

Speaker B: And he started a website recently about what uh, he calls um, Phoenix Architecture or Phoenix Programming. I forget right now. And one of his posts was about rigor and he was saying okay, so if we don't want to care about the code anymore, like the implementation code, then that rigor that we used to apply to that code needs to go somewhere else. It can't just like poof, go away and everything's fine.

Speaker C: Right.

Speaker B: And he's talking about basically this rigor has to go to verification. And I was like, just uh, like you Sven, when I read more about the harness stuff I immediately thought about that because this article for example by the OpenAI team was giving very concrete example where the rigor might go or where some of the rigor might go. Right. Um.

Speaker A: So yeah, yeah, it's actually called the Phoenix Architecture. Um, I must say it's also one of those articles which are ah, forever in, in my list of open tabs and I haven't read it yet. Yeah but um, I think the idea of uh, the rigor is I like it in a sense and I haven't tried it yet. Right. But I like it in the sense that if I have acceptance tests, I have cucumber acceptance tests, lots of them. For example, I write my unit tests, for example, um, and the agent has to fulfill both of those tests. Maybe I also have end to end tests and it also has to run them. But then I have some sort of functional confidence. I can also imagine that property, uh, um, based testing maybe now becomes a new, you know, let's say new, new life. Because it, it might be a very good idea to, to catch uh, bugs by uh, and just you know, restrict the, the, the, the, the, the code output. But one question I, it, it's future thinking because I, you know, I don't program that much and if I program, I'm this guy, you know, give me, I, I get a little code and then I tweak it. Sometimes I don't have to tweak, but I'm not in the camp yet where large amounts of code are ah, produced. Uh, so I hope soon. But um, until now, be careful what you wish for. Yeah, yeah, no, but one question I'm thinking about is how can I make sure that for example some sort of what you said, data structure is always in the same place or I have, have you know, I, I have some sorts of, of logical uh, structure. You know, if you have 1 million lines of code or you know, it's. Let's say you have a monolith, well structured monolith with one. I don't have it, but let's just imagine it for a second. And I just don't want to, to let the agent go off. I just want to say, hey, look here, that is the way we structure it. And now for example, we also use ARC units to structure our services. And if you stay in ARC unit, so maybe we have to say what ARC unit is. So ARC unit is obviously architecture unit test. You can say, you can define structures and axis of your code.

Speaker B: So for example, a class from this package is never allowed to call a class from that package, right?

Speaker A: Yeah, yeah, exactly. And, and you can tell it, you know, the, your, let's say your aggregates remain in the domain package and stuff like that.

Speaker C: Right. So.

Speaker A: And you know, I, I know, I know now I have to be careful what I wish for. But to me, when I, when I kind of understood this, or maybe understanding is too much, when I heard about it, about this rigor, I immediately felt much better because m. I'm always like ah, you uh, know, I just cannot let m. This thing get off the rails.

Speaker B: I mean, yeah, if, if we want those large quantities of code generated in the future with AI autonomous, like unsupervised, um, and still feel good about it. So we talked also that we treat a lot about like how do we increase the trust and the confidence that we can push this. And I think, um, there's almost seems like a split right now. A lot of the public discourse is just about how do we create more code faster. Um, and then when you start talking about yeah, but what about maintainability or testing or something, it just gets brushed off. Right? That's almost like, oh no, let's talk about the future. Yeah, but don't we want software to work in the future? Right, because when you were touching on acceptance tests and all of that before, right. The examples I gave before from the harness from the OpenAI team, they only talk about stuff that maybe I would call maintainability and internal quality or something. They don't even talk in the article about how they verify the functional behavior. Right. So that's also like uh, and of course AI generates tests, but they're often over mocked or they're based on our specification, which might be wrong. So there's a lot of energy happening right now on let's create lots of code, but much less energy spent in public discourse and ideas on the verification part and maybe also the specification part. Um, I think there are, like I said, there are a bunch of things that we can do to increase our confidence about large change sets. But at the same time, I keep coming back to it's actually a good idea to work in small batches. We've learned that over and over and over again, right? So I keep like, going, yeah, we're in this phase right now where we're trying a lot of things, right? Um, but, yeah, let's see how far we can push the whole harness idea. And at least I think it's nice to have a word for it and then you can take that word and see, okay, what would our definition of healthy be? And you know.

Speaker A: Yeah, I mean, to me, uh, uh, I feel the pressure, you know, when you are half a year behind with the tooling and then you, I mean, you shouldn't be on LinkedIn or something like that. It drains energy because everyone, uh, is producing 37,000 lines of code per day. Something like that.

Speaker B: You have to look at the use cases, right? Like, there are so many people writing about this that, oh, I haven't coded in 20 years. I'm doing it again. And it already, like, a lot of the examples are, uh, one person working on one thing. And yes, that is super useful, right? Like Heinrich, before we started the recording, we were talking about people building lots of tools and stuff like that, right? Like scripts, utilities, like lots of these things. That is super useful, right? But so we always have to talk about the context in which we're talking about this. Because if it's like business software that runs in production that I'm on call for, that just needs to work, it doesn't matter if it's safety. It doesn't matter if it's safety critical or not. I want my CRM to work, right? At least to a certain level. Then it's different, right? So, um, because I haven't been on a real team in a while, I always try to ask myself, when I experiment with these things, would I push this if this was real software, Right. Uh, so that I can kind of reflect on this and it's just like, different. You know, AI will be used at different extremes in different use cases and situations, right?

Speaker A: Yeah, uh, exactly, yeah. Um, there with the small batches. You know, uh, I lately, uh, heard a podcast from Gene Kim. You know, you mentioned Gene Kim being there. And I also bought now his book. He wrote a book with Steve Yegi. It's already out there, I think for half a year. But I bought it yesterday. I am ashamed, but it just feels old. It feels old. But, um, you know, I was. I got a. I got a push when I heard Gene Kim, you know, because if you listen to Steve Yegi, it's like he's kind of. It feels like, you know, I don't want to put it in his mouth, but it just felt like this. A gazillion trillion lines of code per millisecond we are producing. And I was like, okay, this is too much for me. You know, I live in the world of generating small pieces of code and tweaking it. Because as you say, you know, how would you feel? I mean, I cannot. I mean, I'm also not writing production code, but I cannot. How can I safely recommend this? I don't know. And then Gene Kim was saying, well, you know, reduce the typing, work in small batches. It's still the best thing you can do. And I was like, uh, I've had kind of a relief.

Speaker B: Yeah, yeah. Don't let yourself. Like all of this. Like I said, a lot of this is public experimentation right now. Right. And that's m. Fine. Like, I. I love that, you know, getting all of these different learnings from it. And we also now have to try all of these things to show what if it does not work? Because if we don't try it, then people will keep claiming that it will work. You know what I mean? It's always like that.

Speaker A: Exactly.

Speaker B: But like in, um. Yeah, it's like, just because, like, maybe also to the people listening, like, just because you might think, oh, my God, this is. This is a bit crazy. This is unrealistic. You should still figure out how to use AI for what you're doing in a responsible way. Just because that extreme case might put you off doesn't mean that there's not use for it in daily work. And especially when you change existing code bases. And again, like software that actually goes into production in a real business that does not want to lose money because of incidents. Right. Um, yeah, go and like, uh, keep going in small batches and m kind of like figure out how you make. How you use AI to make those small steps, um, more effective. Right. And, uh, yeah, I always talk about it right now that for me, I've noticed that I'm doing this constant micro risk assessments. Right. What's the probability that AI gets, uh, this right or wrong? What's the impact if it gets it wrong? Uh, also depends on Criticality of use case and all of that. And then how will I be able to detect if it goes wrong? Right. Like, what's my feedback loop? Do I have to manually test 100 journeys, or is there like a good feedback loop that I can trust?

Speaker C: Right.

Speaker B: And then all of those things, like the probability assessment, depends a lot on my experience that I have with AI. So that's a new skill that we have to build up. Uh, the rest is a lot like criticality of use case, risk assessment of our feedback loops and tests. That's something that we know how to do. And so then when I do that, then I decide which workflow do I use? Do I just give the agent a short prompt and then I go back and forth with it because it's a small change or something? Um, or do I plan first break down? Do I do more stuff beforehand or do I just manually do. Uh, also determines, uh, how much review I do. Do I fully vibe code or do I look at every single line of code and it can determine sometimes how long I let it go without supervision? So if it's something like super straightforward for something uncritical that I've seen AI do a few times before, I just try to let it go for five minutes or something and I don't care. Right. Uh, in a sandboxing, in a safe way, of course. Right. But if it's something where, let's say I myself don't really know yet how to do it because maybe the requirements are a little vague or something, then I'm like a lot closer and maybe go back and forth and once I know what to do, then I let it go for a bit longer again. So it's this constant, um, trying out a balance of this.

Speaker A: Yeah, there is. I just have it open here from Ralf D. Muller.

Speaker C: Ah.

Speaker A: So, uh, in Germany, let's say he's a known architect with the isaqb. Uh, uh, he is very active and um, I share it in the show notes. He has this vibe coding risk radar, which I find quite interesting, where he basically, you have sliders on different kinds of categories. Uh, code type, language, safety, deployment, context, data sensitivity, blast, radio, similar what you say and also what are required mitigation, uh, measures and yeah, so he also has, uh, examples. Now, uh, if you have a landing page, a CSS landing page where there is minimal risk.

Speaker B: Yeah.

Speaker A: If you, uh, write medical device firmware. Well, yeah, yeah.

Speaker B: It's like, I mean, uh, you just mentioned stuff like tech, uh, seg or something that would affect my probability. Right. If I know which tech stacks it's good at or not, then the blast radius is kind of like the impact that I was talking about. Right. And you had some stuff there about detectability as well. Right. So it's basically those categories and then you can kind of, and it sounds a little bit academic right now maybe to some people saying this. Oh yeah, probability, impact, all of that, that. But um, it's like the more you use AI, the more it becomes intuitive to do this little assessment. And maybe in the beginning it helps to have a more explicit mental model about it. Uh, but at some point you can feel how it becomes a bit more intuitive.

Speaker C: What really strikes me just listening to this is the similarity between this problem and the problem of you're managing a team and you want to know that this stream produces high quality output. And the important uh, parallel is you're not reading the code. So you're a proficient software engineer, but as a manager you must not step in and actually read the code. So um, basically can you lift your level of abstraction from looking at the code? And how do you kind of orchestrate a machine that produces reliable outputs or fast outputs or whatever dimensions uh, you are striving for?

Speaker B: I mean the question is the trust again, right? Like as the manager of a team, I trust my team members to certain extents. Maybe I trust some of them more than others. Right. Like maybe I review some of their code more than others. If we think of like a technical engineering manager or a tech lead or something. Right. And with AI, it's like so much like hit and miss at the moment that uh, that sometimes becomes hard. Right. But you know what, like what's really interesting, I recently realized, I saw like months ago, my colleague Eric Durnenburg posted something on LinkedIn about like, oh, is what we're doing here what we used to do with offshoring? Right, exactly. If only we write a, uh, detailed specification, send it over to some other country, surely then we just have to test what they give us back. Right. And at the time I was a bit like, yeah, I see what you mean. But I mean I'm still sitting in front of the session and looking at ah, what's happening. So it's not quite that, but now that the talk about the, these unsupervised agents has really, really taken off, it becomes a lot more similar. And I see there's a lot of stuff that feels waterfall that's happening right now. The spec driven development discussion. We just write a lot of spec in the beginning. I recently caught Myself thinking about the V model which already when I was in university it was kind of old school, right? So for those of you who don't know where you have like imagine a V shape and on the left you have different levels of abstraction of requirements, right? Like very high level requirements. Then gets more detailed, more detailed. And on the right side of the, at the tip of the V is the implementation and on the right side of the V you go up the abstraction levels again and have tests on each of those levels, right? And I caught myself thinking okay, let's say the tip of the V is like a black box. Then how do I go up the V and like have all of those tests, right? And then I was like oh my God, what am I doing? Yeah, we're not used. I mean D model is still very common in like automotive software and stuff like that. I'm not saying it doesn't have a place anymore at all.

Speaker C: But yeah, I mean draw a line between having um, still reading the code and basically uh, working with the model closely, co creating the code versus moving next section higher. And for example for this command line tools effective is like you co create a readme. You basically say here's the look and feel I want this to have and I discuss m maybe some high level architecture, but maybe not even for the things. And then I write tests. So I co create tests, right? I basically say okay, implement this and maybe review the tests and then the rest I don't care. And this is a throw and miss thing. And if there's something doesn't work, I say please debug, fix it. And then I have another thing. It kind of looks great. Uh, but it's also like I know it's pretty shit because I keep running into bugs so it's kind of good enough. But I would like to get to this level of abstraction that I'm just providing kind of human readable thing. I just discuss like a product manager would discuss how this thing should be shaped and then I delegate and then the whole implementation effort. And uh, like versus this, um, I'm embedded as an engineer, as maybe a tech lead with a team. I review what the kind of the agents do and I'm kind of supervising this more closely. And the letter I think 100% works, but they are really scaling limits. The review becomes the bottleneck, right? Am I reviewing a thousand line of code every hour? No way.

Speaker B: Um, but also with the specification. I think uh, we're going through this again. I mean I actually 15 years ago I worked a lot with model driven development. And I built code generators, deterministic code generators. So I went through this exercise of coming up with the language, like a model, a DSL that describes the requirements in a comprehensive way. And then also you run into things like, oh, what if these things are being reused? I don't want to specify it twice, right. So you really have to come up with a comprehensive way to describe this. And uh, we landed on programming languages there in the past, right? That is our kind of, uh, um, uh, dense way of describing this. Right. So I think we'll have to see like, what is that abstraction level that we can get without having like duplication and with making it consistent and all of those things that really gives us ultimately, uh, an advantage. Right?

Speaker A: I mean, yeah, one word on the reviews. I think it's, you know, I wasn't in that situation. Now we are talking about the future. But. But, um, there is this thing. If you review 50 lines of code, you get lots of comments. If you review a thousand lines of code, it looks good to me, right? Because it's very hard. And also the question is, does it really work? Because I think Uwe Fritschten wrote an article about it. The Perils of Automation, I believe is the name. So if you automate too much and you just have a human with which is the inspector, the quality goes down because the inspector just cannot catch all the problems. It's too hard.

Speaker B: And there's also this talk right now about, uh, coming from different directions. I was just talking here in the office to a colleague of mine about this, of burnout, uh, and the cognitive load that all of this leads, uh, to like the same Steve Yegi who created Gastown recently wrote a really interesting blog post called the AI Vampire, where he talks about a bunch of different things, but he also talks about how this type of working mode that he was in when he built Gastown with multiple agents running at the same time in parallel, and he was managing all of them, basically. He says that realistically he can do that maybe for two, three hours a day and then he's like done for the brain energy and uh, you know, he did it for more. But he's saying this is not sustainable, this is not responsible. You know, so there's also. So in general, recently I've been thinking a lot about speed, right? Like everything's like, faster, faster, more throughput. And we're basically speeding up knowledge work, right? So we're speeding up the IO on our brain. And I think there's like a limit to like both what we can do, but also what's responsible or what's like sensible for like having good quality outcomes with minimum waste and stuff like that. Right. So what's the, you know, what's this like the Goldilocks speed? Right. Like just fast enough but not too fast or something like that. Right. Like um, or as also like a colleague from Thailand keeps telling me, like, can somebody explain to me why English is a better way of describing software than code? Which is interesting especially coming from a non English native speaker. Right. So there's. Yeah, it's very interesting times indeed.

Speaker A: Indeed. I mean uh, yeah, Eric Meyer actually wrote a, I believe he wrote a paper about it that you know, he as a language designer wrote a paper. In the future it will be English. But I think the interesting part is also what he says to come back to your small batches, it will be English, but it will still be small batches and small. Uh, it will be small. Right. So you need a small batch.

Speaker B: It would still be in a way like I think, but I might be wrong. Uh, the stuff that he's doing is still in a way structured. So it's still not like uh, fully random natural language, let's say. Right. There's still like a structure to it. So, so the question is all about why do we want this? For example, if we want this so anybody can just create software on the fly, then that might be a limitation. If you still have some format that might still exclude some people because it's too. Or are we doing this just so that we can create more software? Then maybe, I don't know if English language would be good. But yeah, it's kind of like about do we want better software? Do we want uh, I don't know, more high performing software? Um, and then always like what would English language vague descriptions give us to reach that stage? Right.

Speaker C: I think the acceleration is really the core value that we get. I think the bottlenecks are shifting. It's no longer typing, it's no longer learning. I have all this programming languages book programming Ruby. Wouldn't it be nice if I can write a Ruby project? And it's like, okay, it will take me three weeks to get up to speed and halfway productive or maybe three days, but it's kind of a ton of work. And now I can just go spin this up in Ruby and spin it up in Perl and spin it up in Bash and then just compare and have these things. So I find this incredibly liberating. But I think I love the point you made about Vigor. Where does the vigor go? I think it's an excellent question. Also the question, uh, what is the source code? Right? If it's no longer thing I produce used to be the program code which is really the source, it's more something else, but it's also something that is kind of co created if you think about this iterative software delivery, right. Your next iteration step depends heavily on what the output of the first was. So um, I'm kind of evolving my program along a path. But the navigational steps, basically the short prompts that, that I provided to kind of do the next things, they are not an adequate description of the output that I want. They're kind of incremental correction to the last iteration that the model provided me. So that's probably not quite the right thing just to store the prompts that were used. So maybe it goes into the direction of validation actually. And the thing I would say is I don't actually think program code is a description of intent so much much as a recipe, right? I want a nice cake with glass on top and some sprinkles. Something else than saying, hey, I need flour and I need to mix it. And then it's like SQL or the implementation of fetch the data and do this kind of precise counter arithmetic to produce an average over those groups. So declarative versus uh, imperative. So maybe there is something like this TLA plus thing validation language or something.

Speaker B: You still have to define the right model even when you use formal methods. Right? But um, what will also be really interesting to see in the future and at the moment I don't know how it's going to play out, but uh, Heinrich, when you say, oh, you could just do it in Ruby and all of these other languages and compare in part, the three of us can do that because we have all of this experience, we understand the, the building blocks. Like we know, oh, let's say this is a for loop or like this is a web server and this is a build system and this is a. We learn that over time, right? Like I actually once read this thing about learning where it says that you learn abstractions by jumping back and forth between details and abstractions, details and abstractions, right? So there's this whole discussion about learning and right now a lot of the prompting and I would say like the engineers that, that use these things to a super high level, like let lots of code get generated, they still steer a lot from a technical perspective. They know where to look, they know what to mention. And I've Already noticed that when I don't look at the code anymore for a prototype or a POC and I don't know the structure of the code base, my prompts get worse because I can't tell it. Oh, by the way, we already implemented that over there. Please use that. Or, uh, be careful, there's a contradiction which xyz, which we need to resolve in this way. When I know the code base, I can already put that in the prompt. Right. And so it's going to be really interesting. Like if we are that generation, that last generation that has that experience and what will it look like in the future? It's almost like I'm of the generation where I didn't have Internet and social media in school, but I still grew up kind of like, you know, with all of these technical things, like in my late teens and early twenties. Right. There will never be another generation like that. Right. That still had to look like, uh, see how to fix the router. But, uh, you know, still didn't have computers in school. You know what I mean? Like, I almost feel like we see like this special generation that's somewhere in between and we're going to look back at that in 20 years. I don't know.

Speaker C: Well, I think a very interesting aspect is really Darwinism in this regard. Like, first of all, with regards to software engineering practices, because you can be too slow and too risk sensitive and you will die because you're not fast enough, or you can be too fast and get burned and you will also die. So I think we will see the people survive and expand that kind of get this right and this will be the way we figure this out. It will not be, oh, we think so hard and we kind of come up with the ideal model. It will be. We try everything out in parallel and the survivors are telling us how it's done. Maybe we can learn before we die and just converge. Right. But I think there's a little bit of Darwinism at play.

Speaker B: Uh, this also in part applies to the organizational level. Right. Like, will organizations burn out if they try to push too many things through the pipeline? You know, just drown in waste and shitification. Right, yeah.

Speaker A: Which I look at the time, which brings me to, um, my last question. And, uh, as we already mentioned, uh, you're giving a talk at the Agile and Architecture conference in Berlin.

Speaker B: Agile means architecture. Yeah.

Speaker A: What did I say?

Speaker B: Agile and architecture.

Speaker A: Uh, agile means architecture. Sorry, ama. Exactly. We also have a voucher for every listener who wants to go. But 10% voucher. But you give a talk, um, what AI augmentation means for technical leaders and. Yeah, so what does it mean? You already mentioned speed of the organization in one talk of yours you were talking, uh, one presentation of yours, you were talking about cultural aspects like balancing skeptics and uh, let's say enthusiastic.

Speaker B: So yeah, I think it's like uh, maybe three categories like technical leaders, I think of like architects, like, I think more on the technical side of things that just to scope it a little bit because there's operational organizational things as well. Of course. Right. But I think like one category is like, how do you help yourself? Right. Like when you um.

Speaker C: Yeah.

Speaker B: How do you use AI as a sparings partner? Uh, the second category is like how do you amplify your experience? And that goes a lot into this, uh, what we were talking about before with context engineering and harnesses and stuff like that. Right. So like how do I take my experience and kind of with AI now I have new ways to codify it and spread it to people. Right. Be it with like uh, a prompt that helps people write better decision records or a prompt that helps people do threat modeling or, or thinking about how do we build up this harness. So how do I amplify my experience? There's a new way of automating some of it. And then the third one is maybe then when you think about initiatives like migration, modernization, those types of things, how can AI help in those initiatives? It's actually a really, really good use case area, like uh, migrating code either from one version to another when the deterministic code mods are not enough. But you actually need a little bit more than combining code mods with uh, AI or um. We do a lot of work with clients who have uh, cobol, uh, code or old applications that now there's new opportunities to replace them faster or to speed up these modernization programs. And again that's then almost more than just giving a prompt here and there. But it's about how can AI help with the forensics of what we have and what it does and then, and how can we turn that into new designs and then forward engineer it into something new? And so for technical leaders, so how can AI help you? How can it amplify your experience? And the third one was like, what role does it play in different initiatives and stuff in the organization?

Speaker A: I had to immediately think about open rewrite. Um.

Speaker B: Yeah, that's what I meant. Yeah, exactly. Yeah. I mean there was this famous like two years ago. So Amazon famously claimed hundreds or thousands of developer hours saved with their Java upgrade AI thing, and it was like 80, 90% open rewrite under the hood. Right. And then it was presented as if AI saved all of those hours. Right.

Speaker A: Yeah. But Open Rewrite also has some AI, uh, features.

Speaker B: I recently, yeah, the, um, the creators of Open Rewrite have this company, Modern, Moderna Modern. And yeah, they, there's like a lot more in this combination with AI. But I've also seen teams like, do

Speaker C: stuff like, I'm learning so many tools,

Speaker A: unfortunately it costs money. So. But yeah, yeah, yeah.

Speaker B: I mean, AI costs money. That's a whole additional conversation. Yeah, but I've seen teams do stuff like, um, go from Angular one, like transforming Angular one components into React components, including, uh, this team, I think, also had to support a new standard and was changing some functionality. So they built themselves like a whole AI assisted process of doing this transformation and then could do this much faster than they could have without AI.

Speaker A: Yeah, yeah. I think, uh, it's a thought a colleague of mine, Robert Klasser, put into all of our heads at Innocue. It's kind of, he said the end of too expensive, which is probably a little bit, uh, over the top. But I still think at the client where I am currently, we have a huge problem with acceptance tests, like thousands of them are not in a very good, good shape. And it's, you know, we just have to live with a technical depth here, um, with the problems. But we already said, no, we don't. Right. Because we can actually, with AI, we can transform them. But so far the models were too weak and we are like every new model generation. I mean, you said you're not looking at the models, but we are so much behind and we still look at the models because if you are half a year behind, you really see, I mean, it's an incredible difference. Right? So, you know, you think, ah, this crap doesn't work, and then three months later it's like, yeah, and so, but that's something we, we look heavy into. We, we have our ideas where we say, didn't work last year, didn't work six months ago, doesn't work today, but maybe in three months we have it on our watch list because there is, uh, this big chunk of problems we cannot solve. But maybe the transformation is a nice problem.

Speaker B: Yeah, I wouldn't say it's the end of too expensive, but it definitely does a shift and it depends on which area. I, um, mean, I heard a great quote the other day. We still have to be careful with total cost of ownership and not underestimate It. So the quote was, um, uh, free software in the sense, like, oh, it's now almost free to just rebuild this application or build an extra application. Right. It's like free puppies, right?

Speaker C: Yeah.

Speaker B: So you get a puppy, but then you have to take care of it for years. Right?

Speaker A: Yeah. I'm thinking about this very often when I read this. Uh, it's the end of software as a service. I'm thinking, yeah, I can build a lot of things, uh, what I need, but.

Speaker C: But then it's the ops.

Speaker A: Yeah, it's the ops. Yeah. But also, I'm, you know, I'm all very often not sure do I have the domain knowledge to do this properly. Besides, I have to. I have to run it and I have to maintain it. And you know that.

Speaker B: Don't run your own security stuff. Don't roll your own security stuff. Right.

Speaker C: For example, my career. I mean, it's a little bit funny, right? Because just very brief aside, right. I'm a mathematician by training. And then I started going into software and I had my WordPress blog. I was organizing a conference, interestingly, on rigor and openness in science. But, uh, different aside. And I had this WordPress blog, and it was going down all the fucking time because I hadn't figured out the ops. So. And then I said, okay, well, software. I'm doing that step more professionally now. Let's kind of figure out how to do software better and then really get deep into log management, monitoring and so on. And here I am, like 15 years later, still doing the ops. And what I have learned from it, that never voluntarily sign up to operate anything. If you can pay somebody else to run it, that's so much, so much worth. I will not sign up for any operational roles unless I'm paid. That's for sure. So although I kind of know a lot about this stuff, and I do run a lot, when I tell my wife, don't use my stuff. Right. You won't call me to fix that when I'm on a business trip. So you pay Dropbox for your safety of your files. Right. So it's that kind of. For the MCPS I see totally value for SaaS. Although I can code it up myself or something. I write tools that I use. I augment myself. I'm not providing services for other. That's, um, somebody else's business.

Speaker B: Yeah.

Speaker A: Um, maybe I come back to my culture part. Um, like, how do we deal with enthusiasts and, um, skeptics? Because maybe a few words about the background. Because you were talking about it, it, Right. And Martin Fowler on his latest, uh, I don't know how he calls it, fragments.

Speaker B: Mhm.

Speaker A: He was also uh, saying, yeah, I'm a hardcore skeptic and I also need to be skeptical. I'm skeptical about LLMs, but I also need to be skeptical about my skepticism and stuff like that. And we had actually today and yesterday heated discussions. Actually we have always heated discussions, but it's, it seems that many companies have these heated discussions between people who are at the forefront. They're evangelists. You know, if you're an evangelist, if you um, are an innovator, um, you know, you, you maybe look at only the positive side. And if you are, um,

Speaker B: say, I

Speaker A: mean we are early adopters, we are not leggards in a sense. Right, but if you're an early adopter, and you know, that's what I would call me, but I would still be on the skeptic side, you know, looking what the innovators do. But there is a big tension. You know, I just feel this big tension. What I also said earlier, you know, I, I, I almost feel bad if two days after Gastown comes out, I haven't written my own brain compiler with Gastown. Uh, and I, you know, because all the, the enthusiasts say, are you living under a rock?

Speaker B: Yeah, it's like, I mean it's definitely polarizing. Right. And the uh, So I think that's the one side. Oh, we soon will not need developers anymore as the one extreme. And the other one is like, this is total bullshit. This is not useful at all for coding. Right. And we're being pushed, uh, into both sides somehow. I even find myself on either side sometimes. One day like this, the next day like that. And um, yeah, I mean, I guess this goes back, your question goes back to what you asked me about technical leaders.

Speaker C: Right.

Speaker B: Like I forgot to, to cover that there. But I think part of AI governance almost in a way in an organization is a culture where you kind of try to reward both sides. Maybe not the total extremes, but where you try to reward enthusiasts and skeptics because you need the enthusiasts to stay on top of this, for example, to catch when you can finally use it for your acceptance tests or just to innovate. And you need the skeptics to kind of like keep things uh, in a realistic way, let's say, or something. Right. And a culture that rewards both is also like how people talk to each other. Right? Like that. If I'm a skeptic and somebody tries something that seems ridiculous to me, I don't tell them. Why would you even try that? You're so naive about AI. Right? I would. Don't say that. Right. Say like, oh, yeah, cool. That you try to push it a little out there. Right. Or the other way around. Um, when a skeptic is kind, um, of like actually using AI, kind of say, oh, it's great, uh, um, that you did that. And kind of the enthusiast should look like, what are these people doing that they're trying to keep this more, um, in a realistic way or something. I don't know, um, ramping a bit. But you know, like, it, it needs to be both rewarded and they shouldn't dismiss each other and, and they should call each other out when they accidentally do that. And then that's almost like for me, a kind of governance to get both innovation, but also like, uh, a little bit of safety, um, into it as well.

Speaker A: Yeah, yeah, it's. I, I find it difficult. Right. So, um, because I understand

Speaker B: there's all of this like, keep up, keep up communication. Right. It's also coming out of Silicon Valley and all of these different blog posts and that's very stressful. Right. But, um,

Speaker A: yeah, it is the vampire. It's the vampire. Yeah. I think it's a totally new, it's new. I mean, I think I had several, let's say, waves of new technology or methodologies, uh, in my career. Uh, Right. So Agile software development, I mean that took years, um, at least, you know, until the Agile Manifesto and then things. But that still became, until it became mainstream. It took several years. Also I can.

Speaker B: Are we all agile now?

Speaker A: No.

Speaker B: Another podcast episode.

Speaker A: Yeah, exactly. And, and one thing I was, you know, I was also one of those guys I'm now criticizing for like being too hard on others was the idea of continuous delivery. So I was a huge fan. I just couldn't imagine how someone could be stupid not to do it. Which is kind of true. But that's not the way how you, you know, bring this, how you bring innovation to life. Right. To m. Uh, to put other people under pressure.

Speaker B: Yeah. Or how you influence and change people's minds.

Speaker C: Right.

Speaker B: Like, it's more about like showing. Like there's a lot of show and tell. It's like show, not tell. In AI, that's super important because we have to learn from each other's use cases and examples from things. Because it's like it unexpectedly doesn't work in some areas and it unexpectedly works in others. So it's very unintuitive. And uh, yeah, it's, I always call it. And it's a manipulative technology in a way. Right. Which maybe, uh, shows up in all of these, um. Yeah. And all of this polarization and like. Like stress almost, when you hear all of this, uh, polarizing communication.

Speaker A: Yeah, that's true. That's true. All right. There are still lots of other questions.

Speaker B: Um, so many things to talk about.

Speaker A: Yeah, but we are already. We are 20.

Speaker B: We have to stop at some point. Yeah.

Speaker A: Ah, we have 30 minutes more than, um. Than we already said.

Speaker B: Well, anybody who's still listening, thanks for bearing with us.

Speaker A: Yeah, exactly. Pet Kua is always saying. I'm repeating that, um, if you made it this far. Ah, thank you very much. All right, Birgitta. Um, yeah, thank you very much. Uh, also thank you to all the listeners who made it, uh, so far. Um, and, uh, Heinrich is. I don't know where he is. He needed a break and. Yeah, I don't see you at the ama, unfortunately.

Speaker B: But, uh, agile meets architecture in Berlin.

Speaker A: Exactly. Um, but hopefully at Goto Copenhagen.

Speaker C: Mhm.

Speaker A: All right, so thank you and, uh, see you then. Cheers.

Speaker B: Thanks for having me again.