
Arrested DevOps · 2025-10-01 · 40 min
Key moments - from our scoring
Substance score
38 / 100
Five dimensions, 20 points each
Hannah Foxwell, an AI fluency advocate who leads a 1,000-member community called AI for the rest of us, and Robert Werner, a 25-year enterprise veteran now building an AI startup, join Matty Stratton to unpack the real challenges of adopting generative AI in software development. Rather than hype or doom, they anchor the conversation in concrete parallels to previous industry transformations: Agile, DevOps, and cloud migration all faced resistance, required cultural shifts around trust, and eventually stabilized into best practices. The key difference now is that LLMs are non-deterministic and still rapidly evolving, making it harder to codify playbooks. Hannah emphasizes that AI fluency - understanding the technology without dumbing it down - is critical for making good organizational decisions. Robert warns that we're in "super early days" and that management must shift from distrusting distributed decision-making to trusting AI that's right "often enough." A major emerging problem neither guest glosses over: if AI writes code, who owns it operationally? The throw-it-over-the-wall problem resurfaces, but now from AI to ops. They also touch on perverse incentives - teams checking boxes by using AI tools without real intent - and the gap between code generation and code understanding.
Both require cultural shifts and rethinking trust - with DevOps teams had to push decision-making to the edges and automate; with AI, management must trust systems that are non-deterministic and often-but-not-always-correct. The key difference is that AI tools are still rapidly evolving, so unlike DevOps, there isn't a stable playbook yet.
AI fluency means understanding generative AI and LLMs at a foundational level without dumbing it down to black-box magic or overselling capabilities. Hannah argues it's critical because teams need to make intelligent decisions about where AI adds real value versus where it's merely checking boxes.
That's an open problem: if developers don't write or understand AI-generated code, ops teams inherit services they don't own and can't troubleshoot, recreating the throw-over-the-wall silos that DevOps solved.
LLMs are non-deterministic, and new model releases (GPT-4 to GPT-5) and fine-tuning happen constantly, making capabilities and reliability shift unpredictably - very different from tools that stabilize.
The hosts note that checkbox compliance (e.g., mandatory weekly AI usage reports) creates perverse incentives similar to writing tests just to hit a quota, producing low-quality outcomes; success requires aligning incentives around real business value, not tool adoption.
Our reviewer’s read on each dimension, with quotes from the episode.
There are occasional non-obvious observations buried in a lot of meander - most notably the parallel between owning AI-generated code and the old dev-to-ops wall of confusion, and the point that documentation quality drives agent effectiveness. However, the episode is padded with lengthy sponsor reads, biographical introductions, jokes, and the host's own tangential anecdotes that crowd out actual insight per minute.
If prompting your way to a solution requires you to already know what you want, then it's, yeah, the AI has taken the actual typing off you, but it's not taken the thinking, the knowing and the understanding.
Does anybody really want to own a service if they didn't write it and they don't understand how it works?
The dominant frame - 'AI adoption mirrors the early DevOps/cloud transformation, expect resistance and trust gaps' - is one of the most recycled takes in the industry right now. The 'trust but verify for agents' observation is flagged by the host himself as occurring to him in real time, which says something about how thin the pre-prepared thinking is; no genuinely contrarian or first-principles arguments emerge.
trust, but verify really applies to agents. This has not occurred to me until right this very second.
there are some companies who are doing incredible, amazing things. And those are glimpses of what the future will look like for everyone. But there's resistance and there's new challenges.
Both guests are genuine practitioners - Hannah with 10+ years in DevOps/platform engineering and a community of 1,000 around AI fluency, Robert with 25 years in enterprise software and a co-founded AI startup - which is credible, but neither is operating at a scale that unlocks proprietary or hard-won insights; Robert's startup appears early-stage and Hannah has transitioned more to community-building than hands-on engineering.
I've been working in DevOps and platform engineering for over a decade... I got swept up with all of this craziness that's going on with AI in software development
I'm probably 25 years in enterprise computing... founded in my decent age together with some friends, an AI startup
Concrete details are almost entirely absent: no productivity metrics, no named customer outcomes, no dollar figures, no before/after data. The few specifics that appear are incidental - a community membership count, a conference date and discount code, and a second-hand anecdote about assert-equals-true tests from Jez Humble - rather than evidence in service of a substantive claim.
I built a community called AI for the rest of us... We're now 1,000 members
every developer has to add three more tests, which meant they ended up with a whole bunch of tests that were assert equals true
The host asks multi-part, sprawling questions that give guests an easy out to answer the easiest sub-question, and he frequently interrupts his own questions with personal anecdotes and jokes that eat the clock. There is no meaningful pushback on any guest claim, and the closing 'what's your number one tip' format is a soft landing rather than a probe.
What's similar? What's different? What are you seeing in how people are adopting or struggling as well?
what would be like your number one bit of advice to say as an explorer who's trying to learn more, what's the most effective way, most effective tip you would give somebody to keep in mind?
Computed from the transcript - who did the talking, and the words that came up most.
The Trust Problem Returns Hannah Foxwell, who has spent over a decade in DevOps and platform engineering, draws a striking parallel to earlier transformations: "It used to be that testers didn't trust developers and ops didn't trust testers and there were all these silos. Now we're putting AI agents in the mix. Can we trust them? Should we trust them?" This isn't just déjà vu - it's a fundamental challenge that resurfaces with every major shift in how we build software. As Robert Werner points out, management had to give up control and push trust to the edges of organizations during the agile transformation. With cloud adoption came self-service and automation. Now, with AI, we're dealing with non-deterministic black boxes that we need to trust to be "right often enough." The Fluency Gap One of the biggest challenges isn't the technology itself - it's the lack of shared understanding. Hannah launched "AI for the Rest of Us," a community now with over 1,000 members, after realizing that AI fluency is essential for making good decisions about where and how to use these tools.
Transcribed and scored by The B2B Podcast Index.
It used to be that testers didn't trust developers and ops didn't trust testers. And there were all these silos. And now we're putting AI agents in the mix. Can we trust them?
Should we trust them? It's time for Arrested DevOps, the podcast that helps you achieve understanding, develop good practices, and operate your team and organization for maximum DevOps awesomeness. I'm Matty Stratton. We have a remarkably awesome show for you today with two great guests.
But before we jump into that, let's have a word from our sponsors. Today's episode is sponsored by Attribute. I met the team, saw a demo, and honestly, this approach is quite clever. They call it FinOps without tagging.
It's the first FinOps runtime technology that analyzes cloud costs based on infrastructure traffic instead of relying on billing reports or tagging. For teams who need visibility into per team, per product, or per customer cost, Attribute enables this visibility with one line of code. Think of a list of your teams or products or customers broken down to RDS, BigQuery, Kubernetes, OpenAI, data transfer, and over 35 multi-cloud services. Based on actual usage, fully automatic.
No spreadsheets, no tagging. They've been recognized in six Gartner hype cycles and are working with companies like Akamai and Monday.com. Arrested DevOps listeners, reach out by the end of 2025 and get their highest tier at the price of the base one for the first year.
Check them out at ArrestedDevOps.com slash attribute. Got robots writing your code? Great!
But you're going to need a safe place to run it. Fly machines are fast-launching, hardware-isolated VMs that spin up in milliseconds. Perfect for sandboxing even the sketchiest LLM-generated code. Dynamic subdomain routing and zero-cost scaling to idle?
Yep, got those. Robots break things. Fly machines restart cleanly and scale effortlessly. Fly.
io is built for humans and loved by robots. Check out ArrestedDevOps.com slash fly and make your robots happy. I am joined today by Hannah and Robert.
I'd love to welcome you to the show. And why don't you both introduce yourselves a little bit? Hannah, we'll let you go first. Oh, thank you.
Thank you for having us. So my name is Hannah Foxwell. And I can't believe this is the first time that I'm doing Arrested DevOps. How did that happen?
So I've been working in DevOps and platform engineering for over a decade. I like to think of myself as a bit of a DevOps elder. These days, I don't do as much of that as I used to because I got swept up with all of this craziness that's going on with AI in software development at the moment. And to be quite honest, I am like so here for this roller coaster.
Like we don't even know what the future is going to look like, but we know it's not the same as what it looked like just like a few years ago. I work with a number of AI startups. startups. I've built a community around AI fluency.
We're now 1,000 members, which I'm really proud of. And that's really what I'm doing these days. I consider myself a bit of an AI explorer. So I'm here to have a little chat about that today.
That's great. Yeah, I had the same thing when we were talking about having you on. I was like, Hannah has to have been on the show before. And I looked at you, sure wouldn't.
So that is an oversight we are remediating. So welcome to being on ADO here. And Robert, This also is your first time, but tell us a little bit about yourself for the audience. Hi, I'm Robert Werner.
Thanks so much for having me and us. Yeah, a little different background. I'm a little older than Hannah, so when she talked about her 10 years, I was reminded, oh my God, I'm so old. I'm probably 25 years in enterprise computing.
Started with the Commodore 64 some time ago. saw the internet bubble and now I've decided to join the AI bubble. Let's hope it's not a bubble. What I'm currently doing is actually founded in my decent age together with some friends, an AI startup.
And this is where Hannah is helping us. We both know each other from working together at Pivotal. What brought me there is that I spend a lot of time in enterprise computing, selling software into large enterprises and learning the pain. And now with AI, I think stuff is so interesting again, reminding me of when the internet started.
And so, like I said, we are up to building a new way of building software. And that's probably what we want to talk about today. Absolutely. And let's jump right into this.
We're not going to talk about being elder statespeople of the industry. We'll all just start aging ourselves if we play that game. So we just don't do that. Although I will drop one thing I think Hannah will get a kick out of.
Especially when I was looking for another position recently, when I was on the job market, I was doing a lot of interviews. And you have to do your, let me give you your background and everything. And I always would say, oh, I feel like I joined the DevOps movement a little bit late. And then I do the math.
And first of all, every year, I've been a part of it for a greater percentage. But it was like, yeah, it really was. We've been around most of the time. So just let's not talk about this.
But let's talk about AI. So everyone is talking about AI and people seem to have very strong opinions. It's something Kat Morgan and I talked about on the episode recently. I've talked about this with other folks is that at least publicly, it's very extreme.
We either have on one side, like on one end of the discussion, it is like no one's going to have a job anymore. and AI can send us to the moon tomorrow. And it's thinking like a PhD and smarter than your grandmother and blah, blah, blah, blah. On the other end, this is the worst thing in the world.
And it's a massive crisis and everything. And like most things, there's nuance. And like most things, humans don't deal with nuance very well. So we do love a divisive topic amongst ourselves, apparently, as humans in a society.
And as we know, systems are complex and we don't do well with complex systems. But that's a different podcast episode. But for this one, so like you said, people are saying a lot. People are talking a lot.
Many people don't know that much about what they're talking about because these are tentpoles. Hannah, I think you've talked about this idea of AI fluency. And I'm really intrigued to hear a little bit more about that because I'm an explorer myself and trying to find, okay, there's uses that are not so great. There's uses that are helpful.
What's real? And how do we develop that fluency to be able to, for lack of a better word, I almost said speak intelligently, but I'll get yelled at for that. But at least come from a place that I'm understanding. It's a huge need right now.
And I feel a little bit guilty sometimes because I say there's too much noise around AI. Everyone's saying things and not many people are saying very much at all. And that frustrates me. But I think about where I was like 18 months ago.
I went to a talk at a conference and I was like, I'm going to learn about AI today in this one talk. And I couldn't understand a word out there. Teach me everything about AI. Yeah, I was like, I'll just learn about AI today and then tomorrow I'll be an expert.
And it just didn't happen like that. And I've been in tech for a long time and it was a completely new domain, new vocabulary, new concepts, new techniques. And it was like it was a light bulb moment for me. And I was like, actually, there's a whole load of people in tech or who work in our industry who this sort of AI is coming for us, whether we like it or not.
And being able to make good decisions about how and where to use it is dependent on us having good conversations. conversations. And so that's where I started to really work on my own AI fluency. And it just happened that at the time I wasn't working.
And so I was like, I'm just going to take a few more people with me. So that's what I did. I built this community called AI for the rest of us, where we focus on that. We focus on providing talks and content about AI that don't dumb it down.
This is not about us dumbing it down to all the magical black box does a magical thing. But it is about making it accessible as a domain to which is very dense in new vocabulary. Because I do believe that like, we can all understand it. But we need someone to take the time to explain it to us in simple language.
And then we're and then we're away. And so yeah, personally, for me, I think the more people who have that foundational level of AI fluency, the better conversations we have. And it's such a pivotal time in our history, where we need to really make decisions about what the future looks like with AI. And if we're not all engaged in that conversation, or having an intelligent conversation, like you said, then we're not likely to make good decisions.
So it's a bit of a mission driven thing as well. It was selfish to start with, it was like, I want to learn this thing. So I'm gonna I'm gonna learn about it. But now it's very much actually, this is just incredibly important.
And the more people I bring into this community to help share knowledge, the better. I think that's really helpful. And it reminds me a little bit, I said this thought, because AI can be such a broad thing. We joked about how to teach me everything about it, but just in general, even conversationally, and it's a little bit like the early, like cloud, because depending upon who you ask five different people, what cloud means, you'll get six different answers because to some people cloud is where I put my phone is photo or whatever.
Some people it AWS it and at its core it one thing And that all those things are true but some but some things are better used than others and the same thing When we think about AI and LMS and agents and all these places there are certainly use cases and what people look at and think about it where you like yes but that a very different conversation than how we using that tool And it gets very hard, especially in our soundbite, short post, short bait world on either side, or I'm trying to get a bunch of money.
So I'll say what I need to say. So when I think about Gen I, and I think for purposes of this conversation, we're thinking about using generative AI and LLMs and agentic work around building and operating and working with software. So in that context, which, by the way, is a whole other thing. My wife uses AI a lot and she's an operations manager at an HR.
She runs HR and ops and things like that. And her experience is completely different than mine, just because it's a different use case. Anyway, but we're going to talk about it in terms of that. And I think when we think even before this wave of this kind of work, as an industry, developer experience, DevEx, it's interesting to think about like 15 years ago, even what even was, that was not a thing we would think to care about.
It was like tools are tools. they are. So DevEx is a thing. It's continued to be a thing.
How does that really work when we're talking about how people are changing in the way that they work and navigating this through all the different kinds of things that we've gone through with working with Agile and how well it worked, maybe it didn't, you know, DevOps, how we talk about all these things. What's similar? What's different? What are you seeing in how people are adopting or struggling as well?
There are definitely like some lessons that we can take from like these other transformations that we've been through. And again, like both Robert and I were from this like very enterprise transformation backgrounds. We weren't part of startups who were able to like adapt and evolve very, very rapidly. We were up against a lot of bureaucracy controls.
And even, I even remember the first sort of agile transformation I went to, there was a lot of resistance. And then with the cloud, there was a lot of resistance. And then there was DevOps and people like, people didn't believe that continuous delivery was possible. Like, they were like, it will never work here.
And now you fast forward a decade, and it's, it's common practice. And we've accepted that this is a better way to deliver software. But I often think about those early days where you had to be a bit of a believer. And I feel like AI is kind of a generative AI in the software development lifecycle was sort of at that early stage again, where it's like there are some companies who are doing incredible, amazing things.
And those are glimpses of what the future will look like for everyone. But there's resistance and there's new challenges. We had to, with DevOps and continuous delivery, we had to completely rethink how we did ops. Like reliability and stability was derived from like the lack of change, the consistency.
And then when there was frequent change being pushed through for a short while there, stability and reliability suffered. And we had to build our platforms in different ways. And we had to invest in observability and self healing and things like that. And I think with DevOps back in the day, and developer experience becoming central to that, it's about how you take people with you, isn't it?
And I think that's probably what a lot of teams are feeling at the moment, because there is unfortunately, there's this massive top down pressure to adopt AI and do it quickly. Like we're going through a transformation, everyone, but it's a transformation where like the where we haven't got to the point as an industry where the best practices are really solidified around it and the tools are still evolving and we're coalescing around certain patterns but it's not like there's no playbook yet and like i said i'm like i'm really here for it like i'm here for this roller coaster okay what does the future look like where are we seeing patterns for success what could we teach what could we learn from what other people are doing but yeah the customers that we used to work with at pivotal robert they were sometimes like resistance to our methodology.
We've introduced them to things like platform engineering and platform as a product thinking, having a dedicated team that really only cared about developer experience. And I'm sure some of those patterns will continue to help those companies through this AI transformation. What do you think? I totally agree.
The interesting thing, if I look back, is that management in enterprises, they had to take so many decisions and change their mind over what cloud brought. For instance, do you remember this trust issue that management had to give up trust in the sense that in an agile way, you push trust to the edges of your organization and individuals have decision-making power, when to ship code, and you have automation and self-service in the cloud. And it's so interesting. Now we're dealing with AI and all of a sudden you're dealing with llms that black boxes and with everyone is learning now they are non-deterministic and all of a sudden we have to trust an llm to be right often enough and this trust world comes up up all of a sudden again what i draw from that is that we are in the super early days if i compare that with the agile transformation or the devops transformation and tools have to get more mature.
The underlying technology has to get more mature. And even if they would reach a plateau, I think only after then management can draw their conclusion and there has to maybe become a common sense of what's feasible and not many more ideas will be created. So that's why, Hannah, I'm totally with you. I thought I had to do that.
I have to be there because there's so much to discover right now and we can be creators in terms of tools and approaches yeah i think the trust thing is huge isn't it enterprise tech is so like success or failure based on based on trust it used to be that like testers didn't trust developers and ops didn't trust testers and they were all these silos and now we're putting ai agents in the mix can we trust them should we trust them i don't know There is a massively interesting parallel in a couple different ways, because all of these end up being cultural transformation.
And I'm trying to have three different thoughts in my head here. So we'll see if I can remember all of them. But one, we don't love Ronald Reagan and everything. But he was, I would talk a lot about even back in when we first started talking about DevSecOps, which will always give me anxiety, because I'm like, that's what it was the whole time.
Thanks for giving this a stupid name, Andrew. But trust, but verify. And trust, but verify really applies to agents. This has not occurred to me until right this very second.
Then maybe everyone else was like, of course, Maddie. But I was just like, but that particular phrase. But also, when we think about the change, and Robert, you're talking about getting the trust and the management having to change. And people would always ask, say, Maddie, what's the most important DevOps book I could read?
And it's not continuous delivery. Sorry, Jez and Dave. It's a great book. what's not accelerated.
It's Freakonomics. Go learn about incentives. And then you can learn how to do a DevOps transformation in your organization. And it's similar, right?
Because the more that we think about it, I used to quote this guy all the time. I forgot his name. He was it in a go, but you can't change culture, but you change behavior and you change behavior by changing incentives. And sometimes, and then you think about, we can apply Nash Pareto to all of the AI stuff too, which is the incentives don't always work towards the incentives of the organization.
So we're looking at when we think about adapting and adopting these things, what are the unintended things that we are incentivizing perhaps? And there's some, and I actually, Robert, I really want to hear about, I want to talk about Trust Gap. I want to talk about AI generated code, but this reminded me of a thing that Jess Humble said, and that he said, was talking about when he was at ThoughtWorks, was working with a customer and they had come up with, they said in every sprint, we're going to add, every developer has to add three more tests, which meant they ended up with a whole bunch of tests that were assert equals true.
and they're like, I did the tests. And I cannot tell you how many people I talk to who are in organizations now that say, we are required to use Gen I. And they're just looking for bullshit ways to do it so they can tick the box that said, I can fill out my AI TPS report this week to my manager that says, I used an AI tool and I just did it to do it. And this all goes back to outcomes.
It's all the same shit we've been talking about for years and everybody's screwing it up. But that's the cynic in me there. But Robert, you talked about trusting the agents and I've in my explorations and we're going to using it, Gen AI in various ways. I built things with it.
And I, one thing I will say is it's very, we talk about non-deterministic. My take on how helpful a coding agent is. You ask me two days in a row and I will give you two very different answers. because I will either be like, wow, that was really awesome, or oh my God, this thing fucking sucks.
It is so bad. And that's being hyperbolic. But when we think about what we can do with this code that's created as we're vibe coding, as we're building, whatever it is that we're doing, like, how should we be thinking about that? What are the successful ways to think about this?
And what are maybe the pitfalls to watch out for when we're thinking about how we can use Gen.AI with our software development? Oh, that's a very good question. And at the same time, I'm thinking I have to readjust my thinking every other week and every other month because it's moving so fast.
And like you said, things that worked last week might not work this week because someone updated from GPT-4 to GPT-5, sorry. And at the same time, there are other things possible that were impossible last week To give you an example us being a startup we are in contact with our angel investors and the pool leads from our VCs And they always share experiences and knowledges from other startups. And almost every other week, we hear that the hiring strategy changes because all of a sudden, maybe last week, it became feasible to use agentic coders.
We all heard about Devin last year, but it was not usable and not very practical. but just whatever a few months later all of a sudden it's feasible to do certain things with AI that weren't feasible last week so if you ask me what's it's gonna be like I don't know to be honest I think the AI that I have right now looks reminds me of my old Nokia cell phone from from 25 years ago I think it was one that started with a 10 so it was a monster heavy thing and if I compare that to my whatever iphone whatever you have there's no link to that it's totally different nevertheless we think ahead and we try to envision we as a team we try to envision what the future will be and i strongly believe that number one the core technology will get better i don't think we will ever get rid of hallucinations but if it gets better and plateaus then we can start building our procedures and our habits around that.
And that means we get, for instance, we get used to the fact that you can't simply prompt and bytecode a workday on SAP or Salesforce. That's nonsense, right? If the AI ever becomes good enough to, from a prompt, build you a SaaS solution, then your prompt has to be mega complicated and super fine grained. And people are right now used to YouTube videos where you prompt a little game and you get a 3D shooter out of whatever, an airplane simulator out of a few lines of text.
But that's not an enterprise application. That's not what we're aiming for. So I think let's get the core technology stable. And not every other month a new LLM is coming out and everyone is expecting miracles like we had the last 12 months.
Let's build new tools on top of that as a foundation. And then let's create a culture and workflows around that that are based on human experience, how to deal with this black box. Because like you said earlier, Hannah, we're all still learning what's feasible and how to trust it and how not to trust it. And that's my take on that.
And we've had conversations about this before. The big question is, will AI be better and faster at writing code than a developer? And then we have to think about what is the role of a developer? it because somebody needs to own the code.
Like somebody needs to own the service. And then we, it's like, it's the ops, it's the ops challenge again. Does anybody really want to own a service if they didn't write it and they don't understand how it works? And I think that's another gap that's emerging as well.
It's the understanding like that you need to be able to trust and operate. So it's very relevant to the DevOps conversation because I used to be throwing over the wall from dev to ops. It's now AI throwing the code at us potentially. And I think that, again, things will evolve.
And we look at the current state of something, things can change, but maybe not. But that's a real interesting problem. Because if you're like, I haven't really thought about it until you said that, Hannah, but yeah, we're back to the wall of confusion. It's almost worse in certain ways.
And I'm going to speak from my own experiences. So I have a project I'm working on that I'm building with substantial help from Cursor because I suck at React. But from an infrastructure and architecture perspective, I understand the problem and pieces and parts. And one of the things I will tell you that I absolutely learned, and shout out to the wonderful Heidi Waterhouse, who will be like, duh, Maddie, don't you even ever think of having an AI write your docs.
But I was like, okay, I can do this. And it's really bad at this. I shouldn't say it's bad. It's a bad idea because there's so much information it's trying to be.
So you talk about the hallucination, the hallucinations, like it doesn't do any of these things. But there were docs I had in my repo that were my PRD about, I think I'm going to do this. And then it's, oh, that must be what it does. And so I'll put that in the documentation.
So as it becomes, and it's interesting because great documentation is what makes agents work really well. But that requires like, and that's a lesson we learned at Turbo where I'm at, because we went through an experience of how do we make it so that people who are using Gen AI to develop against our APIs, against our tools, have a better experience with that and went through kind of basically what we found was the best way to do that was to have, you know, kick-ass documentation, because that's how it's all trained.
And then we're all guessing it's like the old black box SEO days. Oh, do you have an LLM.txt. And how do you do this or whatever?
And we're like, the other thing, and Robert, you kind of lose is what we found for our own practices was, went through a lot of experience, like, oh, do you write cursor rules? Do you do this? And it was like, no, just to be very explicit, and small in what you're at. And with my joke, I said, depending upon which day it is.
No, the truth is, the days when I get frustrated using a coding agent are the days that I asked it to do too much. Where I sat there and said, this is when I was too vague, I was too broad. I had some GitHub issue I created that was like half-assed. And then because it was once it's built in a way to be like, I have to come up with an answer.
I'm going to do a thing. I'll do the thing you said. But I'm like, maybe the thing I said wasn't really well-baked. But when I'll sit and say, okay, we need to refactor it exactly this way, that's helpful.
The idea here is to be successful with these tools requires an understanding of how it will be operated, of how to build the architecture, how to do that. Nobody leaps out from Zeus's head fully formed as a senior software architect. We all got here by doing shit work, by doing all this stuff. And so if we're like, that's cool, the agents are going to do that.
So nobody has to do that anymore. We're not going to hire people to do that. You're never going to get to that point. And we're not going to solve this problem on this show.
But I think there's probably ways because the same argument was probably made when we think about DevOps, when we think about this, how could you possibly know how to operate? You're like, you do, you learn. And like, maybe it's the evolution of thinking about how you guide these places. And maybe you start a little bit further down the conversation, but we're not there, I think.
If prompting your way to a solution requires you to already know what you want, then it's, yeah, the AI has taken the actual typing off you, but it's not taken the thinking, the knowing and the understanding. One of the things I really like about Leap2, your product, Robert, is it's quite opinionated. It's quite opinionated and constrained in what it will and won't write for you. So it's prompt driven and it's visualized, but it's also constrained in that it can't go too far wrong.
Like we talk about guardrails. I don't know whether you've been to like an AI security talk, I'm sure there are loads of them and people jump up and down about guardrails and no one ever gets really specific about how to implement them. But I think it's like the necessary constraints about what's an acceptable solution in your context so that you can constrain the LLMs and the agents. And all of that knowledge that's internalized within you, Matty, about how to architect things becomes external and like implementable by anyone.
But I don't know whether we're there yet. Like I said, like everyone, everyone knows that we can't just YOLO this AI generated code into production, but we're still wrestling with the like, but how do we do it with reliability and security and speed? It's funny, we've been talking about this for forever. In Infracode world, all this stuff is guardrails bring you freedom, because then you have trust.
this is not a foot gun. I can't accidentally drive this car over the cliff. So that makes me feel really safe and comfortable to explore. Whereas if you're like, okay, wait a minute, I don't know if I could accidentally drop the main table and prod, then I'm going to tiptoe everywhere and be very conservative.
And I'm not going to be able to be as creative. And this goes all the way back to blameless, right? The whole idea was not that somebody made a mistake. It's why did the system allow it to happen.
So to do that, you have to have a system that can create with that. So actually, I would love to hear a little bit about your philosophy around this, Robert. And like, how are you thinking about that product? Obviously, I said everything is very nascent, and we're trying lots of things.
But what have you seen that's effective with that? And what's the guiding principle that you're seeing around that and what you're doing with your product? Yeah, the thing is that we try to think backwards from not knowing how to get into the AI future, but what would be the end vision? And the end vision is accepting that lamps are fallible and non-deterministic.
And even if they would be, we as humans, we might not be clever and prompting. So one thing is to add the guardrails. But the other thing is that we thought is if you use AI to generate code, even if you guardrail it like hell, we all know the difference between a plus and a minus. Even if one symbol in your code is wrong, what would that mean for an insurance company or for a bank if there would be a plus instead of a minus or vice versa?
And that started our thinking that no matter what, for important pieces of code, maybe code as a whole but at least for important pieces of the code there will be in what I believe no other way but some human to take responsibility for that and some human to validate that code and look at that code, even if it's tedious work. And I'm convinced not many developers will like that. I grew up as a developer. I don't know about you, but the job that I don't like is verifying or reviewing code from someone else.
But realistically, the LLMs will be much quicker producing code compared to any human in the world. So we'll probably all end up to some degree verifying AI-generated code much more than everything else. But our idea is just let's make that as convenient and enjoyable as possible. So basically, we're going to build a tool, a platform that allows humans to verify and validate AI-generated code in the most convenient way and the best user experience.
Because we believe there is no other way. Guardrails plus AI generates code and put humans in the loop and make it as convenient as possible. It's really interesting, something that you said when you said about taking responsibility and we don't want to do that. And that's an industry failing.
This goes back to the old thing about why in some countries and in some states, you can't use the word engineer to refer to what people do in software because engineers have a very specific thing. And one of the biggest differences, as we've all talked about before, about like a structural engineer who builds a bridge versus a software engineer is if that bridge falls down, the engineer is responsible for that, liable for that versus we have no liability. And a big part of that was because of the industry boom and nobody would want to do it.
And it's interesting because from a doer perspective, we're not in the new Kingmaker era anymore. As the engineers, as the doers, as the stuff, we are all fungible. We are all, but the people controlling the companies of that, they don't want to take liability either. So it's still not going to get any better when it comes to that.
But still, ultimately, we do have a liability. We do have a responsibility. And I really, I like the idea of saying like, how do we make that easier? Any transformation, make the right way the easy way.
Make it harder to do it wrong is how I always think about it. You have to go out of your way to have it up. But it's interesting too, because I was just thinking, saying about, you know, how we're doing the reviewing. I always get a kick out of the fact, because again, being in Explorer mode, I don't know that I would do this if I was working on real production code.
But I'm always trying every single thing I can find. And so on my project, generating code with cursor or whatever in myself, and then it goes in and then I've got the Codo bot that's commenting on the PR. And then we're taking the feedback from this bot to that bot and back and forth. Now, there's still a human in the loop, which is me.
I'm not sitting there and saying, hey, okay, just you put in a PR, wait for feedback from this other robot and do whatever. I'm taking that feedback and then half the time going, okay, I see where you're coming from with this, but no, that doesn't apply. And then being able to connect. But especially, I always, again, think about incentives and think about what people have to do.
And it's one thing for me on my hobby project to do this. But if I'm under the gun to deliver and deliver faster, because my leaders are like, well, you use AI, so why aren't you shipping faster? Then maybe I'm just gonna be like, sure. It actually reminds me of a really silly thing way back in the day when the Markov chain bots were popular, the e-books things or whatever.
And so, Hannah, I don't know if you remember, I had a bot called Pete Chesbot that was trained on our friend Pete Cheslock. And it ended up, just because I didn't code it right, to do this, got into a loop with another e-book bot. And it just... Wait a minute, don't talk to these other bots because they'll just sit and have a whole conversation with each other.
And it was hilarious, but it's not necessarily hilarious when it's code that you're shipping. No, exactly. And I think a lot of the teams that have invested in good engineering practices, such as like blue-green deployments, A-B testing, making it really safe to fail in production. They're probably going to benefit from this sort of step change in code production and velocity because they've got those safety nets in place.
and folks who don't, who are still relying on those fallible humans, then it probably sounds higher risk, doesn't it? But yeah, there's loads of ways. There's loads of ways that you can envisage the benefits, but also the new problems that this technology and this speed of code generation brings to us all. But I do think that if we step back, a lot of the engineering practices, and I think they exist already, they're just not as well.
are just only adopted by the real sort of leaders in the industry. And there's a lagging majority that haven't quite got there yet. But I hope that they do. Otherwise, we're just, again, we're just YOLOing AI-generated code in production.
And our customers will suffer for it. And our businesses will therefore suffer. And we'll all be like, oh, no, AI was such a bad idea. I want an artisan human-written piece of software, please.
We don't want that to happen. It would be a shame. You never know. Maybe that's becomes a boutique industry where like organic when you say this is USDA organic code.
I don't know. As I say, get your bag, whatever way you need to get paid, figure it out. As is often the case with any pod, right? You're like this spun off possibly 17 different deeper dive conversations.
I would love to just get some final thoughts from each of you about, especially folks who are explorers, because I figure there's definitely folks who are like, this is just bad. not into it, whatever. And Hey, you, that's good. I'm not here to judge anybody's how they look at stuff.
But if you're like, I got my main made up, this is crap. Then we have any advice for you, right? Then continue. And if you're going the other way, whatever, but for probably what is the majority of people who are explorers who are like, I have to learn, I have to figure out how to do this and where this goes, what would be, so we'll start with you, Hannah, I'm going to ask the same question to both of you.
What would be like your number one bit of advice to say as an explorer who's trying to learn more, what's the most effective way, most effective tip you would give somebody to keep in mind? So can I say two? I'm going to say two. My first one is really simple.
It's a guardrail. You can only do one. It's a constraint. No, you can do two.
I'm cheating. I'm cheating. So the first one is just to have your eyes open. I do think that With any transformation, there will be some folks who are out there experimenting and sharing what they're doing and having your eyes open to see what the patterns emerge, like who's being successful and what are they doing?
What do they have in common? And I guess the second one is related to that, because the early days of DevOps, for me, the biggest benefit was sharing stories of success and of failure with the community. And I think community is a huge thing. And having that network of people who are going through similar changes and transformation together and sharing what works and what doesn't work.
I think that's increasingly important. And one of the reasons that I wanted to start building community again, because I thought what we need is a place where we can get together and talk about these things, where we don't have to pretend that everything's great. And so if I could do a tiny little plug for the folks who are based in London, listening to this episode, we've got our annual conference. AI for the Rest of Us is in London on the 15th, 16th of October.
If you're in London, you want to come along. there is a discount code for a 20% off your ticket, which is going to be part of the show notes for this podcast. 20 is the code that you need. We are not live streaming, but all of the talks will be available online afterwards.
But if you do happen to be in London, I'd love to see you there. Like I said, I think for me, community is the main thing that we need right now. We need a place where we can talk about this. And I'd love it if some of you could join us.
Robert, what is your pro tip that people should take away? as they want to explore and learn. Yeah, I think it was really clever what Hannah said. And to add maybe something else, some personal experience, I started to follow a handful of news outlets and YouTube channels, really, because it's so fast-paced.
It's probably you don't read it in the newspaper. But my additional tip is there's so much noise. I think there's so much overhyped stuff or niche stuff or paid content that you have to be careful and not to waste your time. just sneak around and look for the quality outlets and then maybe ignore the stuff for a week or a month and then stuff that comes up again then you should maybe spend your time with and and practice unless you practice yourself like you said you played with cursor and you probably have to do it every other month because on your model coming and an updated version if not if that's not your main job then maybe do it every other three months and do a refresher So yeah, that's what I usually do.
I love it. If you head over to ArrestedDevOps.com slash AISDLC, you can find this episode show notes where we'll have links to everything we talked about, including Hannah's online community work that she's doing and maybe some of the great resources from Hannah and Robert. If you visit ArrestedDevOps.
com slash iTunes, you can leave us a review in the Apple Podcast Store that can help other people find the show. You can also listen to us on Spotify, iHeartRadio and Audible and pretty much everywhere that fine and less fine podcasts can be found. So Hannah and Robert, thank you so much for joining me today. This has been a great conversation.
Thank you for having us. Thank you. This is Arrested DevOps. And remember, there is always DevOps in the banana stance.