614: AI Code Audits

Speaker A: Giant robots smashing into other giant robots. Hello, everyone, and welcome to the Giant Robots Smashing into Other Giant Robots podcast, where we talk about the design, development

Speaker B: and, and business of great products. I'm your host, Chad Pytel.

Speaker A: And I'm your other host, all the way from London, Sammy Birnbaum. Chad, what is it like being in the US as the World cup kicks off?

Speaker B: Uh, so I just took the train from New York back to Boston yesterday, and the train was full, and there were so many Europeans on the train. There was several people from Scotland, uh, one of whom was wearing a kilt. Uh, and so, uh, yeah, it's great. Uh, I do have, uh, my biggest beef with the World cup is unrelated to it being in the US Actually. So we are running, we do our monthly tech leader meetup in London. I've mentioned it on the show before. The next one is scheduled for June 23rd. And we knew that that is an England match that day that overlaps, uh, with our event. Uh, our regular venue wasn't available, so we found this really great large outdoor place. We were saying, okay, we're going to do, like, summer drinks outdoors. We knew that they were showing the game, but they took our reservation and it's a big play. So then, like a month later, they contacted us and saying, you know, we're showing the game, right? Like, uh, we think you're not going to want to be here for that. And we're like, no, we're sort of expecting it. Like, you know, it'll be fine. Uh, and then we're like, no, really, we don't want you here. Uh, and so we lost our venue, uh, for the event. And to be perfectly honest, we may actually need to cancel because we're doing an event in Brighton the next day. We have Brighton Ruby, like, that same week. There's not another day we can really reschedule it to. And, uh, there's no other venues that we can find, as far as we can tell, because everybody is showing the game. So we'll see what happens. Stay tuned for announcements about whether our June event in London is canceled.

Speaker A: But you know what, but we could have married my two favorite things. Right Thought. And England, like, we could have just watched the game together.

Speaker B: That's. Well, that's what I. That's what I was saying. And that's what we told the venue. Like, no. And they're like, no, you're not going to want to be here. Like, we don't want you here. So, yeah, I don't know.

Speaker A: That's really interesting. Do you know that it's causing you even more problems? I don't know if you know about these problems. The World cup, it's causing you more. More problems. Well, they've announced in Scotland that there's going to be a bank holiday on Monday because, sorry to be the breaker of bad news. So we're recording now on 12 June, and the Scottish Government have announced that on 15 June there will be an impromptu bank holiday because Scotland play Haiti in their first fixture in the World cup, which they haven't got to for a number of years at 2am and so I think they've just kind of accepted that none of the Scottish are going to be turning up for work. So, um, we have some developers based in Scotland and, um, they're going to be having a great long weekend.

Speaker B: But, yeah, it's exciting. It is a little weird because it's all over North America, so it's not really, uh. I don't know why they did it this way, but the venues are very far apart. In other words, they're all over the U.S. canada, I think. Mexico too.

Speaker A: Yeah. Do any come close to you?

Speaker B: Yeah. So the reason why there was people on the train is because the Boston venue, which is actually not in Boston, uh, is probably about 40 minutes away from me. It's where the Patriots, the NFL team, play.

Speaker A: Oh, but you could still try and get tickets for a World cup fixture, for a World cup game.

Speaker B: I could go to that Scotland game.

Speaker A: Oh, you have to. What an experience to be at the World Cup. It's difficult for me to explain to you, but I can map my lifetime based on World Cups. Like, I know I can remember my first one and my dad telling me that I'm not old enough because I won't sit quietly to watch, uh, World Cup 98. So I was born in 91. So I was seven years old for, like, World Cup 98. And that's my first memory.

Speaker B: But sit quietly. Why do you have to sit quietly? Because you should be up hollering and excited.

Speaker A: That's. That's only when a goal goes in. The real connoisseurs, they would like to sit there and assess the game, uh, from a tactical perspective. Um, there's an element of kind of, um, you know, sitting there and being nervous for a long time. That's part of watching the game.

Speaker B: Okay.

Speaker A: Um, but that's. I can, like, so from 98. And. And 1 of my best World Cups, um, was in the World cup in Qatar in 2022. Coincided with when My youngest son was born and that meant that I had six weeks paternity leave from Thought Bar, which happened to magically coincide with the whole entire tournament. And what that meant is I would be a selfless husband and offer to take the baby any time of day and I would bottle feed the baby downstairs. And I watched every single game of that World cup from beginning to end and it was incredible. I immersed myself in the tournament for a number of weeks.

Speaker B: So are you excited for England this year? What's it looking like?

Speaker A: I'm pretty excited. Yeah. It's looking. The kickoffs are quite late for everyone in Europe. Um, yeah.

Speaker B: Because they're often in the evening here.

Speaker A: Yeah.

Speaker B: Or the afternoon, I guess.

Speaker A: Yeah. Afternoon towards late evening. So it ends up being in the nighttime here. Um, but yeah, really looking forward to it. The chances we have are decent. Um, there's obviously kind of always the favorites around Spain, France, Brazil, Uh, probably put Argentina up there. Um, but England are strong and if we can get deeper into the tournament, then there's a chance. But actually, um, we did a sweepstake in the company. Are you aware of what that is?

Speaker B: I know what a sweepstake is. Yeah.

Speaker A: Okay. So we essentially, uh, I got a few people together and we've. Everyone's got certain teams. It worked out. We only had 16 people and I think there's 48 teams. So everyone got three countries each. Um, and I organized the whole thing and then I managed to come out with the best teams. But it was random. But I've got people on my back like telling me surely this is rigged. Um, because I managed to get Spain and Portugal, uh, which are two very good teams. So I'm looking forward to winning.

Speaker B: Nice.

Speaker A: And there's a little prize for the winner which is a Thoughtbot mug, also available at the ThoughtBot shop. Wow. Yeah. So it's exciting times. But aside from that, there's been in my own head and this is something which I wanted to get to. I've been pondering about AI code bases and I know this has come up a little bit with the work you're doing with Ready, Set, Go. Um, and one of the driving forces behind that is can we have a code base that is built with or using AI that is then we're able to pick up and work with. It's almost like it's developer friendly and it's built with best practices in mind.

Speaker B: Right. Uh, and the reason why we thought it might be possible or interesting to try was because we knew that with a ThoughtBot developer in the Loop, like not vibe coding, where you're just generating a bunch of stuff without looking at it, but when you're actually taking control over it and you have a person in the loop. We were seeing we can produce really high quality stuff fast, that you still understand the code base and know what is part of it. And we said, well, okay, instead of having a human in the loop, if we have another AI agent in the loop keeping things on track, you would lose that understanding or you would no longer have everything in your head as if you had been in the loop. But could we solve the quality problem at least? Um, so that's part of our angle on it. Yeah.

Speaker A: And that's really interesting because I think, um, I'm seeing quite a lot of questions and queries about code, uh, bases that are built using AI. And for example, I, um, recently did a, ah, code audit on a code base that was built with um, that was built with AI to try and get an understanding of, you know, where do we go from here? Like, what's the difference?

Speaker B: Was it built with AI with like actively uh, with a human, like in the loop, or was it more like vibe coded where that person wasn't even really looking at the code, wasn't doing test driven development.

Speaker A: So the code was never looked at. M. But it was uh, built by an engineer who has experience and used something called spec driven development.

Speaker B: Okay.

Speaker A: So I haven't even looked up what that is, but my understanding was it's basically giving clear directions. Think of putting claw code in planning mode, but someone with experience.

Speaker B: And that was actually was it asking the AI to do test driven development where you do red, green. Okay. Yep.

Speaker A: No, uh, no, there are test suites, but I don't believe it was necessarily done in a test driven way. And that point is really interesting. So I had a few takeaways from it. One of my takeaways, which is something you just touched on as well, we spoke about the developer out the loop is everyone in the room needs to treat this code base as someone else's. We can't come in with the perspective of even the person who has utilized AI to build this code base. Don't think should be comfortable saying this is my code base when they haven't looked at the code. So we should all come in with a perspective of we've all just been delivered this code base because someone else has written it for me. That helped me to change the way I approached the code base. Instead of thinking of this as something, as someone in this room has a really good Understanding and has written it and knows what's going on to someone has just delivered this and we need to come at it externally to understand what it's doing. Um, and I guess that's the difference between having a developer being hands on or like you said with Ready, Set, Go, where you have a separate agent who then does the checking. Like when you're left, when you're left with that code base at the end, it's as if someone else has delivered you a code base as opposed to you've actively written.

Speaker B: It is an interesting take because like that has been happening for a long, long time. Like I mean that's what people hire us to do, right? Is to write applications.

Speaker A: Yeah. But where it gets interesting is if I as a developer am um, using AI to write code, but I'm not actively writing the code. So I'm using AI whether I do it through plan mode and I get Claude to write it. Even when it comes to me deciding how much I might trust this code base, I think I still need to view it for myself personally as a code base someone else has written. Call that person Claude or call them whatever you want. Um, because that was one of the biggest issues we had and actually saw, uh, Kent Beck wrote about this recently and I'll try and link to that article in the notes. Um, is the concept of being able to trust a code base. Um, and there seems like there was a real lack of trust from the people who had written the code base. So questions were being asked like is this usable? Could this go to production? Does my test suite do what it says it does and it's, it's very difficult to trust the code.

Speaker B: Yeah.

Speaker A: And that does.

Speaker B: Another big area is security concerns that people often have is like I don't know since I don't know what's there. I don't know that there's not big security holes in it.

Speaker A: Okay, but what if you were then to say to Claude, can you go and check if there's big security flaws? And it came back and it found a few. Does that make you feel more certain?

Speaker B: I think that's the question. Right. It's like, well why were they there in the first place and how were you able to find them? And then are there more?

Speaker A: But that's one of the biggest issues though because if I can't trust mhm, the LLM, which by nature of it being non deterministic is going to be difficult, then I'm not sure how someone moves forward with that product which hasn't had Human oversight and someone actually going through the code.

Speaker B: So one way is to hire us to do an audit, which is what happened.

Speaker A: Yeah, that's right.

Speaker B: Yeah. And that's been true even before. Like, the audit is a, uh, standard sort of package that we've had and we've been doing for years and years and years. Because there was always a case where someone hired somebody to write an application and they didn't trust it, or they needed to verify that it was suitable for whatever, you know, continuing to use or going to production. Oftentimes there'd be a trigger. Like, we're finding this really, uh, buggy. Uh, can you take a look to figure out, is this just fundamentally flawed? So these concerns have existed for a long time, but I think they're particularly acute when you are able to generate so much so quickly.

Speaker A: So let me tell you my five takeaways, some of which we've touched on, and I'm interested to hear what your opinion is. So takeaway number one, how the AI is used and who is using the AI makes a world of difference. I think that's fair. I don't think that's controversial.

Speaker B: Right. Um, I would assume that someone who's technical, for example, is much more able to request things or word things in a way that takes into account common pitfalls of building applications that prevents them from happening in the first place. Yeah, there were times where, uh. And the other thing is, for example, uh, it was a while ago, but Clarissa and I were on an AI in Focus Stream doing something, and we were, you know, we asked the AI to build it and it did everything like, quote, unquote by hand. And then I was like, do you realize this is a feature that's built into Rails? Uh, and it was like, oh, you're absolutely right. And it rebuilt it, uh, you know, in much less code by using a feature that's built into Rails. And someone who doesn't have that experience would never have identified that that was the case and probably wouldn't even prompt in such a way that even Claude would then discover it and offer to fix it.

Speaker A: Yeah. Which is no different to how developers had the ability to kind of search through Google and Stack Overflow, um, and kind of ask the right questions. One of the biggest things for me, as I learned more, was not that I knew more, but I knew what to ask or how to phrase the question to get closer to the answer.

Speaker B: Right, okay. What's number two?

Speaker A: So number two is positive because you might listen to this and think, like, I'm Hating on AI. My actual conclusion was that in this case, the cobase could be built on top of and not actually thrown away. And I felt like, because it was built by someone with an engineering background and therefore had, I'm assuming, like the plans or the specs were quite thorough, I felt like in the cost benefit analysis of what to do here, the cost of throwing it away and restarting would be higher than the cost of actually spending six weeks of having a developer go in there, see how tricky it is to work with, and potentially have like a phase of, um, like technical debt pay down would be, would be the right way to go, which I think was a really good outcome because at the end of the day that meant they had got some sort of a product, call it a prototype that they could play with. It had confirmed that a lot of the things they wanted to do were possible. And overall a lot of the code was kind of there, even if it wasn't in a great place. And so I still think the benefits kind of outweigh the cost. And so my recommendation was get it to a developer, uh, tell them to build a feature and see how hard it is to work with the actual code. And then maybe, yeah, you have six weeks of technical debt pay down, or maybe you do that iteratively over development so you kind of get a little bit of slowdown. Maybe it's, let's say it's a 20% tax, but I still think that would work out better, uh, than building from scratch, which is good news.

Speaker B: Yeah, I'm not, I'm not surprised. Yeah, not only is it good news, but I'm not entirely surprised. So.

Speaker A: Okay.

Speaker B: I think things have gotten, uh, to the point where that's the case, particularly when, to the first point, it's being used by someone who is experienced in the first place.

Speaker A: Yeah, I think, like, that was a win kind of for everyone. Like, that's. People have to understand this. Like, even though we work in the industry, that's really positive for us. We don't want to do things that are easy to do or that have already been done. Um, we want to actually pick up where we can add value. And I see a lot of that in the next stage is making sure this code base is resilient to change. It can be picked up, it can be understood. So that was an interesting takeaway. I'm not sure if you'll agree with my next takeaway. So there's a test suite. Because AI has written the test suite. Yeah, I think it should all be deleted. Delete the whole thing. And I'll tell you why. I think that you want to jump in there. Do you want to jump in there and then just say that I'm wrong?

Speaker B: No, uh, go ahead, hear me out.

Speaker A: Hear me out. Okay. So the whole purpose, I think, of a test suite besides documentation, right. Is I trust that test suite is actually testing things work. If I don't know what that test suite is testing, then what value does it have for me? What value does it produce to me? So I see this green dots, but I don't know. And I can be quite good at this. You know, if it sees a failing test, it can adjust like the test setup to make sure that it goes green without necessarily ensuring know the test is testing what you want it to test. So I personally felt that those tests don't provide value and just get rid of them.

Speaker B: So I'm not sure that. So, so in terms of the first part about AI just adjusting the test, I've seen that too, but more often I've seen that it actually is a failure. And uh, especially as these tools have gotten better and better over the last few months. But, um, since, but, but, but since you say that the plan is that people would start to work in this code base, I think that's where I would most pause in terms of if you really expect people to work in the code base, then having the tests there is going to aid them. And that's true that you, that you want.

Speaker A: So my actual wording was this. I think in order to trust the test suite, it has to be read by a human and make sure it's testing what it says it tests. Yeah, but is that accurate? Is that true? Does that make sense to have someone spend a week or two actually going through? I mean, it's an arduous task to go through and read tests.

Speaker B: I probably wouldn't do that. Um, I would be more inclined to say, here are the concerns, we're not going to just take a week or two and do it all at once. Um, I would probably couple it with when we touch code, we're going to evaluate the tests there and really do it as you go.

Speaker A: But do you think you could trust the tests as they are to deploy to production?

Speaker B: Well, I mean, you looked at the code. What did just spot checking it? Did it seem okay or do you see concerning things there?

Speaker A: Um, the bits that I did see seemed okay. Um, but it's no more than a spot check. Like with this, um, audit, we only had a week to do it and There was specific kind of. It was more about looking at the high level architecture of features. Like M. Most of actually my time was spent in data modeling. Like, is it, is it doing this data modeling in the correct way?

Speaker B: Mhm.

Speaker A: Which was really interesting because it actually was really good. I think that it, it used some really clever ways, um, for handling certain features, uh, which was really encouraging to see.

Speaker B: Yeah, yeah. I think that. Is that the way you frame the question of like, would you be okay just going to production? That's a big leap of faith. So what are some ways we could mitigate that? One would be, well, you don't just go to production right away, like deploy a staging server, get some early testers. You could do QA on it to really gain a little bit more confidence that it works the way that expected the person who built it should go through and make sure from the user perspective that everything is working the way that they expect. I think if I were to have done that and that, you know, there was a handful of beta users too who were doing it, I think at that point I would probably have the confidence that, uh, the product works. That would give me the confidence to go to production. But then I also have the experience to know that I can likely fix anything that comes up.

Speaker A: It's a challenging one because for a lot of our clients there's also like a reputational cost.

Speaker B: Mhm.

Speaker A: Which makes it more challenging. But then have we then sped up the development of things or have we shifted the bottlenecks?

Speaker B: Yeah, um, I think the jury is out still a little bit on that. I think this framing of are we taking on a bunch of technical debt to build something quickly is a. Is. Is a good one because it's a, it's a framing that we've had for, for a while too. Right. Um, in some ways this is like a trust debt as well. Like there's just stuff there that we don't maybe don't understand or know whether it's good enough yet. Um, so it's going to slow you. It potentially is going to slow you down as you go. But I also think that things are evolving pretty quickly and there could be a world in which, well, the other things are getting faster too, because say it's you or I working on this app and we're able to have agents do a significant part of the work, then that changes the equation too.

Speaker A: Yeah. So, um, I mentioned this to you kind of off air. I got to a point during this code audit where it was a bit of an existential crisis, where I realized, so there was a code base that had been written by Claude. And then a lot of the time I was using Claus to ask things about the thing that Claude had written. And it just felt like a dystopian existence where here I was sat in the middle of really, what is one AI speaking to another AI, perhaps through me. And, um, it was a strange place to be.

Speaker B: Maybe someday you will. You won't even need to be there.

Speaker A: It's a strange world. I mean, there were parts of me that thought, why wouldn't the client ask Claude then to do this audit? Because all the documentation that we were provided actually came from Claude anyway, which said, these are the things I want you to look at. Um, and I guess the difference there is, um, almost like a second pair of eyes where you've got someone who is in the weeds and it's difficult for them to look at it externally. And also it's a different set of expertise. This person happened to be an engineer, um, working in Ruby on Rails. But, um, you know, coming to us gives someone, at least if we are using Claude, to look at the code, which I think makes sense to do because it's pretty accurate and it can give us really concise information in a quick amount of time. Um, again, it's about who's using Claude and how we use our expertise to understand what to ask it, um, and where to drill down and where to kind of say, no, that's okay. But, um, it was an interesting exercise.

Speaker B: Uh, was there another takeaway? Uh,

Speaker A: my last one, My last one is that it feels like we're moving back to waterfall development.

Speaker B: Yeah.

Speaker A: Um, and that's because someone had gone away and built this product. And I don't think that's a secret. Right. I think people are upfront about that. But it ended up, it was very complex to use the UI and the UX was pretty complex. Um, and that's because a lot of stuff is being done upfront, because it's quick and easy to do. But then you're left evaluating, uh, a product which in the past might have taken six months worth of development. And so, like, you're much further down the line than you realize.

Speaker B: But it's super hard to remove features from an application once it's in the hands of users. But this is a thought that has occurred to me as well, as I've seen this, um, is that we are going back to waterfall development is a good way of putting it. And I'm not entirely sure that from a technical Perspective it has the same problems that waterfall development did. But from a product perspective it definitely does. So what I mean by that is one of the problems with waterfall development. Um, from a, from a. So for folks, waterfall is when you do all the planning and specification and everything up front and then implement everything. And the pro. One problem with waterfall development from a technical perspective or from a development perspective is that there would be times where you literally the project would fail because you couldn't do it within the time and the budget that existed to deliver that big against that specification. And that seems to not be the case. Uh, by doing the specification in advance, it's possible to generate the application. In the end you end up with this, like you said, complicated product that probably does more than is actually necessary and that can affect whether the product is even successful in the market.

Speaker A: Yeah. And I think that's like that. From a UX UI perspective it was more apparent that like, so you don't end up consulting users or spending time on metrics about what um, feature do we think actually provides value? What are people calling out for? Or what gives us a return.

Speaker B: Right.

Speaker A: And then you end up with this. You can end up with a product like you said, which is, you know, doing too much and is too complex. And so I think that's just something which is important to like look out for. Is this is, this is why I still think, you know, we should be very careful about doing things in small vertical slices. And even, even from a development perspective, giving a spec, like a big spec up front and having it go away and generate code. There's part of me when I do test driven development where I'm only looking to parse the spec for exactly what I need.

Speaker B: Mhm.

Speaker A: That I end up with less lines of code and developing less and it kind of keeps it more controlled.

Speaker B: Yep.

Speaker A: Which feels better than doing the spec, than doing the development where you put it into plan mode. This is what I want to build and this is what I want it to look like. Go ahead and do it. Sometimes the journey is important. The journey matters. Going through the process is what makes you realize that this is not a good way to build this feature or this is not actually what we need. I don't have a good example for this. We think about it like this. I've always felt like the best people to lead a team or a business. Let's bring this back to the World Cup. The best person to be the captain of a team is someone who knows what it's like to be in the Team themselves has kind of gone through the highs, the lows, the peaks, the troughs. Um, and I feel like the reason they're in the best place to lead is because they've, they've had their. There's definitely saying since they felt the grass between their toes. And there's an element of like, you cannot, uh, change or you cannot substitute being in, hands on in the code and feeling things as you go along, feeling the tensions, the things that cause you pain, the things that pull you different directions to sitting at like a higher level and just being. It's like being the conductor as opposed to being in the actual orchestra.

Speaker B: Right. And I agree with you. Uh, but I don't know that it needs to be a developer either. Like a designer who's actually using the product and feels the pain and says this doesn't quite feel right and adjust it, or a product person who's doing that can be that person as well.

Speaker A: Yeah, yeah, no, I think that's fair. And so I think that kind of wrapped up like, how I felt about the whole project. But on the whole it came out quite positive because it was okay, this is something we can work with and build on top of. Whereas at the beginning it was the question of do we throw this away? And I don't think that was the case, which is good news for people who are, uh, trying to use AI to get something off the ground. Or we will accept you with open arms and we will tell you honestly where the product is. Um, and if we can work with it, we will, um, we won't look down on something just because it's used AI. It was great to see and we're

Speaker B: doing this as well. I mean, people are coming to us founders, uh, with an idea and we help them rapidly hone that idea and then build real working application quickly by leveraging these tools.

Speaker A: That's exactly right. I use Claude, uh, daily, uh, with my work. Um, I've got like, tasks where I like to let it run and do its own thing. And there's some tasks where I feel it's better for me to be hands on and so that's what I'll do. Um, I think the most important thing, which I've spoken to a lot, uh, on the podcast, is like, quality always has to sit as the top priority. I don't really care how you get there. I just care that at the end of the day it's a code base which has a level of quality that it's easy to work with. And I mean, that's something. I don't know if this is true. I've said this before but I don't know if it's true. Um, let's see if it's true. Um, is it true that uh, if you have a startup and you want to exit at some point and sell that anyone worth their soul buying startup will do an audit of the co base and depending on the quality that could make a difference in the purchase. Is that true?

Speaker B: It's generally true but there are always exceptions to that. I'm sure that there have been sometimes major acquisitions where they're actually not considering that at all because they plan on oh well we have the same product, we're buying you and we're going to sunset you and migrate everybody over to us.

Speaker A: I see, yeah, that's a good example.

Speaker B: Um, Google was notorious or famous for replatforming everything that they buy.

Speaker A: Right.

Speaker B: Um, and so that doesn't, so the technical quality doesn't concern them because they're going to rewrite everything in Python to run at Google scale. You know, it doesn't matter what it was written in before. Um, so I think those are the exceptions though. I think most people who buy a business that has a code base that they uh, expect to continue on and to run with uh, should be doing uh, a ah, review.

Speaker A: Yeah, right. Because I think that's okay. So firstly, um, for everyone who I told that to already in the past I was right and it is a thing I assumed it was. It made sense to me logically that it would be. Um, but that's an important part of this right? Like if you are using AI to develop um, and you're planning on building something and you want that to have value, um, the quality of the code base is going to be important.

Speaker B: Well especially because I do still think quality matters to be able to, the higher quality you strive for makes it easier for agents to work on it just as well as it does makes it easier for people to work on it.

Speaker A: That's true because they will exacerbate current patterns. Um, I've actually found. Well I don't know how you did this on Ready Set Go because I found it much easier in a project which um, during my investment time I am building with out touching the code because I want to see what I can do. So I have actively said I'm not touching the code. Um, I violated that rule once today because I decided to let myself change something quickly. But in general I'm not touching the code. Um, it's interesting to see how it's got better once I've embedded like a few controllers, a few, let's say, uh, database views, like just, just got a few of the ground stuff, um, a few presenters, maybe some service objects. Like once you've got some of that stuff done, it's much easier for it to do things. I can rely on it much more. You know, copy this pattern over here because we're doing a similar thing.

Speaker B: Yeah. Yep.

Speaker A: But that must be quite a challenge on Ready Set Go.

Speaker B: M. The way that we have dealt with that so far is that we have, um, like we've baked in our guides and guidance and everything into it, but we have just. And, and the results are pretty good. But we actually have discussed the idea of having an example Rails app built in, like side of Ready Set Go that just shows common patterns and everything and using that as like, we've got an example of our standards over here before we get started. Give it a look. But we haven't. So that's an idea that sort of, if we start to, uh, have problems, maybe we would do that. Um, but we haven't needed it so far.

Speaker A: It's so interesting. I just saw that this week where someone is doing that, where they are selling their production. I think it's React native apps that other people can purchase to then be a guide for, uh, AI agents. So they're basically saying, here I've got five apps that span, um, they were like, um, image editing. So they span that niche. And so go ahead and you can purchase these, use them as the guides and then you can go and use AI. It's a wild time to be alive.

Speaker B: I think one of the things that we've talked about on the show before and that we still grapple with is, well, okay, so why are we trying to automate our jobs away? Why is that? Okay, uh, and where do we fit in that? Like, I don't have all the answers to that. Uh, but one big thing is this is always for me been about building great products in the best way possible. Um, not necessarily writing code in and of itself. There's always been a purpose to that and if I can maintain that purpose, then I think I can still be fulfilled and happy in my work.

Speaker A: Yeah, I like the way you said we are, uh, automating our jobs away. I don't think we are doing that.

Speaker B: Okay, yeah, I meant the collective.

Speaker A: No, there's no, it's not a collective. We, um, it's. It's a, it's a, it's big tech.

Speaker B: We're participating in it.

Speaker A: How would you say we are participating in it?

Speaker B: By actively using it.

Speaker A: Yeah. So we are actively using it. But we didn't. We didn't. We. Okay, let's pretend AI didn't exist. Right. Uh, how comfortable would we feel if we were to take everyone else's code and then use that code to kind of like train a machine to write similar code and then kind of sell it back to them?

Speaker B: I mean that is what has happened.

Speaker A: Yeah. Is it just me or is that like, what's the word? Uh, unfair? Strange.

Speaker B: For me it's not because one, that is what has happened. But uh, the reality is we've been trying to do that since the dawn of computers.

Speaker A: Right. I hear what you're saying.

Speaker B: Computers have always gotten better and faster. Programming languages have always gotten more and more high level and those things have made us more productive and more software, not less.

Speaker A: And we've always tried to automate things. Mhm. Yeah. It's interesting that kind of the focus on product, it makes me think what my focus is, is my focus on good products or is my focus on the craft itself or enjoying so much of what I enjoy is in the code. Um, which is interesting. Something for me to ponder.

Speaker B: This idea of craft is important because that has always been very important to me too. To thought about it in general is the quality of what we produce down to the code level. The engineering practices that we have, um, are super important to us. Um, and we've gotten a lot of fulfillment from that. We've also been, you know, created a reputation which makes our consulting business successful because of that. So it's important not to lose that along the way.

Speaker A: Mm, yeah, that, that's what is that. That part of it is really important to me especially coming into thought bottom five years ago and not having a clue what people were doing. Like what are all these abstractions and patterns and test driven development and you know, that wasn't something which I spent too much time focusing on and then learning over time the value that it brings to products and therefore businesses like working with people where you can estimate well what a change involves and understand like how to do it. Um, yeah, it was amazing. It's definitely a special part. We're coming up to the end. Um, we've had loads of emails that have come in asking about how my uh, cycling training is going. Ah, maybe you forgot about that, Chad. I'm surprised you haven't asked me. Um, but for those listening it's going really well. It's taken me time. I'VE probably done about 20 rides, which is, which is really good. So I've averaged about two or three a week. Um, and we are, uh, three weeks off from the big day. The most I've done is 60 kilometers, and that took me about 2 hours and 45 minutes, which I'm happy with that pace. And so I'm not far off the big 100. I hope to get in maybe a 75 kilometer, uh, this Sunday, maybe the next Sunday, and then I'll be good to go. But obviously I appreciate all the messages and all the donations that have been flying in.

Speaker B: Donations?

Speaker A: Oh, yeah, yeah, yeah. It's big. It's big. It's gone big. I never expected it to get this big.

Speaker B: Well, by the time we have the next one of these episodes, uh, I think it will have happened. So good luck and you can report back on the result.

Speaker A: Yeah, if there's any hook to get you in for the next one, that is it. Join us. You can find out, uh, about how my ride went.

Speaker B: Well, ah, if you're looking for someone to audit, uh, your, uh, product, your code base definitely, uh, go over to thoughtbot.com and, uh, click that hire us button. Get in touch with us. Like I said, it's a standard package that we offer. Uh, not super complicated, pretty straightforward to get started and get started quickly.

Speaker A: Yeah. And it can really help answer those critical questions that kind of block people, which is like, what do I do now? Can I go to production or do I throw it away? And very often we can find somewhere in between, which puts you on a path towards, uh, getting onto production and getting your product live. Cool. Yeah. So thank you for joining us. I don't have the conclusion notes, Chad.

Speaker B: Okay, well, show notes for this episode, which will include links to everything that we have, you can find@giantrobots FM. You can find me, uh, um, Mastodon@CPTelotBot Social.

Speaker A: You can find me on LinkedIn, um, or my own website, Sammyburnbaum.com.

Speaker B: that about does it for this week. We'll see you next time.