Stop Vibe Coding, Start Architecting! Our AI Workflows

Speaker A: Hear that? That's the sound of busy. To a restaurant, all that shouting and banging might as well be a symphony. It means the long days and longer nights are paying off. Sure, it's noisy, but there's a worse sound. This not busy. Busy means business. Which is why Toast gives restaurants the tools and tech they need to help them perform under pressure. Sounds pretty good, right? Toast built for busy.

Speaker B: AI doesn't magically make your software better. It actually accelerates whatever workflows you already have. So today, we're talking about the number one most requested topic that we get. And we brought on two special guests to help us out with this one. In particular, we're talking about AI workflows. If your workflow is clear, AI makes you faster in that regard. If your workflow is chaotic, AI is only going to emphasize and really amplify that chaos a little bit louder. The biggest shift for me around this has been not treating A.I. ah, like magic. And if you've listened to some of our past episodes around this, you'd already know that, right? So everybody talks about this agentic era. We're spinning up A.I. ah, agents in parallel, generating thousands of lines of code before we even finish our morning coffee. Or sometimes we're setting it to run, uh, while we sleep. But here is the dirty little secret that a lot of people won't admit. Throwing an AI at a broken foundation in a broken architecture doesn't make you a better developer. It just gives you very quick, very loud, terrible output. So today, we're going to kind of hone in on what we do with our workflows and our setups to kind of get you past that. So how are we going to discuss this today? Well, if this is your first time coming to the podcast, this is the programming podcast. I'm one of your co hosts. My name is Danny Thompson. I am a senior developer advocate and the founder of the Commit yout Code conference happening in Dallas, Texas, September 2nd, 3rd and 4th. Hope to see you there. And I'm also one of the authors for the developer's guide to AI. Check out DevelopersGuide AI to find out more.

Speaker C: And I'm, um, Leon Noel, managing director of Resilient Coders. And we are now at 82% placement with $109,000 starting salary coming out of our free nonprofit bootcamp. And also a community member at 100 devs.

Speaker B: Let's go.

Speaker D: Awesome. Uh, I am Chris Griffing. I'm developer advocate at Getkraken. Uh, most people probably know me from my Twitch Stream and other social Media as CM Griffing. And uh, I just I building stuff. It's really one of my favorite things to do.

Speaker E: Awesome. And I'm Jeremy Castile, so I am VP of Developer Research at gitkraken. So my team's responsible for working with software teams and developers to understand how can we unlock their flow, how can we improve the developer experience. So we spend a ton of times understanding how devs want to work best and we bring all of that information back to our product and engineering team and all of that goes right into our products. We do a ton of research. Um, so that's a little bit about me.

Speaker B: So let's get cracking into this episode. Oh yeah, it just came to me. I'm sorry, I couldn't. Intrusive thoughts. Well, I guess let's start breaking this down. What are some of the tools that you're leveraging on a regular basis within your AI workflows?

Speaker D: I mean for me my biggest thing is I'm using like spec driven development. So I've tried like a bunch of those tools like um, you know, GSD or BMed and the one I've like landed on is open spec because it's just, it gives me this like grilled phase of the explore. I create the proposal and then I can hand that off to maybe another agent that's a little cheaper to run or something. So yeah, just having that spec and being able to act on it is really powerful.

Speaker B: Mhm. What about you, Jeremy?

Speaker E: Yeah, so my core tools are going to be Claude Replit. Lovable. I do a lot of front end work. Uh, Vercel has been amazing for getting stuff up and running quickly. Um, usually at that point Chris jumps in and helps me get stuff running. Um, but yeah, I mean even on my side, just the increase of friction free workflows has been incredible, uh, with some of the tools. And the thing I love about the tools is they keep getting better. Uh, and you're not at least kind of on my side. The work that I'm doing, I don't feel like I get locked into one specific tool. I love that the tools we're using, uh, get better across the board and so you can kind of like jump in and use the right tool for the right job. Um, so that's been really awesome, Leon.

Speaker B: You know, the community's been dying for this one and we've been postponing it. What does your workflow look like?

Speaker C: Oh, uh, it's so funny because I want to, I want to like put all these caveats and things. It's like it literally changes every week. There's something new that I realize that I have to incorporate or pull out. And kind of like what Jeremy's saying, they're just getting better. So even the tools that I'm using, they're pulling new things in to really simplify before we go. Deep cursor still kind of my harness of choice. Within cursor, I'm using so many different features, tools, models, you name it. I'm kind of deep on local stuff right now too, that we can talk about a little bit later. Um, but for me, I also am in this point where it's not just about the code anymore. And I'm sure this is what we're going to talk a lot about. It's how do I manage everything, Especially when I have teams of 20 plus engineers versus me. By myself, things are a little bit different, but I am finding more and more. I need layers to see what's going on across the code base. I need layers to make sure that I'm doing things responsibly. I need all these different pieces. So cursor is kind of my main harness. I'm using a lot of different tools within that. I'm using things like, uh, bug bot on each kind of commit. I am using sync for security audits of all the code I am using, uh, one tool I can't talk about yet. Then, uh, I think that's kind of the core foundation. And vercel. Right. And convex. That kind of tooling enables me to spin up very, very fast to interact with my auth and databases in ways I never thought I could actually do. And so that kind of stack plus the tooling, plus a few other sprinkles here and there that we'll talk about I think is kind of my day to day and it gets me the most work done. But I got some problems and we need to talk about those problems.

Speaker B: All right, let me jump into mine. My approach is a little bit different and uh, if you're in my discord, you already probably know this. We did like a whole workshop on this one in particular. But I am in this area where I'm not the biggest fan of spec driven development and I'm not the biggest fan of like prompt driven development or just prompting. Just in general. I do something that's somewhat in the middle of a hybrid, but essentially my entire career has revolved around one subject and that's scaling. So I'm constantly thinking about what one to many situations look like. Like when I build something, I'M often thinking that a thousand people, ten thousand people, one hundred thousand people, a million people will interact or a million transactions or a million interactions. I don't trust spec driven development at that level to guide me there. So for example I did this. Um, I'm very big in voice AI. That's why the microphone is always here. I'm constantly using voice and one thing in particular is having strong, strong architecture on things. My belief is you can't out prompt a bad architecture. It just doesn't work that way. And so if I spend a bulk of my time building the architect that I need when I then give it context engineering solutions like I'm um, big on context docs or just documentation in general that can guide a lot of this. It can run miles with what I need because it's there. So for an example, I did this enterprise level demo of using voice to control my agents. And uh, I was using several tools in the mix and with that it was a 14 step architecture and only two of the steps involved AI. 12 of the steps involved me building strict guardrails and just strong, strong, strong scalable architecture that can work at length. So I'm still the person that's not 100% codes off. I'm still maybe 60% of the time I'm you know, prompting or something and 40% I'm in the code. But I'm constantly in an IDE, which drives me nuts by the way that some of these IDEs now have or uh, harnesses have moved into like we don't have a way for you to even check the diffs anymore. In here you have to use open Vs, uh code or something. Drives me wild. But I use voice AI. So I like super whisper. Whisper flow is cool too. I think both of them are great. I use them for that. I have macros that I set up around that so that way I can like have in like little custom snippets. I will use um, something like uh, depending on what it is. And the other thing is I'm big on evals or LLMs, judging LLMs. So that way I check the audit. But instead of it just being an LLM versus lm, I, if you remember last year I said we're at the stage where the infrastructure around the prompts are what's manipulating it. If I can give each LLM, um a prompt, I'll get a different response, but maybe somewhere in the same realm. But if I give that same LLM the same prompts in different IDEs or harnesses, I will still get different results because the infrastructure around is what's manipulating the output. I think everyone started realizing that about a year ago. That's why these harnesses have kind of gotten to the area where they are now. I will honestly, uh, use different harnesses to create different examples or different responses to my thing so I can judge where one is. And then I will take the results of said responses and have them vetted, cross referenced to other models. So now I have that actually streamlined instead of me having to manually do it. Previously, I built a little copy and paste tool that kind of puts it together and has it audited for me, so that way I can then view the results together. And now I'm in this area where I used to have it create HTML pages to kind of see the visuals of the outputs. Now I use MDX for that. So I have a whole streamlined flow for that. And that's honestly how, like, I'm auditing quite a bit of what's happening now. I am very big on testing, but of course, one thing that we'll cover in this episode in particular is my number one tool that if you spent 10 million tokens on something, how to make sure you save it. If y' all are longtime listeners, you're probably familiar with it, but we'll definitely go over this one for sure. I know I went through a lot. I don't know if it was a hot take or not, so I apologize if it was. But I think that kind of, like, puts us in an area to kind of, like, talk further about some of the things that we're doing.

Speaker C: You drop so many other, like, little accessory things too, that I like. I don't even think about them anymore. They're just part of my process. I use whisper flow. I'm using whisper flow right now. Right. Like, it's just so many of these little things are just like our processes fundamentally change so much, and all these little things that we're kind of pulling together. And, um, yeah, there's just a lot of different ways that we can kind of kick off the conversation. But you said something at the very beginning of the podcast that I really kind of want to just take a second to pause on as we kind of talk maybe a little bit more about our workflows. But you mentioned this idea of whatever your system is, the AI amplifies. So if you have a poor workflow, your poor workflow is getting optimized. If you have a great workflow, your great workflow is getting optimized. And I love reading all this stuff that comes out of Google's Dora, they have this kind of research initiative where they kind of look at what's happening to developers and what's happening. And they came up with like, that same thesis that, like, good engineering systems become faster and poor ones become overwhelmed. Right. And so as you're kind of thinking about, as we're thinking about our workflows, have you felt, I'm curious, because we all have slightly, slightly different. Have you felt like your productivity is increasing? Are you getting overwhelmed? Where are you at? Because I flew too close to the sun. I burnt out. I was juggling too much, managing too much. So I'm curious if you're, like, happy with your workflows, do you feel like they're making you 10x more productive? Or, uh, where are you at with that, that kind of thought process?

Speaker D: I mean, for me specifically, it seems like there are things I can try that I wouldn't have tried before because the effort to get involved and get started would be too much, like right now for one of my projects where I care about the code. So I' still kind of like manually coding a bunch of it. I actually turned a model loose on creating some of these, like, SVG illustrations and stuff that I'm really bad at.

Speaker C: Right.

Speaker D: And I would have paid someone else to do at this point, but instead AI can at least like, help me realize the idea and then maybe I actually recognize what's like, good and I can have an illustrator make it actually great, you know?

Speaker C: Yeah.

Speaker B: Uh, for me, I could definitely say in some regards, I'm much faster. I don't think I've ever hit the level where it's like 10 productivity, but I'll even tease this. And it's, uh, since last week I've been building a new application that I absolutely love. And everyone I show it to, um, geeks out and it's like, has nothing to do with tech, which is awesome because, like, finally a breath of fresh air for a little bit. But what I'm building is a Spanish language learning application with the idea that instead of it teaching you the basics like we're all used to, you're actually trying to survive a group chat. And it's basically friends roasting friends. And you're jumping into it and you're like, oh, my God, like, they're making fun of me because I'm like, speaking too formal because, like, maybe I listen to other applications, uh, where they just hit you with vocabulary after vocabulary. Or like, one of them is like a noisy, uh, bar situation. So, like, you've learned a lot of phrases and words, but now you're in a noisy environment. Can you actually pick up what it is that they're saying, even though it's something that you've already learned? Uh, the other one is I have a passive aggressive food truck owner who judges everything that you order, and so, like, funny scenarios like that. And in a week, I feel like I've gotten so much done with this application that it'd probably take me an entire team to do, which blows my mind, number one. But number two, the bigger thing about this is I literally have at all times. I had to close it just to do this podcast episode with y' all three harnesses open that are all working in tandem, that are judging things, that are judging the outputs, that are judging the curriculum. I ended up finding out through a ton of research that there's actually an international standard for learning Spanish called Sephir. And now I'm judging all the curriculum that I wrote against it. And the LLMs are helping me find where those happy paradigms are. And the best part about it is when I finally go to a person to say, hey, can you review this curriculum? Because you're a native speaker and you actually know what you're talking about, it's actually worth their time to sit down and go through everything instead of it being like 15 times back and forth. Just to get at a level where now, like, we're on a similar standpoint. That has been, like, just. I keep saying it over and over. That's been amazing to me. But you know what else was amazing? I'm in Dallas, Texas, and obviously this is World cup time, depending when you listen to this episode. And it's wild over here. We got people from all over the world over here. And I went down for one of the watch parties right next to the stadium. The vibe was wild, right? I was driving back home. My car full self driving is driving me. I literally have my Bluetooth connected to my phone. And for this application, again, this is my thing, right? I'm literally giving my agents voice commands without having to see anything. And it's doing everything. And then when I get home, I'm finally able to see the entire set of things that are being done to me. I literally said. I had a pause and say, I'm literally in the future, as wild as this is. And it's blowing my mind that I'm able to get this stuff done now. It's funny because I tweeted this out, uh, and one of the people Said, if this is the future, this sounds like a nightmare to me. And I'm like, I could totally see what that looks like, like, if your future is me being productive all the time. But I was like, this is me doing stuff for me. Like, how much am I able to offload off my shoulders now and make myself either more attractive or building the business that I need to, or whatever it may be on my own free time to get this stuff done. Now imagine if I were to use this in like a work setting, the same kind of scenarios. I could still get so much done, be more productive, whatever it is that I need to do. But like, uh, it's unlocking so much for me, but it goes back to the same thing. The only reason why I'm able to run like this is because I've set so much in motion that guarantees my outputs are going to be that way. It's not fun to build that part as it is, to just say, hey, one shot. This one thing, which I don't even say anyway. But, uh, you know how social media talks about it. And so it goes back to, like, if you don't have great systems in place, you're always going to feel like you don't see what everybody else is talking about on social media. And that's what I've started to realize more and more of. There's a subset of people that talk about, I'm going on way too long. There's subsection of people that talk about, like, oh my God, AI is amazing. And there's a portion of people, like, I don't see what they're seeing. And I often believe, like, there's some truth in both circles. And I think the people that don't see it, I think they're absolutely correct, they don't see it. And I think partially it's because either A, expectation is too high or too low, or B, they just have terrible systems around it. So when they finally get the response back, they're like, what is this? And I agree with them because I've been on calls with them and I'm seeing it. And then the other side, where like, AI is amazing part of it, I do agree with them. Whatever they did was probably amazing to them. There's also a subset of the group where it's like, they know nothing technical. And so the fact that they see something impressive, anything could have came back and they would have been like, oh my God, this is the future. So there's like a happy medium between both.

Speaker E: Yeah, that's. That's super interesting. I will say it's like going back to the, you know, um, like AI amplifies everything piece. We have absolutely seen that on our side. So we do a lot of research. Some of our research started to get into code quality, right? So now AI can generate tremendous amounts of code. Right? So it's no longer generating code, is no longer the bottleneck. What we saw in some of our research was, uh, sort of the difference between teams that have some of those guardrails that you're talking about and the context, uh, to be able to feed these agents versus teams that are a little bit more. They don't have the guardrails set up quite right or yet. One of the most striking things that came back was what we kind of called was code churn. And this is code that once it gets merged, has to, you have to revisit it within some short amount of time. Let's just say like you merge it, two weeks later you're revisiting that code. You're going to have to rework it. So we saw, you know, so, um, a tremendous increase in terms of the volume of code being generated using AI, you know, coding tools and harnesses. Um, but if you don't have the right guardrails in place, it can turn into a code quality nightmare where it's just like now we're stuck in fixing a bunch of this AI generated code. Um, and so that piece was interesting. And then another piece on the kind of like productivity part. So one of our most recent, uh, pieces of research that we did, we asked developers, do you feel more productive? Right? So just like qualitative, like how do you feel about it? Not necessarily any other quantitative way. And what we saw was 84% of developers feel more productive, which is a great result because if you're using these tools, you want to feel more productive. And, and Danny, to your point about like, hey, I'm driving home, self driving, I can do voice, you know, to these agents. And by the time I get home it's, they're, they're building stuff and I can see it and I think so that feeling of, you know, feeling more productive is absolutely real. We're seeing that in the data. We. And 43% say they feel much more productive. Um, so just some really, really interesting um, research that we've seen.

Speaker B: Can I dig into that slightly of the people that you're seeing that feel more productive, are there certain things that you're noticing that when they approach xyz they get this feeling or what is Giving that.

Speaker E: Yeah. So there are a number of different factors playing into this, a couple that we looked at specifically because what we were trying to understand now that we're moving into sort of this uh, you know, multi agent world where you're running multiple agents at once, it's no longer just like autocomplete. Um, what we found one sort of slice of the data, you know, to kind of get to your. The question you're answering, you're asking was teams, uh, that are um, just starting with agents versus teams, uh, that. So we called it assistant. We broke it into three cohorts. The first one is what we called assistive. So this would be more like ChatGPT, kind of like code complete. Emerging would be like maybe running an agent or letting uh, it go, you know, letting the sessions go a little bit longer. So the depth of session and then to what the third cohort is what we call agent native. And so this is where you are running multiple agents and maybe you are giving them voice commands and maybe they are running for you know, minutes, hours, you know, overnight type of, type of setup. And so what we saw was the uh, there was a clear correlation with how productive you feel to how agentic you are becoming, which is really cool. So like the assistive, you know, they felt the least, you know, bump in productivity, uh, emerging was in the middle. And then agent assistive was the highest in terms of uh, you know, devs that are, you know, uh, agent native, sorry. Um, feeling the most productive. So pretty cool, like cut of the data.

Speaker D: Just having data to back that up is really powerful.

Speaker C: I think it's really cool that y' all are doing the study, uh, to like look through all that stuff and talking to so many developers, it's cool to have that stuff coming in. I feel like I've gone, I feel like I've gone like this, I don't know, like upside down to where. When I was first doing a lot of agency stuff, I, I like, I felt super productive. Like more stuff is happening. I am getting more things done. I am um, my output is higher than it's ever been in my, my career and that's great. Solo projects. I'm the most productive I've ever been in my life. Like I feel like I can crush anything that I want to get done. I pay for assess. No more like when I'm going to create it, it's going to be done. I'm like, I'm going to figure it out now with teams. And this is new. This is raw feelings on the podcast. Don't flame me on Twitter, please. I'm starting to feel the amount of code being generated, the amount of issues that are arising, and I am spending more time reviewing and talking about things that have been made. Uh, Jeremy, you kind of talked about, like, the AI creating more. Like they're creating more files, they're touching more pieces, and I'm spending more time doing that. The fun, like agencic, like we're creating everything is not there for me at the moment. I am middle manager extreme, and I feel like I'm doing a lot of review. And so am I productive? Yes. Is my output staying where I want it to be? I'm not sure at this point. That's why I want to talk about it. And so that's kind of why I like what y' all are working on at Getkraken, because I think it's solving this very real problem that I'm starting to get into. And I'm really curious, kind of your thoughts on that idea of, yes, we're being productive, yes, we're getting more stuff done, but there's this new layer of problem that I think I'm not alone with, and a lot of other people are kind of starting to see come to fruition as well. And how are we going to tackle that long term when we have have four different agents going while Danny's driving their Tesla home? You know, four?

Speaker B: Well, who you think I am, rookie numbers man? Like, I even want to add one extra layer to this. You know, uh, Simon Wilson, who's pretty big in, like, the AI space, he wrote a blog article not too long ago that I read, and I found it really interesting. Or he. It was a blog article. He said something somewhere, and he said something to the effect of, like, um, you know, I'd like to imagine he's like, really, really well versed in the whole AI space. And he's like, by 11am My brain is fried. There's so much going on, so much context switching, so much controlling of agents, that I'm like, I'm finding it mentally taxing to even try and be productive because of all this. So, like, with that being there, and I know, like, we've had, like, some conversations about what it is that y' all are doing, uh, at Gleek Hurricane. So, like, I'm curious as, like, how do you see your solutions potentially working in this? Like, is this. Would the Simon Wilson's of the world find it be like, finally, I can get some breathing room? Or is this more along the lines of like, hey, look, if you want to be done by 11am, you're going to be done by 11am Yeah, I

Speaker E: mean, I think like, you know, Leon mentioned some uh, of it, but, um, you know, in terms of like the bottleneck has shifted, right? So, you know, years ago, uh, maybe five years ago, could, uh, be less, certainly, um, more. A lot of time the bottleneck was the code itself. How quickly can we generate code that is just completely evaporated? That is no longer the case whatsoever. We have absolutely shifted the bottleneck depending on your workflows. Now it's where it may land. Sometimes it is review, uh, because let's say you have um, a team of 10 and maybe each developer on your team is running multiple agents. That quickly multiplies the amount of code that's coming in. So you may have shifted the bottleneck now to merge, uh, or it could be uh, in poor request review. So the bottleneck has absolutely shifted. It's no longer code. Um, historically, gitkraken started for folks that may be familiar with gitkraken. Our core product was our git client, gitkraken Desktop. And that is still true today. Um, but we launched that 10 years ago and that was to solve, we've always been trying to solve friction in the developer experience. That was why we built Gitkraken Desktop first place 10 years ago. But back then it was, we've got a bunch of uh, developers on the team all working within Git and we might be running into these merge conflicts. Um, how do we actually visualize the commits across our repositories? How do we untangle these merge, uh, conflicts that we're running into? And that's really our internal engineering team. That was the friction that they were running into as a team. And that was really kind of the origin story of Gitkraken desktop. That is where Gitkraken came from. Gitkraken the tool. Now you fast forward 10 years and we're almost kind uh, of evolving again into kind of what we're calling the code flow company. But just like there was this need to uh, improve developer experience 10 years ago with version control now. Uh, you know, Danny, to what you were describing of sort of like, I feel overwhelmed by 11am with everything that's going on. That is exactly the friction that we're wanting to solve with uh, some of the latest capabilities that we're building and kind of this re evolution of gitkraken into what we're calling the Code flow company. So really it shows up in a Few different ways. Chris can absolutely talk to this as well and I'll just hit on it quickly. One is so gitkraken Desktop, our core product, that's a great place to start. So this is the git client where you can use this for uh, version control. And what we've introduced are all these AI capabilities to help you speed up things like um, writing, commit messages, writing PR descriptions. So you can do all of that with AI today in gitkraken Desktop. And with our latest release it now gives you the ability to manage uh, an agent that's working within your repository. So you can do that today in gitkraken Desktop. What we just launched is what we're calling Kepler. So this is a brand new product, completely separate and this is built uh, for AI native workflow development. So it is specifically built to run multiple agents. And to your point about this person that's getting overwhelmed at 11am, I think a lot of that has to do with tab switching, agent switching. Now you've got six agents running on 12 different repositories and trying to understand what's the status of each one. Do I need to restart? Did it crash? And you know, there's only so much context we can keep and if you're having to jump from tool to tool model to model, it's just a lot. I mean getting to 11am on some days is an achievement, you know. And so that's where something like our latest tool, gitkraken Kepler can really, really be super powerful to improve the developer experience when you're running multiple agents. It is absolutely, you know, what we're calling an agentic development environment. So ADE instead of an IDE where you can literally go in, plug in the different models and tools that you're using and get statuses and everything right there in one view. So you can kick off work with one agent, get it working in one area and it creates a completely isolated get work tree environment for that uh, agent, let it run and then you can move on to the next one. And so you could have 6, 10, 12 running. And to be able to have all of that in one single pane of glass is really just about reducing the friction. And so we see that as really the future. The developer will always be at the center of software development. That's what we believe is true and what we've seen. Um, and so this is kind of our latest tool to help with that.

Speaker B: Chris, do you want to elaborate on that? Before, like I got so many follow ups that just came on my mind. But like, ah, I'd love to give you. Maybe you're going to answer.

Speaker C: I feel seen. So just.

Speaker D: I def. I definitely have like, I think maybe like ah, a bit of a parallel to what he's saying. So like as an engineer, even before AI, As I leveled up and became more senior, I spent way more of my time doing code review. You right? And now with all of us running, all these agents, everyone, every individual contributor is now their own little like m manager of their own little team. And how do you see what that team's doing? How do you make sure you're not leaving one of these things like hanging in a tab somewhere in your terminal, right? Like instead you can see all of it in like one workflow, one like pane of like all of these things seeing like your team working side by side, you know? You know, team. Right. So yeah, like that's what Kepler is really trying to do. Like bring all of these things back into one surface so you have actual visibility and you're not like dropping a PR and letting it sit for two weeks. Kind of like that same parallel, you know?

Speaker B: Yeah, you know this. My brain's going in like a hundred different directions. Like, I guess number one is like, do you see the average software developer using Kepler? Or is this like you work in Fortune 500 and this is the tool that you're pulling in Now?

Speaker D: Anyone who's managing multiple agents really, like just being able to make sure that they're all not stalled on one little like, oh, hey, can I use this tool? Oh, hey, you know, uh, is it. Is this the right approach? Like, let me quiz you kind of stuff like just being able to make sure that you're not like letting it, you know, sit stale and not be productive.

Speaker B: Right?

Speaker E: Uh, yeah. And I would say like, yeah, go ahead, Leon, please.

Speaker C: Sorry, I was gonna say multiple agents. But also like, as we're thinking about how we evolve to do this more efficiently using the work trees and being able to see the different work trees and be like, I'll be honest, I was a command, uh, line purist for a very long time. That's gone. I need to be able to see this. It's too much now. I can't keep that mental model in my head. And so being able to see the work trees, the different agents come together, that just makes sense. It needs to be more visual now. We need to be able to see it. And so I just wanted to add that layer there. The way that we interact with all this is changing. And the Best practices are changing too and it's just creating a higher volume of, of disjointed things that come back together. But you still need to be able to see those different things, uh, before they come back together. And so like the, the ADE makes a lot of sense to me.

Speaker E: Yeah, absolutely. And I was just going to throw in too, you know, to, to your point, like um, or, or Danny, when you mentioned like, hey, do you have to be in an enterprise using this? So part of our product strategy again, going back to being, you know, developer first, uh, you know, for gitkraken is we wanted to be able to have tools that meet every developer where they are. We know not every developer is, you know, at the same, you know, step in the journey of agentic development. So, you know, getting started with gitkraken desktop, you can manage agents within gitkraken desktop, but it also gives you that really like git native feel. And so if that's more comfortable to start there. Absolutely, you can do that. Um, and Kepler is really the next step. You know, as Chris mentioned, when you're running multiple, multiple agents and you need to see the status and what they're changing side by side, all in one view, when you've got many agents running, that's where Kepler comes in. And so it really creates a nice uh, step, you know, as you move up that, that AI journey.

Speaker B: How do I get those updates? Like, uh, I'm um, generally always, especially like when I'm in work mode, I'm near my computer. But I will say more and more as time has kind of evolved, I'm leveraging my phone a lot more, um, especially because of voice. Right. So with Kepler, like if I'm orchestrating all these agents and tools and this and that, like you're describing, and I'll be transparent, I haven't seen it. So I don't know. But like, what is the imagination here that like I'm getting updates only on my actual machine, or am I getting updates wherever it is that I see fit? Is that something that's in the work, like the pipeline? Like, talk to me about that.

Speaker E: Yeah, it's both. And Chris, please jump in here as well. But so like um, for Kepler, the product, the tool itself. So imagine having every task visible that the agent is working almost in like a Kanban view. So when you go to Kepler, you can see every agent, what it's working on and you have a view of all of those, almost like a Kanban view. So you can see exactly what's Going on. You can filter, obviously you're going to get to the point to where you could have 10 agents running 100 different tasks. So the ability to quickly sort and filter on what's going on here, that is what we've created. And you've got all of that in one pane of glass and you can quickly sort and filter to the views that are most important to you. That is what you know, really is kind of the core of Kepler. Now to the point of like, well, sometimes I want to consume this the way that I want to in my workflow as a developer. Maybe it is through the cli. Um, we absolutely have an MCP with many different apps that plug directly into gitkraken into Kepler. So you can pull all of that directly into your workflow so you don't have to break whatever flow you're in in and still get those that really rich agent data exactly to where you're working.

Speaker D: So I have one thing to add that's completely separate from like the MCP in that. But we are, we're working on it. It's very early but a way to ah, like ssh into the session that is running in Kepler. Right. So if you have a terminal app on your phone, sshing into and getting that like status and all that stuff is really powerful.

Speaker B: Right.

Speaker C: I want gitkraken.de uh V but for Kepler.

Speaker D: Yep, exactly.

Speaker B: Now that, that's interesting actually to me.

Speaker D: Um, now we'll say real quick there's a lot of like work to be done to make this like fully like enterprise ready. Right. Like something like a tailscale or a private network that's like fully vetted through your like entire like infrastructure. That's to be determined still. But I think that's a really important step.

Speaker B: Yeah, that's the interesting part to me. Like, you know, we're at this weird stage of AI, uh, where like if you are in the bubble of social media, whatnot, it seems like everybody and the mother is making like the AI startup of the world right now. Right. But like I was just in a conversation not even 24 hours ago and they're like, yeah, our entire team's AI spend is $2,000. I was like oh, like per member. They're like, no, no, in total. I was like what are you even talking about? Like two grand. Like you're not even using it at that point when you have uh, 150 plus members. Like nobody's using anything. And so it's like how does this translate to them? Is this going to be something that's for them or is this something like further down the road you'll adopt this then?

Speaker D: I think that's a little tougher, right? Like, I think it's really for everyone, but it's tough to make that sell. I think the biggest sell for like an org like that, for Kepler, for us would actually be that like, we're focusing on using like, issues and some of those things as like a great starting point point. And we don't really see a lot of like other ADE style tools doing that. Um, and you can still start from scratch if you want, but being able to take an issue and provide that context immediately is really powerful.

Speaker C: Yeah.

Speaker D: And I think a lot of orgs would benefit from that eventually. Just like, you know, making sure that they are using the tools they're already paying for, like Jira, etc. And like actually leveraging those into the tools that they want to start using as well.

Speaker C: I'm curious just from like a, uh, like a programming standpoint, how was putting together all those integrations? Like all the tools are changing, they're updating. Like, what's that look like internally on the team? I'm just curious for nerd's sake.

Speaker D: So I mean, we've kind of already been doing that with GitKraken Desktop, right? Like, we've been supporting GitLab, GitHub, uh, BitBucket, Azure, DevOps, so we have a lot of that infrastructure already. So that like really primes us to be like the one that does that well with an ADE as well.

Speaker C: Well, yeah, but I guess more like all the harnesses, right? Like, like communicating with and, and pulling those all in. And do they, do they want to play nice? Like, I, I'm curious, like, like that piece too.

Speaker D: You know, most of them support the acp, the agent client protocol, and that communication layer is like really helpful. Um, Claude code made that a little tougher for a while, but it seems like they're walking back that decision, which is cool. Um, but yeah, like the ACP is really like the thing that drives the power behind like orchestrating all this stuff from like a desktop gui.

Speaker E: That's dope.

Speaker B: You know, at the beginning of this podcast episode, I kind of alluded to one thing in particular, that I was going to share something and this is like one of my biggest like, tips that I try to give people. If you've attended like the AI, ah, masterclasses that we do, this is like a big one that I always go over and that's let's say you had a very successful session with your LLM tool, AI harness, whatever you use to get something done, especially coding related, I wouldn't recommend this as much for non coding related. There's like a deviation that I would use. But especially if like you're a software developer and you did this to get like a coding output, let's say you burned like 20 million tokens, 10 million tokens doing it. It's a lot of tokens, right? Well generally the way most people look at it is like oh, uh, I built this thing. It ah, went well. I got the results, I got a good component, I got a good page, whatever it is that I built.

Speaker C: Cool.

Speaker B: I look at it as like that's the lowest thing that you can get out of it. It. If I spend 20 million tokens, I don't want it to be for a one off scenario. And so what I always go back to is after a very successful session I then go to my tool, my LLM, whatever it is that I'm leveraging, my slm, local models, whatever and say hey, make me a context document of everything that we learned, all the successful interactions and things like that. So now if I'm using let's say uh, ade, ide, uh, a harness, whatever you want to call it, it, um, I can then start a new conversation and have my jumping point be those 20 million tokens that I already spent. But now I get to reuse it again and again and again. But as it reads that context doc, instead of spending 20 million tokens to get there again, human readability is not the priority. LLM readability is. It is now taking me 500 1000, 2000 tokens to get there. It's a fraction of what I've spent. But now I get to use it again and again and again. And the best part about it is it's comprising that 20 mil into all the lessons that we use to get to that end result. And the best part is now every single time that I have, let's say I built, I don't know, a newsletter component on my website. Well, anytime I interact with that newsletter component, whether it's the APIs that I'm interacting with, the information, whatever it may be, anything that comprises of it, I can just reference that context doc and it gives me the starting point that I need. And then when I'm done with that next interaction, hey, let's update that context doc with all that stuff that we just got out of it. Now I have this running source of information that gives me everything that I need for my LLM or harness within that newsletter feature. So now it knows all my APIs, it knows the interaction, it knows the service message, it knows what the tables, it knows everything. And so I can go back to it over and over again and just save all that token. And so like to me, game changer. Like if you go into some of um, my bigger projects like Event Convo for example is like 500 ish. Now context docs each naming conventions matter so much on the spot by the way. So please don't, don't do the dev thing like X, Y. You're going to have a nightmare trying to find out which one is which. Name it. Well, use Java naming conventions if you need to name it something really long that you know what it's doing right away. But when you have that, you're going back to that. So now like if I'm thinking about like how do I measure my outputs? Or like how do I even know like for example, token maxing being a thing, which is, you know, when you put a leaderboard up in your company, that's going to happen, right? People are going to game it one way or shape or form. And so now if I start thinking about like what is the best way to make sure if I'm spending tokens, I'm maximizing it. This is one of the key things that I always go back to. And the thing that I always go back to with this is this. If I am looking at the actual output for token ratio, this is going to be the way that I guarantee I'm getting the most output every single time. Time. If I'm, if I'm like the type of shopper, like if I go grocery shopping, I'm trying to stretch a dollar. This is how I'm trying to stretch a token or two tokens or a thousand tokens. This is the way that I maximize it in that regard.

Speaker D: I'm curious how you deal with context rot in that situation, right? Because like as the context fills up it seems like it waits. Like the beginning and the end and the stuff in the middle kind of disappears, right?

Speaker B: Yeah. So like if I'm doing something with my tool, uh, instead of it deviant, that's why I'm always like one window should have a task. I almost treat it like a uh, the, it's the thesis of a microservice. One thing should do one job really, really well and not deviate. If I'm in a contact or a chat window, I should have one Focus on this, and it should do it really, really well. And then I jump ship. I should not be in there like, oh, we finished this. Now let me check out this. We finished working on the newsletter. Let me now check out the footer of this website. Why would I go there? Or, hey, we finished. We fixed this newsletter component. Fantastic. Now I want to talk about the sponsorship page. Why are we even talking about the sponsorship? Why are we talking about our clients page? Why are we talking about our signup forms? Like, where does that. It doesn't correlate. And the problem is a lot of people will go into these chat windows and create these really, really, really, really, really long, um, conversation sessions that deviate into the 15 different areas to where now all the outputs suck. But the problem is I've had so much tunnel vision for 30 minutes, an hour, two hours, that I'm just frustrated. And I'm like, this thing sucks because of course, it's going this. You're minimizing your experience every single time. And the other thing, too, is with a lot of these tools now, especially, like, harnesses, they have, like, these, uh, context, um, uh, uh, my brain is going to the Java compaction. Thank you. I was going for garbage collection, but compaction, it goes and it starts, uh, getting rid of the earlier parts of the conversation. The part where you are the most focused to lay everything out is the thing that's starting to get reduced and it's saving the nuances. But if we were to have 50 conversations, and of those 50 conversations, you remember little pieces of each one, you're it. You'll. You'll remember how you felt in the conversation. You remember what we talked about. Maybe this, the snippets. But, like, the best movie that I ever saw, I'm only gonna immediately think of the highlight reel. I'm never gonna think of, like, the intricacies of the dialogue. This is the same thing in this regard. Right. So, like, when I'm letting all this context just get overwhelmed, it'll never be great. So, like, if I'm focused on this and I create my context, stop, and I jump into another conversation, it's the best way to avoid that scenario.

Speaker D: Yep, I agree. It's almost like, um, like micro agents at that point. Like, instead of microservices, right?

Speaker B: Yep. That's literally what I call it. But, like, I know sometimes online there's like, uh, there's this dichotomy or this argument of, like, microservices versus monore, like, all these other things, like microservices were never a great thing. And like, we shouldn't do this. And there's the other side. Like, microservices are great and like, it's like, it's so fast. Um, I'm, I'm on both sides of it where I think the right solution will work no matter what it is that you give it, as long as it's the right one. Um, but I am a microservice fan for a lot of things that we did. Especially like, uh, two companies ago, we rebuilt this entire architecture in golang, and the primary focus was having microservices that were just performant and you could see the site speeds night and day difference between version one versus version two. So, uh, I stand by that work, but I don't necessarily think a microservice is the solution for everything. And just because, uh, someone high up learned that word or was like, all engineering teams now do this. It's not the right way to do it either.

Speaker D: I think one thing, like, you know, I'll let others people talk in a second. But yeah, like, for microservices, agents are actually really good at building those because it has a very clear input and output. It's like a pure function almost. You know, so you can just turn it loose and actually, like, vibe code your microservices and have like, actually good results out of it.

Speaker B: Yeah.

Speaker C: This is going to be a silly question, but I want to, to, I want to know, so I'm going to ask it. What does your git workflow look like when you're doing these, like, smaller focus sessions? Are you having friction with the. Not even, like, pushing, like, like, what is. I don't put anything on top. But what does your workflow look like in that regard? Yeah.

Speaker B: Are you asking me?

Speaker C: Yeah, yeah, yeah. Just like, when you're. Because you're saying you're doing like, these small, like, focused areas like, like, what is, what is your, like, getting, like, getting those, like, when you feel like it's done its job? How do you get that to mean? Or, like, are you even thinking in that way? You know?

Speaker B: No. Yeah, I am. So, uh, there's, there's two sides of this. When I'm very disciplined, it's a ton of pushes. Like, I'm, like, I'm, I'm pushing up as frequently as possible. To have. Once I know something is a source of truth and it's like, what I want, I want it up there immediately because. Because as soon as the next agent starts working, there's gonna be some deviations. And like I don't want something that I know was correct to be deviated from. But when I have multiple agents working on like the same project, if I don't keep that source of truth as accurate as possible, what ends up happening then is like, number one, diffs. Like you're gonna get things that change that you didn't intend to be changed, or like you're not even gonna notice the change. Cause there's so much that's being worked on that like you, you maybe you checked A, B and C earlier, but now the agents as they started working on it and you didn't pushed up quick I enough D, E and F, ended up changing something in A. But because you've already audited A and it's so far down the line you're not even rechecking that. So now when it's finally in main, you're like, what the heck happened? I think that's number one. Number two, if I know that they're working on so many different parts of the code base, I may just let them run. Like, I just had uh, something pushed up not too long ago that had like 80 files changed, which is like wild, right? But there was like a lot of it was. I'm big on refactoring. I'm constantly refactoring, updating. And the best part is like, once I have my sources of truth or like the things that I want to happen or occur, having docs that I can re reference to say this is what we got out of this. And then having it go remediate that, so much easier to do. Um, so sometimes I have these big, big pushes because I have so many agents working on it that I'm like, instead of creating multiple diffs, let me just create one and just do this massive audit of over every little intricacy. Those code reviews take way longer for me, unfortunately. But I. But it's like if it's 80 files, of course it's going to take longer. But really I want to go back to, um, uh. And this is one thing that I often do is I ask my LLM why these certain changes were made. And the idea is, number one, for it to defend it, explain it, go through it. Number two, I can pose questions then to be like, hey, I see this. What if we did XYZ instead? Why did you choose abc? Number one, it'll either defend ABC or to say XYZ was the better choice. But then it's like, well, if you think it's the better choice, not defend, why you think that is. And so then it starts arguing with itself, debating on back and forth which one was the better process. And we end up finding maybe there's a different one altogether. But like I want that defense to be there. And the other thing through is like I kind of mentioned this earlier, I haven't make MDX docs to show me what it is that was changed visually so we can walk through it together. And so because of those different areas, my testing strategy is far more performant now because number one we have the defenses there. We have LLMs that are able to audit this. We have our, obviously our unit test NN testing that is working strategically. But now we have this MDX docs where like visually I can even see what's happening. So now I can like really dive in deeper.

Speaker D: It's almost like the evolution of like storybook, right?

Speaker C: Mhm. Yeah.

Speaker D: Ah.

Speaker C: Having those, those context files makes it so that the weird stuff like the touching more files, making larger changes that over time hopefully gets minified. Um, as it's learning from the conversation you had with it about like why it did the things that it did did.

Speaker B: Yeah, yeah. There's even like an extra step here where I'm very big on having structured files. So like I have a code standard files which showcases my LLM M how I want them to actually code things out. That's, that's that right there saved me time and time again. There's another one where I have like a UI file where it shows like these are like the common design library, uh, themes, colors, schemes, et cetera. So like anything that we make going on in the future, like a new component, a new button, whatever, it must abide by these. And by having it in an MD file it can easily re reference. Instead of me just saying here's the higher architecture of my design system, like I need quick references. So that way it adopts it very quickly. Uh, the other thing is I have a uh, table of contents file which basically it knows the task that it has and it references it first to see where it would be within a skills file or something. So it immediately jumps to that area and it doesn't deviate from other things. So by having that quick context right away it's able to say like oh, the first hundred lines of this file don't really apply to. Let me not even adopt this context. Let me just go to this section right here, uh, which has been really, really helpful. And the other one is uh, uh, what I call the deviation file. And the idea is if it starts noticing that it's deviating from certain things that I provided. It must immediately recall force, hook itself back to the master file or the table contents file, et cetera, to make sure it goes back on track instead of staying on the file of deviation. Hopefully that was helpful for a lot of people. Uh, okay, it's time for everyone's favorite part of the podcast, including mine. It is to ask Danny, Leon, Chris, and Jeremy a question. Leon, what question do we have today?

Speaker C: All right, uh, question of the day is if you deleted every AI tool you use, which one would genuinely hurt and which one would you be a little relieved to lose?

Speaker B: Good question, Good question. Uh, let's send this to Jeremy first. Jeremy, what are your thoughts on this?

Speaker E: Yeah, so the, the genuinely hurt is easy for me. That would be Claude. I rely. It's such a crutch for me. I use it so much, but it's because it's so good and it keeps getting better. And, um, I love when they add new features and capabilities. It's almost like it's learning from me, like, seeing the friction in my work work. And like, the new models come back and they're. They figure out ways to ease that friction or do something new for me, uh, a new capability. And so Claude would definitely hurt to lose. Um, I don't know, like, in terms of, like, what I would be happy to get rid of, you know, just all the additional work now that we have because of these tools, you know, I think that would be one, uh, in terms of, like, what would hurt too is like, like the what's coming. Like, there are more tools coming, you know, and so, like, you know, they're being introduced, you know, and so just like, that's what I think. Also, I would be afraid to lose of, like, oh, you know, like, what, what I can do with Replit now and what I can do with, uh, you know, Vercel, what I can do with Lovable and some of the new capabilities that they're. They're launching. So just like, the opportunity cost, I think would be better.

Speaker D: Big interesting.

Speaker B: What about you, Chris?

Speaker D: For me, um, I could probably happily lose Claude code at this point. I just feel like as a harness, it kind of lags behind what I expect other tools to do. Um, and I just like being able to, like, I like the tools that are more open and allow me to just like, pick and choose how I want to work. Right. Instead of like, kind of like funneling me towards the thing. Like, like you mentioned with Claude code, how it was learning Ah, they actually have memory turned on. So it is learning. Learning from you. Right. And you have to disable that if you really want to start fresh on new sessions. Right. So I find that interesting. Um, the. Yeah, that would be the one I'd be fine with losing. The one that I would hate to lose. Uh, probably PI at this point. I think PI is just such a cool, minimalist tool. And the fact that it has this, like, SDK you can build from, so you could build your own CLI harnesses on top of the SDK. I think open code has a similar thing too. But, like, man, that's just, like, really cool. I.

Speaker E: Powerful.

Speaker B: You're one of the few people that I now know that has actually mentioned PI. And I feel like I'm in, like, the small bubble. We need to geek out after this podcast, Chris. Uh, I'm not going to deviate here, but that was really cool. I don't know anybody else personally that talked about PI before. Leon.

Speaker C: If I could, like, wave a wand and nobody gets them, it'd be long running agents because the fact that before I have to go to bed, I actually to, like, talk to these agents to spin them up so they can run for seven hours while I sleep. And when I wake up, instead of enjoying my coffee, I'm actually reading all the output of it. Like, if we can just wait one it goes away that I'd be happy to lose. That, uh, one I could not do without is cursor. It just does everything for me. I'm so deep in the ecosystem. Um, yeah, bug by everything just like, it just. And for me, like, I want to get to where this is, like, all I need to, like, manage everything is my travel keyboard. Having cloud agents, having a cloud environment, um, that's just where I want to be. And so they kind of have, uh, pushed a lot of that stuff. So for me, that'd be the hardest to lose in my overall workflow.

Speaker B: Yeah, my answer is a little different. And if every AI tool was deleted, the thing that I would be actually relieved to lose is not even the tools itself. It's the hype cycle around them. Because, oh, my God, I would be so happy for my social media to be back to normal. So I would be happy for that to be gone. The thing. And it's. It's weird, but the thing that I would actually be relieved or like, I would like, it would hurt me to lose, as weird as it is voice to text or like, voice. Uh, AI. I cannot. I like, um, at work, we don't have something that's approved that we can use. Uh, we've created like our own little internal thing one of the software engineers did. So shout out to him, um, but, but like I can't use like super whisper or whisper flow, uh, internally. And so like when I'm having to work on my work stuff compared to like the way that I'm able to speak night and day difference on how much I'm able to get done and it's just blown my mind. And here's the other thing. I'll say I was at AI Engineer Worlds for last year and one conversation with um, and her name is escaping my mind right now. I believe her name was Stephanie, but she was a scientist and the way that she spoke about how she sees and uses voice AI, it totally changed the way that I perceived it and it took me down this entire rabbit hole that I've been down for the last year. I can't imagine going to full development workflows without using voice throughout the day. It's so much harder because even I have a great keyboard. The uh, clickety clacks and all that, right? I have so many clickety clack keyboards that I spent all this money on. And I even tweeted about this that the biggest pain point about using voice is that I no longer use these keyboards the way that I did before. But I'm able to provide so much nuance to the prompt that like or like to the tasks that I'm giving my agents or like even just as I'm writing things out like in docs, that it's so much better that I don't always get in the typing area. Like either I'm overthinking it as I'm typing it or I skip lines that I'm thinking like, oh, maybe this is not really as relevant, but the fact that I can write it all out via voice and then audit it to kind of of mix and match or trim what I need to night and day difference in productivity and quality as far as I'm concerned. All uh, right. Uh, it's been real, it's been fun and we'll see you on the next one. Goodbye everybody.

Speaker E: Peace.