Automated Agile & AI Product Delivery with Paul Glover (Burendo)

Andrew Welcome to byProduct Live where we unbox everything SaaS and product. We talk with experts and leaders in SaaS and product to understand their approach to leadership and ways of working to share their knowledge and understanding. Andrew Joining us today on the byProduct Podcast is Paul Glover, Principal Consultant at Burendo. Today we're going to be talking about the use of AI in the product delivery lifecycle and a concept they're collaborating on called Automated Agile.

Burendo are an agile and product development consultancy who work across multiple sectors, including healthcare, government, financial services, and betting and gaming. They've developed the Blended Agile Delivery or BAD toolkit and have grown rapidly over the last few years. Paul is an experienced leader with a background as a business analyst and product owner. He has a real passion for AI and how it can be applied effectively to improve efficiency and improve the ability to deliver products rapidly.

Welcome to the show today, Paul. How are you? Paul Glover Thanks having me, Andrew. I'm good.

I'm good. Andrew Cool. So let's kick things off straight away and let's dive into automated Agile. So what is it?

Do you want to give us a quick overview? Paul Glover Yeah, so at its most basic, it's kind of an understanding that AI can be utilized in the product delivery lifecycle to get more productivity out of that process, I guess. And so really what we're trying to do with that is understand throughout that product delivery lifecycle, how do we apply the AI principles to get the most out of it each stage? And that might mean tooling, it might mean process, just ways of thinking.

Andrew Okay. Paul Glover and it's about understanding based on what we've got right now what kind of benefit can we give and then pulling that together into something that we can then explain to people. Andrew Yeah, so we think about, I guess we're always trying to solve problems. What kind of problems does Automated Agile address?

Paul Glover Well, it's really about productivity. I think that's the main challenge. But also I think it's about having agile principles at the heart of it all. mean, the primary measure of progress is working software, right?

So I think it's about understanding that and then understanding how we can utilize AI to help us in that situation and keeping those agile principles at heart. So there's inefficiencies in communication, there's slow prototyping processes, there's... Andrew Yeah. Paul Glover time-consuming development cycles.

There's a lot of manual activity that goes on inside of that process. And we like to talk about something that we call a holodeck theory, which is like, if you're a mid-40s nerd, you might know what that means. But it's about the Star Trek kind of thing, where you walk into a room and you say, me this, and it gives you straightaway. Well, I guess that's what we're thinking an Automated Agile really is, is that.

anything in between an idea and the product is friction really and what I do is try and reduce that friction as much as possible. Andrew Yeah, and I guess much of it then is around time and improving delivery times and getting things done quicker. Paul Glover Yeah, well, it's about being more productive. You know, it's about unlocking.

I've worked in a lot of organizations that have got quite long, you know, backlogs and often they tend to say, well, we'd love to do this, but we, you know, we just don't have the time or the ability. Well, it's about unlocking that time and ability to help people deliver more. Andrew Yeah, cool. So if we think about automated Agile in practice and in the real world, how does it work if you want to give us a quick overview?

Paul Glover Yeah, so obviously that's going to change over time. as new tools and methods come out, but currently how it works is we're utilizing a lot of Claude and Claude has something called projects, which is essentially a retrieval augmented generation database. It's like a way of adding bits of information in there so that when you ask the AI a question, it's more informed about what it is that you're doing. And that's the core.

Andrew Yeah. Okay. Paul Glover of Automated Agile really it's about ensuring that the context in which you talk to an AI about a subject is as deep and meaningful for the situation as possible because that drives the best results. Because really what we're looking to do is, you know, save that time so that you can be more productive as you were saying earlier.

So from a practical sense, what you're looking to do throughout that process is utilize tools like this to take you some of the way forward. So one of the key things about Automated Agile is perhaps different than others is what we're trying to do is get an AI answer first and then almost peer review that like it's a junior BA or a junior developer and that comes from kind of thinking of like if you want to get an answer or something on the internet that you don't know post the wrong answer and then other people correct you because it's easier for people from a cognitive lower perspective to you know, what's wrong than it is to create Andrew Okay.

Yep, yep. Paul Glover And so in a situation where we know somewhat what the outcome is going to be. So in a requirements gathering situation, you know somewhat, you're going to gather some requirements. You can help the AI with domain information and with an understanding about how to do that effectively.

And therefore get maybe 60 % of the way forward in that process, you know, and over time that 60 % might turn into 70%, 80%. And that's time taken away from, you know, your BA to be able to really deliver for your customer. Andrew Yeah. Okay.

Yeah, yeah. And Claude's the AI product from Anthropic, right? Yeah, yeah. Paul Glover That's right.

Yeah. I mean, ChatGPT does something, you know, relatively similar. They have memory. It's always a good exercise for anybody who's not a power use for chat GBT to go in there and ask it, you know, what's in my memory.

And you'd be surprised what he's remembering about you. And you can ask it to clear that as well. But I think, yeah, in this, in this situation, really, we're utilizing Claude just as, you know, a stand up around an idea. I think once it gets to the point where Andrew Yeah.

okay. Paul Glover We're actually doing it in anger and enterprise organizations, standing up your own rag database and building this kind of thing off top of it. It'd be a much more preferable ownership of your own data and you get more out of it. Andrew Yep.

Yep. Yep. So for some of those people that might not be as auffay with some of the language like me, so what we're saying is within Claude, you can in effect set up your own organizational database that will store the data that you're inputting, the different prompts over time, then it'll base its answers and its responses on some of the information that you've already put into its system, but that'll just be personal to you. It won't necessarily be available to everybody else that's using Claude.

Paul Glover Yeah. And there's levels of that. So really when we're talking about retrieval augmented generation, we're talking about you actually physically drag and drop things like PDFs or markdown documents into a place. And then what you can do is refer to that in your prompts and then it'll know directly where to look, or you can just ask a general question.

And what the tool will do is search through that database of try and find information that's relevant and then pull that relevant information into the context of your conversation. So that when you're asking a question, Andrew okay. Right, okay. Paul Glover you're getting a better answer at the back of it.

So it's about controlling that context. That's the key is that how do we best control the context? Because every LLM call is essentially made up of the context of the conversation that you've previously had, if there has been any, your question and the answer that it's going to give. So what you want to do is make sure that the information that's utilized in that is the most effective utilization for that particular question.

So rather than there being one for the whole Automated Agile process, you Andrew Yep. Yeah. Yep. Paul Glover can produce these kind of RAG databases on a use case perspective.

you could do it around creating a certain kind of document, or you could do it on a certain kind of question, and then build the context around that as best as you can to get the best possible answer. It's probably another nerdy game in reference, but something we call min-maxing. So if you want to get the most out of your character in World of Warcraft or whatever it is, your weapon of choice isn't these kind of games. You work over time to be able to produce Andrew Yeah, okay.

Paul Glover the best possible character for that particular damage type. You know, you might have the big shiny hammer and all this kind of stuff, you know, so, so, and in Automated Agile, what we're trying to do is understand how do we min max this automated process that sits alongside an agile human process and supports them to get the best possible outcome out of that particular one scenario that we're asking it to do. And the more you break that down and the more you work on those individual scenarios, the better responses you're going to get.

So that's where Automated Agile comes in really is how do we do that? You know, we might know that as a concept, but how do you string it together into kind of product delivery life cycles so that it can be effective and actually utilized. Andrew Yeah. Yeah.

And I guess what we're talking about here is literally speeding up the building of some kind of software application or a software product. Paul Glover Yeah, mean speed is great, but also quality. So for example, we've done a number of tests on this and one of the great things about Automated Agile is we do things like define the standard for what code should look like. And then when we produce some code through this process and we see a bug in that, we can assess it against those standards to be able to say, well, what is it that was the challenge and is it something that exists in our standards that we've not written?

Andrew Yeah. Okay. Paul Glover You know, and certainly in the last test that we did, we produced a working prototype in a little under an hour and a half. We found, we found a bug in that process.

And when we went back and checked the documents, it just wasn't written in there. So all we need to do is fill that hole and then we don't get that bug. And this is how you move it from your 60 % to 70 % to 80%. Cause again, to reiterate, you know, it's about working code.

Andrew Yeah. Right, okay, wow. Okay. Yeah.

Yeah. Paul Glover And so really you could think of Automated Agile as a methodology for helping the AI succeed in helping you succeed. And so a lot of it is around making, you know, the documentation and the information that sits around a project as machine readable as possible. And it kind of flips some of those agile thinking methodologies on its head.

So previously as a business analyst, Andrew Yeah, yeah. Okay. Right, okay. Paul Glover You know, in my early, early days, you'd be writing 20 page requirement documents.

Whereas in your later days, obviously that's a no, no. Why would I spend all that time writing requirements document, trying to understand that I know all the answers when I don't. For then someone else to review it takes weeks, right? Well, you know, in Automated Agile using tools like this, you can produce documents like that in a couple of hours with your stakeholder.

And if they're incorrect, your stakeholder can see them straight away. and make those changes straight away. So it's a much more visceral process, getting the outcome of what's going on in the customer's hands and squeezing down that amount of friction in between that idea and the execution of it. Andrew Yeah.

Yeah, cool. And I think like anybody that's using AI, it's all about your approach and actually some of the fundamentals around how you apply AI, but obviously the information that you give it. So what fundamentals have you identified that are really important to make this process work? Paul Glover It's about layering.

So I think, you know, one of the first conversations you have in a project is always, you've got some domain information and you might have a little bit of information, maybe there's a feasibility bit done or something like that. So, and you start your conversation with your customer and I'm a business analyst. So this kind of thing is like, you know, something I'm used to. So you start your conversation with the customer and gain that initial understanding.

And then what you do is use that initial understanding to build layers of understanding after that. And Automated Agile is no different. Andrew Okay. Paul Glover So you might start with creating a document with the AI, which gives you an overview of that kind of product.

And then from there, you produce whatever documentation is needed. Because really what you're trying to do is fully explain in a machine way, that product to the machine, but also check it with the customer as you're doing it. And rather than that kind of process, traditionally taking your days and weeks, know, it's something that you can do in Andrew Okay. Paul Glover minutes, know, certainly hours, even for large tools.

And so that kind of gets rid of all that, you know, like contrary indicators, I guess, of right, spending all that time on something and pretending you understand. Well, what we can do is almost instantly create this kind of documentation, almost instantly put it to the AI to build it. And then we're checking the production. Andrew Yeah.

Paul Glover That's the bit that matters, not the requirements document, any technical information or any plans. It's, well, what outcome do we get and can we criticize that outcome to get any feedback that we would need? Because it's the documentation that's produced it. Andrew Yeah.

And I'm guessing then that what becomes really important is the prompts that you're giving your AI models, because the clearer that you can be with AI, then the better the results that you'll get. And is that the same for this process? Paul Glover Yeah, you know, prompt engineering is one way you can, you know, improve the process, making sure the right information is in the RAC database is another, but even utilizing the right tool, you know, you might find that if you're trying to fix a bug as a, for instance, Claude might be missing some information in there, which is crucial and you can't fill with the RAC database, but you put it through Check GPT, you'll get an answer that matters.

So it's about, you know, it's about Andrew Right, okay. Paul Glover the steps that you take and understanding what you're trying to get out of the process and making sure that you are self-improving. So I wouldn't want anyone to think that they have to spend a lot of time creating the best prompts because the best thing to do is to give it your best shot, see what the outcome is, and then respond to the results. Yeah.

And based on your results, which are going to be different depending on your domain anyway, you'll get better and better over time. Andrew Right, okay. and keep refining. Paul Glover And then crucially, like this kind of structure and way of thinking of making sure all of your project documentation, your projects in general is built from machine readability perspective allows you to then layer new tools as they come along.

So it's about being fit to fly for the future. Right. So we know these tools are improving over time. We might get a 60 % answer now, but when, you know, chat, CBT five or whatever comes along, you know, it might, we might decide, well, you know, it's doesn't even better job, but what.

it's still going to need is that context and that information to be machine readable and to be understandable. And then you can keep layering those benefits on as the process improves. So it's about started early. Andrew Yeah, yeah, yeah, cool.

And I think one of the important elements to consider with anything around AI is obviously the human impact and then the ethical standpoint. And I guess particularly in consultancy, for as long as consultancy has been around, it's told people that it's a people business. This is about relationships and it's about Paul Glover 100%. Andrew people's ability to solve problem, actually it feels like with AI generally, not necessarily around Automated Agile, we're trying to take a lot away from that human element.

So how do you see Automated Agile impacting that human element of consultancy? Paul Glover I think certainly from an Automated Agile perspective, I know AI in general in other areas is taken away from the human, but I think certainly from an Automated Agile perspective, the human's at the center of it. It's about making sure that you're almost peer reviewing the AI's work. And instead of having to go away and type, it's about making that process of gathering requirements, updating documentation, driving forward towards something which is visible.

like a visceral conversation that you're having directly with a customer. And the human is absolutely central in that. the AI may be able to produce a set of questions that the human can ask, you know, but the understanding of them, you know, the unsaid things, you know, the facial expressions, you know, maybe there's a future where AI can understand those things too, but we're a way away from that yet. So really it's about unlocking the potential of people because Andrew Yeah.

Yeah. Yeah. Paul Glover As we were talking about earlier, know, that kind of backlogs are going on and any, you know, tech professional, especially someone who's been in a business for a year or two, it can really look at that business and understand so much that it could be improved using technology and so many better ways to move forward. So I think while AI, you know, there's some thinking out there at the moment that it's going to cost jobs, you know, and I just think personally, that's the wrong way of looking at it.

I think it should be unlocking opportunity inside your organization and innovation. And while there will be some companies that may say, well, this will save us 40 % of time, we'll lose 40 % of our people. Those companies won't survive very long against other companies that spend that 40 % on improving their business and supporting their customer. So I think, and the humans are at the heart of that because an AI is not going to question your customer and come up with your backlog in the same effective way.

It'll help you do so. Andrew Yeah. Yeah. Yeah.

Yeah, yeah. Paul Glover You know, it'll organize the work, but AI doesn't know anything. You know, it can't make decisions. It's based on probability.

So, you know, it's going to need a person behind that who can really support. Andrew Yeah. Yeah. And I guess one of the things it should do, and it goes back to that efficiency piece, is humans are human, and they do forget to do things, and they do make mistakes, whereas AI is becoming almost mistake-proof because it's just based on the information that you give it, humans reviewing documents, they do miss things, they do misinterpret things, and I guess that'll have a big advantage of another area of efficiency in a business.

Paul Glover Yeah. And that's one of the first places we started is, this getting together a small group of consultants and just testing like from an analysis perspective, how do humans do versus AI? Cause humans forget things too. We hallucinate too.

know, and, and, and it's all about, you know, perception, isn't it? You everyone's got their own truth. So when you're listening as a business analyst, you have, you know, these kinds of biases that you try and work around and the AI works in the same way. Andrew Yeah Paul Glover So you can, between the two, you produce a better product.

But either one of you isn't as good as having both together. Andrew Yeah, cool. And as we just start to wrap up the conversation, one of the things we touched on was innovation. And you did mention it whilst we were talking about the prototype project that you did do.

So I just want to make sure our listeners can hear that. So you took an idea through to a working prototype. So this is code produced. And it's a.

piece of software or a product that's actually working and could go live and how quickly did you do that? Paul Glover I won't go as far as to say could go live. think it was about producing essentially what we tried. We did it twice.

So what we're trying to do is produce, you know, an MVP. something that we can, we can show the customer to say, yeah, some of it. Yeah. So we can show a customer and say, is this, is this what you're looking for?

Are we on the right direction? You know, because what we don't want to do is produce something really big and then ask them that question and get told no. Andrew Right, okay. Yeah.

Yeah. Sorry, I meant an MVP rather than something that's out live and in the wild. Yeah. Yeah.

Paul Glover So we want, you know, give them some of the stairs in that direction. We did this twice. The first time we did it, we did it with a small group of consultants utilizing cloud projects. And that was 75 minutes from idea to working prototype.

Second time we did it. The first time we did it was quite basic. We weren't trying to think about it from an enterprise perspective. We weren't trying to produce a set of code that you could then integrate into somewhere else.

It wasn't to any standards. It was just a test. Can we do it? And we can.

Andrew Yeah. Right, okay. Yeah. Paul Glover So the second time we did it was thinking about those standards.

So we had a document standard for requirements. You know, we had a code standards and that was about, can we do this and then integrate it into an enterprise, you know, kind of structure? And is it something that we can continuously improve against? And that took about, you know, about an hour and a half, little bit less than an hour and a half, you know, a little bit longer.

Andrew no, 90 minutes is an acceptable pause. Paul Glover Well, that's what I mean. It's just wasn't good enough. But to be, be fair to us, we actually taught people how to do it and did it in that 90 minutes.

So it's not like it's a complicated process to even go into an organization and teach, you know, at a fundamental level. Obviously there's a lot deeper you can go. And that produced, instead of it being one code file, it was a highly modular set of code files, all to a set standard. And that allowed us that opportunity to be talked about earlier where Andrew Hi, okay.

Yeah. Paul Glover Once we found a bug, we could go back against that set standard and then make the change that would be better in the future. So that second one was a leap forward really, even though it took a little bit longer. And that's the kind of structure we're doing.

So we're working with other tools now like cursor. So I don't know if you're aware of cursor, it's essentially it's a tool that allows you to utilize models like Claude and add context to them. Andrew Yeah. Yeah, yeah.

No. Paul Glover based off the code base and it will generate code for you. So it's about how do we create enough information so it does that effectively in a good proportion of the times. So we're deepening it with new tools and methods.

You there's other things out there like Replet, which deploy, but I'm not too sure how that works at enterprise departments that still to be looked at. But I think, you know, and this is essentially what Automated Agile is, is that all these new things are coming out all the time. Andrew Okay. Right, okay.

Yeah. Yeah. Paul Glover And it's so confusing for people to understand, what do I do with my organization to get any benefit out of it? So we're trying to shortcut that for people.

We'll do the testing so you don't have to. Andrew Yeah. Yeah, And I guess once you have done it, there's a new tool there that could make it happen even quicker or in a more efficient way. And yeah, yeah.

Paul Glover Yeah, and 100 % that's the kind of thing that, because these things will layer over time. So what we're going to start to see is this gap in between, you know, people who work in the product delivery lifecycle and don't use these tools and people who do. And that gap's going to widen all the time. Because as you get good at these kinds of tools and as you structure your data in the Automated Agile way of making it machine readable, Andrew Yeah.

Paul Glover you know, the people who don't do that are really gonna start falling behind. So I think it's just like the internet coming along, know, stack overflow and stuff like that. You know, it's gonna be disruptive, but ultimately every time that happened, it ended up with more tech people because what we're doing now is that at the moment there's a conversation that's had in an organization, you know, a team of developers and tech people, a product team. Andrew Yeah.

Yeah. Yeah. Paul Glover cost me X and I can get Y value out of it. Well, when that changes and you can get even more value out of it, I think what we've seen in previous incantations of this happening is that that makes the tech industry explode.

So that's why I'm expecting not a lot of redundancies, certainly a lot of improvement. Andrew Yeah. Yeah, A shift in type, the types of roles that people are doing rather than we don't need these people. It's actually, this is where the skills are changing.

Paul Glover Yeah. And, and a shift in how good tech gets in helping you as a customer, you know, and, and how much value it can do, you know, how many websites have you been on? thought so awful, you know, I would never design this. Well, that kind of stuff should hopefully get better over time, you know, so that's the, that's the hope.

Andrew Yeah. Yeah. Yeah. Yeah.

Yeah, Cool. And just before we finish and talk about some takeaways for our listeners, just wanted to, so we've built this prototype in about an hour and a half, and the MVP. Without AI, how long on average would that take somebody to do what, and I know it's going to vary between organization and the type of MVP you're producing, but from your experience, where is the time now that AI is bringing it down to? Paul Glover Mm-hmm.

Yeah. I don't know. I think if you thought about it in this way, but didn't use AI, I think it'd be a lot faster than if you just did it using the normal PDLC that people do all over the country. So I would like to think about it in those kinds of terms rather than trying to, because it'd be easy for me to say, well, this was 500 % faster than anywhere I've ever worked, you know, but really, you know, if you tried to do the thinking without AI, I think the benefit is probably 30 to 40 % at this stage.

Andrew Yeah. Yeah, yeah, yeah. Paul Glover But I think there's still some stuff to be worked out in terms of like deployment and all that kind of stuff. So I think I would say that based on what we know right now, there's about a 30 % improvement on current methodologies and standards that we could apply today.

And I think as we're learning, I think that's going to increase. Andrew Yeah, yeah, yeah. Yeah, cool. So that's a good point for us to wrap up the conversation today.

And as always, we like our listeners to take away three key takeaways. So what do you think are the three key things that our listeners can take away today? Paul Glover I think number one is AI works on data. So if you ever want to use AI in your business, structure your business so that you have the best possible data.

And think about that from a context perspective. So how are you storing information? How are you recording information around your projects in a way that an AI can effectively understand? And I think there's a couple of tools out there that a lot of people use, like JIRA, there's your DevOps.

Andrew Yeah. Paul Glover Those tools are starting to bring generative AI into their processes. And so that's a good place to start for the enterprise organization, you know, making sure that you're putting really solid data that exists in there because it's going to be useful to you in the future. The second one is start now because the longer you wait, the further you're going to get behind.

And the benefits are out there right now. So if you're willing to, you know, spend the time and effort as an organization to do it. And I think the third one is Andrew Okay. Yeah.

Paul Glover If you can build a way of testing these and implementing them inside your organization as a process. So, you know, Burendo as a community of interest around this. And I think that's a good place to start because people are interested. So we'll average that.

And then consider it on a use case by use case basis and just put some data around it. Does it work? Does it not work? Let's try.

And I think what you'll find is that Andrew Yeah. Yeah. Yeah. Paul Glover because different organisations work in different ways, different things that work for you, but you'll find something that does work.

And the benefit, you know, not just in productivity, but in the happiness of people working with those tools is going to be really worth it. Andrew Yeah. Yeah. Brilliant.

Well, look, I really appreciate your time today. And actually, I think it's definitely something that's really thought provoking, but actually something that people are going to have to think about because AI is going to impact everybody's roles, but particularly by the nature of tech and software engineering and product development. It's got people. Paul Glover and keep us in.

Andrew who naturally want to find out more about this, so people do need to pick it up, otherwise they are gonna get left behind. And I think what would be really interesting is if we had a chat in 12 months time and see where you are now with this and where you are in 12 months, because the pace of change is gonna be so rapid, I think, yeah, it's gonna be unbelievable in 12 months time. Paul Glover Yeah. I think so too.

remember it gets more rapid because of tools like this. you know, so if I just quickly plug the website, which is Automated Agile.co.uk, I think on there, we're putting our thinking, as, as we learn and move forward and we're not, you know, the ultimate experts in this thing.

There's a lot of smart people in this space, but we're, this is just our practical application. What we would hope is that other, other, you know, organizations and people do the same thing. And then between everybody, we kind of collaborate together on what the best practices are. Andrew Yes.

Yeah. Yeah. Paul Glover But I think 12 months from now, we'll be in a completely different world from the way that, you know, this kind of technology works and speed. Andrew Yeah.

Yeah. We might both be robots. Definitely cool. So thanks very much for your time.

just to repeat that website again, so it's automatedagile.co.uk. If people want to check that out, pick it up and see what it's all about.

Thank you very much for your time today, Paul. I really appreciate it. Paul Glover Well, I could do with a robot knee, I don't know about her. Thank you.

You have a great afternoon. Andrew Thank you very much for listening today. We hope that you found the byProduct Podcast valuable, useful, and informative. You can subscribe on all major podcast platforms, including Apple, Spotify, and also YouTube.

And if you have the time, we'd love for you to leave us a positive review. As a new podcast, this would really help us out. We'll hope to see you soon on the next episode, and thank you again.