The Elf Foundry - Vibe Engineering, from Agentic AI to Platform AI

Speaker A: Securing the Realm. So, welcome to this episode of Securing the Realm. Um, it's a bit of a more festive themed one today, um, despite the clothing being the usual, but my name is Josh McDonald. Um, I'm a data and AI security architect, um, and technology consultant.

Speaker B: I'm Dr. Diana Wolf. I'm an IO psychologist and I lead AI research and strategy.

Speaker C: And I'm Fergus Kids. I'm the CTO of a small startup and we are AI focused and also very reliant on AI.

Speaker D: Very reliant AI. So, hi, I'm Chris Lloyd Jones. I'm the co host of Securing the Realm and I work at Kyndryl during the day and by night I'm an mvp.

Speaker B: A vigilante mvp.

Speaker C: Yeah, uh, I'm an MVP as well.

Speaker A: I forgot There's a few MVPs in the space.

Speaker D: And a third MVP.

Speaker C: Oh, yeah.

Speaker B: I consider myself important.

Speaker A: Three MVPs and a PhD. That's pretty, pretty good, I'd say.

Speaker D: So what are we here to talk about today?

Speaker A: Yeah, so, you know, we're here to talk about vibe coding. I know it's one that we've talked about on the podcast quite a lot and the kind of umbrage we take with that term sometimes around kind of the connotations of what vibe coding means. But really kind of what we're coming down to is there's been a lot of innovation in the space on the Microsoft platforms in particular. People are wanting to do more and more in plain English. The results can vary, I think it's fair to say. So what we're really kind of coming try and solve for is how do we get the speed and the kind of, sometimes the quality, um, without all the chaos that can sometimes ensue.

Speaker D: So I'll summarize. Vibe coding. I hate it.

Speaker B: Great summary.

Speaker D: It's all about vibe engineering and how we can put stuff into production that isn't going to be immediately hacked. So where do you want to start?

Speaker A: Oh, yeah. How do we capitalise on getting the kind of most out of it in terms of the speed that it clearly realizes the ability to proliferate code like ever before. What it really comes down to is, like, how do we get the speed and all the other benefits of AI generated code without the chaos of AI generated code?

Speaker B: Right. I'll take a swing at it and I'll probably try and land this in a Christmas metaphor. Vibe coding, as we've come to define it, is an emerging definition, but right now we're understanding it as our NLP's giving us like a conversational form of development process where previously it's been code based. And it's going to be a, ah, area where we're able to prototype faster. There's lower barriers to being able to build and it's rapid integration. And if I was going to do this in a Christmas metaphor, um, if we're making a, if the elves are doing a toy factory, right, and they've got to put all the pieces and components together, they need to know where all the parts are. Um, it's a little bit more like we're skipping assembling the toys and they're just kind of being prototyped and 3D printed for us. So 3D printing brings me to you.

Speaker C: I must know about 3D printing. Yeah, I think it's really interesting because like one thing that's changed in my own habits as we move to Vibe Engineering is actually having a, um, git structure, uh, sort of monitoring structure that actually works. Because in especially, you know, I work for a small startup called FieldPal AI. In some areas we're moving really super quickly and in some areas we're building a really solid product foundation. We move quickly in areas which are not as important to our core product, but are things that we need. So where we're moving quickly, not all of the engineers and developers know exactly what is going into the repo. So we have very clearly defined buckets of generative AI is allowed in here. Vibe Engineering and Vibe coding is allowed in here. But we use the same pull request system, the same git systems as everything else. So we, we are always double checking that code. So it's like, although we don't necessarily know what's going into it, we, we are always checking that it works and it doesn't break the core product. I think that's super important. And that really uh, errs on like the engineering side of the Vibe engineering.

Speaker D: Right.

Speaker C: We have to keep the engineering, but we're, you know, we're very much in danger. We're building an AI product. There are parts of that AI product that if we're vibe coding them, AI is writing the prompt and AI is writing the code. We're this kind of danger of, of a uh, snowball effect, to use another wintery metaphor, I missed that one. A sort of snowball effect of Vibe coded stuff being layered onto more Vibe coded stuff to output more. So some testing we did the other day was does our AI respond in a pirate voice if you ask it? Yes.

Speaker D: Why? Why are we doing that?

Speaker B: It's Important we test everything.

Speaker C: We test everything.

Speaker A: Couldn't you get it to ask a Santa?

Speaker D: So I guess sticking to the toys analogy that you mentioned earlier. So toys, when we make them, they have safety marks, they've got compliance to make sure that, you know, kids aren't going to choke on bits of the toy. How do you make sure that what your AI is doing is secure? What do you do today?

Speaker C: Uh, we say please don't put lead in the toys. So what you making? So, you know, it's a really good, good point. It's basic, uh, today it's basically done through prompt engineering at the beginning. So careful prompt engineering, you, uh, know, using custom prompts, saying this is my goal. I can give you some, some examples. But I asked, was trying to build a quick website, asked an AI agent to change something, it misunderstood and deleted the entire page. And I was like, if I hadn't have had that in git, it would have been hours of work lost.

Speaker D: So thankfully this is terrifying. Yeah, yeah, I know.

Speaker B: I also misunderstand and delete entire pages.

Speaker A: Yeah. Did you challenge the AI and then it came back and said, oh, you're absolutely right, I did delete that.

Speaker C: It welcomely admits that it deleted it, but it can't undelete it without git or some.

Speaker D: So I guess my question would be going back to the toys and the framework piece. Do you have testing strategies in place or do you have an existing security framework like OWASP or something that you're extending?

Speaker C: Yeah, so we only use uh, today we only use Azure services which have a lot of the security built into the product. So we know that we're using safe and secure products. We also do things, things in Docker containers to make sure that they're containerized properly and things like that. But I think a really important part of that is, is code review. So reviewing the code that agents are actually writing as part of your normal process and testing is a really interesting topic because testing is very important to everyone and everything. But in a small scale startup, when we're just trying to build a product that works, it's one of the last things we're thinking about. Because we're trying to build something that works. We're not worried about every single microservice working perfectly. But actually that's an area that AI can help with because you can have an agent run over that code base and say, I've noticed that you don't have any tests covering this. I can write some unit tests doing that and it can do that. In the background. And those tests aren't necessarily going to break the product or the way that it's working, but that is going to improve your overall sort of technical.

Speaker D: I guess that would fix what you talked about where if the AI, uh, hand deleted the page, the test would have picked up on it. So it's not about testing every scenario, it's about making sure that features aren't being removed.

Speaker C: Exactly.

Speaker B: Uh, what about the volume of code generation though? If we're saying code review is the viable way to make sure that we're maintaining quality and integrity and security of our code, how much can a human really be expected to process?

Speaker C: I mean, in the toy analogy, it's like if we 3D printed 100 million toy soldiers, we're also going to have to check every single one.

Speaker B: Right.

Speaker C: It's just not realistic. So I think that that really comes back to Chris's point about this is engineering, not just vibe coding. If you're vibe coding a personal product, you're perfectly happy that it is producing thousands of lines of code at once. But it has to be realistic. Uh, you know, we work in small pull requests that don't break the product, basically. So that pull request has to be small enough that one of our developers can check it, whether that's me or someone else. So that's really about the human approach to using the tools, rather than saying to the tools go off and run away and do everything overnight, which they can do and can be powerful, but it's impossible to review all of that. Or developers just change into review machines.

Speaker A: Yeah. Do you think there's a lot of space for actually getting better quality coverage, test coverage in this space than what we previously had? You know, pre AI. I imagine at the moment most organizations are kind of going through a place where they're losing coverage by the day.

Speaker C: Perhaps I'd rather have a. I would rather have a lazy AI than a lazy developer because at least you'll have something that will work and is testable. Whereas, um, if you're, if you have a contractor or a developer who is not outputting, that doesn't help you either. And it's a lot more expensive.

Speaker D: It sounds very anti human.

Speaker C: It's not anti human. It's about how can we best use the tools. I mean, you could make the same argument about, you know, compilers from the 90s. It's like, oh, that's making our process a lot more efficient and saving us all time. Is that a bad thing?

Speaker D: Have any of you tried spec driven development? What do you think of that trend.

Speaker A: So, you know, I'm someone who's used spec driven development in a few different fashions recently and I know we've got some projects on our repo at the moment as well that essentially driven entirely by spec driven development. But, you know, been looking at personal projects and you know, as someone who started out as a business analyst in technology, I love the idea that I can actually write a really detailed document, make it readable for humans, make it readable for machines at the same time. That's the kind of new bit of like, how do I AI ify the documentation, but I'm finding the results are really good. You just kind of go back to writing user stories rather than writing code.

Speaker C: What do you think the difference is between writing that for a human? Because that's just kind of what we're doing. But what's the difference between writing it for a human and writing it for an AI?

Speaker B: I think that potentially when we think about how we're going to be speaking with AIs and some of the ways we've just been able to communicate with each other naturalistically as humans, we need to be making sure that we put boundary setting into the language. Right. So we don't use generalizable terms. Right. Not like make this, I guess, understandable to a user. That's kind of a generalizable thing. But if there are certain features and ways you want to build that out, I would lean into that. And it is actually a really good spectrum in development. And your example is actually a really good example of complementarity, where the human is still there to design and build and have novelty, and the AI is to execute and go a little bit faster with the code. But we're getting to a better outcome quicker is what we're looking for. Because you can test and iterate rapidly in spec driven development. Because you know what you were looking for to begin with. You're not just saying, build me a shopping app, right? You outlined what you're looking for and what your quality is inherently through that.

Speaker A: Yeah, yeah, absolutely. Because the less of those quality markers you put into it, I find with the spectrum side of things, you're more likely to end up with something that is very deeply generic. You'd probably start to call it AI slop because not even a human would make some of this stuff because it's been kind of like an amalgam of all the things it can look at on an E commerce platform.

Speaker D: I think what I like about spec driven development is spec is kind of all encompassing. So a lot of these frameworks include architect, ux, business analysis. There's a researcher, there's a security side, there's a tester. So I guess how does that change the way in which we think about security? It's got the ails running wild, they've got a security framework they're following, got the blueprints. Now what comes next, what's different? Because I'm seeing this space evolve really rapidly.

Speaker A: Yeah, that's a good one. And you know, from a security perspective, I think we're still trying to play catch up at the moment and that's probably broader than just the development side of code. We're trying to get the governance right around the processes, the ways of working but you know, also the traditional tools, they're struggling with some of this, um, a static code analysis of AI driven code does work but it's kind of hard to kind of keep up with the pace of traditional analysis when you are doing more and more proliferation of code.

Speaker D: So how do we stop Fergus product getting hacked? So like should he be building, building a new security framework, doing something like from scratch?

Speaker C: What's the right way forward using off the shelf? Like that's uh, our kind of approach today.

Speaker A: Yeah, I mean, I mean we recently explored the whole point of around application security, around AI broadly and you know, application security is something that has to go across all applications. But there's also some promising stuff around the GitHub space as well at the moment. You know, being able to bring in agents from a variety of different spaces. The GitHub Agent HQ I'm sure that we saw around, you know, the end of last month starting to appear in some of the environments, being able to use agents from a variety of different sources. But you know, we can't put AI back in the box now. It's out there. So we just kind of have to react.

Speaker D: So that's nice. But going back to my question, how do we stop Ferd from getting hacked? Should he be building something from scratch or should he be using an existing framework?

Speaker A: I'd say start with the existing frameworks. You know one of the things we explored was how we can use um, 27034, the standard for app security. That's not an AI standard, that's an AppSec standard. So using whilst the tools are struggling to keep up the frameworks, I guess

Speaker C: and the frameworks can be used by spec driven development. Right. Because you can the framework into apply this. Yeah.

Speaker A: As long as that spec is either kind of available to you or it's open source in some way, um, and can be used with all the proper licenses of course, you know, so you can work with the existing standards feed in some of the new developing challenges. So one of the things we looked at was The OWASP top 10 for LLMs to look at, kind of if you're building an AI app. But not every AI generated app is an AI app. You can look at things like that if you're building an agent. Today they announced an OWASP top 10 for agent, agent, AI. Um, so there's a lot coming up,

Speaker D: so that's really helpful. So you've now got a framework you can take away so you're not going to get hacked. Yeah.

Speaker B: Well, I can tell you you, uh, saved Christmas.

Speaker C: Our website was online for about six hours before bots started trying to find end files. Really Robot txts, env, prod, envs, all the PHP files have just that. Our website has actually been more popular with bots than it has with humans so far.

Speaker D: So have you seen any high profile security failures when people haven't adopted these things?

Speaker A: Well, I think there's a lot that we don't know about.

Speaker D: Yeah.

Speaker A: So I say high profile, but I'm thinking low profile.

Speaker C: Yeah.

Speaker A: Um, because of course there's a lot of high profile stuff around the LLMs themselves being compromised or an agentic app being compromised. And you know, there's probably so much color that is already out there on stuff like the anthropic cyber espionage.

Speaker D: Yeah, I guess. Yesterday you shared with me that there was a large vulnerability in React. So if you aren't using Spectre from development and defining your versions of your packages, you could be using an insecure version of React. But then I guess also bytecoding means that you are not necessarily the subject matter expert in what you're building, but you know, the features. I know recently there was a side channel attack where a security researcher was able to look at the packet size and shape and velocity and identify what was contained within the tokens that being sent to and from the server. Now if you're OpenAI or if you're Microsoft, then I guess you can fix it. But if you fiber coded your own LLM interface, you can be keeping up with these developments. Like how do you defend against that?

Speaker A: Well, it comes down to again, probably a governance issue and starting to think about some of these implications that you touched on there. So one of the things you touched on there is essentially the, I guess the outsourcing of the expertise. So you don't have to be an SME in a particular thing. Like I'm not a front end guy.

Speaker D: Um, you're not?

Speaker B: No.

Speaker C: Well, you are. With AI, everyone's a front end.

Speaker A: And one of the things that a lot of the LLMs and development agents like is React. They love React is often the solution to any kind of vaguely web based problem.

Speaker C: I mean, I think AI agents, and specifically the agents we see with GitHub, Copilot HQ and agents that can kind of run in the background without too much prompting. I think, you know, we've seen from the podcast some really good examples of both. But, um, most examples you see on LinkedIn and on podcasts are about I am typing to an agent, it does some code on. There are agents that can run in the background. Code staleness is a really good example of that. You know an agent that says, you haven't updated this. This is still insecure.

Speaker D: And M. I try and remove code more than I add it because otherwise I find agents are adding thousands of lines that can't be audited. So, Dr. Wolf, you're a researcher, uh, you know a lot about governance. I know that's a big area of interest for you. What have you been seeing recently?

Speaker B: I think we just talked a little bit about domain expertise as it's being spread out through vibe coding.

Speaker D: And I Fergus said, get rid of all the humans.

Speaker B: Yes.

Speaker C: Don't quote me as a psychologist.

Speaker B: No. So we just talked about what does it look like when we democratize expertise and when we vibe code, we essentially are attempting to democratize some field and domain expertise. What previously could have been a team of developers, front end, back end security, all of the ways that we go through building an application, we've now outsourced all of that. In psychology, we talk very specifically about the Dunning Krieger effect. When you learn something new and that is the assumption that you know more than you actually know before you're very aggressively confronted with the knowledge that you do not know more than you actually know.

Speaker C: I'm not familiar with this.

Speaker B: No one here has experienced it. Um, and I think that the dunningkrieger effect is accelerated by the dopamine hits you get from seeing outputs. It's a self reinforcing loop that starts to get pushed out. So there's a governance of self that needs to happen. In these systems and psychology you need to go, am I running too fast? Am I not asking the right questions? That's starting there. But then when we get to systems level governance and how we're thinking about it. Frameworks are incredibly helpful and useful tools, but they're only as useful as they are implemented, as they are, uh, held to fidelity and as they are understood not only by the AI, but also by the people deploying the AI. Why is the security framework here? So it's not enough to just launch a very pretty framework and say, okay, put it into your, uh, spec development and go run. Do they need to understand what good looks like when it comes back? And that's actually some skill development that's going to have to happen. All of us are in the phase

Speaker D: of skill development, so what should people do with that information? Because that makes sense. But how do people know what good looks like? How do people avoid the Dunning Kruger effect? How do people add a framework? What do people need to know?

Speaker B: Okay, everyone, brace yourselves. Learn. You're going to have to learn a little bit, right? If I gave, uh, you all of the knowledge of. Oh, wait, let me make it a Christmas metaphor. If I told you everything you needed to know to be a reindeer scientist and everything about how flight works with reindeers, and you had all of it, that does not mean you know what to do to make sure that Santa gets to where he's going on time. Right?

Speaker D: Absorb it for one. Right?

Speaker B: So much to know about flying reindeer. So that means if I give you expertise, that doesn't mean you know how to effectively use it, scale it, trust it, build it. So there is a level of skill development we need to ask of ourselves when we get to the field of vibe engineering for us to do. So start with, I don't know, a book. But you could also use AI as a way of teaching you back. Right? You can ask for some of that. And then you can also rely on quite a bit of the community around it is surfacing these things and policies are being put into place. So you're just going to have to do some good old book learning.

Speaker D: And how do you know if that learning is correct? How do you know what the AI is telling you isn't slop?

Speaker B: Right. I would probably lean toward if I would use the AI to surface information so you can learn. But I'm sure we're going to see an emergence of a lot of certifications and governance and things like that, that we're going to say that this engineer who is capable of using AI in their development process is actually capable of doing it in a secure way and they understand the framework. We've seen that time and time again.

Speaker D: So what will you be doing now?

Speaker C: It just made me think about, can we certify the AI, make sure it knows what it's talking about? What will I be doing? I think one thing that, you know, we've, we've, we've joked a little bit. So what I took from what Diana was saying is there's a really important place for human because we need to apply that. We need to do that learning. We need to apply that learning. And that's what AI isn't very good at because it will just regurgitate the last thing you said. So if you say that was completely wrong, it usually says yes, I totally agree. Where. So we, we need to, you know, in a world where we're actually using human skills and AI skills, AI skills as tools and human skills, we're kind of. Yeah. In that sort of world where we balance that. I think there's um, a really, that's a really important point is like humans are m. Always going to be much better at learning, whereas AI, we have to wait for OpenAI to retrain it.

Speaker D: So I think that's a good point. From what Dan, I mentioned complementarity, but what are you going to do?

Speaker C: Yeah. So that brings me back to the difference between human and AI. Um, what I'm going to do is learn. I got that. And apply some standards and look into what standards they could be and discuss that with the team and agree on that. But then also think about the role that AI has. Do we treat AI as a human with things like role based access control? Do we treat humans as robots and check everything that they output? Like, where's that line?

Speaker D: So you're going to learn, look at standards. I guess it's a good segue. Over to you, Josh. You've been looking at the security standards in this space and more generally for AppSec. What do you think people need to do?

Speaker A: Well, um, I mean a couple of things. Ah. To Diana's point, one of the kind of critical things I'm seeing is the uncritical use of AI, whether it's development or otherwise. And you know, we've talked about one framework, the kind of AppSec framework. Um, but governance isn't just a security problem and kind of governing good code isn't just a security problem. So you need to find a governance framework that works for you. I worry we're going to have sprawl. Um, of those frameworks there's going to

Speaker C: be more or just AI generated frameworks built on AI generated frameworks.

Speaker A: Yeah, probably something like all the way down.

Speaker C: Yeah, yeah.

Speaker A: AI in Depth rather than defense. In depth. But no, like, as we've seen with the explosion of the mcp, the Model Context Protocol, we now have probably just as many developers of MCP servers as there are MCP servers at this point. Um, and far, far more than there are MCP users.

Speaker D: So we've been working with machine learning since like 2017, 2018. We've been focused on AI since what, 2020, 2021. But there's a lot of people now entering the space who just read something of chat and have no idea what they're talking about. How do you detect that?

Speaker A: Honestly, I don't know how we detect that other than say, to rely on your critical thinking. Um, I think it's an M M dashes detecting AI gone. Sorry. And detecting AI content is definitely one thing. EM M dashes emojis everywhere M and there's definitely telltale signs. But you really just gotta rely on your own critical thought as to um, not whether something is AI generated, but is it something that makes sense regardless of where the output comes from?

Speaker C: I've written code with AI and have been called out by developers saying, oh, I liked your solution to xyz. And I have thought, I don't remember doing X or Y or Z because it was something that an AI fixed in the background whilst I was doing something else. So I think one strategy is just asking people about what they did, what resources they used and if they can't answer any of those questions. The way you see it with a front end developer is exactly what Josh said is if they use emojis instead of an SVG library that is Vibe coded.

Speaker D: I guess that neatly ties to what you mentioned earlier when you alluded at uh, like unconscious incompetence, conscious competence. Conscious incompetence.

Speaker B: And say it three times.

Speaker D: First, how you figured out why. I also missed one as well. How you figured out who shifted from one to the other.

Speaker B: Yeah,

Speaker D: yeah.

Speaker A: Ultimately Vibe coding is, you know, it's not going away. We can do better.

Speaker D: It is going away with Vibe engineering.

Speaker A: Well, we're revolving it then. Fair enough. But you know, it's very powerful. It's not something we necessarily want to avoid, but the question is whether we let it run wild. And the answer is we shouldn't. But what we can do to make it a sustainable pace and aligned with what humans actually want and expect as well. So where can people go to find out a little bit more? So we've got our website securing Quest. We've had some great commentary on kind of the different Frameworks over this and some of the things. So you know, on there, there's also links to Diana's paper around.

Speaker B: Read the paper.

Speaker D: Yeah.

Speaker A: Yes. Diana's famous strap line. Read the paper and learn.

Speaker B: Yeah.

Speaker D: Thank you for watching this on the Festive Tech calendar. Thank you to kinder for supporting the particular episode. Thank you to you, Dan of joining us and uh, Fergus Kidd from fieldpal. Thank you for joining us too.

Speaker C: Thanks for having us.

Speaker B: Yeah, thank you.

Speaker D: Anything.

Speaker C: Well, I've got one Christmas related question which relates to security is like in A Nightmare Before Christmas world when Santa's replaced by Jack Skellington, what happens when that somebody infiltrates the OpenAI models or

Speaker D: as a matter of fact, so, you

Speaker C: know, Nightmare Before Christmas, Jack Skellington kidnaps Santa Claus and gives all the kids a bunch of Halloween presents and it's all horrible. Is that like basically making an analogy between that and something like deep seeking, basically giving out incorrect information?

Speaker D: I'm not sold.

Speaker C: Okay, fine. It was just in my head and

Speaker B: I was like, yeah, I was thinking about how we can think about whether or not like how would we. Sorry, I'm doing it right now because I was lost. The Nightmare Before Christmas, the idea of, uh, the logistics, uh, the very complex system that is Santa's present delivery system. Right. And we need to understand it across context and all of these different things. And if we're trying to optimize it, that doesn't. That we're trying to optimize. But there's so much context missing. How can we be certain that the security frameworks are transferring between the context?

Speaker A: So as we kind of went through this with the term that we kind of have criticized from the get go of vibe coding, of course, the term that's starting to emerge and what we're kind of getting to, I guess, is vibe engineering. So, you know, gives a bit more of an air of sophistication, I think

Speaker D: to it is, uh, it warranted.

Speaker A: That's a very good question. But you know, when it comes to vibe Engineering, we're talking about, you know, hopefully some sophistication in tools, sophistication in practices for humans and for models and agents alike. So, you know, the people factor as well.

Speaker D: So people process technology.

Speaker B: Right?

Speaker D: Because you need experts. Because experts will be better than folks who haven't coded before. I assume m you process. So spec driven development and as you mentioned, the tools.

Speaker B: This all sounds like added rigor to me though. It's what we're adding into the process when we transfer into Vibe Engineering from Vibe coding. Right. Like, we, we know enough to now be, uh, questioning of the outputs, but also we're building rigor into the methodology.

Speaker D: So I guess if that's where Vibe engineering is, where does it transition from Vibe coding to Vibe engineering? Where's the line?

Speaker C: Well, I think Vibe coding is really a person, like one of us sitting with a tool and doing some back and forth. Whereas Vibe engineering is, as you said, how do we put the people and process into that? So, like at fieldpal, we are very much using Vibe coding, that we have engineers doing certain things for Vibe coding, but we need to fit that into a process. So one thing we're doing is we're having agents review pull requests to make sure before a human looks at it, it can highlight some. But that's process. Right? That's us as humans agreeing we will use AI to check some of our pull requests before a human does, because we have agreed that it saves us time on the. So for me, it's that shift between sort of one on one, churning out hundreds of lines of code to let's pick the right tool for the job, put it in a rigorous process where it actually helps and works, and let's have all of the people agree on that and use that process correctly, but also, you know, be able to vote to change that if a better tool comes out, if a better process comes out, if we change to a new framework.

Speaker D: Sounds very democratic. Yeah, I guess as well, if you think about engineering in general, the patterns and practices will evolve over time as people become more expert in their field.

Speaker A: Yeah, yeah, absolutely. And, you know, we've had practices like DevOps for some time now, but there was definitely a time before DevOps. Shocking. I know.

Speaker D: Uh, I may have received a code sent to me in a zip file in one direct cto.

Speaker C: I will reiterate that Vibe coding has made me much better at Git because it deletes stuff.

Speaker B: Okay, Christmas question. Okay, so let's get back to the holiday theme. So, Santa Claude. All right. Christmas is a very complex system of subsystems. We've got to go across user preferences, what the kid wants. Right. Uh, sentiment toward the user, whether they're naughty or nice, time zones. And we have to orchestrate all of that with the supply chain, our elves and toy factory. So as we're looking at the present distribution center for Santa Claude, how do we make sure that we are secure end to end, with each of these contexts and overlapping asks and constraints?

Speaker D: That's a good one for you, given your focus. Security lad.

Speaker A: Yeah, absolutely. So there's a lot to kind of go through there and manage. But, you know, starting with the supply chain side of things, you kind of need to still be cognizant of your own software supply chain, as many of the AppSec frameworks will say. Kind of looking around your libraries and things like that. And as we talked about earlier, a lot of CodeGen tools love React. There's a 10.0 vulnerability out on React at the moment, so if you use the wrong version, you're probably going to get hacked at this point. So, you know, having the knowledge of your software supply chain, building things like a software bill of materials and still doing that by hand, maybe with some AI assistance. I say by hand on a computer.

Speaker C: I was going to say the second

Speaker D: you said by hand, I was like, what?

Speaker C: My hands touch the keyboard.

Speaker A: So kind of like everything you're feeding into that factory or foundry, if you're using something like Azure Foundry to build these apps is kind of crucial. But then as you go to kind of the assembly process, not just kind of the tools and things are you using, you know, the spec driven development, how does that come into it? Um, not just defining a requirements or an architecture or any of the other artifacts we talked about, but enforcing like an adherence to that architecture, I'd say as well. So one of the things I found we have to do sometimes is kind of go back to say Claude or GitHub Copilot and say, hey, so can we just have a look at what we've got so far and how it aligns to our spec? And then it will say, oh, I've done these extra things that weren't on the spec. And you go, okay, but that's not what I asked for. Then it goes, you're absolutely right.

Speaker D: It sounds like to hit your Santa Claude, like supply chain analogy, you need to really focus on the frameworks and processes that we have today and um, not create something new, but build on it iteratively and go where might this differ for AI?

Speaker A: Yeah, yeah, it's, it's evolution, not revolution. I guess if you want to try

Speaker C: and summarize it, Santa Claude leaves his Azure toy foundry with a GitHub copilot instead of a reindeer. Where else we.

Speaker D: I can't handle.

Speaker B: We've gone too far.

Speaker C: Yeah.

Speaker A: Um, okay, so, well, thanks very much everyone for kind of the experiences around AI, generation of code and, and the sprawl and the analogies that we've learned along the way of really forcing in the Christmas message to this. So, uh, no thanks. Very much for playing along on that. Of course, if you want more information, you can go to our website. So that's, for example, where the application security framework we talked about is available. So securing Quest. And, of course, thanks very much to Diana as well.

Speaker B: Yes. And my paper is also hosted on that website. If you want to go ahead and take the lesson of maybe as a human, we should learn, feel free to read my paper on the website. Um, in addition to that, making sure that we are leaning into some of the strategies that we're implementing, and we're considering what that means for the people and human, or the human and AI systems functioning at their best.

Speaker C: Thank you. Awesome. And if my message is, if you have elves on the front line, please visit Feelpal AI and learn more about how we can make those operations on the frontline, present delivery, and other things on the front line more, uh, efficient using some of this super cool AI. And, um, yeah, thanks for having me.

Speaker D: Yeah, thanks, everyone. That was securing the Realm. Now, I'd like to say thank you to Kyndryl for sponsoring this episode, and hopefully you're catching us on the festive tech calendar. So see you all soon. Thank you.

Speaker C: Merry Christmas.